Method, medium, and system for facilitating electronic commercial transactions in an augmented reality environment

• US 9,665,983 B2
• Filed: 04/10/2014
• Issued: 05/30/2017
• Est. Priority Date: 05/01/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Zambala LLLP
• Inventors: Spivack, Nova T.
• Primary Examiner(s): Zimmerman, Matthew
• Application Number:
  US14/249,541
• Publication Number:
  US 20140222558A1
• Time in Patent Office: 1,146 Days
• Field of Search: 705 261- 272
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06Q 30/0623 Item investigation
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence

Abstract

Techniques are disclosed for facilitating electronic commerce in an augmented reality environment. In some embodiments, a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via the augmented reality environment. In some embodiments, a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets. Among other advantages, embodiments disclosed herein provide an intuitive and integrated user experience in shopping using augmented reality devices, thereby reducing the consumer user'"'"'s effort in conducting such activities, reducing necessary sales personnel and their working hours, and increasing sales.

Claims

1. A method for facilitating an electronic transaction in an augmented reality environment of a physical product present in a real world environment, the method comprising:

automatically identifying the physical product present in the real world environment based on a perspective of a user, the perspective of the user being determined by a mobile device used by the user;

wherein, the physical product in the real world environment that is identified, is determined based on a geolocation of the user and a timing parameter;

wherein, the geolocation is determined by one or more sensors associated with the mobile device;

wherein, the timing parameter includes, one or more of, a sale event, store hours, availability of the physical product;

rendering, the augmented reality environment, based on the geolocation of the user on a display of the mobile device;

wherein, the augmented reality environment depicts information associated with the physical product of the real world environment for use in the electronic transaction;

responsive to failure to automatically identify or detect the physical product, entering a learning mode to prompt the user to facilitate disambiguation in identifying the physical product or recognition of the physical product.

2. The method of claim 1, wherein the information is rendered or projected translucently in the augmented reality environment on the display of the mobile device and is disposed adjacent to or partially overlaid with depiction of the physical product of the real world environment in the augmented reality environment.

3. The method of claim 1, further comprising:

acquiring the information from a server that enables the electronic transaction in the augmented reality environment;

wherein the information includes annotation, a review, or a tag of the physical product entered from another user.

4. The method of claim 1, further comprising:

initiating, via the display, the electronic transaction in the augmented reality environment using the information.

5. The method of claim 1, further comprising:

displaying an advertisement relevant to the physical product in the augmented reality environment based on the information.

6. The method of claim 1, wherein, the physical product is identified through image or textual recognition techniques.

7. The method of claim 1, further comprising:

using information about the physical product, querying a server having a database to assist the automatic identification of the physical product.

8. The method of claim 1, further comprising:

receiving assistance to detect or recognise the physical product in the learning mode.

9. The method of claim 1,

wherein, the learning mode is provided as a selectable function in the augmented reality environment.

10. The method of claim 1,

wherein, the learning mode includes using one or more of (I) at least partial input of a name, (ii) metadata (iii) boundary defined using a pointer, or (iv) images from multiple views of the physical product to generate a recognition pattern.

11. The method of claim 1, further comprising obtaining views of the physical product to generate a recognition pattern;

wherein, the recognition pattern is further used to improve image recognition techniques for identifying or detecting physical products in the real world environment.

12. The method of claim 1, further comprising:

automatically performing a search for products in the real world that are available for access to the user in the augmented reality environment via the mobile device based on the geolocation of the user and the timing parameter.

13. The method of claim 1, wherein, the timing parameter includes, one or more of, a current time, a relative time, or a time period.

14. The method of claim 1, wherein, the physical product is identified through image recognition techniques by detecting shape or form of the physical product.

15. The method of claim 1, wherein, the physical product is identified through image recognition techniques by detecting shape or form of the physical product.

16. The method of claim 1, wherein the one or more sensors includes a location sensor comprising a GPS sensor or a proximity sensor.

17. The method of claim 1, wherein, the one or more sensors includes, one or more of, a user stimulus sensor, a motion or gesture sensor, or a wireless sensor;

wherein the wireless sensor includes a radio frequency (RF) sensor to detect RFIDs.

18. The method of claim 1 wherein, the one or more sensors includes, one or more of, accelerometers, gyroscopic gauges, temperature sensors, heart beat sensors, touch sensitive sensors including touch-screens, fingerprint sensors, a light sensor or an image sensor.

19. The method of claim 1, wherein, the mobile device includes one or more of, eyewear, visor, goggles.

20. The method of claim 1, wherein, the mobile device includes, one or more of a mobile phone, a smart phone, a tablet, or a phablet.

21. The method of claim 5, wherein, the advertisement is identified or detected based on one or more of, velocity or speed of movement of the mobile device, an ambient temperature in which the mobile device physically resides, or a pointing direction of the mobile device.

22. The method of claim 5, wherein, the advertisement is identified or detected based on one or more of:

(i) location, (ii) time, (iii) interest or profile of the user of the mobile device.

23. A system for facilitating a transaction of a physical product or real life service via an augmented reality platform, the system comprising:

a processor;

a memory having stored thereon instructions which, when executed by the processor, cause the system to;

automatically identify, in a real world environment, presence of the physical product or the real life service, based on a perspective of a user, the perspective of the user being determined by a mobile device used by the user;

render, the augmented reality platform to facilitate the transaction of the physical product or the real life service,

responsive to failure to automatically identify or detect the physical product or the real life service, enter a learning mode to prompt the user to facilitate disambiguation in identifying the real life service or recognition of the physical product;

wherein, the augmented reality platform depicts information usable to conduct the transaction of the physical product or the real life service via the augmented reality platform;

wherein the presence of the product or service is identified in the augmented reality platform based on a geographic location of the physical product or real life service and further based on a timing parameter, wherein the geographic location is determined by one or more sensors associated with the mobile device;

wherein, the timing parameter includes, one or more of, a sale event, store hours, availability of the physical product.

24. The system of claim 23, wherein the presence of the physical product in the real world environment is detected or identified by a shape or other physical characteristics for presentation in the augmented reality platform to facilitate the transaction.

25. The system of claim 23, wherein the instructions further cause the system to:

receive input from the user to disambiguate an identity of the physical product or real life service, wherein the user'"'"'s input includes at least partially a name of the physical product or real life service;

prompt the user to enter an advisory data for the physical product or real life service; and

share the user'"'"'s entry to another user of the augmented reality platform;

receive advisory data associated with the physical product or real life service, the advisory data being added by another user of the augmented reality platform.

26. The system of claim 23, wherein the instructions further cause the system to:

initiate the transaction of the physical product or the real life service via the system based on responses to the presented information from the user.

27. A non-transitory computer readable medium for facilitating an electronic transaction in a augmented reality environment, the non-transitory computer readable medium including executable instructions which when executed by a computer processor cause the computer processor to perform a method comprising:

automatically identifying a physical product in a real world environment based on a perspective of a user, the perspective of the user being determined by a device used by the user;

wherein, the physical product in the real world environment that is identified is recognized based on a geolocation of the user and a timing parameter;

wherein, the geolocation is determined by one or more sensors associated with the user device;

wherein, the timing parameter includes, one or more of, a sale event, store hours, availability of the physical product;

rendering, a visible display of the augmented reality environment;

wherein, the augmented reality environment depicts information associated with the physical product of the real world environment;

wherein the information is rendered or projected translucently in the augmented reality environment and is disposed adjacent to or partially overlaid with depiction of the physical product of the real world environment in the augmented reality environment;

responsive to failure to automatically identify or detect the physical product, entering a learning mode to prompt the user to facilitate disambiguation in identifying the physical product or recognition of the physical product.