Augmented reality assisted shopping

• US 20080071559A1
• Filed: 09/19/2006
• Published: 03/20/2008
• Est. Priority Date: 09/19/2006
• Status: Abandoned Application

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Arrasvuori, Juha
• Application Number:
  US11/523,162
• Publication Number:
  US 20080071559A1
• Time in Patent Office: Days
• US Class Current: 705/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings

Abstract

Facilitating shopping for a tangible object via a network using a mobile device involves obtaining a graphical representation of a scene of a local environment using a sensor of the mobile device. Graphical object data that enables a three-dimensional representation of the tangible object to be rendered on the mobile device is obtained via the network, in response to a shopping selection. The three-dimensional representation of the tangible object is displayed with the graphical representation of the scene via the mobile device so that the appearance of the tangible object in the scene is simulated.

Claims

1. A method comprising:

facilitating shopping for a tangible object via a network using a mobile device;

obtaining a graphical representation of a scene of a local environment using a sensor of the mobile device;

obtaining, via the network, graphical object data that enables a three-dimensional representation of the tangible object to be rendered on the mobile device, wherein the graphical object data is obtained in response to a shopping selection; and

displaying, via the mobile device, the three-dimensional representation of the tangible object with the graphical representation of the scene so that the appearance of the tangible object in the scene is simulated.

2. The method of claim 1, further comprising:

facilitating user selection of a plurality of tangible objects;

obtaining a plurality of graphical object data that enables a three-dimensional representation of each of the plurality of tangible objects to be rendered on the mobile device;

displaying each three-dimensional representation with the graphical representation of the scene so that the appearance of each of the plurality of tangible objects is simulated in the scene, one after another, in response to user selections.

3. The method of claim 1, wherein obtaining the graphical representation of the scene comprises capturing a still image via a camera of the mobile device.

4. The method of claim 1, wherein obtaining the graphical representation of the scene comprises capturing a video data via a video camera of the mobile device.

5. The method of claim 4, further comprising updating the graphic representation of the object based on changes in a camera view of the mobile device.

6. The method of claim 1, wherein the tangible object comprises a surface covering, and wherein displaying the three-dimensional representation of the tangible object comprises overlaying the three-dimensional representation on a surface detected in the scene.

7. The method of claim 1, further comprising sending the graphical representation of the scene to a service element via the network, and wherein obtaining the three-dimensional representation of the tangible object is facilitated in response to the sending of the representation of the scene.

8. The method of claim 7, further comprising receiving geometry data of the scene from the service element via the network in response to sending the graphical representation of the scene, wherein the geometry data assists the mobile device in accurately displaying the three-dimensional representation of the tangible object with the graphical representation of the scene.

9. The method of claim 7, further comprising establishing a voice call with an operator with access to the service element, and wherein the operator facilitates obtaining the three-dimensional representation of the tangible object in response to the sending of the representation of the scene.

10. The method of claim 1, further comprising facilitating the purchase of the tangible object via the mobile device in response to a user selection of the graphical representation of the tangible object.

11. A mobile device comprising:

a sensor;

a display;

a network interface capable of communicating via a network;

a processor coupled to the network interface, the sensor, and the display; and

a memory coupled to the processor, the memory including instructions that cause the processor to,

facilitate shopping for a tangible object via the network;

obtain a graphical representation of a scene of a local environment using the sensor;

obtain, via the network, graphical object data that enables a three-dimensional representation of the tangible object to be rendered on the display, wherein the graphical object data is obtained in response to a shopping selection; and

present on the display the three-dimensional representation of the tangible object with the graphical representation of the scene so that the appearance of the tangible object in the scene is simulated.

12. The mobile device of claim 11, wherein the instructions cause the processor to obtain the graphical object data in response to a selection of the tangible object made during the shopping via the mobile device.

13. The mobile device of claim 11, wherein the sensor comprises a still camera, and wherein obtaining the graphical representation of the scene comprises capturing a still image via the still camera.

14. The mobile device of claim 11, wherein the sensor comprises a video camera, and wherein obtaining the graphical representation of the scene comprises capturing a video image via the video camera.

15. The mobile device of claim 14, wherein the instructions further cause the processor to update the three-dimensional representation of the tangible object on the display based on changes in a camera view of the mobile device.

16. The mobile device of claim 15 further comprising a location sensor, and wherein the changes in the camera view are detected via the location sensor.

17. The mobile device of claim 11, wherein the tangible object comprises a surface covering, and wherein the instructions further cause the processor to overlay the three-dimensional representation of the tangible object on a surface detected in the scene.

18. The mobile device of claim 11, wherein the instructions further cause the processor to,

send the graphical representation of the scene to a service element via the network;

and

receive geometry data of the scene from the service element via the network in response to sending the graphical representation of the scene, wherein the geometry data assists the mobile device in accurately displaying the three-dimensional representation of the tangible object within the graphical representation of the scene.

19. The mobile device of claim 18, wherein the instructions further cause the processor to facilitate a voice call between the mobile device and an operator via the service element, wherein the operator facilitates determining the geometry data.

20. A computer-readable medium having instructions stored thereon which are executable by a mobile device capable of being coupled to a network for performing steps comprising:

facilitating shopping for a tangible object via the network using the mobile device;

obtaining a graphical representation of a scene of a local environment using a sensor of the mobile device;

obtaining, via the network, graphical object data that enables a three-dimensional representation of the tangible object to be rendered on the mobile device, wherein the graphical object data is obtained in response to a shopping selection; and

displaying the graphic representation of the object with the graphical representation of the scene so that the appearance of the tangible object in the scene is simulated.

21. A server, comprising:

a network interface capable of communicating via a network;

a processor coupled to the network interface; and

a memory coupled to the processor, the memory including instructions that cause the processor to,

receive a request for e-commerce data related to a tangible object from a mobile device that is shopping for the tangible object via the network;

determine, based on the request, graphical object data that enables a three-dimensional representation of the tangible object to be rendered on the mobile device; and

send the e-commerce data and graphical object data to the mobile device so that the three-dimensional representation of the tangible object can be overlaid with a scene taken from a camera of the mobile device.

22. The server of claim 21, wherein the instructions further cause the processor to:

receive a graphical representation of the scene from the mobile device;

determine geometry data of the scene that assists the mobile device in overlaying the three-dimensional representation of the tangible object with the scene; and

send the geometry data to the mobile device via the network.

23. The server of claim 22, wherein the instructions further cause the processor to facilitate a voice call between the mobile device and an operator, wherein the operator facilitates determining the geometry data of the scene that assists the mobile device in overlaying the three-dimensional representation of the tangible object with the scene.