Systems and Methods for Presenting Information Associated With a Three-Dimensional Location on a Two-Dimensional Display

• US 20150170256A1
• Filed: 12/18/2014
• Published: 06/18/2015
• Est. Priority Date: 06/05/2008
• Status: Abandoned Application

• Current Assignee: Aisle411 Inc.
• Original Assignee: Aisle411 Inc.
• Inventors: Pettyjohn, Nathan, Saunders, Ed, Jeffrey, Niarcas, Cannarozzi, Dante
• Application Number:
  US14/575,432
• Publication Number:
  US 20150170256A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/1058 Shopping and product ordering
• CPC Class Codes: H04M 2203/355 Interactive dialogue design tools, features or methods
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/42348 Location-based services which utilize the location information of a target
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

Systems and methods for creating an augmented reality mobile device application and using such an application in a retail environment to display marketing messages to a user.

Claims

1. A method for generating augmented reality area data comprising:

providing an augmented reality data gathering device having a plurality of cameras, a plurality of orientation and movement sensors, and a non-volatile computer-readable storage medium;

providing vendor data comprising;

a vendor data coordinate system;

merchandizing fixture data comprising a plurality of merchandizing fixture data sets, each one of said merchandizing fixture data sets having fixture locational coordinates and/or dimensions for a merchandizing fixture in a retail store, said fixture locational coordinates being coordinates in said vendor data coordinate system;

a plurality of product data sets, each one of said product data sets having product data about a product and product locational coordinates corresponding to the location of said product on at least one of said merchandizing fixtures in said retail store, said product locational coordinates being coordinates in said vendor data coordinate system;

generating in said non-volatile computer-readable storage medium a three-dimensional model of the interior configuration of said retail store, said three-dimensional model comprising;

an internal coordinate system;

an origin point in said internal coordinate system, said origin point corresponding to a location in said retail store;

an internal camera, said internal camera having a default internal location at said origin point in said three-dimensional model and a default orientation in said three-dimensional model;

for each one of said merchandizing fixture data sets in said merchandizing fixture data, translating said fixture locational coordinate for said merchandizing fixture from said vendor data coordinate system to said internal coordinate system of said three-dimensional model and generating in said generated three-dimensional model an opaque collidable object having a volume defined by said translated coordinates;

placing said augmented reality data gathering device at said location in said retail store corresponding to said origin point and orienting said augmented reality data gathering device such that the orientation of said augmented reality data gathering device relative to said retail location corresponds to said default orientation of said internal camera in said three-dimensional model;

moving said augmented reality data gathering device through said retail location;

moving said internal camera within said three-dimensional model in real-time with said movement of said augmented reality data gathering device;

during said movement of said augmented reality data gathering device, determining a location of said augmented reality data gathering device in said retail location and said plurality of cameras capturing a plurality of image datasets about said retail location at said determined location of said augmented reality data gathering device and said plurality of orientation sensors capturing orientation data about said augmented reality data gathering device at said determined location;

storing in said memory area data comprising;

at least one captured image dataset;

at least one captured orientation dataset; and

said detected location of said augmented reality data gathering device in said retail location when said at least one captured image dataset and at least one captured orientation dataset were captured.

2. The method of claim 1, wherein said fixture locational coordinates and/or dimensions comprise x-coordinates and y-coordinates for the location of a merchandizing fixture in said retail store.

3. The method of claim 2, wherein said fixture locational coordinates and/or dimensions further comprise a z-coordinate for the height of a merchandizing fixture in said retail store.

4. The method of claim 1, wherein said at least one stored captured image dataset comprises at least in part data about a visual element of said retail location.

5. The method of claim 4, wherein said visual element is selected from the group consisting of:

an edge, a corner, a merchandizing fixture, furniture, flooring, ceiling, lighting, signage, a door, a doorway, a window, and a wall.

6. The method of claim 1, wherein said determining a location of said augmented reality data gathering device in said retail location comprises determining the location of internal camera in said three-dimensional model, said location of said internal camera being at least a two-dimensional coordinate in said internal coordinate system of said three-dimensional model.

7. The method of claim 1, wherein said plurality of orientation sensors capturing orientation data about said augmented reality data gathering device at said determined location comprises determining the direction said internal camera is facing in said three-dimensional model.

8. A method for providing messages to a consumer comprising:

providing a mobile computing device comprising;

a non-transitory computer-readable medium having thereon an augmented reality software application, said application having access to an augmented reality area description for a retail environment, said area description comprising a plurality of image datasets, each one of said image datasets having a corresponding coordinate in said retail environment, and said application having access to a plurality of messages, each one of said messages having a corresponding coordinate in said retail environment;

a display operable by said application; and

an imaging device operable by said application;

in a retail environment, said application causing said imaging device to capture in real-time image data about said retail environment and said application causing said display to display in real-time said captured image data as images;

locating in said area description at least one image dataset in said plurality of image datasets, said at least one image dataset corresponding to said image data about said retail environment captured in real-time by said imaging device;

selecting one or more messages from said plurality of messages, said one or more messages being selected based upon the proximity of said determined location of said computing device to said selected message'"'"'s corresponding coordinate in said retail environment;

displaying on said display at least one of said selected one or more messages.

9. The method of claim 8, wherein at least one of said selected one or more messages is a marketing message.

10. The method of claim 8, wherein said plurality of messages is stored on said memory.

11. The method of claim 8, wherein said augmented reality area description is accessible over a telecommunications network.

12. The method of claim 8, wherein said plurality of messages is accessible over a telecommunications network.

13. The method of claim 8, further comprising:

wherein said application has access to a previously generated three-dimensional virtual model of said retail environment, said three-dimensional virtual model having an internal coordinate system and an internal camera;

wherein said corresponding coordinates in said retail environment for said plurality of image datasets are coordinates in said internal coordinate system;

wherein said corresponding coordinates in said retail environment for said plurality of messages are coordinates in said internal coordinate system;

moving said internal camera within said three-dimensional model in real-time with said movement of said mobile computing device;

wherein said selecting step comprises calculating, in the internal coordinate system, the distance between the locational coordinates for said internal camera and the corresponding coordinate of each message and select a message for display if said calculated distance is within a pre-defined trigger threshold.

14. The method of claim 8, wherein said message is a coupon.

15. The method of claim 14, wherein said message is a user interface element selectable by a user to redeem said coupon.

16. The method of claim 15, wherein the user selects the element by tapping the displayed message.

17. The method of claim 8, wherein said message is an indication of a product category.

18. The method of claim 17, wherein said product category is selected from the group consisting of:

gluten-free;

heart-healthy;

vegetarian;

vegan;

low-sodium;

low-sugar;

fair trade;

organic;

lactose-free; and

local.

19. The method of claim 8, further comprising:

said mobile computing device comprising an orientation sensor;

in said retail environment, said application causing said imaging device to generate in real-time orientation data about the orientation of said mobile computing device when said real-time image data is capture;

said locating step further comprising locating in said area description at least one image dataset in said plurality of image datasets, said at least one image dataset having orientation data corresponding to said orientation of said generated real-time orientation data of said mobile computing device.