PROVIDING CITY SERVICES USING MOBILE DEVICES AND A SENSOR NETWORK

• US 20170325082A1
• Filed: 07/25/2017
• Published: 11/09/2017
• Est. Priority Date: 12/11/2009
• Status: Active Grant

• Current Assignee: Mentis Services France
• Original Assignee: Mentis Services France
• Inventors: Rowe, Richard E., Fiorucci, Jean-Louis
• Granted Patent: US 10,462,621 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G08G 1/14 indicating individual free spaces in parking areas
• CPC Class Codes: G08G 1/147 where the parking area is within an open public zone, e.g. city centre
• CPC Class Codes: H04M 2203/359 Augmented reality
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 2242/15 Information service where the information is dependent on the location of the subscriber
• CPC Class Codes: H04M 3/42348 Location-based services which utilize the location information of a target
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

Apparatus and methods related providing city services, such as parking, are described. A mobile device can be configured to receive information from local sensor nodes, such as parking sensor nodes, in the vicinity of the mobile device. In a parking application, the mobile device located in a moving vehicle can be configured to locate available parking based upon the information received from the parking sensor nodes. In other embodiments, the mobile device can be utilized in a retail establishment in conjunction with a remote server to display eye-level image data taken at various locations throughout the retail establishment. The eye-level image data can include products displayed throughout the retail establishment and can be augmented with one or more indicators that indicate product placement locations associated with the products.

Claims

1. A method comprising:

receiving a first image having a first plurality of pixels wherein the first image shows a portion of an interior of a retail establishment wherein the first image is captured using a camera on a mobile device carried by a customer within the retail establishment and wherein the first image shows one or more structures for displaying products and one or more products placed on the structures in the retail establishment;

receiving a current orientation of the mobile device based upon accelerometer data obtained from accelerometers on the mobile device;

receiving depth data for each of the first plurality pixels wherein the depth data indicates a physical distance between the camera and a location in physical space associated with each of the first plurality of pixels;

determining a first physical location in the retail establishment associated with a first pixel in the first image;

based upon the depth data, the current orientation of the mobile device and the first physical location associated with the first pixel in the first image;

determining physical locations associated with each of the first plurality of pixels in the retail establishment;

determining a second physical location of a first product placement location within the retail establishment;

based upon the physical location associated with each of the first plurality of pixels, determining a second pixel associated with the second physical location of the first product placement location; and

generating a second image formed from the first image wherein the second image includes an indicator formed from a second plurality of pixels rendered over the first plurality of pixels and rendered near the second pixel in the first image wherein the second image is output to a display on the mobile device.

2. The method of claim 1, further comprising recognizing an object in the first image wherein the first physical location is determined from a known physical location of the object that is recognized.

3. The method of claim 1, based upon the depth data and the current orientation, determining a physical location of the camera in the retail establishment.

4. The method of claim 1, further comprising, based upon the depth data, determining a size to render the indicator in the second image.

5. The method of claim 4, wherein the size of the indicator is rendered smaller in the second image when the second physical location is farther away from the camera as compared to the size of the indicator rendered when the second physical location is closer to the camera.

6. The method of claim 1, wherein the depth data is received from a 3-D depth sensing camera coupled to the mobile device.

7. The method of claim 1, wherein the depth data is derived from images taken from two cameras coupled to the mobile device.

8. The method of claim 1, further comprising receiving a filter setting including a group of products wherein the second physical location of the first product placement location is associated with a particular product within the group of products.

9. The method of claim 1, further comprising receiving a request to identify a location of a particular product wherein second physical location of the first product placement location is associated with the particular product.

10. The method of claim 1, wherein the display is a touch screen display and wherein touching the indicator on the touch screen display causes information about a particular product at the first product placement location to be output to the display.

11. The method of claim 1, further comprising receiving sensor data from a plurality of sensor nodes wherein the sensor data is used to determine a physical location of the mobile device in the retail establishment.

12. A method comprising:

receiving a first image having a first plurality of pixels wherein the first image shows a portion of an interior of a retail establishment wherein the first image is captured using a camera on a mobile device carried by a customer within the retail establishment and wherein the first image shows one or more structures for displaying products and one or more products placed on the structures in the retail establishment;

receiving a current orientation of the mobile device based upon accelerometer data obtained from accelerometers on the mobile device;

recognizing a first object in the first plurality of pixels wherein the first object is associated with a known physical location and a known physical dimension;

based upon the known physical location of the object, determining a first physical location in the retail establishment associated with a first pixel in the first image;

determining a size of the object in pixels;

based upon the size of the object in pixels and a known physical dimension of the object, determining a scaling factor between pixel dimensions and physical dimensions;

determining a second physical location of a first product placement location within the retail establishment associated with a second object;

based upon the scaling factor, the current orientation of the mobile device and the first physical location associated with the first pixel in the first image, determining a second pixel associated with the second physical location of the first product placement location without identifying the second object in the first image; and

generating a second image formed from the first image wherein the second image includes an indicator formed from a second plurality of pixels rendered over the first plurality of pixels and rendered near the second pixel in the first image wherein the second image is output to display on the mobile device.

13. The method of claim 12, further comprising receiving sensor data from a plurality of sensor nodes wherein the sensor data is used to determine a physical location of the mobile device in the retail establishment.

14. The method of claim 13, wherein the sensors nodes are configured to receive wireless signals from the mobile device.

15. The method of claim 12, further comprising generating a mapping between physical coordinates in the retail establishment and pixel coordinates in the first image.

16. The method of claim 15, based upon the mapping determining locations of a plurality of products at different pixel locations in the first image.

17. The method of claim 12, wherein the display is a touch screen display and wherein touching the indicator on the touch screen display causes information about a product at the product placement location to be output to the display.

18. The method of claim 12, further comprising receiving a filter setting including a group of products wherein the second physical location of the first product placement location is associated with a particular product within the group of products.

19. The method of claim 12, further comprising receiving a request to identify a location of a particular product within the retail establishment wherein second physical location of the first product placement location is associated with the particular product.

20. The method of claim 12, wherein the display is a touch screen display and wherein touching the indicator on the touch screen display causes information about a particular product at the first product placement location to be output to the display.