Providing city services using mobile devices and a sensor network

• US 9,749,823 B2
• Filed: 04/19/2010
• Issued: 08/29/2017
• Est. Priority Date: 12/11/2009
• Status: Active Grant

• Current Assignee: Mentis Services France
• Original Assignee: Mentis Services France
• Inventors: Rowe, Richard E., Fiorucci, Jean-Louis
• Primary Examiner(s): CRAWLEY, TALIA F
• Application Number:
  US12/763,077
• Publication Number:
  US 20110143779A1
• Time in Patent Office: 2,689 Days
• Field of Search: 705 28, 705 26, 705 1464
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G08G 1/14 indicating individual free spaces in parking areas
• CPC Class Codes: G08G 1/147 where the parking area is within an open public zone, e.g. city centre
• CPC Class Codes: H04M 2203/359 Augmented reality
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 2242/15 Information service where the information is dependent on the location of the subscriber
• CPC Class Codes: H04M 3/42348 Location-based services which utilize the location information of a target
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

Apparatus and methods related providing city services, such as parking, are described. A mobile device can be configured to receive information from local sensor nodes, such as parking sensor nodes, in the vicinity of the mobile device. In a parking application, the mobile device located in a moving vehicle can be configured to locate available parking based upon the information received from the parking sensor nodes. In other embodiments, the mobile device can be utilized in a retail establishment in conjunction with a remote server to display eye-level image data taken at various locations throughout the retail establishment. The eye-level image data can include products displayed throughout the retail establishment and can be augmented with one or more indicators that indicate product placement locations associated with the products.

Claims

1. A system comprising

a server including a processor and a memory and at least one communication interface;

said processor designed or configured

1) to receive a first image including pixel data showing a portion of an interior of a retail establishment wherein the first image is captured using a camera on a mobile device carried by a customer within the retail establishment and wherein the first image shows one or more structures for displaying products and one or more products placed on the structures in the retail establishment;

     2) receive location information and orientation information from the mobile device wherein the orientation information indicates a current orientation of the mobile device,

     3) store the first image to the memory;

     4) recognize in the pixel data associated with the first image at least one object to identify the portion of the interior of the retail establishment shown in the first image to map the pixel data to physical locations in the interior of the retail establishment;

     5) based upon the identified portion of the retail establishment shown in the first image, determine a pixel location in the pixel data associated with the first image to draw an indicator wherein the indicator points to at least one product placement location visible in the first image;

     6) based upon the orientation information and the pixel data from the first image, generate a second image including a portion of the pixel data associated with the first image and the indicator drawn at the determined pixel location in the first image and

     7) send the second image to the mobile device in communication with the server via the communication interface wherein the second image is output to a display device associated with the mobile device to help the customer to find a product within the interior of the retail establishment;

a sensor node, placed within the retail establishment, configured to communicate with the server via the at least one communication interface and the mobile device via a wireless interface; and

the mobile device including the camera, the display device, the wireless communication interface, a speaker, a microphone, a cellular communication interface, a GPS receiver, a local, a CPU and a memory, the processor configured to communicate with remote devices via the cellular communication, the server via the sensor, receive GPS information from the GPS receiver, determine a mobile device location using the GPS information, receive an input signal associated with the camera, in response to the input signal, cause the camera to capture the first image, receive image data from the camera including the first image, output the first image to the display device wherein the first image includes the at least one product placement location, send the first image data, the location data and the orientation data, to the server via the wireless interface, receive from server the second image and output the second image data to the display device wherein the second image includes the at least one product placement location visible in the first image and the indicator which points to the at least one product placement location added by the server.

2. The system of claim 1, wherein the camera is positioned between 4-7 feet above a floor of the retail establishment.

3. The system of claim 1 wherein the memory is configured to store a physical size, a physical location or combinations thereof of the at least one object.

4. The system of claim 3, wherein the physical size or the physical location of the at least one object identified in the first image is used to determine where to draw the first indicator.

5. The system of claim 4, wherein the physical size or the physical location of the at least one object identified in the first image is used to determine a size the first indicator that is to be drawn.

6. The system of claim 1, wherein the at least one object is a product proximate to the at least one product placement location that is visible in the first image.

7. The system of claim 1, wherein the processor is further configured to determine a size of the indicator to account for image perspective.

8. The system of claim 7, wherein the size of the indicator is determined such that when the determined location of the indicator is at a location that appears closer in the first image, the indicator is drawn larger and when the determined location of the indicator is at a location that appears farther away in the first image, the indicator is drawn smaller.

9. The system of claim 1, wherein the memory is further configured to store an image database including images taken from an eye-level perspective throughout the interior of the retail establishment.

10. The system of claim 9, wherein the images in the image database are updated one or more times per day to reflect changes in an arrangement of products placed on the one or more structures for displaying products.

11. The system of claim 9, wherein the processor is further designed or configured to receive an image from the mobile device carried by the customer within the retail establishment and to determine whether to add the image to the image database.

12. The system of claim 9, wherein the processor is in communication with one or more cameras distributed throughout the retail establishment and wherein the processor is further configured to receive images taken from an eye-level perspective from the one or more cameras and add the images from the one or more cameras to the image database.

13. The system of claim 9, wherein the processor is further designed or configured to 1) receive from the mobile device carried by the customer information identifying a product and in response find an image stored in the image database that shows a product placement location for the identified product;

and

     2) send the found image showing the product placement location for the identified product to the mobile device.

14. The system of claim 1, wherein the processor is further configured to determine locations of a plurality of indicators to draw on the first image wherein each of the plurality of indicators is proximate to a different product placement location.

15. The system of claim 14, wherein at least one of the different product placement locations is empty of the product normally placed at the product placement location.

16. The system of claim 14, wherein the processor is further configured to receive a filter setting from the mobile device and based upon, the filter setting select a set of different product placement locations for which indicators are to be drawn on the first image.

17. The system of claim 14, wherein the server is a kiosk.

18. A system comprising:

a kiosk comprising a processor and a memory and at least one communication interface;

said processor designed or configured

     1) to receive first image including pixel data showing a portion of an interior of a retail establishment wherein the first image is captured using a camera on a mobile device carried by a customer within the retail establishment and wherein the first image shows one or more structures for displaying products and one or more products placed on the structures in the retail establishment;

     2) receive location information and orientation information from the mobile device wherein the orientation information indicates a current orientation of the mobile device,

     3) store the first image to the memory;

     4) recognize in the pixel data associated with the first image at least one object to identify the portion of the interior of the retail establishment shown in the first image to map the pixel data to physical locations in the interior of the retail establishment;

     5) based upon the identified portion of the retail establishment shown in the first image, determine a pixel location in the pixel data associated with the first image to draw an indicator wherein the indicator points to at least one product placement location visible in the first image;

     6) based upon the orientation information and the pixel data from the first image, generate a second image including a portion of the pixel data associated with the first image and the indicator drawn at the determined pixel location in the first image and

     7) send the second image to the mobile device in communication with the server via the communication interface wherein the second image is output to a display device associated with the mobile device to allow the customer to locate a product placed at the product placement location visible in the second image; and

a plurality of cameras is configured to take images within the retail establishment and wherein the kiosk is configured to receive the images via the at least one communication interface.