Data services based on gesture and location information of device

• US 8,467,991 B2
• Filed: 05/08/2009
• Issued: 06/18/2013
• Est. Priority Date: 06/20/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Khosravy, Moe, Novik, Lev, Woodcock, Katrika Marie
• Primary Examiner(s): PAULS, JOHN A
• Application Number:
  US12/437,857
• Publication Number:
  US 20090319181A1
• Time in Patent Office: 1,502 Days
• Field of Search: 340/815.4, 340/995.1, 340/995.14, 340/995.15, 340/995.17, 382/118, 702/153
• US Class Current: 702/153
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01C 21/3811 Point data, e.g. Point of Interest [POI]
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass
• CPC Class Codes: H04W 4/21 for social networking applications

Abstract

With the addition of directional information and gesture based input in a location based services environment, a variety of service(s) can be provided on top of user identification or interaction with specific object(s) of interest. For instance, when a user gestures at or points at a particular item, or gestures at a particular location or place, this creates an opportunity, e.g., an advertising opportunity, for anyone having an interest in that particular item or place to communicate with the user regarding that item or related items at a point in time when the user'"'"'s focus is on the particular item. User context for the interaction can also be taken into account to supplement the provision of one or more interactive direction based services.

Claims

1. A method implemented at a portable device that includes one or more processors and memory storing instructions which, when executed by the one or more processors, implement the method for interacting with representations one or more local points of interest at the device through gestures made with the portable device, the method comprising:

an act of the portable device determining one or more points of interest that are contextually relevant to a physical location of the portable device based on location information, the location information representing a global physical position of the portable device, the one or more points of interest comprising at least one of a stationary physical object or a geographical location;

an act of the portable device determining that the portable device is pointing toward at least one of the points of interest that are contextually relevant to the physical location of the portable device based on direction information, the direction information representing a physical orientation of the portable device within a three-dimensional space;

an act of the portable device displaying at least a representation of the at least one of the points of interest toward which the portable device is pointing at a display of the portable device, including;

in response to detecting that a viewing plane of the portable device is aligned with a substantially horizontal plane that is substantially parallel to a ground plane, displaying a two-dimensional (2D) map-based view of a local geographical area instead of a three-dimensional 3D perspective view, including displaying a representation of the at least one of the points of interest within the 2D map-based view; and

in response to detecting that the viewing plane of the portable device is aligned with a substantially vertical plane that is substantially orthogonal to the ground plane, displaying the 3D perspective view of the local geographical area instead of the 2D map-based view, including displaying a representation of the at least one of the points of interest within the 3D perspective view;

an act of the portable device detecting a gesture through detecting a physical movement of the portable device, detecting the gesture comprising detecting changes in one or more of (i) the location information, (ii) the direction information, or (iii) motion information; and

an act of the portable device initiating at least one operation with respect to the at least one of the points of interest in response to detecting the gesture.

2. The method of claim 1, wherein the representation of the at least one of the points of interest is an image captured by a camera at the portable device and wherein the method includes overlaying information received in response to the at least one operation over the image.

3. The method of claim 1, wherein the one or more points of interest comprises at least one person within a field of view of the portable device.

4. The method of claim 3, wherein initiating the at least one operation comprises one or more of:

(i) displaying information about friends of the at least one person, (ii) initiating a call to the at least one person, (iii) starting a message with the at least one person, (iv) viewing a web site associated with the at least one person, (v) adding the at least one person to a list, (vi) determining an availability of the at least one person, or (vii) sending a friend request with the at least one person.

5. The method of claim 1, wherein the one or more points of interest comprises one or more venues within a determined distance from the portable device and within a field of view of the portable device.

6. The method of claim 5, wherein initiating the at least one operation comprises one or more of:

(i) displaying information events at the venue, (ii) purchasing tickets for events at the venue, (iii) determining operating hours for the venue, (iv) viewing promotional content about the venue, (v) providing feedback for the venue, or (vi) navigating to a website for the venue.

7. The method of claim 1, wherein the one or more points of interest comprises one or more properties for sale within a determined distance from the portable device and within a field of view of the portable device.

8. The method of claim 7, wherein initiating the at least one operation comprises one or more of:

(i) displaying agent contact information for the property, (ii) displaying a listing for the property, (iii) requesting an open house for the property, or (iv) displaying a price for the property.

9. The method of claim 1, wherein the motion information comprises one or more of gyroscopic, accelerometer, or compass information.

10. The method of claim 1, wherein initiating at least one operation comprises sending a request for information about the at least one of the points of interest.

11. The method of claim 1, wherein initiating at least one operation comprises displaying information about the at least one of the points of interest.

12. The method of claim 1, further comprising an act of the portable device receiving user input at a user interface requesting information about the at least one of the points of interest.

13. The method of claim 1, wherein detecting the gesture comprises detecting changes in both the direction information and the motion information.

14. The method of claim 1, wherein detecting the gesture comprises tracking a path of motion of the portable device.

15. A method, implemented at a portable computing device that includes one or more processors, for displaying different views at the portable device depending on a detected orientation of a viewing plane of the portable device, the method comprising:

an act of a portable device determining that a viewing plane of the portable device is aligned with a substantially horizontal plane that is substantially parallel to a ground plane, and based at least in part on the viewing plane being aligned with the substantially horizontal plane;

an act of the portable device displaying a two-dimensional (2D) map-based view of a local geographical area based on location and direction information measured by the portable device; and

an act of the portable device displaying at least one indication of a particular point of interest that is contextually relevant to the portable device within the 2D map-based view of the geographical area; and

subsequently, an act of the portable device determining that the viewing plane of the portable device has changed and is now aligned with a substantially vertical plane that is substantially orthogonal to the ground plane, and based at least in part on the viewing plane now being aligned with the substantially vertical plane;

an act of the portable device displaying a three-dimensional (3D) perspective view instead of the 2D map-based view; and

an act of the portable device displaying at least one indication of the particular point of interest within the 3D perspective view.

16. The method of claim 15, wherein the 2D map-based view of the geographical area comprises a topographical map.

17. The method of claim 15, further comprising:

an act the portable device detecting a gesture interacting with the particular point of interest on the geographical area map, the gesture comprising a combination of a change in one or more of a position, a direction, or a motion of the portable device that indicates that the portable device is physically moving.

18. The method of claim 17, wherein, in response to the portable device detecting a gesture comprising movement of the device, an act of the portable device displaying information about the particular point of interest.

19. The method of claim 15, wherein the 3D perspective view comprises an image view of a real space as captured by a camera at the portable device, and wherein the at least one indication of the particular point of interest is overlaid over the image view of the real space.

20. A computing device, comprising:

one or more processors;

a display device; and

one or more computer-readable media having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to perform the following;

determining that a viewing plane of the display device is aligned with a substantially horizontal plane that is substantially parallel to a ground plane, and based at least in part on the viewing plane being aligned with the substantially horizontal plane;

displaying a two-dimensional (2D) map-based view of a local geographical area on the display device based on location and direction information measured by the portable device; and

displaying at least one indication of a particular point of interest that is contextually relevant to the portable device within the 2D map-based view of the geographical area; and

subsequently determining that the viewing plane of the display device has changed and is now aligned with a substantially vertical plane that is substantially orthogonal to the ground plane, and based at least in part on the viewing plane now being aligned with the substantially vertical plane;

displaying a three-dimensional (3D) perspective view of the local geographic area on the display device instead of the 2D map-based view; and

displaying at least one indication of the particular point of interest within the 3D perspective view.