Obtaining information by tracking a user

• US 8,077,915 B2
• Filed: 10/12/2007
• Issued: 12/13/2011
• Est. Priority Date: 10/12/2007
• Status: Expired due to Fees

• Current Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Inventors: Thorn, Ola Karl
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US11/871,519
• Publication Number:
  US 20090097705A1
• Time in Patent Office: 1,523 Days
• Field of Search: 382/103, 382/104, 382/107, 342/357.23, 342/357.31
• US Class Current: 382/103
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G09B 29/10 Map spot or coordinate position indicators; Map reading aids optical projection apparatus per se G03B

Abstract

A device may obtain tracking information of a face or a head of a user, determine a position and orientation of the user, and determine a direction of focus of the user based on the tracking information, the position, and the orientation. In addition, the device may retrieve information associated with a location at which the user focused.

Claims

1. A method comprising:

obtaining, by one or more devices of a mobile communication device, tracking information of a face or a head of a user of the mobile communication device;

determining, by the one or more devices, a geographic location of the mobile communication device;

determining, by the one or more devices, a position of the user with respect to the geographic location;

determining, by the one or more devices, an orientation of the user with respect to the geographic location;

determining, by the one or more devices, a direction of focus of the user, with respect to the geographic location, based on the tracking information, the position, and the orientation; and

obtaining, by the one or more devices, information associated with a location or geographic feature identified in results of a search, using the direction of focus, of geographic information records related to the geographic location.

2. The method of claim 1, further comprising:

presenting the obtained information to the user based on a user selection of at least one of;

a speaker of a headset coupled to the one or more devices;

or

a display of the one or more devices.

3. The method of claim 1, further comprising:

conducting, based on user input, a transaction related to the identified location or geographic feature based on the obtained information.

4. The method of claim 3, where conducting a transaction includes at least one of:

reserving a seat in a restaurant;

purchasing an item;

obtaining information about businesses associated with the identified location;

crediting or debiting an account of the user;

receiving instructions for traveling to the identified location;

or

receiving descriptions of the identified location.

5. The method of claim 1, wherein obtaining information associated with a location or a geographic feature includes:

transmitting, via a network, the tracking information, the position, and the orientation of the user to a network device; and

obtaining the information from the results of the search of the geographic information records stored on a remote database associated with the network device.

6. The method of claim 1, wherein determining a geographic location includes:

receiving information from a global positioning system (GPS) receiver or a Beidou navigation system (BNS) receiver.

7. The method of claim 1, further comprising;

obtaining tracking information of an eye of the user.

8. The method of claim 7, where obtaining tracking information of an eye includes obtaining the tracking information of the eye based on at least one of:

a reflection on a cornea of the eye;

a reflection on a lens of the eye;

movements of a pupil of the eye;

or

images of a retina inside the eye.

9. The method of claim 7, wherein determining a direction of focus includes:

using the tracking information of the eye and the tracking information of the face or the head to determine that the direction of focus is maintained by the user for a particular amount of time.

10. The method claim 1, wherein obtaining information associated with a location or a geographic feature includes:

obtaining a list of places in an area associated with the identified location;

obtaining, from the list, a set of places that lie in the direction of focus; and

using the direction of focus and height information of each place in the set, to determine a particular place, of the set of places on which the user focused.

11. The method of claim 1, wherein obtaining tracking information of a face includes:

capturing an image of the face or the head with a camera;

matching the captured image to a particular image of a plurality of stored images of the user'"'"'s face or head; and

identifying one or more angles that are associated with the particular image.

12. The method of claim 1, wherein obtaining tracking information includes:

obtaining tracking information via a camera associated with the mobile communication device and is stably mounted in a direction opposite to that of a moving vehicle in which the user is located.

13. The method of claim 1, wherein obtaining information associated with a location or a geographic feature includes at least one of:

retrieving, from the geographic information records, a name of a street that is within a particular threshold distance from the mobile communication device;

retrieving, from the geographic information records, a name of a landmark, a business, or a building that is within the particular threshold distance from the mobile communication device;

or

retrieving, from the geographic information records, information associated with operations related to the landmark, the business, or the building.

18. The method of claim 1, further comprising:

receiving, from the user, a request for particular information associated with the identified location or geographic feature; and

providing, to the user and in response to the request, the requested information.

19. The method of claim 18, wherein receiving the request includes receiving a voice command from the user, and providing the requested information includes audibly providing the requested information based on the voice command.

20. The method of claim 1, further comprising:

determining whether the user is operating a vehicle; and

based on a determination that the user is operating a vehicle, audibly providing at least some of the obtained information to the user.

14. A mobile communication device comprising:

a camera for obtaining an image of a face or a head of a user; and

a processor configured to;

determine a position of the mobile communication device;

obtain orientation information of the face or the head by comparing the image and a plurality of images that are associated with different angles of the face or the head;

determine a direction at which the user is looking based on the obtained orientation information and the position of the mobile communication device;

determine that the user is looking in the direction for a particular amount of time; and

trigger, when the particular amount of time exceeds a threshold, an operation to obtain information associated with a location where the user is looking based on the direction and the position of the mobile communication device.

15. The mobile communication device of claim 14, wherein the processor is further configured to obtain the orientation information that corresponds to the different angles and includes:

a pitch, yaw, and roll;

or

Euler angles.

16. The mobile communication device of claim 14, wherein the processor is further configured to determine whether the user is operating a vehicle, the mobile communication device further comprising:

a speaker for presenting the obtained information associated with the location to the user based on a determination that the user is operating a vehicle.

17. A device comprising:

means for obtaining tracking information of a face, a head, or eyes of a user;

means for determining a location and a direction in which the user is traveling;

means for determining a particular direction of a line of sight of the user based on the tracking information, the location, and the direction in which the user is traveling;

means for determining a duration of time that the user'"'"'s line of sight is trained in the particular direction;

means for triggering an operation, when the duration of time exceeds a particular threshold, for identifying a geographic location or feature within a threshold distance associated with the line of sight of the user.