Gesture based map annotation

• US 9,214,043 B2
• Filed: 03/04/2013
• Issued: 12/15/2015
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Beaurepaire, Jerome
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): Cobb, Michael J
• Application Number:
  US13/783,824
• Publication Number:
  US 20140247282A1
• Time in Patent Office: 1,016 Days
• Field of Search: 345/156, 345/179, 345/419, 345/629, 345/633, 345/661, 715/863
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3682 output of POI information on a road map G01C21/3614, G01C21/3685 take precedence
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass

Abstract

A method, apparatus and computer readable medium provide for gesture based map annotation. In the context of a method, an annotation is made to a map that includes a three-dimensional model of geographic elements. The annotation is based on a free-space user gesture and is positioned at an annotation point on a surface of the three-dimensional model of the map. The annotation point is derived from a geographic location and a gesture direction associated with the user gesture.

Claims

1. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following;

based on a received free-space user gesture associated with a real-world geographic location where the gesture was performed and a gesture direction representing the direction in which the user gesture was made, automatically provide for the making of an annotation to a map comprising three-dimensional models of geographic elements, the three-dimensional models including surfaces that correspond to real-world surfaces of geographical elements represented in the model, wherein analysis of the map of three-dimensional models using the real-world geographic location and the gesture direction provides for identification of an annotation point comprising a particular surface of the surfaces of the three-dimensional models, the annotation being based on said user gesture and positioned at the annotation point on the particular surface of the three-dimensional model of the map to be rendered onto said surface of the three-dimensional model and wherein if a particular surface cannot be identified, the annotation is positioned at a location in the map corresponding to the real-world geographic location.

2. An apparatus according to claim 1, wherein the annotation comprises a graphic element that is a direct representation of the gesture made by the user.

3. An apparatus according to claim 1, wherein the annotation comprises a predetermined alphanumeric note or pre-drawn graphic element that is associated with a predefined user gesture made by the user.

4. An apparatus according to claim 1, wherein the apparatus includes a communication module to receive gesture data representing at least the user gesture from a gesture capture apparatus.

5. An apparatus according to claim 4, wherein the apparatus is configured to determine the geographical location at which the user gesture was made from geographical location data included with the gesture data.

6. An apparatus according to claim 1, wherein the apparatus includes a geographic location determination module and the geographic location is determined by the geographic location determination module using the time that the user gesture was made by the user.

7. An apparatus according to claim 1, wherein the apparatus is configured to make the annotation to the map.

8. An apparatus according to claim 1, wherein the apparatus is a portable electronic device, a wearable device, a ring, a wristband, a laptop computer, a mobile phone, a Smartphone, a tablet computer, a personal digital assistant, a digital camera, a watch, a server, or a module/circuitry for one or more of the above-listed apparatuses or a non-portable electronic device, a desktop computer, a monitor, a server, or a module/circuitry for one or more of the non-portable electronic device, desktop computer, monitor, or server.

9. An apparatus according to claim 1, wherein the apparatus provides for the display of a map including a three-dimensional model of the real world geographic area, wherein the annotation is rendered onto the three-dimensional model.

10. An apparatus according to claim 1, wherein the apparatus provides for the association of particular user-selected or user-provided gestures with pre-prepared notes or graphic elements.

11. An apparatus according to claim 1, wherein the apparatus provides for supplementation of the annotation with extra information, including one or more of a street address or a telephone number.

12. An apparatus according to claim 1, wherein the annotation is provided for rendering onto the surface of the three-dimensional model of the map along with any other surface graphics or textures.

13. An apparatus according to claim 1, wherein the annotation point is based on a map location in the map of three-dimensional models of geographical elements derived from the real-world geographic location and the gesture direction, wherein the annotation point comprises a surface incident with a projection from the map location along the gesture direction.

14. An apparatus according to claim 1, wherein the annotation comprises a predetermined alphanumeric note or pre-drawn graphic element that is associated with a predefined user gesture made by the user determined using a look up table and if such an predefined user gesture is not present in the look up table, the annotation comprises a direct representation of the gesture made by the user.

15. A method, the method comprising

automatically providing for the making of an annotation to a map comprising three-dimensional models of geographic elements, the three-dimensional models including surfaces that correspond to real-world surfaces of geographical elements represented in the model, wherein analysis of the map of three-dimensional models using a geographic location at which the user gesture was made and a gesture direction representing the direction in which the user gesture was made provides for identification of an annotation point comprising a particular surface of the surfaces of the three-dimensional models, the annotation based on a free space user gesture and positioned at the annotation point on the particular surface of the three-dimensional models of the map to be rendered onto said surface of the three-dimensional model and wherein if a particular surface cannot be identified, the annotation is positioned at a location in the map corresponding to the real-world geographic location.

16. A method according to claim 15, wherein the annotation comprises a graphic element that is a direct representation of the gesture made by the user.

17. A method according to claim 15, wherein the annotation comprises a predetermined alphanumeric note or pre-drawn graphic element that is associated with a predefined user gesture made by the user.

18. A method according to claim 15, wherein the method includes receiving gesture data representing at least the user gesture from a gesture capture apparatus.

19. A method according to claim 18, wherein the method includes determining the geographical location at which the user gesture was made from geographical location data included with the gesture data.

20. A method according to claim 15, wherein the method includes determining the geographic location using the time that the user gesture was made by the user.

21. A method according to claim 15, wherein the method includes making the annotation to the map.

22. A non-transitory computer readable medium comprising computer program code stored thereon, the computer readable medium and computer program code being configured to, when run on at least one processor, perform at least the following:

based on a received free-space user gesture associated with a real-world geographic location where the gesture was performed and a gesture direction representing the direction in which the user gesture was made, automatically provide for the making of an annotation to a map comprising three-dimensional models of geographic elements, the three-dimensional models including surfaces that correspond to real-world surfaces of geographical elements represented in the model, wherein analysis of the map of three-dimensional models using the real-world geographic location and the gesture direction provides for identification of an annotation point comprising a particular surface of the surfaces of the three-dimensional models, the annotation being based on said user gesture and positioned at the annotation point on the particular surface of the three-dimensional model of the map to be rendered onto said surface of the three-dimensional model and wherein if a particular surface cannot be identified, the annotation is positioned at a location in the map corresponding to the real-world geographic location.