System and method for selection of an object of interest during physical browsing by finger pointing and snapping

• US 7,885,145 B2
• Filed: 10/26/2007
• Issued: 02/08/2011
• Est. Priority Date: 10/26/2007
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Marti, Stefan
• Primary Examiner(s): Tarcza; Thomas H
• Assistant Examiner(s): Ratcliffe; Luke D
• Application Number:
  US11/925,463
• Publication Number:
  US 20090109795A1
• Time in Patent Office: 1,201 Days
• Field of Search: 367118-190
• US Class Current: 367/118
• CPC Class Codes: G01S 11/14 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A system and method for selecting an object from a plurality of objects in a physical environment is disclosed. The method may include analyzing acoustic data received by a plurality of acoustic sensors, detecting a snapping sound in the acoustic data, determining the direction of the snapping sound, estimating the distance to an object based on loudness measurements of the snapping sound detected in the acoustic data, and determining the location of the object relative to the system based at least in part on the determined direction, the estimated distance to the object and the location of the system. The method may further include identifying the selected object based on its geolocation, collecting and merging data about the identified object from a plurality of data sources, and displaying the collected and merged data.

Claims

1. A system for selecting an object from a plurality of objects in a physical environment, the system comprising:

a plurality of acoustic sensors;

a sound analysis subsystem configured to analyze acoustic data received by the plurality of acoustic sensors, the sound analysis subsystem further configured to detect a snapping sound caused by a source controlled by a user of the system, determine the relative direction of the source of the snapping sound, and estimate the distance to the source of the snapping sound based on loudness measurements of the snapping sound detected in the acoustic data;

an object distance estimation subsystem configured to estimate a distance to an object other than the source of the snapping sound based at least in part on the determined estimated distance to the source; and

an object location subsystem configured to determine the location of the object relative to the system based at least in part on the determined relative direction, the estimated distance to the object and a location of the system.

2. The system of claim 1, wherein the sound analysis subsystem is further configured to determine the direction of the snapping sound based on times of arrival of the detected snapping sound in the acoustic data.

3. The system of claim 1, further comprising a geo-location subsystem configured to sense a geo-location of the system, wherein the object location subsystem is further configured to determine a geo-location of the object based on the determined relative location of the object and the sensed geo-location of the system.

4. The system of claim 3, wherein the geo-location subsystem comprises a GPS sensor.

5. The system of claim 3, further comprising a geo-tagged object database configured to identify the object based on the determined geo-location of the object.

6. The system of claim 1, wherein the sound analysis subsystem is further configured to determine the direction of the snapping sound in a local coordinate system, the system further comprising:

a directional subsystem configured to sense an orientation of the acoustic sensors in a geo-location coordinate system, wherein the object location subsystem is further configured to determine a relationship between the geo-location coordinate system and the local coordinate system, and determine the direction of the snapping sound in the geo-location coordinate system based on the determined relationship and the determined direction of the object in the local coordinate system.

7. The system of claim 1, wherein the sound analysis subsystem is further configured to estimate the distance to the object to be within a plurality of distance ranges.

8. The system of claim 1, further comprising a noise suppression subsystem configured to suppress noise in the acoustic data used to detect the snapping sound.

9. The system of claim 1, wherein the acoustic sensors are attached to the user.

19. The system of claim 1, wherein the source of the snapping sound is the user'"'"'s fingers.

20. The system of claim 1, wherein the source of the snapping sound is a device controlled by the user.

10. A method of selecting an object from a plurality of objects in a physical environment, the method comprising:

analyzing acoustic data received by a plurality of acoustic sensors;

detecting a snapping sound in the acoustic data, wherein the snapping sound is caused by a source controlled by a user of the system;

determining the direction of the snapping sound;

estimating the distance to an object other than the source of the snapping sound based on loudness measurements of the snapping sound detected in the acoustic data; and

determining the location of the object relative to the system based at least in part on the determined direction, the estimated distance to the object and a location of the system.

11. The method of claim 10, wherein determining the direction of the snapping sound comprises determining the direction of the snapping sound based on times of arrival of the detected snapping sound in the acoustic data.

12. The method of claim 10, further comprising:

sensing a geo-location of the system; and

determining a geo-location of the object based on the determined relative location of the object and the sensed geo-location of the system.

13. The method of claim 12, wherein sensing the geo-location of the system comprises sensing the geo-location of the system using a GPS sensor.

14. The method of claim 12, further comprising identifying the object based on the determined geo-location of the object.

15. The method of claim 10, wherein determining the direction of the snapping sound comprises determining the direction of the snapping sound in a local coordinate system, the method further comprising:

determining a relationship between a geo-location coordinate system and the local coordinate system; and

determining the direction of the snapping sound in the geo-location coordinate system based on the determined relationship and the determined direction of the object in the local coordinate system.

16. The method of claim 10, wherein estimating the distance to the object further comprises estimating the distance to the object to be within a plurality of distance ranges.

17. The method of claim 10, further comprising suppressing noise in the acoustic data used to detect the snapping sound.

21. The method of claim 10, wherein the snapping sound is caused by manipulation of the user'"'"'s fingers.

18. A system for selecting an object from a plurality of objects in a physical environment, the system comprising:

means for analyzing acoustic data received by a plurality of acoustic sensors;

means for detecting a snapping sound in the acoustic data;

means for causing the snapping sound, wherein the means for causing the snapping sound is controlled by a user;

means for determining the direction of the snapping sound;

means for estimating the distance to an object other than the means for causing the snapping sound based on loudness measurements of the snapping sound detected in the acoustic data; and

means for determining the location of the object relative to the system based at least in part on the determined direction, the estimated distance to the object and a location of the system.