Vision-guided model-based point-and-click interface for a wireless handheld device

• US 6,738,631 B1
• Filed: 05/06/2002
• Issued: 05/18/2004
• Est. Priority Date: 05/06/2002
• Status: Expired due to Term

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia, Inc. (Nokia Corporation)
• Inventors: Reynolds, Franklin, Adler, Mark
• Primary Examiner(s): Trost, William
• Assistant Examiner(s): LE, DANH C
• Application Number:
  US10/138,489
• Time in Patent Office: 743 Days
• Field of Search: 455/412.1, 455/414.1, 455/414.2, 455/418, 455/456.1, 455/456.3, 455/456.6, 345/326, 345/348, 340/825.29, 340/825.69, 340/825.72, 348/233, 348/211
• US Class Current: 455/456.6
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04W 40/246 Connectivity information discovery
• CPC Class Codes: H04W 60/00 Affiliation to network, e.g. registration; Terminating affiliation with the network, e.g. de-registration
• CPC Class Codes: H04W 76/10 Connection setup
• CPC Class Codes: H04W 8/005 Discovery of network devices, e.g. terminals

Abstract

Aspects of the invention overcome the problem of establishing communications with devices for which the communication protocol is unknown. One aspect of the invention involves the use of a discovery registration in which a portable device is provided with information describing the physical appearance characteristics of objects or devices that may be used to identify an object of interest. Another aspect of the invention involves the use of sensing techniques for establishing the identity of an object for which more information is sought by comparing the sensed physical characteristics of an object with data describing the physical characteristics of the objects located within an environment of the network. Aspects of the invention further involve associating the information describing the physical characteristics of objects or devices with additional information regarding the object, which may include the communication protocol for the device or object with which communication is desired.

Claims

1. A portable device capable of communicating within an environment, comprising:

a processor;

a communication interface;

a sensor that obtains sensed image data representing the actual physical appearance characteristics ol an area of interest within the environment; and

a memory containing discovery data comprising physical appearance characteristics of objects located within the environment and additional data concerning the objects located within the environment;

wherein the processor performs steps of retrieving from the memory discovery data comprising physical appearance characteristics of objects located within the environment and searching the sensed image data for data corresponding to the retrieved discovery data.

2. The portable device according to claim 1 wherein the discovery data further includes data describing the location of the objects.

3. The portable device according to claim 1 wherein the communication interface receives the discovery data wirelessly.

4. The portable device according to claim 1 wherein the communication interface receives the discovery data from a memory card on which the discovery data is prestored.

5. The portable device according to claim 1 wherein the communication interface receives the discovery data from objects located within the environment.

6. The portable device according to claim 1 wherein the portable device comprises a mobile telephone.

7. The portable device according to claim 1 wherein the portable device comprises a personal digital assistant.

8. The portable device according to claim 1 wherein the portable device comprises a laptop computer.

9. The portable device according to claim 1 wherein the processor performs object recognition on the sensed image data to identify one or more objects located in an area from which the sensed image data was acquired.

10. The portable device according to claim 1 wherein the processor processes a subset of the sensed image data in response to a user indication input to the portable device selecting only a subset of sensed image data to be processed.

11. The portable device according to claim 1 wherein the processor performs object recognition on the sensed image data using at least a portion of the discovery data.

12. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object in the sensed image data by comparing features in the sensed image data to the physical appearance characteristics of the objects described in the retrieved discovery data.

13. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object in the sensed image data using a set of instructions that perform color histogram analysis.

14. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object in the sensed image data using a set of instructions that perform image analysis, wherein the image analysis includes edge detection.

15. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object in the sensed image data using a set of instructions for analyzing the sensed image data selected from among a plurality of sets of instructions.

16. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object in the sensed image data using a set of instructions for analyzing the sensed image data selected from among a plurality of sets of instructions, wherein the selection of the set of instructions is performed based on the discovery data.

17. The portable device according to claim 1 wherein based on the discovery data the processor identifies an object and communicates with the object using a communication protocol described in the discovery data.

18. The portable device according to claim 1 further including a display that displays the additional data concerning the objects located within the environment.

19. A method of identifying an object, comprising the steps of:

retrieving discovery data describing physical appearance characteristics of objects within an environment, wherein the discovery data further includes additional data concerning the objects located within the environment;

acquiring, using a sensor associated with the portable device sensed image data representing the actual physical appearance characteristics of an area of interest within the environment; and

searching the sensed image data for data corresponding to the retrieved discovery data.

20. The method according to claim 19, wherein the step of retrieving discovery data further comprises the step of:

receiving the discovery data as a wireless communication.

21. The method according to claim 19, wherein the step of retrieving discovery data further comprises the step of:

receiving the discovery data from a detachable memory on which the discovery data is prestored.

22. The method according to claim 19, wherein the step of retrieving discovery data further comprises the step of:

receiving the discovery data from a memory card on which the discovery data is prestored.

23. The method according to claim 19, wherein the step of retrieving discovery data further comprises the step of:

receiving the discovery data from the objects located within the environment.

24. The method according to claim 19, wherein the step of acquiring sensed image data further comprises the step of:

capturing image data of an area within the environment.

25. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

identifying an object in the sensed image data using a set of instructions that perform image analysis for comparing features in the sensed image data to the physical appearance characteristics of the objects described in the retrieved discovery data.

26. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

performing image analysis on the sensed image data using a set of instructions selected from among a plurality of sets of instructions.

27. The method according to claim 26, wherein the step of conducting image analysis on the sensed image data comprises the step of:

comparing information contained in the discovery data with the sensed image data.

28. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

identifying an object using physical characteristics of the object contained in the retrieved discovery data.

29. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

identifying an object using information relating to the location of the object contained in the retrieved discovery data.

30. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

searching a subset of the sensed image data.

31. The method according to claim 30, wherein the step of searching a subset of the sensed image data further comprises the step of:

searching a subset of the sensed image data on the basis of a user selection input to the portable device.

32. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

identifying the object on the basis of a color histogram analysis.

33. The method according to claim 19, wherein the step of searching the sensed image data further comprises the step of:

identifying the object on the basis of an analysis utilizing edge detection.

34. The method according to claim 19, comprising the additional step of:

retrieving additional data comprising an identified object for establishing communications with the object, based on the results of the search.

35. The method according to claim 19, comprising the additional step of:

retrieving additional data comprising an URL associated with the object, based on the results of the search.

36. The method according to claim 19, comprising the additional step of:

providing a display of the additional data corresponding to an identified object.

37. The method according to claim 19, comprising the additional step of:

executing an application utilizing the additional data corresponding to an identified object.

38. The method according to claim 19, comprising the additional step of:

performing steps to facilitate communication between the portable device and the object using additional data corresponding to an identified object.

39. A method of identifying an object, comprising the steps of:

locating a portable device within a wireless network;

wirelessly receiving discovery data describing physical characteristics of objects within the environment of the wireless network, wherein the discovery data includes additional data concerning the object or objects located within the environment,

capturing image data representing an area within the wireless communication network using an image sensing unit;

comparing the captured image data to data corresponding to the retrieved discovery data using at least one set of instructions;

identifying an object in the captured image data;

retrieving the additional data associated with the identified object, and

utilizing the additional data to communicate with the identified object.

40. A portable device comprising:

a processor;

a wireless communication interface;

a memory;

an image sensor that captures image data,

wherein the processor is programmed with computer instructions that perform the steps of;

(1) receiving from the wireless communication interface discovery data comprising physical appearance characteristics for a plurality of objects located within a wireless network in which the portable device is operating and including additional data pertaining to each of the plurality of objects;

(2) storing the received discovery data in the memory; and

(3) processing image data captured from the image sensor and identifying one of the plurality of objects as an object of interest based on a comparison between the image data captured from the image sensor and the physical appearance characteristics stored in the memory.

41. The portable device according to claim 40,

wherein the discovery data comprises information identifying an object recognition algorithm that should be used to identify one or more of the plurality of objects, and wherein the processor performs step (3) by executing an object recognition algorithm identified in the discovery data.

42. The portable device according to claim 40, wherein the discovery data comprises information identifying a communication protocol that should be used to communicate with one or more of the plurality of objects, and wherein the processor performs the further step of communicating with the object of interest using the communication protocol identified in the discovery data.

43. The portable device according to claim 40, further comprising:

a display for displaying additional data pertaining to the object of interest.