Audio-visual three-dimensional input/output

• US 7,774,075 B2
• Filed: 06/30/2005
• Issued: 08/10/2010
• Est. Priority Date: 11/06/2002
• Status: Expired due to Fees

• Current Assignee: Julius J.Y. Lin
• Original Assignee: Julius J.Y. Lin
• Inventors: Lin, Julius J. Y.
• Primary Examiner(s): Hartman, Jr.; Ronald D
• Application Number:
  US11/174,348
• Publication Number:
  US 20050264527A1
• Time in Patent Office: 1,867 Days
• Field of Search: 345/156, 345/158, 345/168, 700/17, 700/83, 382/103, 382/104, 382/106, 382/286, 356/600, 356/614, 356/625
• US Class Current: 700/17
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Methods and apparatus for determining an object'"'"'s three-dimensional location (i.e. real world coordinates) using the audio-video infrastructure of a 3G cellular phone or a 3C (Computer, Communications, Consumer) electronic device. A first detection device (e.g. a camera) is used to capture images of the objects. The captured image data is used to compute location data of the object in a first two-dimensional plane. A second detection device (e.g. microphone or infrared detector) may be used to collect additional location data in a second plane, which when combined with image data from the captured images allows the determination of the real world coordinates (x, y, z) of the object. The real-world coordinate data may be used in various applications. If the size of an object of interest is known or can be calculated, and the size of the projected image does not vary due to rotation of the object, a single camera (e.g. the camera in a 3G or 3C mobile device) may be used to obtain three-dimensional coordinate data for the applications.

Claims

1. A method of determining the location of an object in three-dimensional space, comprising:

using an imaging device, capturing an image of a real world object;

determining a size of the real world object represented in the captured image; and

based on a size of the real world object and the size of the real world object represented in the captured image, determining three-dimensional coordinates of the real world object.

2. The method of claim 1, further comprising determining the size of the real world object using the data in the captured image of the real world object.

3. The method of claim 2, further comprising:

storing the determined size of the real world object; and

retrieving the determined size of the real world object for use in the determining of three-dimensional coordinates of the real world object.

4. The method of claim 2, wherein determining the size of the real world object comprises a calibration for the real world object of a system used to perform the determining of three-dimensional coordinates of the real world object.

5. The method of claim 2, further comprising modifying the determined size of the real world object to compensate for distortion effects produced by the imaging device.

6. The method of claim 5, wherein the imaging device includes a lens and the modifying of the determined size of the real world object compensates for distortion effects produced by the lens.

7. The method of claim 6, wherein the lens is a wide angle lens.

8. The method of claim 2, further comprising modifying the determined size of the real world object to compensate for nonlinear effects produced by the imaging device.

9. The method of claim 8, wherein the imaging device includes a lens and the modifying of the determined size of the real world object compensates for distortion effects produced by the lens.

10. The method of claim 1 wherein the size of the real world object is predetermined.

11. The method of claim 10, further comprising:

storing the predetermined size of the real world object; and

retrieving the stored predetermined size of the real world object for use in the determining of three-dimensional coordinates of the real world object.

12. The method of claim 1, further comprising determining a distance between the captured image and the real world object.

13. The method of claim 12, further comprising modifying the determined distance to correct for nonlinear optical effects produced by the imaging device.

14. The method of claim 12, wherein the captured image is at a focal length of the imaging device.

15. The method of claim 1, further comprising determining a vector along a line between the captured image and an origin of the imaging device.

16. The method of claim 1 wherein the determined three-dimensional coordinates are used to operate or control a virtual device.

17. The method of claim 16 wherein the virtual device comprises a virtual workstation, virtual conferencing system, virtual control panel, a virtual music station, or a virtual gaming system.

18. The method of claim 1 wherein the imaging device comprises a single camera.

19. The method of claim 18 wherein the single camera comprises a camera in a communications or computing device.

20. The method of claim 19 wherein the communications or computing device comprises a mobile communications or computing device.

21. The method of claim 1, further comprising:

storing the size of the real world object; and

retrieving the stored size of the real world object for use in the determining of three-dimensional coordinates of the real world object.