Head mounted communication system for providing interactive visual communications with a remote system

• US 6,046,712 A
• Filed: 07/23/1996
• Issued: 04/04/2000
• Est. Priority Date: 07/23/1996
• Status: Expired due to Term

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Telxon Corporation (Zebra Technologies Corporation)
• Inventors: O'Hagan, Timothy P., Beller, William E.
• Primary Examiner(s): Lieu, Julie
• Application Number:
  US08/685,395
• Time in Patent Office: 1,351 Days
• Field of Search: 345/7, 345/8, 348/13, 348/14, 348/15, 348/115, 348/207, 348/211, 348/552, 340/505, 340/286.01, 340/506, 379/34, 379/35
• US Class Current: 345/8
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 5/39 Control of the bit-mapped memory
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A head mounted communication system provides interactive visual and audio communications between a user of the head mounted system and an operator of a remote system. The head mounted system includes a camera to pick up an image within the user'"'"'s field of view. A display and optics are provided for projecting an enlarged image of displayed information at a distance from the user so as to be superimposed on the user'"'"'s view of the real world. The system also includes a transmitter and receiver for providing two-way wireless communication between the head mounted system and the remote system so as to allow the operator of the remote system to see what the user of the head mounted system is viewing and to allow the user of the head mounted system to view via the display and optics video inputs received from the remote system so that they appear to be superimposed upon the real world.

Claims

1. A head mounted communication system for providing visual communications between a user of the head mounted system and a remote system having a display and being capable of receiving data from said head mounted system and transmitting data to said head mounted system, comprising:

a support to be worn on a user'"'"'s head;

a camera system mounted on said support to pick up an image within at least a portion of the user'"'"'s field of view and to provide digital signals representative of said image;

a display mounted on said support for displaying information;

optics mounted on said support relative to said display for projecting an enlarged image of said displayed information at a distance from the user;

a transmitter for wireless transmission of data to said remote system;

a receiver for wireless reception of data from said remote system; and

a controller responsive to digital signals representing an image picked up by said camera system for controlling the transmission, to said remote system, of data representing said picked up image for display of said image on said remote system display, said controller being responsive to the receipt of data from said remote system representing said picked up image modified with additional information superimposed thereon for controlling said display to display said modified image for projection by said optics.

2. A head mounted communication system as recited in claim 1 wherein said optics include a partial reflector viewed by said user to perceive said enlarged image superimposed on the user'"'"'s view of the real world.

3. A head mounted communication system as recited in claim 1 wherein said optics include a monocular reflective element viewed by the user to perceive said enlarged image superimposed on the user'"'"'s view of the real world.

4. A head mounted communication system as recited in claim 1 wherein said transmitter and receiver communicate with said remote system via a radio network including a plurality of base stations.

5. A head mounted communication system as recited in claim 1 wherein said camera system includes a charged coupled device photosensor array.

6. A head mounted communication system as recited in claim 1 wherein said controller formats data for transmission in one or more sequential packets including header information containing an identity of the remote system for which the transmitted data is intended and an identity of the head mounted system.

7. A head mounted communication system as recited in claim 1 further including at least one speaker mounted on said support;

and an audio pick up device mounted on said support to pick up verbal communications from the user and to provide digital signals representative thereof, said controller being responsive to said digital signals representing picked up audio to control the transmission thereof to said remote system and being responsive to the receipt of audio data by said receiver to couple said audio data to said speaker.

8. A head mounted communication system as recited in claim 1 including an input device actuable by the user for providing a user input command;

and a read only memory for storing data representing selectable information for display, said controller being responsive to a user input command for controlling said display to display selected information represented by data stored in said read only memory and said modified image.

9. A head mounted communication system as recited in claim 8 wherein said controller is responsive to a user input command for controlling the locations at which the information from said read only memory and said modified image are displayed.

10. A head mounted communication system as recited in claim 1 including an input device actuable by the user for providing a user input command;

and a port for wired connection to at least one of a plurality of external sensing devices to receive data therefrom representing displayable measurement information, said controller being responsive to a user input command for controlling said display to display selected information represented by data from said port with said modified image.

11. A head mounted communication system as recited in claim 10 wherein said controller is responsive to a user input command for controlling the locations at which the information from said port and said modified image are displayed.

12. A head mounted communication system for providing visual communications between a user of the head mounted communication system and a remote system having a display and input device for controlling information input on said display and being capable of receiving data from said head mounted communication system comprising:

a support to be worn on a user'"'"'s head;

a camera system mounted on said support to pick up an image within at least a portion of the user'"'"'s field of view;

a user'"'"'s display mounted on said support for displaying information;

optics mounted on said support relative to said display including a partially reflective element into which the user looks to view an enlarged image of said display information projected by said element so as to be superimposed upon the real world, wherein the real world image picked up by said camera is correlatable to the real world image viewed by said user through said partially reflective element;

a transmitter for wireless transmission of data to said remote system;

a receiver for wireless reception of data from said remote system; and

a controller responsive to said camera system for controlling the transmission, to said remote system, of data representing said picked up image for display of said image on said remote system display, said controller being responsive to the receipt of data from said remote system, said data representing information input at said remote system at an identifiable location on the remote system display with respect to said picked up image, for controlling said user'"'"'s display to display said input information at a location correlated with said identifiable location.

13. A head mounted communication system as recited in claim 12 wherein said data received from said remote system includes information representing said identifiable location.

14. A head mounted communication system as recited in claim 12 wherein said controller operates to transmit a plurality of frames of image data picked up by said camera system substantially continuously so as to provide a real time representation of said portion of the user'"'"'s field of view picked up by said camera.

15. A head mounted communication system as recited in claim 12 wherein said controller is responsive to an input from the user to transmit data representing only a selected frame of an image picked up by said camera.

16. A head mounted communication system as recited in claim 12 wherein said transmitter and receiver communicate with said remote system via a radio network including a plurality of base stations.

17. A head mounted communication system as recited in claim 12 wherein said controller formats data for transmission in one or more sequential packets including header information containing an identity of the remote system for which the transmitted data is intended and an identity of the head mounted system.

18. A head mounted communication system as recited in claim 12 further including at least one speaker mounted on said support;

and an audio pick up device mounted on said support to pick up verbal communications from the user and to provide digital signals representative thereof, said controller being responsive to said digital signals representing picked up audio to control the transmission thereof to said remote system and being responsive to the receipt of audio data by said receiver to couple said audio data to said speaker.

19. A head mounted communication system for providing visual communications between a user of the head mounted communication system and a remote system having a display and input device and being capable of receiving data from said head mounted communication system and transmitting data to said head mounted communication system comprising:

a support to be worn on a user'"'"'s head;

a camera system mounted on said support to pick up an image within at least a portion of the user'"'"'s field of view and to provide digital signals representative of said image;

an image generating and projection system mounted on said support for generating visual information and for projecting said information at a distance from the user;

a transmitter for wireless transmission of data to said remote system;

a receiver for wireless reception of data from said remote system; and

a controller responsive to digital signals representing an image picked up by said camera system for controlling the transmission of data representing said picked up image to said remote system for display of said image on said remote system display, said controller being responsive to the receipt of video data from said remote system representing at least in part, additional information input at the remote system in spatial relation to objects within said picked up image as displayed on said remote system display, and for controlling the image generating and projection system to generate a visual representation of said additional information for projection thereof in said spatial relation to said objects within said picked up image.

20. A head mounted communication system as recited in claim 19 wherein said image generating and projection system includes a liquid crystal display.

21. A head mounted communication system as recited in claim 19 wherein said image generating and projection system includes a partial reflector viewed by said user to perceive said projected information so as to be superimposed on the real world.

22. A head mounted communication system for providing visual and audio communications between a user of the head mounted communication system and a remote system having a display and being capable of receiving data from said head mounted communication system and transmitting data to said head mounted communication system comprising:

a support to be worn on a user'"'"'s head;

a camera system mounted on said support to pick up an image within at least a portion of the user'"'"'s field of view;

an image generating and projection system mounted on said support for generating visual information and for projecting said information at a distance from the user;

at least one speaker mounted on said support;

an audio pick up device mounted on said support to pick up verbal communications from the user;

a transmitter for wireless transmission of data to said remote system;

a receiver for wireless reception of data from said remote system; and

a controller responsive to said camera system for controlling the transmission, to said remote system, of data representing an image picked up by said camera system and responsive to the receipt of audio and visual feedback information for respectively controlling said speaker and said image generating and projection system to provide said audio and visual feedback information to said user, said visual feedback information including modifications to said picked up image as input at said remote system in correspondence with objects in said picked up image and wherein the image generating and projection system presents a display of said modifications in relation to said objects in said picked up image.

23. A head mounted communication system as recited in claim 22 wherein said image generating and projection system includes a liquid crystal display.

24. A head mounted communication system as recited in claim 22 wherein said image generating and projection system includes a partial reflector viewed by said user to perceive said projected information.

25. A head mounted communication system as recited in claim 22 wherein said controller formats data for transmission in one or more sequential packets including header information containing an identity of the remote system for which the transmitted data is intended and an identity of the head mounted system.

26. A head mounted communication system as recited in claim 22 including an input device actuable by the user for providing a user input command;

and a read only memory for storing data representing selectable information for display, said controller being responsive to a user input command for controlling said display to display selected information represented by data stored in said read only memory and said modified image.

27. A head mounted communication system as recited in claim 26 wherein said controller is responsive to a user input command for controlling the locations at which the information from said read only memory and said modified image are displayed.

28. A head mounted communication system as recited in claim 22 including an input device actuable by the user for providing a user input command;

and a port for wired connection to at least one of a plurality of external sensing devices to receive data therefrom representing displayable measurement information, said controller being responsive to a user input command for controlling said display to display selected information represented by data from said port with said modified image.

29. A head mounted communication system as recited in claim 28 wherein said controller is responsive to a user input command for controlling the locations at which the information from said port and said modified image are displayed.

30. A head mounted communication system for providing visual communications between a user of the head mounted system and a remote system comprising:

a support to be worn on a user'"'"'s head;

a display mounted on said support for displaying information;

optics mounted on said support relative to said display for projecting an enlarged image of said displayed information;

a plurality of sources of data representing displayable information including;

a camera system to pick up an image within the field of view thereof and to provide data representative of said picked up image;

a wireless communication system with a receiver for receiving data representing displayable information from said remote system;

a read only memory for storing data representing selectable information for display;

a port for a wired connection to at least one of a plurality of external sensing devices to receive data therefrom representing displayable measurement information;

an input device actuable by the user for providing a user input command; and

a controller coupled to said plurality of data sources and responsive to a user input command for controlling said display to selectively display information represented by data received from said plurality of data sources.

31. A head mounted display communication system as recited in claim 30 wherein said controller is responsive to a user input command for controlling said display to simultaneously display information represented by data received from two or more of said data source.

32. A head mounted communication system as recited in claim 31 wherein said controller is responsive to a user input command for controlling the location at which the information from said two or more data sources is depicted on said display.

33. A head mounted communication system as recited in claim 30 wherein said input device includes at least one switch for positioning a cursor depicted on said display under the control of the controller.

34. A head mounted communication system for providing visual communications between a user of the head mounted system and a remote system comprising:

a support to be worn on a user'"'"'s head;

a display mounted on said support for displaying information;

optics mounted on said support relative to said display for projecting an enlarged image of said displayed information;

a wireless communication system with a receiver for receiving, from said remote system, data representing displayable information from said remote system;

a plurality of non-remote sources of data representing displayable information;

an input device actuable by the user for providing a user input command for selecting display information represented by data received from said wireless communication system and said plurality of data sources; and

a controller for receiving data from said wireless communication system and said plurality of data sources and responsive to a user input command for controlling said display to selectively display information represented by data received from said wireless communication system and one or more of said data sources simultaneously or individually in accordance with said user input command.

35. A head mounted communication system as recited in claim 34 wherein one of said non-remote sources of data includes a camera system to pick up an image within the field of view thereof and to provide data representative of said picked up image.

36. A head mounted communication system as recited in claim 34 wherein one of said non-remote sources of data includes a read only memory for storing data representing selectable information for display.

37. A head mounted communication system as recited in claim 34 wherein one of said non-remote sources of data includes a port for a wired connection to at least one of a plurality of external sensing devices to receive data therefrom representing displayable measurement information.

38. A method of controlling a head mounted communication system having a display for displaying information, optics for projecting an enlarged image of displayed information at a distance from the user, a wireless communication device for transmitting and receiving data, a camera for capturing images, and a support to be worn on a user'"'"'s head, said display, optics, communication device and camera being mounted on said support comprising:

operating said camera to capture an image within at least a portion of a user'"'"'s field of view and to provide image data representative of the captured image;

digitizing a frame of captured image data to provide a frame of digital image data;

compressing said frame of digital image data;

formatting said frame of compressed digital image data for transmission in one or more sequential packets, each of said packets including an identity of said head mounted system and an identity of a remote system for which said data is intended to be transmitted;

transmitting said one or more sequential packets to said remote system; and

further comprising the step of displaying a plurality of frames of images captured by said camera; and

formatting and transmitting only a user selected frame of a captured image.

39. A method of controlling a head mounted communication system as recited in claim 38 wherein said steps of operating said camera, digitizing a frame of data, and compressing, formatting and transmitting said data are performed substantially continuously until interrupted so as to transmit to said remote station real time images falling within said portion of the user'"'"'s field of view.

40. A method of controlling a head mounted communication system as recited in claim 38 including the steps of

receiving a signal transmitted from said remote system to said head mounted system, said signal including data representing an image;

retrieving said image data from said received signal; and

controlling said display to display the image represented by said data.

41. A method of controlling a head mounted communication system as recited in claim 40 further including the steps of

receiving data from a non-remote data source;

and

displaying data received from said non-remote data source with said image represented by data from said remote system.

42. A method of controlling a head mounted communication system as recited in claim 40 wherein said data includes information identifying a location of the image from the remote system with respect to an image captured by said camera and said display control step includes correlating said identified location with a location on said display.

43. A method of controlling a head mounted communication system as recited in claim 38 wherein said head mounted communication system includes a speaker and a microphone for picking up audio and further including the steps of transmitting in real time audio picked up from said microphone;

receiving data representing audio transmitted from said remote system; and

coupling said audio to said speaker to provide two way real time audio communications.