System for transmitting video images over a computer network to a remote receiver

• US 7,257,158 B1
• Filed: 05/17/1999
• Issued: 08/14/2007
• Est. Priority Date: 05/18/1998
• Status: Expired due to Fees

• Current Assignee: Secure Cam, LLC
• Original Assignee: Kendyl A. Roman
• Inventors: Broadbent, Thomas Joseph, Hoomani, Cyrus Javad, Neale, Richard Scott, Raposo, R. Paul, Roman, Kendyl Allan, Figueredo, Vincent Michael
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/312,922
• Time in Patent Office: 3,011 Days
• Field of Search: 600/437, 434/262, 375/240.01, 345/853, 604/20, 128/660.01
• US Class Current: 375/240.01
• CPC Class Codes: A61B 8/565 involving data transmission via a network
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Abstract

A method of and apparatus for transmitting video images preferably allows a specially trained individual to remotely supervise, instruct, and observe administration of medical tests conducted at remote locations. This apparatus preferably includes a source device, a transmitting device, and at least one remote receiving device. Preferably, the transmitting device and the remote receiving device communicate over a network such as the Internet Protocol network. Alternatively, the transmitting device and the receiving device communicate over any appropriate data network. The transmitting device transmits the video images to the remote receiving device either for live display through the source device or for pre-recorded display through a video recorder device. The remote receiving device is also capable of communicating with the transmitting device while simultaneously receiving video images. The source device is preferably a medical test device such as an ultrasound, a sonogram, an echocardiogram, an angioplastigram, and the like. This medical test device preferably generates video images for the transmitting device. The transmitting device captures the video images in real-time from the source device and compresses these video images utilizing a compression algorithm prior to transmitting data representing the video images to the remote receiving device. Remote users utilizing the remote receiving devices are capable of remotely controlling a number of parameters relating to the source device and the transmitting device. Such parameters include image quality, storage of the video images on the transmitting device, manipulating and controlling the source device, and the like.

Claims

1. A system for transmitting data representing a stream of video images, comprising:

a) one or more medical test devices for generating the stream of video images, wherein;

i) each medical test device generates, using a first processor, each image at a device frame rate determined by the settings and capabilities of said medical test device, said device frame rate being at least 8 frames per second and generally less than 100 frames per second,

ii) said video images have a device image width and a device image height determined by the settings and capabilities of said medical test device,

iii) each medical test device displays said video images locally on a display connected to said medical test device at a device display frame rate determined by the settings and capabilities of said medical test device, and

iv) each medical test device outputs said of video images over a standard video link at a standard video frame rate, such as 30 frames per seconds, at a standard image height, having at least 480 lines of resolution per frame, and at a standard image width, having at least 640 square pixels sample points per line;

b) a transmitter removably coupled via said standard video link to any medical test device for receiving and selectively distributing data representing the stream of video images, wherein;

i) said transmitter further comprising video capture device for capturing said stream of video images from said standard video link, said capture device selectively capturing said video images at a capture frame rate, capture width, and capture height, brightness, and contrast, each determined by video capture settings in said transmitter,

ii) each captured frame having a timestamp indicating the time said captured frame was captured, being substantially the time the corresponding video image was generated,

iii) said transmitter further comprising a compressor configured for compressing the data representing the stream of video images, thereby forming a compressed stream of data, said compressor having;

(1) a plurality of clinically lossless compression algorithms implemented as software running on a second processor in said transmitter, and

(2) a compression image width and a compression image height,

iv) said transmitter, further comprising a recorded video device having a record frame rate, a record compression algorithm, a record width and a record height, each determined by record settings in said transmitter, and

v) said transmitter, having;

(1) a transmission frame rate, a transmission compression algorithm, a transmission width and height, and a transmission area, each determined by transmission settings in said transmitter, and

(2) a transmission period of time between the transmission of each transmitted frame, said transmission period determined by said transmission frame rate;

c) one or more remote receivers for communicating with the transmitter and configured to receive the compressed stream of data from the transmitter wherein;

i) each of said one or more remote receivers further comprise a decompressor configured for returning each frame represented by the compressed stream of data into an uncompressed state, forming decompressed frames, said decompressor running a decompression algorithm as software on another processor in each of said receivers, said compression algorithm corresponding to the current transmission compression algorithm,

ii) each receiver further comprising a remote display for displaying the decompressed frames to a user at said receiver, said remote display having a receiver display frame rate, a receiver display image width, and a receiver display image height, said receiver display image width and said receiver display height being determined by display setting determined by said user,

iii) each receiver receives reception settings from said transmitter whenever transmission starts or transmission settings change, said reception settings comprising the transmission frame rate, the transmission compression algorithm, the transmission width, and the transmission height,

iv) each receiver having a reception frame rate,

v) each receiver being configured to receive remote control commands from each respective user for remotely controlling said medical test device or said transmitter, and

vi) said remote control commands including commands specifying changes to the capture settings, record settings or transmission settings; and

d) a packet switched data network coupled between the transmitter and the one or more receivers for transporting the compressed stream of data representing the stream of video images, the reception settings, and the remote control commands, wherein;

i) the compressed stream of data, the reception settings, and the remote control commands are transported over the same digital transmission channel, and

ii) at least one of said receivers is connected to said network with standard phone lines, wherein the maximum sustainable bandwidth between the transmitter and at least one of said receivers is less than one thousand five hundred bits per second;

wherein said transmitter changes capture settings, record settings, or transmission settings when any remote receiver sends predetermined remote control commands,

wherein said transmitter starts transmitting the compressed stream of data when any remote receiver sends a remote control command to start transmission,

wherein said transmitter stops transmitting when any remote receiver sends a remote control command to stop transmission,

wherein any remote receiver dynamically sets the transmission frame rate to match the device frame rate,

wherein any remote receiver dynamically specifies the transmission compression algorithm to optimally compress the clinical content of the image being generated, and

wherein any remote receiver is able to control the capture settings, record settings, and transmission settings,

whereby the user views live diagnostic quality moving video images at a location which is remote from the medical test device, and

whereby the user dynamically balances the tradeoffs between transmission frame rate and image quality to achieve the optimal image in a given circumstance.

2. The system according to claim 1 further wherein:

a) within the transmitter, each potentially transmitted frame having a transmission deadline, being the time of an immediately previously transmitted frame plus the transmission period;

b) within each receiver;

i) each frame having a reception deadline, determined by the timestamp of the immediately previous receiver displayed frame plus a reception period, determined by the reception settings, and

ii) each frame further having a decompression deadline, determined by the timestamp of the immediately previous remotely displayed frame plus a display period, determined by the reception settings; and

c) the packet switched data network is subject to congestion causing delays in transmission of said compressed stream of data;

wherein the transmitter transmits each potentially transmitted frame and its corresponding timestamp if the transmission deadline for each said frame has been met, otherwise the transmitter drops the frame and transmits a dropped frame indicator with the timestamp of the dropped frame,

wherein said receiver notifies the user when a dropped frame indicator is received,

wherein said receiver decompresses each received frame if the reception deadline has been met, otherwise drops the frame and notifies the user, and

wherein said receiver displays, using current display settings, each decompressed frame if the decompression deadline has been met, otherwise drops the frame and notifies the user,

whereby the user is notified when network congestion interferes with transmission at requested transmission settings, and

whereby the user is able to remotely direct a study being performed using the medical test device and assure optimal image recording when network congestion prevents continuous display of live diagnostic quality motion video and later receive and playback recorded video having a continuous full-frame, full-motion, lossless diagnostic quality.

3. The system according to claim 2

wherein said transmitter records to a recorded video file data representing the captured video frames, using current recording settings, when the user sends a remote control command to start recording,

wherein said transmitter stops recording when the user sends a remote control command to stop recording,

wherein said transmitter transmits said recorded video file to the user at the receiver when the user sends a remote control command to transmit said recorded video file, and

wherein said receiver plays said recorded video file at a playback fine rate, a playback height, and a playback width, specified by the user,

whereby the user controls the transmitter to record only portions of the transmitted video, said portions having optimum diagnostic quality motion video of interest to the user, whereby recording storage space is reduced and the time to transmit the recorded video file is reduced,

whereby the user has immediate access to the study, and

whereby a patient who is a subject of the study comprising the video images receives a faster and more accurate report of the study and is scheduled for surgery sooner, when necessary.

4. The system according to claim 1 fixer comprising a recorder device coupled to the medical test device and configured for storing the stream of video images generated by the medical test device.

5. The system according to claim 1 wherein the medical test device is one of an ultrasound, a sonogram, an echocardiogram, and an angioplastigram.

6. The system according to claim 1 wherein the network is an Internet Protocol network.

7. A system for allowing a user to remotely control a medical device, the system comprising:

a) a medical device for generating a stream of video images;

b) a transmitter coupled to the medical device for selectively distributing the stream of video images, wherein said transmitter receives and interprets remote control commands from said user and wherein said transmitter alters the characteristics of the stream of video images in response to the remote control commands;

c) a remote receiver coupled to the transmitter for selectively receiving the stream of video images and allowing the user to remotely control the medical device through the receiver by issuing the remote control commands; and

d) a packet switched data network,

wherein the remote receiver is coupled to the transmitter through the network,

wherein the distributed stream of video images and the remote control commands are transported over a transmission channel in the network,

whereby the distributed stream of video images and the remote control commands are transported over the same low cost, readily available transmission channel,

whereby the user sees the results of the remote control commands in substantially real-time.

8. The system according to claim 7 wherein the medical device is one of an ultrasound, a sonogram, an echocardiogram, and an angioplastigram.

9. The system according to claim 7 wherein the network is an Internet Protocol network.

10. The system according to claim 7 wherein the user remotely controls transmission parameters of the stream of video images including transmission frame rate and transmission frame size,

whereby the user dynamically and remotely controls the transmission parameters of the stream of video images as they are being transmitted.

11. The system of claim 7, said system further comprising:

d) a robotic device coupled to said transmitter,

wherein said transmitter is configured to control said robotic device, and

wherein said transmitter is configured to receive control commands from said user through said remote receiver, and

wherein said robotic device responds to said control commands in substantially real-time, and

wherein said stream of video images comprises a substantially live video, whereby said remote receiver receives and displays said live video substantially in real-time, and

whereby the remote user can control said robotic device with control commands while viewing said live video,

whereby the remote user can perform procedures with the robotic device and the medical device with substantially real-time control and real-time visual feedback.

12. A system for transmitting a real-time video and remote control commands over a digital network, said system comprising:

a) a transmitter containing one or more digitized frames of said real-time video being transmitted,

b) the digital network connected to said transmitter, and

c) one or more remote receivers connected to said network for receiving said video from said transmitter,

wherein at least one of said receivers is configured to receive one or more control commands from a user,

wherein said transmitter is configured to receive and interpret at least one of said control commands from said one of said receivers over said network, and

wherein, upon interpretation of said control command, said transmitter dynamically changes the operation of said transmitter while said video is being transmitted,

whereby said user can remotely control the operation of said transmitter in substantially real-time.

13. The system of claim 12 wherein said control command specifies a subset of the area of said digitized frames,

wherein said transmitter selectively operates on said subset of the frame area.

14. The system of claim 12 wherein said transmitter further comprises a compressor which can be configured to use a plurality of video compression algorithms and,

wherein said control command allows the remote user to select or change the selection of one of the plurality of video compression algorithms to be used by the transmitter to process said digitized frames.

15. The system of claim 12 wherein said control command allows the remote user to start or stop the transmission of said video.

16. A system for transmitting data representing a stream of video images, comprising:

a) a medical test device for generating the stream of video images;

b) a transmitter coupled to the medical test device for receiving and selectively distributing data representing the stream of video images;

c) one or more remote receivers for communicating with the transmitter and configured to receive the data representing the stream of video images from the transmitter; and

d) a digital network coupled between the transmitter and the one or more receivers for transporting the data representing the stream of video images,

wherein said transmitter is configured to control the medical test device,

wherein said transmitter comprises a compressor configured for compressing the data representing the stream of video images, thereby forming a compressed stream of data, and

wherein said one or more receivers further comprise a decompressor configured for returning the compressed stream of data into an uncompressed state, whereby a remote user sees the stream of video images in substantially real-time.

17. The system of claim 16 wherein said one or more remote receivers allow the user to remotely control the medical device through the receiver by issuing remote control commands,

wherein said remote control commands are transported over the same digital network transmission channel as said compressed stream of data,

whereby the user sees the results of the remote control commands in substantially real-time.

18. The system of claim 17, said system further comprising:

d) a robotic device coupled to said transmitter,

wherein said transmitter is configured to control said robotic device, and

wherein said transmitter is configured to receive the control commands from said user through said remote receiver,

wherein said robotic device responds to said control commands in substantially real-time, and

wherein said stream of video images comprises a substantially live video, whereby said remote receiver receives and displays said live video substantially in real-time,

whereby the remote user dynamically controls said robotic device with control commands while viewing said live video, and

whereby the remote user performs procedures with the robotic device and the medical device with substantially real-time control and real-time visual feedback.

19. A system for transmitting data representing a stream of video images and control commands, comprising:

a) a video source;

b) a transmitter coupled to the video source for receiving and selectively distributing data representing the stream of video images;

c) one or more remote receivers for communicating with the transmitter and configured to receive the data representing the stream of video images from the transmitter and to send data representing control commands;

d) a data pipe coupled between the transmitter and at least one receiver for transporting the data representing the stream of video images;

e) a control link coupled between the transmitter and at least one receiver for transporting the data representing the control commands;

wherein said transmitter comprises;

i) a video image capture device with associated video settings;

ii) a video server connected to said video image capture device and further comprising a video compressor for compressing the stream of video images, a first buffer for use by the compressor, and a second buffer for holding at least a portion of the compressed stream of video images,

iii) a listener connected to the video server for making socket connections for the data pipes to said one or more receivers; and

iv) a transmitter video control for receiving control commands from said one or more receivers and altering said video settings and settings of said compressor,

wherein at least one of said receivers comprises;

v) a video client connected to at least one of said data pipes for receiving said stream of compressed video images, said video client further comprising a decompressor and a third buffer for use by the decompressor, whereby said stream of compressed video images is decompressed and displayed to a remote user;

vi) a receiver video control for receiving control commands from said user;

whereby said remote use dynamically and remotely controls the transmitter video capture settings and transmitter compressor settings while viewing said decompressed stream of video images at said remote receiver.

20. The system of claim 19, wherein said transmitter further comprises:

vii) a video recorder connected to the video control and video server for recording the stream of video images for later playback as a recorded video; and

viii) a recorded video transmitter for transmitting said recorded video to at least one of said receivers via a recorded video data pipe;

wherein at least one of said receivers further comprises;

ix) a video player connected to said recorded video data pipe and said video client and said video control whereby said recorded video is received and displayed to said user;

whereby said remote user dynamically and remotely controls the recording of portions of said stream of video images in one or more instances of said recorded video and remotely controls selection and playback of at least one of said instances of said recorded video.

21. The system of claim 20, wherein said video recorder further comprises an edit list, said edit list comprising a list of one or more segments of the recorded video, whereby specified portions of the recorded video are selected for transmission, and

whereby said remote user remotely controls the portions of the recorded video that is transmitted from the transmitter to at least one remote receiver.

22. The system of claim 20, wherein said video recorder further comprises an edit list, said edit list comprising a list of one or more segments of the recorded video, whereby specified portions of the recorded video are selected for special processing, and

whereby said remote user remotely controls the portions of the recorded video that is specially processed.