Computerized information collection and processing apparatus

• US 10,028,646 B2
• Filed: 02/14/2018
• Issued: 07/24/2018
• Est. Priority Date: 03/01/1999
• Status: Expired due to Fees

• Current Assignee: West View Research LLC
• Original Assignee: West View Research LLC
• Inventors: Gazdzinski, Robert F.
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US15/897,062
• Publication Number:
  US 20180168438A1
• Time in Patent Office: 160 Days
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00016 using wireless means
• CPC Class Codes: A61B 1/0002 provided with data storages
• CPC Class Codes: A61B 1/00032 internally powered
• CPC Class Codes: A61B 1/00036 Means for power saving, e.g. sleeping mode
• CPC Class Codes: A61B 1/0004 for electronic operation
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00059 provided with identification means for the endoscope
• CPC Class Codes: A61B 1/00156 using self propulsion
• CPC Class Codes: A61B 1/00158 using magnetic field
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/273 for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes
• CPC Class Codes: A61B 1/31 for the rectum, e.g. proctoscopes, sigmoidoscopes , colonoscopes
• CPC Class Codes: A61B 10/02 Instruments for taking cell samples or for biopsy A61B10/0038 and A61B10/0045 take precedence; needle locating or guiding means A61B17/3403; samplers for enzymology or microbiology C12M1/26; sampling or preparing biological specimens G01N33/48
• CPC Class Codes: A61B 18/20 using laser
• CPC Class Codes: A61B 2562/162 Capsule shaped sensor housings, e.g. for swallowing or implantation
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0084 : for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/073 : Intestinal transmitters
• CPC Class Codes: A61B 5/411 : Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/4255 : Intestines, colon or appendix
• CPC Class Codes: A61B 5/6861 : Capsules, e.g. for swallowing or implanting
• CPC Class Codes: A61B 5/7232 : involving compression of the physiological signal, e.g. to extend the signal recording period
• CPC Class Codes: A61B 5/7282 : Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 8/12 : in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/4245 : involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
• CPC Class Codes: A61B 8/4472 : Wireless probes
• CPC Class Codes: A61B 8/4488 : the transducer being a phased array
• CPC Class Codes: A61B 8/56 : Details of data transmission or power supply
• CPC Class Codes: A61N 2/002 : in combination with another treatment
• CPC Class Codes: A61N 2005/1005 : with asymmetrical radiation pattern
• CPC Class Codes: A61N 5/1014 : Intracavitary radiation therapy
• CPC Class Codes: G06T 2207/30028 : Colon; Small intestine
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G16Z 99/00 : Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04L 1/004 : by using forward error control H04L1/0618 takes precedence; coding, decoding or code conversion, for error detection or correction H03M13/00

Abstract

Computerized information acquisition and processing apparatus for use in endoscopic analysis. In one embodiment, the apparatus includes a swallowable apparatus having a digital processor apparatus and video apparatus with image capture and digitization capability, and a wireless interface for accomplishing various purposes, including e.g., transferring the digital image data to a second computerized apparatus having various algorithmic analysis functions operative thereon, including for example pixel intensity analysis, shape recognition, and preview functions, so as to enhance efficiency of the review process. In another embodiment, portions of the algorithmic analysis (and other functions) are performed by the digital processor apparatus on the swallowable apparatus, including selective deletion or omission of data prior to transmission.

Claims

1. Computerized information acquisition and processing apparatus configured for efficient data processing and review, comprising:

first computerized apparatus, the first computerized apparatus swallowable by a human being and comprising;

an electrical power source;

at least one integrated circuit-based camera apparatus;

video processing circuitry in communication with the at least one integrated circuit-based camera apparatus and configured to generate digital image data from signals received from the at least one integrated circuit-based camera apparatus;

first digital processor apparatus in data communication with the video processing circuitry;

digital data storage apparatus in data communication with the first digital processor apparatus; and

a wireless data interface having at least one antenna and configured for digital data transfer, the wireless data interface being powered at least in part by the electrical power source; and

second computerized apparatus configured to communicate with the first computerized apparatus via the wireless data interface to receive at least a portion of the digital image data while the first computerized apparatus is within an intestinal tract of the human being, the second computerized apparatus comprising a second digital processor apparatus and at least one computer program operative to run on the second processor apparatus and configured to, when executed, process at least a portion of said received at least portion of said digital image data, said processing comprising;

production of a plurality of frames from said at least portion of said received at least portion of said digital image data;

algorithmic analysis of said plurality of frames to automatically identify a subset of said plurality of frames of potential interest to a user of the second computerized apparatus, the identification of the subset based at least in part on algorithmic identification of an artifact rendered in at least one of the plurality of frames of the subset; and

causation of generation of a display on a computerized monitor apparatus associated with the second computerized apparatus, the display comprising identification of at least a portion of said subset of said plurality of frames.

2. The computerized information acquisition and processing apparatus of claim 1, wherein:

the display further comprises a display having at least a portion of the subset of the plurality of frames displayed in a temporal sequence, the temporal sequence corresponding to the travel of the first computerized apparatus within the intestinal tract; and

the causing generation of a display provides the user with an algorithmically selected preview of the data, the preview comprising said at least portion of said subset selected based at least in part on scores or values attributed to said plurality of frames of said subset by said at least one computer program and based on said algorithmic analysis.

3. The computerized information acquisition and processing apparatus of claim 1, wherein:

the display further comprises a display having at least a portion of the subset of the plurality of frames displayed in a temporal sequence; and

the display of at least a portion of the subset of the plurality of frames displayed in a temporal sequence comprises display of the plurality of frames of the subset in their temporal sequence of occurrence, with any intervening frames that are not part of said subset automatically removed.

4. The computerized information acquisition and processing apparatus of claim 1, wherein the at least one at least one computer program operative to run on the second digital processor apparatus is further configured to, when executed:

access one or more stored template data structures comprising data relating to one or more shapes of potential relation to the artifact; and

cause algorithmic analysis of at least a portion of the subset of the plurality of frames to evaluate a correlation of the artifact to the one or more shapes, the algorithmic analysis of at least a portion of the subset of the plurality of frames to evaluate a correlation of the artifact to the one or more shapes comprising mathematical correlation analysis of at least one of;

(i) plurality of nodal points of each of the artifact and the one or more shapes; and

(ii) one or more mathematical functions or curves of each of the artifact and the one or more shapes.

5. The computerized information acquisition and processing apparatus of claim 1, wherein the algorithmic analysis of said plurality of frames to identify a subset of said plurality of frames of potential interest to a user of the second computerized apparatus comprises algorithmic identification of image intensity data associated with at least one region of at least one frame of the subset which has a difference in the intensity of at least one color as compared to at least one of:

(i) another region of the same at least one frame of the subset, or (ii) to another frame of the subset.

6. The computerized information acquisition and processing apparatus of claim 1, wherein the production of the plurality of frames comprises:

algorithmic sampling of the received at least portion of said digital image data according to a prescribed sampling rate or periodicity to produce sampled digital image data; and

generating the plurality of frames only from the sampled digital image data;

wherein the algorithmic sampling of the received at least portion of said digital image data, and the generating the plurality of frames only from the sampled digital image data, cooperate to reduce a volume of data which must be subsequently processed and reviewed by the user.

7. The computerized information acquisition and processing apparatus of claim 6, wherein the prescribed sampling rate or periodicity comprises a rate or periodicity automatically determined by the at least one computer program based on data input to the at least one computer program via a user interface generated by the second computerized apparatus and relating to a degree of temporal compression desired by a user of the second computerized apparatus.

8. The computerized information acquisition and processing apparatus of claim 1, wherein the second computerized apparatus is further configured to:

sample the received at least portion of said digital image data according to a prescribed sampling rate to produce sampled digital image data; and

causing display of at least a portion of the subset according to a prescribed display rate which is different than the prescribed sampling rate; and

wherein the production of a plurality of frames from said at least portion of said received at least portion of said digital image data comprises production of the plurality of frames from said sampled digital image data only.

9. The computerized information acquisition and processing apparatus of claim 1, wherein:

the production of a plurality of frames from said at least portion of said received at least portion of said digital image data comprises production of a plurality of frames based only on sampling of the received at least portion of the digital image data at a prescribed sampling rate;

the display further comprises a display having at least a portion of the subset of the plurality of frames displayed in a temporal sequence; and

the display of at least a portion of the subset of the plurality of frames displayed in a temporal sequence is displayed as a video data stream having a display rate different from the sampling rate.

10. The computerized information acquisition and processing apparatus of claim 1, wherein the second computerized apparatus comprises an object-oriented software environment, and the at least one computer program is configured to, upon execution, cause generation of a plurality of display regions on the computerized monitor apparatus, each of the display regions comprising respective software objects within the object-oriented software environment, each of the display regions comprising display of only a portion of the subset of the plurality of frames, each of the software objects being independently controllable by the user.

11. The computerized information acquisition and processing apparatus of claim 1, further comprising:

at least one computer program having a data compression algorithm, the at least one computer program having a data compression algorithm operative to run on the first digital processor apparatus and to compress at least portions of the digital image data according to the data compression algorithm prior to transmission of the digital image data via the wireless interface, the compression of the digital image data providing at least a reduction of a communications bandwidth necessary to transmit the digital image data over the wireless interface as compared to a communications bandwidth of uncompressed digital image data; and

wherein the compression of the digital data comprises selective elimination of one or more portions of the digital data based on a determination made by computerized logic operative to be executed the first digital processor apparatus of the first computerized apparatus, the determination comprising at least one of;

(i) an algorithmic shape analysis, and (ii) an algorithmic pixel intensity analysis.

12. The computerized information acquisition and processing apparatus of claim 1, wherein the first computerized apparatus further comprises at least one computer program operative to run on the first processor apparatus, the at least one computer program configured to, when executed:

generate two or more portions of the digital image data;

cause transmission of a first of the two or more portions of the digital image data via a first one of at least two communication channels of the wireless data interface; and

cause transmission of a second of the two or more portions of the digital image data via a second, different one of the at least two communication channels of the wireless data interface.

13. The computerized information acquisition and processing apparatus of claim 1, wherein the first computerized apparatus further comprises at least one computer program operative to run on the first processor apparatus, the at least one computer program configured to, when executed, selectively delete or omit one or more portions of the digital image data prior to transmission via the wireless of data interface, the selective deletion or omission of the one or more portions of the digital image data prior to transmission via the wireless data interface based at least on an algorithmic analysis by the at least one computer program operative to run on the first processor apparatus which compares a first portion of data within the one or more portions of digital image data to data previously stored in the digital data storage apparatus.

14. The computerized information acquisition and processing apparatus of claim 13, wherein the algorithmic analysis by the at least one computer program operative to run on the first digital processor apparatus which compares a first portion of data within the one or more portions of digital image data to data previously stored in the digital data storage apparatus comprises a similarity analysis between data within the first portion of data corresponding to at least a portion of a field of view (FOV) of the at least one integrated circuit-based camera apparatus, and the previously stored data.

15. The computerized information acquisition and processing apparatus of claim 1, wherein the first computerized apparatus further comprises at least one computer program operative to run on the first digital processor apparatus, the at least one computer program configured to, when executed, selectively delete or omit one or more portions of the digital image data prior to transmission via the wireless interface, the selective deletion or omission of the one or more portions of the digital image data prior to transmission via the wireless interface based at least on data relating to a then-present location of the first computerized apparatus within the intestinal tract.

16. The computerized information acquisition and processing apparatus of claim 1, wherein the second computerized apparatus further comprises receiver apparatus in data communication with the second digital processor apparatus, the receiver apparatus comprising at least one apparatus configured to be worn around at least an abdominal portion of the human being;

wherein the first computerized apparatus is configured to transfer the at least portion of the digital image data over the wireless interface using electromagnetic signals, and the at least one apparatus configured to be worn around an abdominal portion of the human being comprises a plurality of individual receiver devices, the individual receiver devices each corresponding to at least one respective data channel.

17. The computerized information acquisition and processing apparatus of claim 16, wherein the second computerized apparatus is further configured to algorithmically evaluate data relating to at least one of:

(i) a signal strength of respective ones of at least two of the plurality of individual receiver devices, and (ii) a signal quality of respective ones of at least two of the plurality of individual receiver devices, so as to enable selection of at least one of the at least two individual receiver devices over one or more of the others of the at least two receiver devices, the selection causing use of data of a data channel from the selected at least one individual receiver device for said processing of said at least a portion of said received at least portion of said digital image data by said second digital processor and said at least one computer program.

18. The computerized information acquisition and processing apparatus of claim 17, wherein the second computerized apparatus further comprises multiplexer apparatus in signal communication with the at least two of the plurality of individual receiver devices, the multiplexer apparatus configured to temporally multiplex signals generated by the at least two individual receiver devices into a single output of the multiplexer apparatus;

and

wherein the algorithmic evaluation of the data relating to at least one of;

(i) the signal strength of respective ones of the at least two of the plurality of individual receiver devices, and (ii) the signal quality of respective ones of the at least two of the plurality of individual receiver devices, is performed on the single output of the multiplexer apparatus.

19. The computerized information acquisition and processing apparatus of claim 1, wherein the first computerized apparatus further comprises at least one computer program configured to execute on the first digital processor apparatus, the at least one computer program configured to execute configured to, when executed, perform an algorithmic analysis of at least a portion of the digital image data, the algorithmic analysis of the at least portion of the digital image data enabling the identification of the artifact rendered in at least one of the plurality of frames of the subset and comprising detection of image intensity data associated with at least one color, the at least one color selected from the group consisting of red, green, and blue.

20. Computerized apparatus configured for processing data derived from an integrated circuit-based visual band sensor apparatus disposed on an autonomous intestinal probe, said data having been generated while said probe moved autonomously through the intestinal tract of a human being, the computerized apparatus comprising:

a data interface configured to receive at least a portion of the data derived from the integrated circuit-based visual band sensor;

digital processor apparatus in data communication with the data interface;

a digital data storage device in data communication with the digital processor apparatus;

a display device in data communication with the digital processor apparatus; and

at least one computer program configured to run on the digital processor apparatus, the at least one computer program configured to, when executed, process said received at least portion of the data, said processing comprising;

evaluation of the received at least portion of the data to identify at least a first portion that is of possible interest to a human reviewer, said identification comprising algorithmic analysis of said at received at least portion of the data to identify one or more artifacts corresponding to one or more known physiologic conditions associated with an intestinal tract of the human being, said one or more artifacts comprising at least one of;

(i) one or more characteristic shapes associated with said one or more known physiologic conditions; and

(ii) one or more color intensity values associated with said one or more known physiologic conditions; and

presentation, at least initially, of only said first portion to the human reviewer for further review via the display device;

wherein said computerized apparatus is configured to perform said algorithmic analysis automatically as part of said processing.

21. The computerized apparatus of claim 20, wherein said analysis further comprises applying one or more mathematical corrections or adjustments for a visual perspective associated with the integrated circuit-based visual band sensor when the data was generated thereby.

22. The computerized apparatus of claim 20, wherein said identification of at least one of:

(i) one or more characteristic shapes associated with said one or more known physiologic conditions; and

(ii) one or more color intensity values associated with said one or more known physiologic conditions, comprises identification of said one or more characteristic shapes associated with said one or more known physiologic conditions based at least in part on a multi-nodal spatial analysis of the received at least portion of the data.

23. The computerized apparatus of claim 20, wherein the processing further comprises generating a plurality of frames from the received at least portion of the data, and the identification of the first portion comprises selecting a subset of the plurality of frames based at least in part on data corresponding to scores or values attributed to at least a portion of said plurality of frames of by said at least one computer program;

wherein at least some of said scores are based at least in part on recognition of the one or more artifacts as correlating to data of one or more previously stored templates, the one or more previously stored templates associated with the one or more know physiological conditions of the intestinal tract, the one or more known conditions of the intestinal tract comprising at least one of;

(i) polyps;

(ii) carcinomas;

or (iii) ulcerations.

24. The computerized apparatus of claim 20, wherein:

the processing further comprises generating a plurality of frames from the the received at least portion of the data; and

said analysis of said received at least portion of the data to identify one or more artifacts corresponding to one or more known physiologic conditions further comprises use of a multi-scale image processing algorithm to identify at least portions of the one or more artifacts within one or more of the plurality of frames, the multi-scale image processing algorithm configured to select portions of the one or more of the plurality of frames for the analysis which are not within a focal region of the integrated circuit-based visual band sensor.

25. The computerized apparatus of claim 20, wherein:

the processing further comprises generating a plurality of frames from the received at least portion of the data, each of the plurality of frames having a plurality of pixels; and

said analysis said at least portion of the received data to identify one or more artifacts corresponding to one or more known physiologic conditions associated with an intestinal tract of a human being further comprises algorithmic analysis of at least one intensity of at least a portion of the plurality of pixels in at least one of the plurality of frames to identify at least portions of the one or more artifacts within the at least one frame.

26. Computerized apparatus for processing data derived from signals of an integrated circuit-based visual-band sensor apparatus disposed on a swallowable intestinal probe, said signals having been generated while said swallowable intestinal probe moved autonomously through the intestinal tract of a living being, the computerized apparatus configured to enhance an efficiency of review of the data by a human reviewer, the computerized apparatus comprising:

a data interface configured to receive at least a portion of the data derived from the visual band sensor;

digital processor apparatus in data communication with the data interface;

a digital data storage device in data communication with the digital processor apparatus;

a display device in data communication with the digital processor apparatus; and

at least one computer program configured to run on the digital processor apparatus, the at least one computer program configured to, when executed, process said received at least portion of the data, said processing comprising;

identification of a plurality of frames from the received at least portion of the data;

automatic selection of a subset of said plurality of frames that are likely to be of interest to the reviewer, the automatic selection being (i) without user input identifying the subset, and (ii) based at least in part on detection of the presence of a physiological artifact rendered within at least a first one of the plurality of frames, the automatic selection of the subset further comprising selection of one or more frames temporally contiguous to said first one of the frames; and

causing generation of a display on the display device, the display comprising at least two of said subset of said plurality of frames displayed simultaneously as a temporal sequence in respective ones of at least two regions of a display screen of the display device.

27. The computerized apparatus of claim 26, wherein:

the at least two of said subset of said plurality of frames displayed simultaneously in respective ones of at least two contiguous regions of the display screen of the display device comprise the first one of the plurality of the frames, and two frames temporally contiguous to said first one of the frames; and

the simultaneous display as a temporal sequence comprises display of additional frames both temporally before and after the two temporally contiguous frames, the at least one computer program further configured to select the additional frames based on a prescribed temporal window or duration value.

28. The computerized apparatus of claim 26, wherein the detection, using the at least one computer program, of the presence of a physiological artifact rendered within at least a first one of the plurality of frames, comprises detection of the presence of the physiological artifact based at least on an algorithmic pixel intensity analysis of at least a region of the at least first one of the plurality of frames.

29. Computerized information acquisition and processing apparatus, comprising:

first computerized apparatus, the first computerized apparatus swallowable by a human being and comprising;

a power source;

at least one camera apparatus configured to generate digital image data from signals received from one or more semiconductor-based sensor elements of the at least one camera apparatus;

first processor apparatus in data communication with the at least one camera apparatus;

digital data storage apparatus in data communication with the first processor apparatus;

a data interface configured for digital data transfer, the data interface being powered at least in part by the power source; and

at least one computer program having a data compression algorithm operative to run on the first processor apparatus and, upon execution thereof, to compress at least portions of the digital image data according to the data compression algorithm prior to transmission of the digital image data via the data interface, the compression of the digital image data providing at least a reduction of storage space necessary to store the digital image data within the digital data storage apparatus as compared to uncompressed digital image data;

second computerized apparatus configured to communicate with the first computerized apparatus via the data interface to receive at least a portion of the compressed digital image data, the second computerized apparatus comprising a second processor apparatus and at least one computer program operative to run on the second processor apparatus and configured to, when executed, process at least a portion of said received compressed digital image data, said processing comprising;

decompression of at least portions of the received compressed digital image data;

production of a plurality of frames from said decompressed at least portions of said received digital image data;

algorithmic analysis of said plurality of frames to identify a subset of said plurality of frames of potential interest to a user of the second computerized apparatus, the identification of the subset based at least in part on identification of an artifact rendered in at least one of the plurality of frames of the subset; and

causation of generation of a display on a computerized monitor apparatus associated with the second computerized apparatus, the display comprising at least a portion of said subset of said plurality of frames.

30. The computerized information acquisition and processing apparatus of claim 29, wherein the algorithmic analysis of said plurality of frames to identify a subset of said plurality of frames of potential interest to a user of the second computerized apparatus comprises identification of image intensity data associated with at least one region of at least one frame of the subset which has at least a prescribed level of intensity of at least one color, the at least one color selected from the group consisting of Red, Green, and Blue;

and

wherein the data interface is configured to communicate with the first computerized apparatus via the data interface to receive at least a portion of the compressed digital image data after the first computerized apparatus has been excreted by the human being, the communication comprising use of a third computerized device to establish a data communication link with the first computerized apparatus in order to transfer the at least portion of the compressed digital image data from the digital data storage apparatus to the third device.