Method of Monitoring a Surface Feature and Apparatus Therefor

• US 20090213213A1
• Filed: 10/13/2006
• Published: 08/27/2009
• Est. Priority Date: 10/14/2005
• Status: Active Grant

• Current Assignee: Aranz Healthcare Limited
• Original Assignee: Applied Research Associates New Zealand Limited
• Inventors: Preddey, James Telford George, McCallum, Bruce Clinton, Fright, William Richard, Nixon, Mark Arthur
• Granted Patent: US 8,755,053 B2
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/1073 Measuring volume, e.g. of limbs
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61B 5/14539 for measuring pH
• CPC Class Codes: A61B 5/444 Evaluating skin marks, e.g. mole, nevi, tumour, scar
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: A61B 5/447 specially adapted for aiding the prevention of ulcer or pressure sore development, i.e. before the ulcer or sore has developed
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: A61B 5/748 : Selection of a region of interest, e.g. using a graphics tablet
• CPC Class Codes: A61B 5/7485 : Automatic selection of region of interest
• CPC Class Codes: A61B 8/488 : involving Doppler signals
• CPC Class Codes: A61B 8/5223 : for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: G01B 11/002 : for measuring two or more coordinates
• CPC Class Codes: G01B 11/02 : for measuring length, width or thickness G01B11/08 takes precedence
• CPC Class Codes: G01B 11/14 : for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01B 11/22 : for measuring depth
• CPC Class Codes: G01B 11/285 : using photoelectric detection means
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30088 : Skin; Dermal
• CPC Class Codes: G06T 7/0004 : Industrial image inspection
• CPC Class Codes: G06T 7/0014 : using an image reference approach
• CPC Class Codes: H04N 1/00209 : Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

Dimensions of a surface feature are determined by capturing an image of the surface feature and determining a scale associated with the image. Structured light may be projected onto the surface, such that the position of structured light in the captured image allows determination of scale. A non-planar surface may be unwrapped. The surface may alternatively be projected into a plane to correct for the scene being tilted with respect to the camera axis. A border of the surface feature may be input manually by a user. An apparatus and system for implementing the method are also disclosed.

Claims

1-78. -78. (canceled)

79. A method of producing a planar form of a non-planar surface including a surface feature comprising:

a. projecting structured light onto the surface;

b. capturing an image including the surface feature;

c. determining the three-dimensional coordinates of structured light elements within the image; and

d. unwrapping the image based on the three-dimensional coordinates of to the structured light elements to produce a planar form of the surface.

80. A method as claimed in claim 79 further comprising determining the area and/or depth of the surface feature.

81. A method as claimed in claim 79 wherein the image is unwrapped according to either a model of an inelastic surface draped between the three-dimensional coordinates of the structured light elements, or a model of an elastic surface stretched between the three-dimensional coordinates of the structured light elements.

82. A method as claimed in claim 79 wherein the image is unwrapped according to a model of physical anatomy.

83. A method as claimed in claim 81 wherein the model is a developable surface model.

84. A method of producing a projection of a surface feature comprising:

a. capturing an image of a surface feature;

b. determining from the image the coordinates of a plurality of points of the surface feature in three-dimensional space;

c. determining a plane in which at least a subset of the coordinates lie; and

d. projecting the image onto the plane to produce a transformed image.

85. A method as claimed in claim 84 further comprising determining one or more of:

the area of the surface feature from the transformed image; and

the depth of at least one coordinate with respect to the plane.

86. A method as claimed in claim 84 wherein the coordinates of a plurality of points of the surface feature are determined by projecting structured light onto the surface feature and determining the coordinates based on information from the image.

87. A method of determining at least one dimension of a surface feature, including:

a. capturing an image including a surface feature;

b. determining a scale associated with the image;

c. manually inputting at least part of an outline of the surface feature; and

d. determining at least one dimension of the surface feature using the manually input outline data.

88. A method as claimed in claim 87 wherein a scale associated with the image is determined by projecting structured light onto the surface feature, determining coordinates of points of the surface feature based on attributes of the structured light, determining a scale associated with the image from the coordinates.

89. An apparatus including:

a. a camera for capturing an image including a surface feature; and

b. a portable computing device including;

i. a display configured to display the image and to allow a user to manually input at least part of an outline of the surface feature; and

ii. a processor configured to determine a scale associated with the image and to determine at least one dimension of the surface feature using the manually input outline data.

90. An apparatus as claimed in claim 89 including a structured light projector disposed at an angle to the optical axis of the camera.

97. A healthcare system including one or more apparatuses as claimed in claim 89, a server and a central database, wherein each portable apparatus is configured to communicate over a communications network with the server, which is adapted to receive patient to data and images from the portable apparatuses and to store them in the central database, allowing historical patient data and images to be retrieved.

98. A healthcare system as claimed in claim 97 wherein the processor or one of the apparatuses is configured to perform one or more of:

processing patient data and or images retrieved from the database in order to assess the effectiveness of a particular treatment;

receiving current patient data and suggesting a treatment based on that patient data and on historical patient data retrieved from the database;

receiving patient data and providing one or more predictions for wound

healing based on that patient data and on historical patient data retrieved from the database; and

issuing an alert when actual wound healing differs from the predictions by more than a threshold.

91. A portable apparatus including:

a. a camera for capturing an image of a surface feature;

b. a portable computing device including a processor adapted to determine a scale associated with the image; and

c. a positioning module allowing the position of the apparatus to be determined.

92. A portable apparatus as claimed in claim 91 wherein the positioning module includes a GPS receiver.

93. A system including a portable apparatus as claimed in claim 91, the system being configured to determine the position of the apparatus using the positioning module and to retrieve one or more of:

personal details associated with the position; and

address information in accordance with the position.

94. A system as claimed in claim 93 configured to associate the image with the address and/or position and/or personal details.

95. A healthcare apparatus including:

a. camera for capturing an image of a surface feature on a patient;

b. one or more auxiliary sensors for determining a physical or chemical parameter associated with the patient; and

c. a portable computing device configured to receive image data from the camera and output from the auxiliary sensors, including a processor adapted to determine a scale associated with the image.

96. A healthcare apparatus as claimed in claim 95 wherein the auxiliary sensors include one or more of:

a Doppler Ultrasound Probe;

a thermometer;

a pH sensor;

a moisture sensor;

an odour sensor; and

an optical probe.

99. An apparatus including:

a. camera for capturing an image including a surface feature; and

b. one or more structured light projectors configured to project structured light onto the surface, the structured light including two or more structured light components, each projected at a different angle to the camera'"'"'s optical axis.

100. An apparatus as claimed in claim 99 further including a portable computing device having a processor, wherein the structured light components are three or more spots projected onto the surface, and the processor is configured to fit a plane through at least three points where the spots lie on the surface.

101. An apparatus as claimed in claim 99 wherein the structured light components 5 are stripes projected onto the surface.

102. An apparatus as claimed in claim 101 wherein the structured light components include a first stripe and a second stripe projected so as cross before reaching the surface.

103. An apparatus as claimed in claim 101 further including a portable computing device having a processor, wherein the processor is configured to map the surface feature into a plane based on the positions of the stripes on the surface.