User-assistance information at least partially based on an identified possible non-imaged portion of a skin

• US 8,644,615 B2
• Filed: 12/07/2011
• Issued: 02/04/2014
• Est. Priority Date: 12/07/2011
• Status: Expired due to Fees

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Hyde, Roderick A., Kare, Jordin T., Leuthardt, Eric C., Lieberman, Erez, Rivet, Dennis J., Sweeney, Elizabeth A., Wood, Lowell L. Jr.
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/373,998
• Publication Number:
  US 20130148869A1
• Time in Patent Office: 790 Days
• Field of Search: None
• US Class Current: 382/190
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

Described embodiments include a system, method, and program product. A described system includes a circuit that determines a substantial correspondence between a human-perceivable feature included in a border region segment of a selected medical skin image and a human-perceivable feature included in each other medical skin image of a plurality of medical skin images. A circuit gathers the determined substantial correspondences. A circuit generates data indicative of a border region-overlap status of the selected medical skin image. A circuit adds the data to an omitted-coverage list. A circuit iteratively designates a next medical skin image as the selected digital image, and initiates a processing of each of the iteratively designated next medical skin images. A circuit identifies a possible non-imaged portion of the region of interest. A circuit outputs user-assistance information based on the identified possible non-imaged portion of the skin.

Claims

1. A system comprising:

(a) a feature-detection circuit configured to extract at least one respective human-perceivable feature included in each medical image of a plurality of medical images of the skin of an individual human (hereafter “

medical skin images”

), the each medical image of plurality of medical skin images includes a respective portion of a region of interest of a surface of the skin of the individual human, and was acquired by a handheld digital image acquisition device;

(b) a feature-matching circuit configured to determine a substantial correspondence between

(x) an extracted human-perceivable feature included in a border region segment of a selected medical skin image of the plurality of medical skin images and

(y) an extracted at least one respective human-perceivable feature included in the each medical skin image of the plurality of medical skin images other than the selected medical skin image;

(c) a data collection circuit configured to gather the determined substantial correspondences for the extracted human-perceivable feature included in the border region segment of the selected medical skin image;

(d) an overlap-analysis circuit configured to generate data indicative of a border region-overlap status of the selected medical skin image, the data generated at least partially in response to the determined substantial correspondences;

(e) a list management circuit configured to add the data indicative of the determined border region-overlap status for the border region segment of the selected medical skin image to an omitted-coverage list;

(f) an iteration control circuit configured to

iteratively designate a next medical skin image from the plurality of medical skin images as the selected medical skin image until each medical skin image of the plurality of medical skin images has been designated, and

initiate a processing of each of the iteratively designated next medical skin images by the feature-matching circuit, the data collection circuit, the overlap-analysis circuit, and the list management circuit;

(g) a coverage-analysis circuit configured to identify a particular portion of the skin surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the region of interest”

), the identifying the possible non-imaged portion of the region of interest at least partially based on the omitted-coverage list; and

(h) a reporting circuit configured to output user-assistance information at least partially based on the identified possible non-imaged portion of the skin.

2. The system of claim 1, further comprising:

an image receiver circuit configured to receive the plurality of medical skin images.

3. The system of claim 1, further comprising:

a border region detection circuit configured to detect at least one border region segment of a medical skin image of the plurality of medical skin images.

4. The system of claim 1, wherein the coverage-analysis circuit includes:

a coverage-analysis circuit configured to identify a particular portion of the skin surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the skin”

) and to identify at least one medical skin image of the plurality of medical skin images as spatially proximate to the possible non-imaged portion of the region of interest (hereafter “

signpost medical skin image”

), the identifying the possible non-imaged portion of the skin and the identifying the signpost medical skin image at least partially based on the omitted-coverage list.

5. The system of claim 4, wherein the reporting circuit includes:

a reporting circuit configured to output user-assistance information indicative of the possible non-imaged portion of the region of interest adjacent to the selected medical skin image and of the signpost medical skin image.

6. The system of claim 1, wherein the coverage-analysis circuit includes:

a coverage-analysis circuit configured to

(i) identify a particular portion of the region of interest of a surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the region of interest”

), the identifying the possible non-imaged portion of the region of interest at least partially based on the omitted-coverage list;

(ii) identify at least three medical skin images of the plurality of medical skin images immediately adjacent to the possible non-imaged portion of the region of interest, the identifying the at least three medical skin images at least partially based on the omitted-coverage list;

(iii) define a substantially simple closed curve formed by linking at least one border region segment of each of the at least three identified medical skin images wherein the border region segments each have an overlap status of likely not overlapped; and

(iv) determine whether the possible non-imaged portion of the region of interest lies inside or outside of the region of interest, the determination at least partially based on the positions of the identified at least three medical skin images relative to the closed curve.

7. The system of claim 6, wherein the reporting circuit includes:

a reporting circuit configured to output user-assistance information indicative of the identified possible non-imaged portion of the region of interest and indicative of the determination whether the possible non-imaged portion of the region of interest lies inside or outside of the region of interest.

8. The system of claim 1, wherein the user-assistance information includes a user-assistance information corresponding to a location of the possible non-imaged portion of the region of interest.

9. The system of claim 1, wherein the user-assistance information includes user-assistance information corresponding to spatially orientating the handheld digital image acquisition device in an alignment to acquire a medical skin image of the possible non-imaged portion of the region of interest.

10. The system of claim 1, wherein the user-assistance information includes user-assistance information corresponding to selecting a parameter facilitating an acquisition of a medical skin image of the possible non-imaged portion of the region of interest.

11. The system of claim 1, wherein the user-assistance information includes user-assistance information corresponding to initiating an acquisition by the handheld digital image acquisition device of a medical skin image of the possible non-imaged portion of the region of interest.

12. The system of claim 1, wherein the user-assistance information includes user instructions corresponding to operating the handheld digital image acquisition device in acquiring a medical skin image of the possible non-imaged portion of the region of interest.

13. The system of claim 1, wherein the user-assistance information includes user-assistance information responsive to a request entered by the user in conjunction with acquiring a medical skin image of the possible non-imaged portion of the region of interest.

14. The system of claim 1, wherein the reporting circuit includes:

a reporting circuit configured to output user-assistance information in substantially real-time, the user-assistance information at least partially based on the identified possible non-imaged portion of the surface of the skin.

15. The system of claim 1, wherein the reporting circuit includes:

a reporting circuit configured to output user-assistance information usable in displaying a human-perceivable indication of the possible non-imaged portion of the region of interest of the surface of the skin.

16. The system of claim 1, wherein the reporting circuit includes:

a reporting circuit configured to output a rendering of the user-assistance information in a form facilitating a human-perceivable representation of the user-assistance information.

17. The system of claim 1, further comprising:

a communications device configured to display a particular human-perceivable depiction of the user-assistance information.

18. A method implemented in a computing device, the method comprising:

(a) extracting at least one respective human-perceivable feature included in each medical image of a plurality of medical images of the skin of an individual human (hereafter “

medical skin images”

), the each medical image of plurality of medical skin images includes a respective portion of a region of interest of a surface of the skin of the individual human, and was acquired by a handheld digital image acquisition device;

(b) determining a substantial correspondence between

(x) an extracted human-perceivable feature included in a border region segment of a selected medical skin image of the plurality of medical skin images and

(y) an extracted at least one respective human-perceivable feature included in each medical skin image of the plurality of medical skin images other than the selected medical skin image;

(c) gathering the determined substantial correspondences for the extracted human-perceivable feature included in the border region segment of the selected medical skin image;

(d) generating data indicative of a border region-overlap status of the selected medical skin image, the data generated at least partially in response to the determined substantial correspondences;

(e) adding the data indicative of the determined border region-overlap status for the border region segment of the selected medical skin image to an omitted-coverage list;

(f) iteratively designating a next medical skin image from the plurality of medical skin images as the selected medical skin image until each medical skin image of the plurality of medical skin images has been designated;

(g) processing of each of the iteratively designated next medical skin images, the processing including operations (b), (c), (d), and (e);

(h) identifying a particular portion of the skin surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the skin”

), the identifying the possible non-imaged portion of the skin at least partially based on the omitted-coverage list; and

(i) outputting user-assistance information at least partially based on the identified possible non-imaged portion of the skin.

19. The method of claim 18, wherein the outputting user-assistance information includes:

outputting user-assistance information in substantially real-time and at least partially based on the identified possible non-imaged portion of the skin.

20. The method of claim 18, wherein the outputting user-assistance information includes:

outputting user-assistance information usable in displaying a human-perceivable indication of the possible non-imaged portion of the region of interest of the surface.

21. The method of claim 18, wherein the outputting user-assistance information includes:

transforming the user-assistance information into a particular visual depiction of a possible non-imaged portion of the region of interest of the surface, and outputting the transformed user-assistance information.

22. The method of claim 18, further comprising:

receiving the plurality of medical skin images.

23. The method of claim 18, further comprising:

detecting the border region segment of a medical skin image of the plurality of medical skin images.

24. The method of claim 18, further comprising:

maintaining the user-assistance information in computer-readable storage media.

25. A computer program product comprising:

(a) a non-transitory computer-readable medium storing program instructions thereon which, when executed by a processor of a computing device, cause the computing device to perform a process including;

(i) extracting at least one respective human-perceivable feature included in each medical image of a plurality of medical images of the skin of an individual human (hereafter “

medical skin images”

), the each medical image of plurality of medical skin images includes a respective portion of a region of interest of a surface of the skin of the individual human, and was acquired by a handheld digital image acquisition device;

(ii) determining a substantial correspondence between

(x) an extracted human-perceivable feature included in a border region segment of a selected medical skin image of the plurality of medical skin images and

(y) an extracted at least one respective human-perceivable feature included in each medical skin image of the plurality of medical skin images other than the selected medical skin image;

(iii) gathering the determined substantial correspondences for the extracted human-perceivable feature included in the border region segment of the selected medical skin image;

(iv) generating data indicative of a border region-overlap status of the selected medical skin image, the data generated at least partially in response to the determined substantial correspondences;

(v) adding the data indicative of the determined border region-overlap status for the border region segment of the selected medical skin image to an omitted-coverage list;

(vi) iteratively designating a next medical skin image from a plurality of medical skin images as the selected medical skin image until each medical skin image of the plurality of medical skin images has been designated;

(vii) processing of each of the iteratively designated next medical skin images, the processing including operations (ii), (iii), (iv), and (v);

(viii) identifying a particular portion of the skin surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the skin”

), the identifying the possible non-imaged portion of the skin at least partially based on the omitted-coverage list; and

(ix) outputting user-assistance information at least partially based on the identified possible non-imaged portion of the skin.

26. The computer program product of claim 25, wherein the outputting user-assistance information includes:

outputting user-assistance information in substantially real time and at least partially based on the identified possible non-imaged portion of the skin.

27. The computer program product of claim 25, wherein the process further includes:

receiving the plurality of medical skin images.

28. The computer program product of claim 25, wherein the process further includes:

detecting the border region segment of a medical skin image of the plurality of medical skin images.

29. The computer program product of claim 25, wherein the process further includes:

displaying the user-assistance information.

30. The computer program product of claim 25, wherein the process further includes:

maintaining the user-assistance information in computer-readable storage media.

31. The computer program product of claim 25, wherein

the non-transitory computer-readable medium includes a tangible computer-readable medium.

32. The computer program product of claim 25, wherein the non-transitory computer-readable medium includes communications media.

33. A system comprising:

(a) means for extracting at least one respective human-perceivable feature included in each medical image of a plurality of medical images of the skin of an individual human (hereafter “

medical skin images”

), the each medical image of the plurality of medical skin images includes a respective portion of a region of interest of a surface of the skin of the individual human, and was acquired by a handheld digital image acquisition device;

(b) means for determining a substantial correspondence between

(x) an extracted human-perceivable feature included in a border region segment of a selected medical skin image of the plurality of medical skin images and

(y) an extracted at least one respective human-perceivable feature included in each medical skin image of the plurality of medical skin images other than the selected medical skin image;

(c) means for gathering the determined substantial correspondences for the extracted human-perceivable feature included in the border region segment of the selected medical skin image;

(d) means for generating data indicative of a border region-overlap status of the selected medical skin image, the data generated at least partially in response to the determined substantial correspondences;

(e) means for adding the data indicative of the determined border region-overlap status for the border region segment of the selected medical skin image to an omitted-coverage list;

(f) means for iteratively designating a next medical skin image from the plurality of medical skin images as the selected medical skin image until each medical skin image of the plurality of medical skin images has been designated;

(g) means for initiating a processing of each of the iteratively designated next medical skin images, the processing including operations at means (b), (c), (d), and (e);

(h) means for identifying a particular portion of the skin surface as likely not included in the plurality of medical skin images (hereafter “

possible non-imaged portion of the skin”

), the identifying the possible non-imaged portion of the skin at least partially based on the omitted-coverage list; and

(i) means for outputting user-assistance information at least partially based on the identified possible non-imaged portion of the skin.

34. The system of claim 33, wherein the means for outputting user-assistance information includes:

means for outputting user-assistance information in substantially real-time and at least partially based on the identified possible non-imaged portion of the skin.