LONGITUDINAL DERMOSCOPIC STUDY EMPLOYING SMARTPHONE-BASED IMAGE REGISTRATION

• US 20140313303A1
• Filed: 05/28/2014
• Published: 10/23/2014
• Est. Priority Date: 04/18/2013
• Status: Abandoned Application

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F., Rhoads, Geoffrey B.
• Application Number:
  US14/288,890
• Publication Number:
  US 20140313303A1
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/444 Evaluating skin marks, e.g. mole, nevi, tumour, scar
• CPC Class Codes: A61B 5/68 Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The evolution of a skin condition over time can be useful in its assessment. In an illustrative arrangement, a user captures skin images at different times, using a smartphone. The images are co-registered, color-corrected, and presented to the user (or a clinician) for review, e.g., in a temporal sequence, or as one image presented as a ghosted overlay atop another. Image registration can employ nevi, hair follicles, wrinkles, pores, and pigmented regions as keypoints. With some imaging spectra, keypoints from below the outermost layer of skin can be used. Hair may be removed for image registration, and restored for image review. Transformations in addition to rotation and affine transforms can be employed. Diagnostic correlations with reference image sequences can be made, employing machine learning in some instances. A great variety of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

capturing a first image of skin at a first time, using a camera of a smartphone;

capturing a second image of said skin at a second time later than the first time;

identifying plural features in one of said images;

locating said plural features in the other of said images;

transforming one or more of said images to co-register said plural features therebetween; and

after said transforming, presenting a time-based sequence of images, on a display of the smartphone, to a viewer.

2. The method of claim 1 that further includes capturing a third image of said skin at a third time later than the second time, and including said third image in said presented time-based sequence of images.

3. The method of claim 1 in which said plural features comprise one or more nevi.

4. The method of claim 1 in which said plural features comprise one or more hair follicles.

5. The method of claim 1 in which said plural features comprise one or more wrinkles.

6. The method of claim 1 in which said plural features comprise one or more pigmented regions.

7. The method of claim 1 in which said plural features include one or more features from below an outermost layer of skin.

8. The method of claim 1 that includes determining an area of a lesion on said skin, and avoiding said area in identifying said plural features.

9. The method of claim 8 that includes flooding said area with a uniform color, so that features are not identified therein.

10. The method of claim 1 that includes processing said captured images to remove depictions of hair therein before identifying and locating said plural features.

11. The method of claim 10 wherein said time-based sequence of images presented to the viewer includes depictions of hair.

12. The method of claim 1 in which said transforming further includes applying color correction to one or more of said images.

13. The method of claim 1 in which said presenting includes presenting different images in said sequence in response to a sliding-finger motion by the viewer on said display.

14. The method of claim 1 in which said transforming includes a transformation other than rotation and/or affine transformation, to account for curvature of a surface of the skin.

15. The method of claim 1 that includes presenting one of said presented images for a first duration, and presenting another of said presented images for a second duration different than the first duration, in accordance with spans of time between captures of different images.

16. The method of claim 1 that includes overlaying measurement information on said presented sequence of images.

17. The method of claim 1 in which said presenting includes overlaying one of said images on another of said images, as a ghosted augmentation.

18. The method of claim 17 that further includes displaying a user interface that enables the viewer to select a particular image for said overlaying.

19. The method of claim 1 that includes analyzing said time-based sequence of images by comparing associated data with information from reference time-based sequences, for diagnostic purposes.

20. The method of claim 1 in which at least one of said images includes color data other than red, green or blue.

21. A method comprising the acts:

capturing a first image of skin at a first time;

capturing a live image of said skin at a second time later than the first time;

identifying plural features in one of said images;

locating said plural features in the other of said images;

transforming one of said images to co-register said plural features between the first and live images; and

presenting said first and live images, after said transforming, on a display to a viewer.

22. The method of claim 21 in which said presenting includes presenting one of said images as a ghosted overlay on the other of said images.

23. The method of claim 21 in which said plural features comprise features selected from the list consisting of nevi, hair follicles, wrinkles, pores, pigmented regions, and skin textures.

24. The method of claim 21 in which said plural features include one or more features from below an outermost layer of skin.

25. The method of claim 21 in which said transforming employs a hardware processor in a smartphone.

26. The method of claim 21 that includes determining an area of a lesion on said skin, and avoiding said area in identifying said plural features.

27. The method of claim 21 that includes processing said captured images to remove depictions of hair therein before identifying and locating said plural features.

28. The method of claim 27 wherein said presented images include depictions of hair.

29. The method of claim 21 in which said transforming includes a transformation other than rotation and/or affine transformation, to account for curvature of a surface of the skin.

30. The method of claim 21 that further includes:

capturing another image of said skin at a time between the first and second times;

locating said plural features in said another image;

transforming said another image to co-register said another image with the first and live images; and

including said transformed another image in said presentation of images on the display to the viewer.

31. A non-transitory computer readable medium containing software instructions for configuring one or more processors of a smartphone to perform acts including:

capturing a first image of skin at a first time, using a camera of the smartphone;

capturing a second image of said skin at a second time later than the first time;

identifying plural features in one of said images;

locating said plural features in the other of said images;

transforming one or more of said images to co-register said plural features therebetween; and

after said transforming, presenting a time-based sequence of images, on a display of the smartphone, to a viewer.