Recording dose data from drug injection devices using optical character recognition (OCR)

• US 10,169,536 B2
• Filed: 10/15/2015
• Issued: 01/01/2019
• Est. Priority Date: 10/21/2014
• Status: Active Grant

• Current Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Original Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Inventors: Klemm, Thomas, Hammen, Dietmar
• Primary Examiner(s): Frech, Karl D
• Application Number:
  US15/520,057
• Publication Number:
  US 20170316158A1
• Time in Patent Office: 1,174 Days
• Field of Search: 23546204, 23546213
• CPC Class Codes: A61M 2005/3126 Specific display means related to dosing
• CPC Class Codes: A61M 2205/3306 Optical measuring means
• CPC Class Codes: A61M 2205/505 Touch-screens; Virtual keyboard or keypads; Virtual buttons; Soft keys; Mouse touches
• CPC Class Codes: A61M 2205/52 with memories providing a history of measured variating parameters of apparatus or patient
• CPC Class Codes: A61M 2205/6081 Colour codes
• CPC Class Codes: A61M 5/24 Ampoule syringes, i.e. syringes with needle for use in combination with replaceable ampoules or carpules, e.g. automatic ampoules or carpules A61J1/06
• CPC Class Codes: A61M 5/315 Pistons; Piston-rods; Guiding, blocking or restricting the movement of the rod or piston; Appliances on the rod for facilitating dosing ; Dosing mechanisms
• CPC Class Codes: A61M 5/31525 Dosing burettes, pipettes B01L3/02
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1478 of characters or characters lines
• CPC Class Codes: G16H 20/17 delivered via infusion or injection
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A method of recording a medicament dose using a data collection device comprises capturing an image of a medicament dose indicator of a medicament delivery device, adjusting a scale of said image, adjusting said image for skew of one or more characters displayed by the medicament dose indicator, determining the position of at least one of said one or more characters in the image, identifying the at least one character using optical character recognition and determining a medicament dose indicator by the medicament dose indicator based on a result of the optical character recognition. The method may be performed using a handheld electronic device comprising a camera, such as a cellphone, a tablet computer or other device. A computer program for controlling a data collection device to perform the method may be provided in the form of a software application or “app”.

Claims

1. A method of recording a medicament dose using a handheld data collection device, the method comprising:

capturing, by a camera of the handheld data collection device positioned at a distance from a medicament dose indicator of a medicament delivery device, an image of the medicament dose indicator, wherein the distance between the camera and the medicament dose indicator is variable;

adjusting a scale of said image;

adjusting said image for skew of one or more characters displayed by the medicament dose indicator;

determining a position of at least one of said one or more characters in the image;

identifying the at least one character using optical character recognition; and

determining a medicament dose indicated by the medicament dose indicator based on a result of said optical character recognition.

2. A method according to claim 1, wherein said handheld electronic device is a cellphone or a tablet computer.

3. A method according to claim 1, wherein capturing an image comprises combining multiple images captured by said camera having different exposure levels, to provide said image with a high dynamic range.

4. A method according to claim 1, comprising obtaining color information from said image and identifying a type of medicament to be dispensed based on said color information.

5. A method according to claim 4, wherein obtaining color information comprises obtaining a color balance measurement based on reference color information provided on the medicament delivery device.

6. A method according to claim 1, wherein said medicament delivery device is an injector pen comprising a movable component for selecting said medicament dose to be dispensed.

7. A computer program comprising computer-readable instructions that, when executed by a processor, causes a method according to claim 1 to be performed.

8. An application for one or both of a smartphone and a tablet computer, wherein said application comprises a computer program according to claim 7.

9. A handheld data collection device comprising:

a camera; and

a processing arrangement configured to;

capture an image of a medicament dose indicator of a medicament delivery device using said camera when the camera is positioned at a distance from the medicament dose indicator, wherein the distance between the camera and the dose indicator is variable;

adjust a scale of said image;

adjust said image for skew of one or more characters displayed by the medicament dose indicator;

determine a position of at least one of said one or more characters in the image;

identify the at least one character using optical character recognition; and

determine a medicament dose indicated by said medicament dose indicator based on a result of said optical character recognition.

10. A data collection device according to claim 9, wherein said handheld electronic device is a cellphone or a tablet computer.

11. A data collection device according to claim 9, wherein said processing arrangement is configured to combine multiple images captured by said camera, said multiple images having different exposure levels, to provide said image with a high dynamic range.

12. A data collection device according to claim 9, wherein said processing arrangement is configured to identify a color of at least one component of the medicament delivery device and to determine a type of said medicament dose based on said color.

13. A method of recording a medicament dose using a handheld data collection device, the method comprising:

capturing, by a camera of the handheld data collection device, an image of a medicament dose indicator of a medicament delivery device;

adjusting a scale of said image;

adjusting said image for skew of one or more characters displayed by the medicament dose indicator by removing slanting of the one or more characters;

determining a position of at least one of said one or more characters in the image;

after adjusting said image for skew by removing slanting of the one or more characters, identifying the one or more characters using optical character recognition; and

determining a medicament dose indicated by the medicament dose indicator based on a result of said optical character recognition.

14. A method according to claim 13, wherein said camera of said handheld data collection device is positioned at a distance from the medicament dose indicator of the medicament delivery device, wherein the distance between said camera and said medicament dose indicator is not fixed.

15. A method according to claim 13, wherein said handheld electronic device is a cellphone or a tablet computer.

16. A method according to claim 13, wherein capturing an image comprises combining multiple images captured by said camera having different exposure levels, to provide said image with a high dynamic range.

17. A method according to claim 13, comprising obtaining color information from said image and identifying a type of medicament to be dispensed based on said color information.