Recording dose data from drug injection devices using optical character recognition (OCR)

• US 10,146,910 B2
• Filed: 10/15/2015
• Issued: 12/04/2018
• Est. Priority Date: 10/21/2014
• Status: Active Grant

• Current Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Original Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Inventors: Riedel, Stephan, Gerken, Till
• Primary Examiner(s): LE, THIEN MINH
• Application Number:
  US15/520,053
• Publication Number:
  US 20170316157A1
• Time in Patent Office: 1,146 Days
• Field of Search: 235454
• CPC Class Codes: A61M 2005/3126 Specific display means related to dosing
• CPC Class Codes: A61M 2205/3306 Optical measuring means
• CPC Class Codes: A61M 2205/505 Touch-screens; Virtual keyboard or keypads; Virtual buttons; Soft keys; Mouse touches
• CPC Class Codes: A61M 2205/52 with memories providing a history of measured variating parameters of apparatus or patient
• CPC Class Codes: A61M 2205/6081 Colour codes
• CPC Class Codes: A61M 5/24 Ampoule syringes, i.e. syringes with needle for use in combination with replaceable ampoules or carpules, e.g. automatic ampoules or carpules A61J1/06
• CPC Class Codes: A61M 5/31 Details
• CPC Class Codes: A61M 5/315 Pistons; Piston-rods; Guiding, blocking or restricting the movement of the rod or piston; Appliances on the rod for facilitating dosing ; Dosing mechanisms
• CPC Class Codes: A61M 5/31525 Dosing burettes, pipettes B01L3/02
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 2201/02 Recognising information on displays, dials, clocks
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 20/13 delivered from dispensers
• CPC Class Codes: G16H 20/17 : delivered via infusion or injection
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/70 : Circuitry for compensating brightness variation in the scene

Abstract

A method of recording a medicament dose using a data collection device comprises capturing, by a video camera of said data collection device, a video showing a medicament dose indicator of a medicament delivery device, adjusting a scale of an image of said medicament dose indicator in said video, adjusting said image for skew of one or more characters displayed on a component of the medicament delivery device in said video, determining the position of at least one of said one or more characters in the image, identifying the at least one character using optical character recognition and determining a medicament dose shown by the medicament dose indicator based on a result of said optical character recognition. The method may include determining whether more than one delivery of medicament is recorded in the video and, if so, whether said more than one delivery includes one or more prime shots, so that an overall dosage delivered to a user may be determined based on multiple determined medicament doses. A wearable electronic device comprising a video camera may be used to obtain and analyze the video, for example, using software provided in an “app”. The wearable electronic device may be configured to be worn on the head of a user, to capture the video from the user'"'"'s point of view.

Claims

1. A method of recording a medicament dose using a data collection device, comprising:

capturing, by a video camera of the data collection device, a video showing a medicament dose indicator of an injector pen;

adjusting a scale of an image in said video;

adjusting, using a processing arrangement, said image for skew of one or more characters displayed on a component of the injector pen in said video;

determining a position of at least one of said one or more characters in the image;

identifying the at least one character using optical character recognition software; and

determining a medicament dose indicated by the medicament dose indicator based on a result of said optical character recognition software, wherein the injector pen comprises a movable component for selecting said medicament dose to be dispensed.

2. A method according to claim 1, wherein said data collection device is a wearable electronic device.

3. A method according to claim 1, comprising determining whether more than one delivery of medicament is recorded in said video.

4. A method according to claim 3, comprising, in response to a determination that more than one delivery of medicament is recorded in said video, determining at least a second medicament dose based on at least one character identified using the optical character recognition software on at least a second image of said medicament dose indicator in said video and determining an overall medicament dose received by a user based on said determined medicament dose and said determined second medicament dose.

5. A method according to claim 3, comprising, in response to a determination that more than one delivery of medicament is recorded in said video, determining whether said more than one delivery of medicament includes one or more prime shots.

6. A method according to claim 1, comprising obtaining color information from said image and identifying a type of medicament to be dispensed based on said color information.

7. A computer program comprising computer-readable instructions that, when executed by a processor, causes a method according to claim 1 to be performed.

8. An application for a wearable electronic device, wherein said application comprises a computer program according to claim 7.

9. A data collection apparatus for recording a medicament dose, the data collection apparatus comprising:

a video camera; and

a processing arrangement configured to;

capture a video using said video camera, said video including at least one image of a medicament dosage indicator of an injector pen;

adjust a scale of said image;

adjust said image for skew of one or more characters displayed on a component of the injector pen;

determine a position of at least one of said one or more characters in the image;

identify the at least one character using optical character recognition software; and

determine a medicament dose indicated by said medicament dose indicator based on a result of said optical character recognition software, wherein the injector pen comprises a movable component for selecting said medicament dose to be dispensed.

10. A data collection apparatus according to claim 9, comprising a wearable electronic device including said video camera.

11. A data collection apparatus according to claim 10, wherein said wearable electronic device is configured to be worn on the head of a user.

12. A data collection apparatus according to claim 9, wherein said processing arrangement is configured to determine whether more than one delivery of medicament is recorded in said video.

13. A data collection apparatus according to claim 12, wherein the processing arrangement is configured to respond to a determination that more than one delivery of medicament is recorded in said video by determining at least a second medicament dose based on at least one character identified using the optical character recognition software on at least a second image of said medicament dose indicator in said video and determining an overall dosage received by a user based on said determined medicament dose and said determined second medicament dose.

14. A data collection apparatus according to claim 12, wherein said processing arrangement is configured to respond to a determination that more than one delivery of medicament is recorded in said video by determining whether said more than one delivery of medicament includes one or more prime shots.

15. A medicament delivery system comprising:

a medicament delivery device; and

a data collection apparatus according to claim 9.