Supplemental device for attachment to an injection device for recording and displaying a dose value set by the user using optical character recognition (OCR)

• US 10,043,093 B2
• Filed: 01/22/2015
• Issued: 08/07/2018
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Original Assignee: Sanofi-Aventis Deutschland GmbH (Sanofi )
• Inventors: Allerdings, Alexander, Hoffmann, Christian, Fischer, Rainer
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US15/113,213
• Publication Number:
  US 20170032211A1
• Time in Patent Office: 1,293 Days
• CPC Class Codes: A61M 2005/3126 Specific display means related to dosing
• CPC Class Codes: A61M 2205/3306 Optical measuring means
• CPC Class Codes: A61M 2205/52 with memories providing a history of measured variating parameters of apparatus or patient
• CPC Class Codes: A61M 2205/8206 battery-operated
• CPC Class Codes: A61M 5/24 Ampoule syringes, i.e. syringes with needle for use in combination with replaceable ampoules or carpules, e.g. automatic ampoules or carpules A61J1/06
• CPC Class Codes: A61M 5/31525 Dosing burettes, pipettes B01L3/02
• CPC Class Codes: A61M 5/3155 by rotational movement of dose setting member, e.g. during setting or filling of a syringe
• CPC Class Codes: G06V 2201/02 Recognising information on displays, dials, clocks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/18095 Summing image-intensity values; Projection and histogram analysis

Abstract

A supplemental device (2) for attachment to a pen-type drug injection device, the supplemental device comprising an optical sensor array or camera (25) configured to capture an image of one or more numbers representing dose values on a dose dial sleeve of the injection device, and a processor (24) and memory (240,241) arrangement configured to run an algorithm to receive image data from the sensor/camera and extract the dialed dose value from said image via optical character recognition (OCR). The algorithm is able to determine the set dose value which is at the centre of the image (902), also if the dose scale only comprises integer values in steps of two (see FIG. 9).

Claims

1. A supplemental device for attachment to an injection device, the supplemental device comprising:

an imaging sensor configured to capture an image of one or more visible numbers displayed on a moveable component of the injection device;

a processor; and

a computer-readable memory storing instructions executable by the processor to;

receive data representing an image;

determine a position of at least one visible number in the image by determining a boundaries of each visible number;

select one of the visible numbers to be identified;

perform a pattern matching process to identify the value of the selected visible number; and

determine a dose dialed into the injection device using the determined position of the selected one of the at least one visible number in the image by;

determining a position offset of the selected visible number from the center of the image; and

applying a weighting function based on the position offset to the value of the selected visible number.

2. The supplemental device according to claim 1, wherein the position offset is expressed as a number of pixel rows.

3. The supplemental device according to claim 1, wherein one or more of the visible numbers in the image is only partially visible.

4. The supplemental device according to claim 1, wherein the computer-readable memory further stores instructions executable by the processor to perform a binarization process on the received image data.

5. The supplemental device according to claim 1, wherein determining the boundaries of each visible number comprises recording a sum of black pixels in each pixel row of the image and identifying one or more rows in which the sum is below a threshold value.

6. The supplemental device according to claim 1, wherein determining the boundaries of each visible number further comprises recording the sum of black pixels in each pixel column of the image and identifying one or more columns in which the sum is below a threshold value.

7. The supplemental device according to claim 6, wherein the threshold value is 1.

8. The supplemental device according to claim 1, wherein performing the pattern matching process comprises:

determining that the visible selected number comprises two or more digits;

in response to determining that the selected visible number comprises two or more digits, separating the digits; and

comparing each digit to stored digit templates to determine a best match.

9. The supplemental device according to claim 8, wherein the computer-readable memory further stores instructions executable by the processor to determine whether the selected visible number comprises two or more digits by determining whether the selected visible number is wider than a predetermined maximum digit width value.

10. The supplemental device according to claim 1, wherein selecting one of the visible numbers to be identified comprises selecting the visible number having a greatest height.

11. The supplemental device according to claim 10, wherein two visible numbers are of equal height, and wherein selecting one of the visible numbers to be identified comprises selecting an uppermost of the two visible numbers of equal height.

12. The supplemental device according to claim 1, wherein the processor is pre-programmed with values for an expected height and width of the visible numbers in the image.

13. The supplemental device according to claim 12, wherein the computer-readable memory further stores instructions executable by the processor to use an expected width for a number to predict the point or points at which a separation of the two or more digits should occur.

14. The supplemental device according to claim 1, wherein the computer-readable memory further stores instructions executable by the processor to cause a value corresponding to a result of the application of a weighting function to the value of the selected visible number to be displayed on a display of the device.

15. The supplemental device according to claim 1, wherein the computer-readable memory stores instructions executable by the processor to asses and improve image data quality.

16. The supplemental device according to claim 15, wherein assessing and improving image data quality comprises defective and bad pixel correction, light correction and distortion and slant correction.

17. The supplemental device according to any claim 1, wherein the computer-readable memory stores instructions executable by the processor to:

determine that a confidence level in the result of the pattern matching process is below a threshold level; and

in response to determining that the confidence level is below the threshold level, perform a second pattern matching process on another of the one or more visible numbers in the image.

18. The supplemental device according to claim 1, wherein the computer-readable memory stores instructions executable by the processor to display a determined dose value which does not correspond to any of the one or more visible numbers present on the moveable component of the injection device.

19. A system comprising:

a supplemental device for attachment to an injection device, the supplemental device comprising;

an imaging sensor configured to capture an image of one or more numbers displayed on a moveable component of the injection device;

a processor;

a computer-readable memory storing instructions executable by the processor to;

receive data representing an image;

determine the position of at least one number in the image by determining the boundaries of each visible number;

select one of the visible numbers to be identified;

perform a pattern matching process to identify the value of the selected number; and

determine a dose dialed into the injection device using the determined position of the selected one of the at least one visible number in the image by;

determining a position offset of the selected visible number from the center of the image; and

applying a weighting function based on the position offset to the value of the selected visible number; and

an injection device having a moveable component with numbers present thereon.

20. The system according to claim 19, wherein only even numbers are present on the moveable component.

21. A method of determining a dose dialed into an injection device using a supplemental device for attachment to an injection device, the method comprising:

receiving a captured image of one or more visible numbers present on a moveable component of the injection device;

receiving data representing an image;

determining a position of at least one visible number in the image by

determining a boundaries of each visible number;

selecting one of the visible numbers to be identified;

performing a pattern matching process to identify a value of the selected visible number; and

determining a dose dialled into the injection device using the determined position of the selected one of the at least one visible number in the image by;

determining a position offset of the selected visible number from the center of the image; and

applying a weighting function based on the position offset to the value of the selected visible number.