Method for computer vision to recognize objects marked for identification with a bigram of glyphs, and devices utilizing the method for practical purposes

• US 9,443,158 B1
• Filed: 06/22/2015
• Issued: 09/13/2016
• Est. Priority Date: 06/22/2014
• Status: Expired due to Fees

• Current Assignee: Kristopher Haskins
• Original Assignee: Kristopher Haskins
• Inventors: Haskins, Kristopher
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/746,829
• Time in Patent Office: 449 Days
• Field of Search: 382181-231
• US Class Current: 1/1
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/262 using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information

Abstract

A method of identifying bigrams of glyphs within a digital image. Classifiers are configured through training to differentiate between glyph orientations, allowing for accurate pairing and positive identification of the bigram. Embodiments include assistive technology devices which read playing cards held in-hand by visually impaired users and convey the rank and suit of each card to the player by non-visual means.

Claims

1. A process of operating a computer, which comprises one or more processing units connected to one or more memories, to computationally identify those bigrams of glyphs within a digital image which belong to a known, finite set of glyph bigrams, by:

a. isolating and labeling each contiguous blob within said image,

b. computing a linkage vector between each of the labeled blobs,

c. for each of the blobs linked by each of said linkage vectors, calculating a rectangular bounding box with two edges parallel to said linkage vector,

d. for each of said linkage vectors, transforming raster data contained within said rectangular bounding box into a feature set suitable for input to a classifier which is configured to be sensitive to glyph orientation,

e. for each of said feature sets, assigning a classification according to said classifier'"'"'s output when provided said feature set as input,

f. for each of the labeled blobs, selecting one linkage vector, corresponding blob which forms a bigram, and classification,

whereby raster data contained in said image, representing luminous intensity projected upon an imaging sensor, is transformed into actionable information comprising the identities of each glyph bigram within said image.

2. The process of claim 1 wherein said classifier comprises one or more instances of one or more machine learning algorithms configured to recognize members of said finite set of glyph bigrams, and a means to unify output from each of said instance into a single classification.

3. The process of claim 2 wherein said classifier also provides a confidence score.

4. The process of claim 1 with additional means to selectively cull glyphs and/or linkage vectors based upon measured or calculated properties of individual glyphs, linkage vectors, or collections of glyphs and/or linkage vectors, whereby the computational workload is reduced, and the results are available to the user more quickly.

5. The process of claim 1 wherein the transformation of raster data includes rotation to a predetermined orientation such as a vertical alignment of the containing bounding box'"'"'s primary axis.

6. The process of claim 1 wherein a single bounding box is calculated for each linkage vector such that said bounding box fully encompasses both glyphs of said linkage vector while said bounding box'"'"'s primary axis is aligned parallel to said linkage vector, whereby subsequent operations identify the rank and suit after transformation of the raster information contained within said single bounding box rather than, or in addition to that from the individual bounding boxes created for each blob of the linkage.

7. The process of claim 1 wherein the transformation of raster data includes an affine transformation to compensate for perspective skew in the original image and its derivatives.

8. A device comprising one or more processors connected to one or more memories, one or more imaging sensors, and a means of non-visual output, programmed to:

a. acquire one or more images from the connected image sensor, b. identify the rank and suit of each playing card present by means provided in claims 1, 2, 3, 4, 5, 6 or 7, c. convey rank and suit of each card thus identified to the user via non-visual output means, whereby a visually impaired user can participate in playing card games utilizing decks of widely-available, standard playing cards.

9. The device of claim 8 wherein said means of non-visual output is an electro-mechanical audio output, whereby a user may wear comfortable headphones and participate even if they are not braille-literate.

10. The device of claim 8 wherein said means of non-visual output is an electronically refreshable braille display, whereby a braille literate user can discreetly read the rank and suit of each card at their leisure.

11. The device of claim 8 which also incorporates supplemental lighting, whereby the device can operate properly even in poorly lit environments.

12. The device of claim 8 wherein multiple cameras are connected and oriented such that:

a. one or more cameras have a clear view of the common area containing communal face-up cards,

b. each user'"'"'s cards, when held in-hand, are presented with a clear view to one or more cameras,

and the processor is additionally programmed to provide the identities of all communal cards to each user, whereby no sighted player is required to read aloud the communal cards for the non-sighted players.