×

Systems and methods for sign language recognition

  • US 10,580,213 B2
  • Filed: 09/12/2017
  • Issued: 03/03/2020
  • Est. Priority Date: 09/13/2016
  • Status: Active Grant
First Claim
Patent Images

1. A wearable system for sign language recognition, the wearable system comprising:

  • a head-mounted display configured to present virtual content to a user;

    an audio sensor configured to detect audio from an environment of the user;

    an outward-facing imaging system configured to image the environment of the user; and

    a hardware processor in communication with the head-mounted display and the imaging system, and programmed to;

    receive a plurality of images captured by the outward-facing imaging system;

    detect at least one set of hands in the plurality of images with an object recognizer;

    determine a relative size of the at least one set of hands;

    identify a source of the at least one set of hands among a plurality of persons based on the relative size of the at least one set of hands;

    detect at least one gesture by the at least one set of hands in the plurality of images with the object recognizer;

    determine that the source of the at least one gesture belongs to a person other than the user;

    recognize a meaning of the at least one gesture in a sign language in response to the determination;

    identify a target language based on contextual information associated with the user, wherein the contextual information comprises;

    speech of the user obtained from analysis of audio data obtained by the audio sensor;

    translate the at least one gesture into the target language based on the recognized meaning and further based on determining that the source of the at least one gesture belongs to the person other than the user;

    generate virtual content based at least partly on a translation of the gesture into the target language; and

    cause the head-mounted display to render the virtual content to the user.

View all claims
  • 3 Assignments
Timeline View
Assignment View
    ×
    ×