Object tracking for artificial vision

• US 9,697,746 B2
• Filed: 09/30/2010
• Issued: 07/04/2017
• Est. Priority Date: 09/30/2009
• Status: Active Grant

• Current Assignee: National ICT Australia Limited
• Original Assignee: National ICT Australia Limited
• Inventors: Shen, Chunhua, Barnes, Nick
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Jiang, Zaihan
• Application Number:
  US13/498,667
• Publication Number:
  US 20120212594A1
• Time in Patent Office: 2,469 Days
• Field of Search: 348 62, 348E07085
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61N 1/36046 of the eye
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098

Abstract

This invention concerns the tracking of objects in video data for artificial vision; for instance for a bionic eye. More particularly, the invention concerns a vision enhancement apparatus for a vision-impaired user. In other aspects, the invention concerns a method for enhancing vision and software to perform the method. The image processor operates to process video data representing images of a scene. Automatically detect and track a user selected object, such as a face, in the images. And, automatically modify the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the scene. The separate video tile remains in the selected area despite movement of the camera relative to the scene, or movement of the user relative to the object or the scene.

Claims

1. A vision enhancement apparatus for a vision-impaired user, comprising:

a high resolution video camera on the vision-impaired user for filming a scene in front of the vision-impaired user;

a video display viewable by the vision-impaired user; and

an image processor for processing high resolution video data from the video camera and displaying the resulting images on the video display, the processor automatically detecting one or more objects and human heads in the displayed images of the scene, the processor processing the images to identify human heads and causing an alert to the user of newly identified human heads the processor receiving a signal indicative of the vision-impaired user selecting an object or a head from the one or more objects or human heads in the displayed images, the processor automatically detecting and tracking the user selected human head, and automatically modifying the video data, by displaying the selected head as a separate video tile within the scene, the separate video tile located in a part of the displayed images that is more visible to the vision-impaired user.

2. A vision enhancement apparatus according to claim 1, wherein the object or human head is identified within a first box and the separate video tile is a second box, the same size and shape as the first box.

3. A vision enhancement apparatus according to claim 2, wherein the object or human head in the first box is displayed in the separate video tile.

4. A vision enhancement apparatus according to claim 3, wherein the user selects the location for the separate video tile such that it is located in a part of the displayed images where it is visible to the vision-impaired user.

5. A vision enhancement apparatus according to claim 4, wherein in the event the user is suffering from macular degeneration the separate video tile is located in their peripheral vision.

6. A vision enhancement apparatus according to claim 3, wherein the separate video tile will remain in that part of the user'"'"'s field of vision despite movement of the user, the camera or the tracked object.

7. A vision enhancement apparatus according to claim 1, wherein while a human head remains in the scene, it is continually tracked so the image of it is continually updated in the separate video tile.

8. A vision enhancement apparatus according to claim 3, wherein the user is able to select more than one object or human head to be displayed in respective separate video tiles of the displayed images.

9. A vision enhancement apparatus according to claim 8, wherein the user selects which objects or human heads are displayed and which video tile each is displayed in within the displayed images.

10. A vision enhancement apparatus according to claim 2, wherein the user is able to switch between the detected objects or human heads, to select the one which is displayed in the video tile.

11. A vision enhancement apparatus according to claim 3, wherein the object or a human head is enlarged, or otherwise manipulated, to increase the user'"'"'s perception of it.

12. A vision enhancement apparatus according to claim 3, wherein the modified video data is provided to a visual display, being vision spectacles, a retina, cortical or optic nerve implant.

13. A vision enhancement apparatus according to claim 1, wherein the user selected object is the user'"'"'s hands.

14. A vision enhancement apparatus according to claim 13, wherein the user selected object is in the users hands and the image processor operates to segment the object, to enable the user to interactively recognize it.

15. A vision enhancement apparatus according to claim 1, wherein the user selected object includes text or an object with text on it.

16. A vision enhancement apparatus according to claim 15, wherein the user is able to selectively switch between a still image of the text and a video image of the scene including the object or human head.

17. A vision enhancement apparatus according to claim 1, wherein the apparatus automatically obtains and presents information to the user about the selected object.

18. A vision enhancement apparatus according to claim 1, wherein the entire apparatus is distributed about the body of the user.

19. A method for enhancing vision for a vision impaired user, comprising the steps of:

processing high resolution video data representing images of a scene in front of the vision-impaired user;

displaying the resulting images to the user;

automatically detecting one or more objects or human heads in the displayed images;

identifying the objects or human heads;

alerting the user of newly identified objects or human heads;

the user selecting an object or human head in the displayed images;

automatically detecting and tracking the user selected human head; and

automatically modifying the video data, by displaying the tracked human head as a separate video tile within the scene, the separate video tile within the scene located in a part of the displayed images that is more visible to the vision-impaired user.

20. A non-transitory computer readable medium that provides instructions, which when executed by a processor, cause the processor to perform the processing steps of claim 19.