Object tracking for artificial vision

• US 10,062,303 B2
• Filed: 05/31/2017
• Issued: 08/28/2018
• Est. Priority Date: 09/30/2009
• Status: Active Grant

• Current Assignee: National ICT Australia Limited
• Original Assignee: National ICT Australia Limited
• Inventors: Barnes, Nick, Shen, Chunhua
• Primary Examiner(s): Vo, Tung
• Assistant Examiner(s): Jiang, Zaihan
• Application Number:
  US15/609,953
• Publication Number:
  US 20170263153A1
• Time in Patent Office: 454 Days
• Field of Search: 348143, 348 62, 348E07085
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61N 1/36046 of the eye
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098

Abstract

This invention concerns the tracking of objects in video data for artificial vision; for instance for a bionic eye. More particularly, the invention concerns a vision enhancement apparatus for a vision-impaired user. In other aspects, the invention concerns a method for enhancing vision and software to perform the method. The image processor operates to process video data representing images of a scene. Automatically detect and track a user selected object, such as a face, in the images. And, automatically modify the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the scene. The separate video tile remains in the selected area despite movement of the camera relative to the scene, or movement of the user relative to the object or the scene.

Claims

1. A vision enhancement apparatus for a vision-impaired user, comprising an image processor arranged to:

process high resolution video data representing images of a scene and display the resulting images to the user;

automatically detect and track a user selected object in the images; and

automatically modify the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the images of the scene to provide modified video data that includes;

the displayed images of the scene that includes the tracked object; and

the separate video tile, at the user selected area within the displayed images, to display the tracked object,

wherein the tracked object remains displayed at the separate video tile despite movement of the user or the tracked object.

2. A vision enhancement apparatus according to claim 1, wherein the object is identified within a first box and the separate video tile is a second box.

3. A vision enhancement apparatus according to claim 2, wherein the object in the first box is displayed in the separate video tile.

4. The vision enhancement apparatus according to claim 2, wherein the second box is the same size and shape as the first box.

5. A vision enhancement apparatus according to claim 1, wherein the user selects the location for the separate video tile such that it is located in a part of the displayed images where it is visible to the vision-impaired user.

6. A vision enhancement apparatus according to claim 5, wherein in the event the user is suffering from macular degeneration the separate video tile is located in their peripheral vision.

7. A vision enhancement apparatus according to claim 1, wherein the separate video tile will remain in that part of the user'"'"'s field of vision despite movement of the user, the tracked object or a camera mounted on the head of the user for filming the scene in front of the user.

8. A vision enhancement apparatus according to claim 1, wherein the user has a high resolution video camera mounted on their head so that it films the scene in front of them.

9. A vision enhancement apparatus according to claim 1, wherein the processor automatically detects faces in the scene and processes the images to identify the faces.

10. A vision enhancement apparatus according to claim 9, wherein the apparatus announces newly identified faces to the user and then tracks their movements within the scene.

11. A vision enhancement apparatus according to claim 9, wherein while a face remains in the scene it is continually tracked so the image of it is continually updated to show its changing facial expressions in the separate video tile.

12. A vision enhancement apparatus according to claim 3, wherein the user is able to select more than one object to be displayed in respective separate video tiles of the displayed images.

13. A vision enhancement apparatus according to claim 12, wherein the user selects which objects are displayed and which video tile each is displayed in within the displayed images.

14. A vision enhancement apparatus according to claim 1, wherein the user is able to switch between the detected objects, to select the one which is displayed in the video tile.

15. A vision enhancement apparatus according to claim 1, wherein the object is enlarged, or otherwise manipulated, to increase the user'"'"'s perception of it.

16. A vision enhancement apparatus according to claim 1, wherein the modified video data is, provided to a visual display, being vision spectacles, a retina, cortical or optic nerve implant.

17. A vision enhancement apparatus according to claim 1, wherein the user selected object is the users hands.

18. A vision enhancement apparatus according to claim 17, wherein the user selected object is in the users hands and the image processor operates to segment the object, to enable to user to interactively recognise it.

19. A vision enhancement apparatus according to claim 1, wherein the user selected object includes text or an object with text on it.

20. A vision enhancement apparatus according to claim 19, wherein the user is able to selectively switch between a still image of the text and a video image of the scene including the object.

21. A vision enhancement apparatus according to claim 1, wherein the apparatus automatically obtains and present information to the user about the selected object.

22. A vision enhancement apparatus according to claim 1, wherein the entire apparatus is distributed about a body of the user.

23. A method for enhancing vision, comprising the steps of:

processing high resolution video data representing images of a scene and displaying the resulting images to the user;

automatically detecting and tracking a user selected object in the images; and

automatically modifying the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the images of the scene to provide modified video data that includes;

the displayed images of the scene that includes the tracked object; and

the separate video tile, at the user selected area within the displayed images, to display the tracked object,

wherein the tracked object remains displayed at the separate video tile despite movement of the user or the tracked object.

24. A non-transitory computer readable medium that provides instructions, which when executed by a processor, cause the processor to perform the processing steps of claim 23.