Virtual focus feedback

• US 10,498,976 B2
• Filed: 12/05/2014
• Issued: 12/03/2019
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Corlett, Barry
• Primary Examiner(s): Rao, Anand S
• Assistant Examiner(s): Edwards, Tyler B
• Application Number:
  US14/562,292
• Publication Number:
  US 20160165151A1
• Time in Patent Office: 1,824 Days
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A system and method for focusing a camera is disclosed. The system may generate a proxy image that is blurred to an extent that is correlated to a degree to which a camera is out of focus. A user may be asked to adjust a focusing mechanism to attempt to bring the proxy image into focus. This allows the camera to be focused without the user needing to see an image from the camera. This can be used, for example, to focus an infrared camera. The infrared camera could be a tracking camera in a device such as a head mounted display device.

Claims

1. An apparatus comprising:

a display;

a camera; and

processing logic in communication with the display and the camera, the processing logic configured to;

receive image data associated with the camera;

determine a degree to which the camera is in focus based on the image data;

blur a visible base image having content that is independent of content in the image data from the camera to generate a proxy image that has a same degree of blurring across an entirety of the proxy image that inversely correlates with the degree to which the camera is in focus;

display the proxy image on the display; and

instruct a user to adjust a camera focus mechanism to better focus the proxy image,

wherein the processing logic receives updated image data associated with the camera after displaying the proxy image, determines a new degree to which the camera is in focus, modifies the degree of blurring of the proxy image to inversely correlate with new degree to which the camera is in focus, and displays an updated proxy image based on the modified degree of blurring.

2. The apparatus of claim 1, wherein the image data associated with the camera that the processing logic receives is the image from the camera, the processing logic determines a degree to which the camera is in focus based on a degree to which the image is in focus.

3. The apparatus of claim 2, wherein the image from the camera is an infrared (IR) image.

4. The apparatus of claim 1, wherein the image data associated with the camera that is received by the processing logic includes a distance that the camera is from an object;

the processing logic determines a degree to which the camera is in focus based on the distance.

5. The apparatus of claim 1, wherein the display is a near-eye, see-through display, the apparatus is a head mounted display (HMD), the camera is focused on a user that is wearing the HMD.

6. The apparatus of claim 1, further comprising a graphics processing unit (GPU), the processing logic blurs the visible base image to create the proxy image on the GPU.

7. The apparatus of claim 1, wherein the processing logic is further configured to:

continue to update the degree of blurring in the displayed proxy image responsive to changes in the degree to which the camera is in focus until the camera is focused to an adequate degree.

8. The apparatus of claim 2, wherein the processing logic is configured to blur the same visible base image regardless of the image from the camera.

9. The apparatus of claim 1, wherein the visible base image is blurred according to the same degree based on a function across each pixel in the visible base image.

10. A method comprising:

receiving image data associated with a camera;

determining a degree to which the camera is in focus based on the received image data;

accessing a visible base image from processor readable storage, wherein the visible base image has content that does not match content in an image from the camera;

blurring the visible base image to generate a proxy image that has a same degree of blurring across an entirety of the proxy image that inversely correlates with the degree to which the camera is in focus;

displaying the proxy image on a display;

instructing a user to adjust a focus mechanism that is configured to focus the camera to bring the proxy image into better focus;

repeating said receiving, determining, blurring, and displaying until the camera is focused to an adequate degree, including receiving updated image data associated with the camera, determining a new degree to which the camera is in focus, modifying the degree of blurring of the proxy image to inversely correlate with the new degree to which the camera is in focus, and displaying the modified proxy image on the display.

11. The method of claim 10, wherein the receiving the image data associated with a camera includes receiving the image from the camera;

and wherein the determining a degree to which the camera is in focus based on the image data includes determining a degree to which the received image is in focus.

12. The method of claim 11, wherein the image from the camera is an infrared (IR) image.

13. The method of claim 10, wherein the receiving the image data associated with a camera includes receiving data that indicates a distance that the camera is from an object;

and wherein the determining a degree to which the camera is in focus includes determining a degree to which the camera is in focus based on the distance.

14. The method of claim 10, wherein the blurring the visible base image that is to generate a proxy image that has a degree of blurring that correlates with the degree to which the camera is in focus comprises:

mathematically blurring the visible base image to create the proxy image, wherein the same visible base image is blurred regardless of the data received associated with the camera.

15. The method of claim 14, wherein the mathematically blurring the visible base image to create the proxy image comprises using graphics image processing on a graphics processing unit (GPU).

16. The method of claim 10, wherein the displaying the proxy image on a display comprises:

displaying the proxy image on a display of a head mounted display (HMD).

17. A head mounted display (HMD), comprising:

a near-eye, see-through display;

an infrared (IR) camera;

processor readable storage;

an IR camera focus mechanism configured to focus the IR camera; and

a processor in communication with the infrared camera and the near-eye, see-through display, the processor configured to;

receive an infrared image from the IR camera;

determine a degree to which the infrared image is out of focus;

access a visible base image from the processor readable storage, wherein the visible base image is an image other than an image from the IR camera and has content that does not match content in the image from the IR camera;

blur the visible base image to create a proxy image that has a same degree of blurring across an entirety of the proxy image that correlates with the degree to which the infrared image is out of focus;

display the blurred proxy image on the near-eye, see-through display; and

determine whether the IR camera is adequately focused responsive to a change in the IR camera focus mechanism after the blurred proxy image is displayed,

wherein the processor is further configured to receive updated image data associated with the IR camera after displaying the proxy image, determine a new degree to which the IR camera is in focus, modify the degree of blurring of the proxy image to inversely correlate with new degree to which the IR camera is in focus, and display an updated proxy image based on the modified degree of blurring.

18. The head mounted display of claim 17, wherein the processor is further configured to instruct a user to manually adjust the camera focus mechanism to better focus the proxy image.

19. The head mounted display of claim 18, wherein the IR camera is an eye tracking camera on the HMD.

20. The head mounted display of claim 17, further comprising a graphics processing unit (GPU), the processor mathematically blurs the visible base image to create the proxy image using the GPU.