Smart head-mounted projection system

• US 10,061,137 B2
• Filed: 11/06/2017
• Issued: 08/28/2018
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Gerard Dirk Smits
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Abraham, Samantha K
• Application Number:
  US15/804,392
• Publication Number:
  US 20180059428A1
• Time in Patent Office: 295 Days
• CPC Class Codes: G02B 30/35 using reflective optical elements in the optical path between the images and the observer
• CPC Class Codes: G02B 5/124 plural reflecting elements forming part of a unitary plate or sheet

Abstract

Embodiments are directed towards a system for enabling a user to view an image on a surface. The system may include projector(s), sensor, projection surface or screen, and processor. The projectors may project light for an image onto the surface. The sensor may detect light reflected off the surface. The surface may include multiple types of surface elements, such as multiple first elements positioned as border of a display area on the surface to provide feedback regarding the surface and multiple second elements positioned within the border of the display area to reflect the image to the user. The processor may determine characteristics of the border of the display area based on light reflected to the sensor from first elements. And it may modify parameters of the image based on the characteristics of the border of the display area.

Claims

1. A system for enabling a user to view an image on a surface, comprising:

one or more projectors that project light for the image onto the surface;

one or more sensors that detect light reflected off the surface;

a plurality of first elements that are located on the surface and border a display area on the surface, wherein each of the first elements reflect light projected from the one or more projectors to the sensor and one or more portions of this reflected light being non-visible to the user, and wherein a portion of the plurality of first elements are retro-reflective elements that are fiducial markings that are located on a border of the display area on the surface; and

a plurality of second elements that are positioned within the border of the display area, wherein the plurality of second elements reflect light for the image from the one or more projectors to the user for viewing the image; and

one or more processors that performs actions, including;

modifying one or more parameters of the image based on one or more characteristics of the border of the display area.

2. The system of claim 1, further comprising one or more scanners for three-dimensionally scanning one or more physical objects into a memory, wherein the one or more scanned objects are automatically separated from a background and provided for compositing with the image on the surface in real time or at another time.

3. The system of claim 1, further comprising:

determining the one or more characteristics of the border of the display area based on light reflected to the one or more sensors from the plurality of first elements.

4. The system of claim 1, wherein the processor performs further actions, comprising:

determining a distance from the one or more projectors to the surface based on a triangulation of light from at least a first projector that is reflected off one or more of the second elements on the surface onto at least a first sensor.

5. The system of claim 1, wherein modifying at least one parameter of the image, further comprises:

tracking the border of the display area by continuously determining the one or more characteristics of the border; and

dynamically modifying one or more of a size, a shape, or an orientation of the image to stay within the tracked border.

6. The system of claim 1, wherein the surface includes a plurality of third elements that reflect light from the one or more projectors onto the one or more sensors and that represent an embedded code of surface information, including one or more of a type or a model of a screen that provides the display area.

7. The system of claim 1, wherein one or more of the plurality of the first and the second elements include one or more include full retro reflector (FRR) elements, modified retro reflector (MRR) elements, or wide modified retro reflector (WMRR) elements.

8. The system of claim 1, further comprising:

a second surface that includes another plurality of elements, wherein the second surface provides a working surface that is smaller than the surface and positioned physically closer to the user, wherein the working surface is employed by the user to select one or more virtual drawing implements, including one or more pens, brushes, or magic wands, and wherein the selected one or more virtual drawing implements are employed to edit one or more images projected on the surface.

9. A method for enabling a user to view an image on a surface, comprising:

employing one or more projectors to project light for the image onto the surface;

employing one or more sensors to detect light reflected off the surface;

a plurality of first elements that are located on the surface and border a display area on the surface, wherein each of the first elements reflect light projected from the one or more projectors to the sensor and one or more portions of this reflected light being non-visible to the user, and wherein a portion of the plurality of first elements are retro-reflective elements that are fiducial markings located on a border of the display area on the surface; and

providing a plurality of second elements that are positioned within the border of the display area, wherein the plurality of second elements reflect light for the image from the one or more projectors to the user for viewing the image; and

employing one or more processors to modify one or more parameters of the image based on one or more characteristics of the border of the display area.

10. The method of claim 9, further comprising one or more scanners for three-dimensionally scanning one or more physical objects into a memory, wherein the one or more scanned objects are automatically separated from a background and provided for compositing with the image on the surface in real time or at another time.

11. The method of claim 9, further comprising:

determining the one or more characteristics of the border of the display area based on light reflected to the one or more sensors from the plurality of first elements.

12. The method of claim 9, wherein the processor performs further actions, comprising:

determining a distance from the one or more projectors to the surface based on a triangulation of light from at least a first projector that is reflected off one or more of the second elements on the surface onto at least a first sensor.

13. The method of claim 9, wherein modifying at least one parameter of the image, further comprises:

tracking the border of the display area by continuously determining the one or more characteristics of the border; and

dynamically modifying one or more of a size, a shape, or an orientation of the image to stay within the tracked border.

14. The method of claim 9, wherein the surface includes a plurality of third elements that reflect light from the one or more projectors onto the one or more sensors and that represent an embedded code of surface information, including one or more of a type or a model of a screen that provides the display area.

15. The method of claim 9, wherein one or more of the plurality of the first and the second elements include one or more include full retro reflector (FRR) elements, modified retro reflector (MRR) elements, or wide modified retro reflector (WMRR) elements.

16. The method of claim 9, further comprising:

a second surface that includes another plurality of elements, wherein the second surface provides a working surface that is smaller than the surface and positioned physically closer to the user, wherein the working surface is employed by the user to select one or more virtual drawing implements, including one or more pens, brushes, or magic wands, and wherein the selected one or more virtual drawing implements are employed to edit one or more images projected on the surface.

17. A computer for enabling a user to view an image on a surface, comprising:

a memory that stores instructions;

a processor that executes the instructions to perform actions, including;

employing one or more projectors to project light for the image onto the surface;

employing one or more sensors to detect light reflected off the surface;

modifying one or more parameters of the image based on one or more characteristics of a border of a display area on the surface;

wherein a plurality of first elements are located on the surface and adjacent to the border of the display area on the surface, wherein each of the first elements reflect light projected from the one or more projectors to the sensor and one or more portions of this reflected light being non-visible to the user, and wherein a portion of the plurality of first elements are retro-reflective elements that are fiducial markings located on a border of the display area on the surface; and

wherein a plurality of second elements are positioned within the border of the display area, wherein the plurality of second elements reflect light for the image from the one or more projectors to the user for viewing the image.

18. The computer of claim 17, wherein the processor performs further actions, comprising:

employing one or more scanners to three-dimensionally scan one or more physical objects into a memory, wherein the one or more scanned objects are automatically separated from a background and provided for compositing with the image on the surface in real time or at another time.

19. The computer of claim 17, wherein the processor performs further actions, comprising:

determining a distance from the one or more projectors to the surface based on a triangulation of light from at least a first projector that is reflected off one or more of the second elements on the surface onto at least a first sensor.

20. The computer of claim 17, wherein the processor performs further actions, comprising:

employing a second surface that includes another plurality of elements to provide a working surface that is smaller than the surface and positioned physically closer to the user, wherein the working surface is employed by the user to select one or more virtual drawing implements, including one or more pens, brushes, or magic wands, and wherein the selected one or more virtual drawing implements are employed to edit one or more images projected on the surface.