System and method of visual layering

• US 10,229,538 B2
• Filed: 07/29/2011
• Issued: 03/12/2019
• Est. Priority Date: 07/29/2011
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Sievert, Otto K.
• Primary Examiner(s): Cobb, Michael J
• Application Number:
  US14/126,352
• Publication Number:
  US 20140125704A1
• Time in Patent Office: 2,783 Days
• Field of Search: None
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A camera identifies a physical object positioned in a workspace. A display displays first digital information into the workspace. A layering module treats the physical object as a first layer in the workspace and treats the first digital information as a second layer in the workspace. A controller controls the visual adjacency of the first and second layers via display of the first digital information.

Claims

1. A system, comprising:

a camera to identify a physical object positioned in a three-dimensional workspace;

a display to display first digital information in the workspace;

a processor;

a memory to store instructions that, when executed by the processor, cause the processor to;

treat the physical object as a first layer of a plurality of depth ordered visual layers in the workspace and treat the first digital information as a second layer of the plurality of depth ordered visual layers in the workspace, wherein the plurality of depth ordered visual layers comprises a foreground layer and a background layer, and the first layer comprises a projected image of the physical object;

control the visual order of the first and second layers via display of the first digital information;

receive user input, within the workspace, representing physical interaction with at least one of the physical object or a surface of the workspace to indicate a change in the visual order of the first and second layers; and

alter the visual order of the first and second layers in response to the user input, wherein altering the visual order comprises one of changing the visual order of the first and second layers so that the first layer is behind the second layer, or changing the visual order of the first and second layers so that the first layer is in front of the second layer.

2. The system of claim 1, wherein the camera is at least one of an infrared camera or a visible-light camera.

3. The system of claim 1, wherein the instructions, when executed by the processor, cause the processor to alter the display of the first digital information to maintain an existing visual order between the first and second layers in response to a change in position of the physical object relative to the first digital information.

4. The system of claim 1, wherein:

the display to display second digital information in the workspace;

the instructions, when executed by the processor, cause the processor to treat the second digital information as a third layer in the workspace; and

the instructions, when executed by the processor, cause the processor to control display of the first and second digital information to alter the visual order between the first, second, and third layers.

5. The system of claim 1, wherein the display comprises a digital projector.

6. A computer-readable non-transitory storage medium containing instructions that, when executed, cause a computer to:

use a camera to identify a physical object positioned in a three-dimensional workspace;

associate the physical object with a first layer of a plurality of depth ordered visual layers comprising a foreground layer and a background layer, wherein the first layer comprises a projected image of the physical object;

display digital information as a second layer of the plurality of depth ordered visual layers in the workspace;

receive user input, within the workspace, representing physical interaction with at least one of the physical object or a surface of the workspace to indicate a change in the visual order of the first and second layers; and

alter the visual adjacency between the first and second layers in response to the user input, wherein altering the visual adjacency between the first and second layers comprises one of changing the visual adjacency of the first and second layers so that the first layer is behind the second layer or changing the visual adjacency of the first and second layers so that the first layer is in front of the second layer.

7. The computer-readable storage medium of claim 6, comprising further instructions that cause the computer to:

maintain an existing visual adjacency between the first and second layers in response to a change in position of the physical object relative to the displayed digital information.

8. The computer-readable storage medium of claim 6, comprising further instructions that cause the computer to:

display additional digital information as a third layer in the workspace; and

alter the visual adjacency between the first, second, and third layers.

9. A method for a projector-camera system, comprising:

using a camera to identify a physical object positioned in a three-dimensional workspace;

associating the physical object with a first layer of a plurality of depth ordered visual layers comprising a foreground layer and a background layer, wherein the first layer comprises a projected image of the physical object;

projecting digital information as a second layer of the plurality of depth ordered visual layers into the workspace;

receiving user input, within the workspace, representing physical interaction with at least one of the physical object or a surface of the workspace to indicate a change in the visual order of the first and second layers; and

altering the visual adjacency between the first and second layers in response to the user input, wherein altering the visual adjacency between the first and second layers comprises one of changing the visual adjacency of the first and second layers so that the first layer is behind the second layer or changing the visual adjacency of the first and second layers so that the first layer is in front of the second layer.

10. The method of claim 9, further comprising using infrared sensing to identify the physical object.

11. The method of claim 9, wherein the digital information comprises a digital image or digital video feed.

12. The method of claim 9, further comprising:

maintaining an existing visual adjacency between the first and second layers in response to a change in position of the physical object relative to the projected digital information.

13. The method of claim 9, further comprising:

projecting additional digital information as a third layer into the workspace; and

altering the visual adjacency between the first, second, and third layers.