Graphical element placement on a display surface

• US 9,146,668 B2
• Filed: 01/31/2013
• Issued: 09/29/2015
• Est. Priority Date: 01/31/2013
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Walsh, Robert, Lasslo, Laurel
• Primary Examiner(s): ABDUL-ALI, OMAR R
• Application Number:
  US13/755,035
• Publication Number:
  US 20140215389A1
• Time in Patent Office: 971 Days
• Field of Search: 715/702, 715/808
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

An example processor-implemented method for placing a graphical element on a display surface in accordance with the present disclosure is receiving an image of at least part of a display surface, detecting in the received image a token placed by a user on the display surface to specify an area on the display surface, and placing the graphical element within the area specified by the placement of the token.

Claims

1. A method comprising:

receiving, by a system including a processor, an image of at least part of a display surface;

detecting, by the system, in the received image a token comprising a physical object placed by a user on the display surface, the token having a visual marking and an arrow;

determining, by the system based on the visual marking on the token, a boundary of an area for placing a graphical element to be displayed on the display surface;

placing, by the system, the graphical element corresponding to the token within the area specified by placement of the token on the display surface; and

determining, based on the arrow, a direction relative to the token where the area is to be placed in a displayed image displayed on the display surface.

2. The method of claim 1, further comprising:

receiving a selection by the user of a graphical element item, wherein the graphical element item is selected from a set of graphical element items in the graphical element, and the graphical element item identifies an action to be performed; and

performing the action identified by the graphical element item.

3. The method of claim 2, further comprising:

receiving user data from the user;

retrieving a user profile associated with the user data, the user profile identifying a security level for the user; and

determining whether to perform the action identified by the graphical element item based on the security level.

4. The method of claim 1, wherein the display surface is horizontal and the token rests on the display surface.

5. The method of claim 1, further comprising:

identifying token data associated with the token based on the received image; and

generating the graphical element based on the token data.

6. The method of claim 5, wherein generating the graphical element based on the token data comprises:

checking the token data for graphical element information;

in response to determining that the graphical element information does not exist in the token data, generating the graphical element with generic graphical element items; and

in response to determining that the graphical element information exists in the token data, using the graphical element information to generate the graphical element.

7. The method of claim 5,

wherein the token data comprises information defining dimensions of the graphical element;

and

wherein generating the graphical element based on the token data comprises generating the graphical element based on the information defining the dimensions of the graphical element.

8. The method of claim 5,

wherein the token data comprises at least one graphical element item and graphical element settings;

and

wherein identifying the token data associated with the token based on the received image comprises;

examining an image of the token to recognize the token; and

obtaining the at least one graphical element item and the graphical element settings associated with the recognized token.

9. The method of claim 1, wherein placing the graphical element within the area specified by the placement of the token comprises:

identifying dimensions of the graphical element;

comparing the dimensions of the graphical element to the area within the boundary determined based on the visual marking on the token;

in response to the dimensions of the graphical element being greater than the area, adjusting the dimensions of the graphical element according to the area; and

in response to the dimensions of the graphical element being equal to or less than the area, maintaining unchanged the dimensions of the graphical element.

10. The method of claim 1, further comprising:

receiving user data from the user;

retrieving a user profile associated with the user data, the user profile identifying at least one graphical element item and graphical element settings; and

generating the graphical element based on the user profile using the at least one graphical element item and graphical element settings.

11. The method of claim 1, wherein receiving the image comprises receiving the image captured by a camera, the image projected onto the display surface by a projector.

12. The method of claim 1, wherein determining the boundary of the area based on the visual marking on the token comprises determining the boundary of the area based on visual code on the token.

13. A projector-camera system comprising:

a projector to project an image onto a display surface, wherein the projector has a field of projection on the display surface;

a camera to capture an image of at least a portion of the display surface on which a token comprising a physical object has been placed by a user, wherein the camera has a camera field of view that intersects the field of projection of the projector on the display surface;

at least one processor to;

receive the captured image from the camera;

detect in the received captured image the token placed by the user on the display surface, the token having a visual marking;

determine, based on the visual marking on the token, a boundary of an area for placing a graphical element to be displayed on the display surface; and

place the graphical element corresponding to the token within the area specified by placement of the token on the display surface, wherein to place the graphical element within the area specified by the placement of the token, the at least one processor is to;

identify dimensions of the graphical element;

compare the dimensions of the graphical element to the area within the boundary determined based on the visual marking on the token;

in response to the dimensions of the graphical element exceeding the area, adjust the dimensions of the graphical element to fit within the area; and

in response to the dimensions of the graphical element not exceeding the area, maintain unchanged the dimensions of the graphical element.

14. The system of claim 13, wherein the at least one processor is to:

identify token data associated with the token based on the received captured image; and

generate the graphical element based on the token data.

15. The system of claim 13, wherein the graphical element has a set of graphical element items, wherein the set of graphical element items identify actions to be performed, and

wherein the at least one processor is to:

receive a selection by the user of a graphical element item from the set of graphical element items; and

perform the action identified by the graphical element item.

16. The system of claim 15, wherein the at least one processor is to:

receive user data from the user;

retrieve a user profile associated with the user data, the user profile identifying a security level for the user; and

determine whether to perform the action identified by the graphical element item based on the security level.

17. The system of claim 14, wherein the at least one processor is to:

check the token data for graphical element items;

in response to determining that the graphical element items do not exist in the token data, generate the graphical element with generic graphical element items; and

in response to determining that the graphical element items exist in the token data, use the graphical element items to generate the graphical element.

18. A non-transitory computer-readable storage medium storing instructions that when executed cause a system to:

receive an image of at least part of a display surface;

detect in the received image a token comprising a physical object placed by a user on the display surface, the token having a visual marking;

determine, based on the visual marking, a boundary of an area for placing a graphical element to be displayed on the display surface; and

place the graphical element corresponding to the token within the area specified by placement of the token on the display surface, wherein placing the graphical element within the area specified by the placement of the token comprises;

identifying dimensions of the graphical element;

comparing the dimensions of the graphical element to the area within the boundary determined based on the visual marking on the token;

in response to the dimensions of the graphical element exceeding the area, adjusting the dimensions of the graphical element to fit within the area and

in response to the dimensions of the graphical element not exceeding the area, maintaining unchanged the dimensions of the graphical element.

19. The non-transitory computer-readable storage medium of claim 18, wherein the instructions when executed cause the system to further:

receive a selection by the user of a graphical element item of the graphical element, wherein the graphical element item specifies an action to be performed;

retrieve a user profile identifying a security level for the user; and

determine whether to perform the action specified by the graphical element item based on the security level.

20. A projector-camera system comprising:

a projector to project an image onto a display surface, wherein the projector has a field of projection on the display surface;

a camera to capture an image of at least a portion of the display surface on which a token comprising a physical object has been placed by a user, wherein the camera has a camera field of view that intersects the field of projection of the projector on the display surface;

at least one processor to;

receive the captured image from the camera;

detect in the received captured image the token placed by the user on the display surface, the token having a visual marking;

determine, based on the visual marking on the token, a boundary of an area for placing a graphical element to be displayed on the display surface; and

place the graphical element corresponding to the token within the area specified by placement of the token on the display surface,

wherein the visual marking on the token comprises one of a barcode and a QR code.