System and method for providing object location information and physical contact information

• US 8,878,821 B2
• Filed: 04/29/2010
• Issued: 11/04/2014
• Est. Priority Date: 04/29/2010
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Robinson, Ian N, Tan, Kar-Han, Gelb, Daniel George
• Primary Examiner(s): CERULLO, LILIANA P
• Application Number:
  US13/695,261
• Publication Number:
  US 20130050145A1
• Time in Patent Office: 1,650 Days
• Field of Search: 345173-178, 178 1801- 1904
• US Class Current: 345/175
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

The present invention provides a method (500) executed on a processor for providing object information about an object in proximity to a display screen. The method includes the step of determining whether an at least first object is located within a proximal distance of the first surface of the display screen based on information provided by a depth camera (step 510). The method also includes the step of determining whether the at least first object has physically contacted a first surface of the display screen based on information provided by a physical contact sensing component (step 520). The information about (1) the proximal distance of the at least first object from the screen and (2) whether the at least first object has physically contacted the display surface of the screen to provide proximal event information and based on the information provided, it is determining whether the user interface should be modified (step 560).

Claims

1. A system for providing object information, the system comprising:

a screen having a display surface;

a physical contact sensing component to generate physical contact information regarding whether at least an object has come into physical contact with the display surface of the screen; and

a depth camera physically positioned to generate object location information by capturing a location of the object located in front of the display surface, the object location information including a region of the screen that the object is located in front of and further including object distance information related to the distance the object is located away from the front of the screen; and

a user interface component coupled to the physical contact sensing component and to the depth camera, the user interface component to, in response to detecting a physical touch via the physical contact sensing component, determine a type of touch event based on the physical contact information and based on the object location information, the type of touch event including a first type associated with a single proximal object and a second type associated with multiple proximal objects.

2. The system recited in claim 1, wherein responsive to the physical contact information, object location information, and object distance information provided, the user interface component is to determine whether to modify a user interface.

3. The system recited in claim 1, wherein the physical contact sensing component is a resistive touch screen element.

4. The system recited in claim 1, wherein the object distance information from the depth camera and the physical contact information from the physical contact sensing component is used to determine whether the object has touched the display surface of the screen.

5. The system recited in claim 1, wherein in response to a determination of the second type of touch event, the user interface component is to determine whether the physical touch is associated with a centroid of the multiple proximal objects.

6. A system for providing object information, the system comprising:

a screen having a display surface;

a physical contact sensing component to provide physical contact information regarding whether an object has physically contacted the display surface;

an image capture device positioned to capture a location of the object located in front of the display surface, the image capture device to generate object location information including a region of the screen that the object is located in front of and further including object distance information about whether the object is within a hover distance from the display surface; and

a user interface component, coupled to the physical contact sensing component and to the image capture device, to;

in response to detecting a physical touch via the physical contact sensing component, determine a type of touch event based on the physical contact information and based on the object location information, the type of touch event including a first associated with a single proximal object and a second type associated with multiple proximal objects; and

in response to detecting the object being within the hover distance and no registration of physical contact by the physical contact sensing component, determine whether to modify a user interface.

7. The system recited in claim 6, wherein in response to a determination that the user interface is be modified, the user interface component is to modify the user interface by providing a touch sensitive menu.

8. The system recited in claim 7, wherein the touch sensitive menu is displayed on the screen in the region of the screen where the object is hovering in front of.

9. The system recited in claim 6, wherein in response to a determination that the user interface is to be modified, the user interface component is to modify the user interface by magnifying data displayed on the screen in the region of the screen where the object is hovering in front of.

10. The system recited in claim 6, wherein in response to a determination that the user interface is to be modified, the user interface component is to modify the user interface by changing content displayed on the screen in the region of the screen that the object is hovering in front of.

11. A non-transitory computer readable storage medium instructions when executed cause a processor of a computer system to:

determine whether an object is located within a proximal distance of a display surface of a screen of the computer system based on object location information provided by a depth camera of the computer system, wherein the object location information includes a region of the screen that the object is located in front of and further including object distance information about whether the object is within a hover distance from the display surface;

determine whether the object has physically contacted the display surface based on physical contact information provided by a physical contact sensing component of the computer system;

determine whether a user interface is to be modified based on the object location information and the physical contact information; and

in response to detecting a physical touch based on the physical contact information, determine a type of touch event based on the physical contact information and based on the object location information, the type of touch event including a first type associated with a single proximal object and a second type associated with multiple proximal objects.

12. The computer readable storage medium recited in claim 11, wherein the instructions when executed further cause the processor to determine whether the object is hovering in response to a determination that the object has not contacted the display surface of the screen.

13. The computer readable storage medium recited in claim 11, wherein the instructions when executed further cause the processor to:

in response to a first determination that the object is at a first sensed proximity of the screen, instruct the screen to display a shadow on a displayed image; and

in response to a second determination that the object is at a second sensed proximity of the screen, instruct the screen to darken the shadow.

14. The computer readable storage medium recited in claim 11, wherein the instructions when executed further cause the processor to, in response to a determination of the second type of touch event, determine whether the physical touch is associated with a centroid of the multiple proximal objects.

15. A method executed on a processor for providing object information, the method comprising:

determining whether an object is located within a proximal distance of a display surface of a screen of a computer system based on object location information provided by a depth camera of the computer system, wherein the object location information includes a region of the screen that the object is located in front of and further including object distance information about whether the object is within a hover distance from the display surface;

determining whether the object has physically contacted the display surface based on physical contact information provided by a physical contact sensing component of the computer system;

determine whether a user interface is to be modified based on the object location information and the physical contact information; and

in response to detecting a physical touch based on the physical contact information, determine a type of touch event based on the physical contact information and based on the object location information, the type of touch event including a first type associated with a single proximal object and a second type associated with multiple proximal objects.