INTERACTIVE INPUT SYSTEM AND METHOD

• US 20140354602A1
• Filed: 08/18/2014
• Published: 12/04/2014
• Est. Priority Date: 04/12/2013
• Status: Abandoned Application

• Current Assignee: uSens Inc.
• Original Assignee: Impression PI Incorporated
• Inventors: HE, Anli, FEI, Yue
• Application Number:
  US14/462,324
• Publication Number:
  US 20140354602A1
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/042 by opto-electronic means

Abstract

A method for generating and displaying a graphic representation of an object on a display screen. The method includes capturing at least one image of the object using at least one image sensor, determining, according to the at least one image, three-dimensional (3D) coordinates of a 3D point on the object in a 3D coordinate system defined in a space containing the object, defining a touch interactive surface in the space, performing a projection of the 3D point onto a projection point on the touch interactive surface, determining 3D coordinates of the projection point in the 3D coordinate system according to the projection, determining a displaying position of the graphic representation on the display screen according to the 3D coordinates of the projection point, and displaying the graphic representation at the displaying position on the display screen.

Claims

1. A method for generating and displaying a graphic representation of an object on a display screen, the method comprising:

capturing at least one image of the object using at least one image sensor;

determining, according to the at least one image, three-dimensional (3D) coordinates of a 3D point on the object in a 3D coordinate system defined in a space containing the object;

defining a touch interactive surface in the space;

performing a projection of the 3D point onto a projection point on the touch interactive surface;

determining 3D coordinates of the projection point in the 3D coordinate system according to the projection;

determining a displaying position of the graphic representation on the display screen according to the 3D coordinates of the projection point; and

displaying the graphic representation at the displaying position on the display screen.

2. The method of claim 1, further comprising:

calculating one of a first distance between the 3D point and the touch interactive surface or a second distance between the 3D point and the projection point.

3. The method of claim 2, wherein displaying the graphic representation includes:

displaying a 2D position indicator at the displaying position; and

displaying an object indicator at an offset position on the display screen, the object indicator representing the object, and an offset between the displaying position and the offset position is a function of the first distance or the second distance.

4. The method of claim 3, wherein displaying the object indicator includes displaying an object indicator having at least one of a size or a transparency that depends on the first distance or the second distance.

5. The method of claim 3, wherein displaying the graphic representation further includes displaying an object shadow at the displaying position, the object shadow representing a shadow of the object.

6. The method of claim 5, wherein displaying the object shadow includes displaying an object shadow having at least one of a size or a transparency that depends on the first distance or the second distance.

7. The method of claim 3, wherein:

the offset position is a first offset position,

the offset is a first offset, and

displaying the graphic representation further includes displaying an object shadow at a second offset position on the display screen, the object shadow representing a shadow of the object, and a second offset between the displaying position and the second offset position is a function of the first distance or the second distance.

8. The method of claim 3, further comprising:

detecting a click or tap action performed by the object, the click or tap action including a sudden move of the object toward the touch interactive surface; and

causing the 2D position indicator to animate according to the click or tap action.

9. The method of claim 8, further comprising:

holding a position of the 2D position indicator when the click or tap action is detected.

10. The method of claim 1, wherein performing the projection includes performing an orthogonal projection.

11. The method of claim 1, wherein performing the projection includes performing a non-orthogonal projection.

12. The method of claim 11, further comprising:

detecting a position of a user'"'"'s head,

wherein performing the non-orthogonal projection includes determining an intersection point at which a line connecting the position of the user'"'"'s head and the 3D point intersects the touch interactive surface, the interaction point being the projection point.

13. The method of claim 1, wherein determining the displaying position includes:

mapping the 3D coordinates of the projection point to 2D coordinates of the displaying position in a 2D coordinate system defined on the display screen.

14. The method of claim 13, wherein:

defining the touch interactive surface includes defining four corner points of the touch interactive surface in the space, and

mapping the 3D coordinates of the projection point to the 2D coordinates of the displaying position includes;

obtaining a mapping function by a fitting method according to correspondences between the four corner points of the touch interactive surface and four corners of the display screen, and

determining the displaying position according to the 3D coordinates of the projection point and the mapping function.

15. The method of claim 13, wherein determining the displaying position includes:

calculating 2D coordinates of the projection point in a first 2D coordinate system defined on the touch interactive surface; and

transforming the 2D coordinates of the projection point in the first 2D coordinate system to 2D coordinates of the displaying position in a second 2D coordinate system defined on the display screen.

16. The method of claim 15, wherein:

defining the touch interactive surface includes defining four corner points of the touch interactive surface in the space, and

transforming the 2D coordinates of the projection point in the first 2D coordinate system to the 2D coordinates of the displaying position in the second 2D coordinate system includes;

calculating 2D coordinates of each of the four corner points of the touch interactive surface in the first 2D coordinate system,

defining 2D coordinates of each of four corners of the display screen in the second 2D coordinate system,

obtaining a homography transform matrix according to correspondences between the 2D coordinates of the four corner points of the touch interactive surface in the first 2D coordinate system and the 2D coordinates of the four corners of the display screen in the second 2D coordinate system, and

determining the displaying position according to the 2D coordinates of the projection point in the first 2D coordinate system and the homography transform matrix.

17. A non-transitory computer-readable storage medium storing a program for generating and displaying a graphic representation of an object on a display screen, the program, when executed by a computer, instructing the computer to:

capture at least one image of the object using at least one image sensor;

determine, according to the at least one image, three-dimensional (3D) coordinates of a 3D point on the object in a 3D coordinate system defined in a space containing the object;

define a touch interactive surface in the space;

perform a projection of the 3D point onto a projection point on the touch interactive surface;

determine 3D coordinates of the projection point in the 3D coordinate system according to the projection;

determine a displaying position of the graphic representation on the display screen according to the 3D coordinates of the projection point; and

display the graphic representation at the displaying position on the display screen.

18. An apparatus for generating and displaying a graphic representation of an object on a display screen, the apparatus comprising:

a processor; and

a non-transitory computer-readable storage medium storing a program that, when executed, instructs the processor to;

capture at least one image of the object using at least one image sensor;

determine, according to the at least one image, three-dimensional (3D) coordinates of a 3D point on the object in a 3D coordinate system defined in a space containing the object;

define a touch interactive surface in the space;

perform a projection of the 3D point onto a projection point on the touch interactive surface;

determine 3D coordinates of the projection point in the 3D coordinate system according to the projection;

determine a displaying position of the graphic representation on the display screen according to the 3D coordinates of the projection point; and

display the graphic representation at the displaying position on the display screen.