Method of controlling virtual object or view point on two dimensional interactive display

• US 8,839,136 B2
• Filed: 05/07/2009
• Issued: 09/16/2014
• Est. Priority Date: 12/10/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoo, Byung In, Choi, Chang Kyu
• Primary Examiner(s): ALVESTEFFER, STEPHEN D
• Application Number:
  US12/453,352
• Publication Number:
  US 20100146389A1
• Time in Patent Office: 1,958 Days
• Field of Search: 715/702, 715/757, 715/781
• US Class Current: 715/773
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method of controlling a viewpoint of a user or a virtual object on a two-dimensional (2D) interactive display is provided. The method may convert a user input to at least 6 degrees of freedom (DOF) structured data according to a number of touch points, a movement direction thereof, and a rotation direction thereof. Any one of the virtual object and the viewpoint of the user may be determined as a manipulation target based on a location of the touch point.

Claims

1. A method of controlling a virtual object or a viewpoint of a user on a two-dimensional (2D) interactive display, the method comprising:

verifying, when a number of touch points by a user input is at least two, two touch points most adjacent to each other among the at least two touch points;

determining whether a distance between the verified two touch points is less than a predetermined value;

pairing the verified two touch points when the distance between the verified two touch points is less than the predetermined value;

converting the user input to at least 6 degrees of freedom (DOF) structured data according to at least one of a movement direction of the paired touch points and a rotation direction of the paired touch points; and

controlling a manipulation target using the at least 6DOF structured data.

2. The method of claim 1, wherein the at least 6DOF comprises a movement along an X axis, a movement along a Y axis, a movement along a Z axis, a roll based on the X axis, a pitch based on the Y axis, and a yaw based on the Z axis, in a three-dimensional (3D) space comprising the X axis, the Y axis, and the Z axis.

3. The method of claim 1, wherein the converting of the user input comprises comparing the number of the touch points with a predetermined particular value to convert the user input to the at least 6DOF structured data.

4. The method of claim 1, wherein, when the number of the touch points is at least two, the converting of the user input comprises pairing the at least two touch points and converting the user input to the at least 6DOF structured data according to a movement direction of the paired at least two touch points and a rotation direction thereof.

5. The method of claim 1, wherein the controlling of the manipulation target comprises applying a video effect or an audio effect to the manipulation target that is any one of the virtual object and the viewpoint of the user.

6. The method of claim 1, further comprising:

displaying a virtual manipulation medium as a user interface to manipulate the virtual object, displayed on a screen, or the viewpoint of the user, wherein the virtual manipulation medium is displayed in a region including the virtual object when the virtual object is determined as the manipulation target, and the virtual manipulation medium is displayed in a region excluding the virtual object when the viewpoint of the user is determined as the manipulation target.

7. The method of claim 6, wherein, when the number of the touch points is two, the displayed virtual manipulation medium has a spherical form which has a diameter which is determined by a distance between two fingers, and the displayed virtual medium includes display of three-dimensional (3D) coordinate axes.

8. A non-transitory computer-readable recording media storing program instructions to control at least one processor to implement the method of claim 1.

9. The method of claim 1, wherein:

the at least 6DOF comprises a movement along an X axis, a movement along a Y axis, a movement along a Z axis, a roll based on the X axis, a pitch based on the Y axis, and a yaw based on the Z axis, in a three-dimensional (3D) space comprising the X axis, the Y axis, and the Z axis; and

when the number of touch points is at least two, the converting of the user input comprises pairing the at least two touch points and converting the user input to the at least 6DOF structured data according to at least one of a movement direction of the paired at least two touch points and a rotation direction thereof to provide movement along the X-axis, movement along the Y-axis, movement along the Z-axis, roll, pitch, and yaw.

10. The method of claim 1, further comprising:

determining, as the manipulation target, any one of the virtual object and the viewpoint of the user, based on the touch points,

wherein the virtual object is determined as the manipulation target when a location of the touch points exists in a region including the virtual object, and the viewpoint of the user is determined as the manipulation target when the location of the touch points exists in a region excluding the virtual object.

13. The method of claim 8, wherein the at least 6DOF comprises a movement along an X axis, a movement along a Y axis, a movement along a Z axis, a roll based on the X axis, a pitch based on the Y axis, and a yaw based on the Z axis, in a 3D space comprising of the X axis, the Y axis, and the Z axis.

11. A method of processing multi-touch structured data, the method comprising:

receiving, from a multi-touch driver, the multi-touch structured data that is associated with a user input via a virtual manipulation medium;

extracting, from the multi-touch structured data, a number of touch points, a movement direction thereof, a location thereof, and a rotation direction thereof;

verifying, when the number of touch points is at least two, two touch points most adjacent to each other among the at least two touch points;

determining whether a distance between the verified two touch points is less than a predetermined value;

pairing the verified two touch points when the distance between the verified two touch points is less than the predetermined value; and

converting the multi-touch structured data to at least 6 degrees of freedom (6DOF) structured data, based on at least one of a movement direction of the paired touch points and a rotation direction of the paired touch points.

12. The method of claim 11, wherein the converting of the multi-touch structured data comprises comparing the number of touch points with a predetermined particular value to convert the multi-touch structured data to the at least 6DOF structured data.

14. A non-transitory computer-readable recording media storing program instructions to control at least one processor to implement the method of claim 11.

15. A method of processing a virtual manipulation medium that is used to control a virtual object or a viewpoint of a user on an interactive display, the method comprising:

receiving, from a user input via the virtual manipulation medium, at least 6 degrees of freedom (6DOF) structured data, wherein the at least 6DOF structured data is generated based on a number of touch points, a movement direction thereof, a location thereof, and a rotation direction thereof;

determining, as a manipulation target, any one of the virtual object, displayed on a screen, and the viewpoint of the user, based on the location of the touch point, wherein the virtual object is determined as the manipulation target when a location of the touch points exists in a region including the virtual object, and the viewpoint of the user is determined as the manipulation target when the location of the touch points exists in a region excluding the virtual object;

displaying, as a user interface, the virtual manipulation medium in any one of a region including the virtual object and a region excluding the virtual object, depending on whether the virtual object is determined as the manipulation target or the viewpoint of the user is determined as the manipulation target; and

applying a video effect or an audio effect to the manipulation target that is any one of the virtual object and the viewpoint of the user.

16. The method of claim 15, wherein the at least 6DOF comprises a movement along an X axis, a movement along a Y axis, a movement along a Z axis, a roll based on the X axis, a pitch based on the Y axis, and a yaw based on the Z axis, in a 3D space comprising the X axis, the Y axis, and the Z axis.

17. The method of claim 15, wherein, when the number of touch points is two, the displayed virtual manipulation medium has a spherical form which has a diameter which is determined by a distance between two fingers, and the displayed virtual medium includes display of three-dimensional (3D) coordinate axes.

18. A non-transitory computer-readable recording media storing program instructions to control at least one processor to implement the method of claim 15.

19. A computing system for controlling a virtual object or a viewpoint of a user on a two-dimensional (2D) interactive display, the system comprising:

a multi-touch sensor to recognize one or more touches of the user and to generate one or more touch signals, wherein a virtual manipulation medium is displayed on the interactive display after the multi-touch sensor recognizes the one or more touches;

a multi-touch driver, using at least one processor, to generate multi-touch structured data based on one or more touch signals received from the multi-touch sensor;

a six degree of freedom interaction engine to receive the multi-touch structured data from the multi-touch driver and to generate six degrees of freedom structured data;

a virtual manipulation medium processing engine to generate a command to apply at least one of a video effect and an audio effect to the virtual manipulation medium to correspond to a user gesture using the six degrees of freedom structured data; and

a three-dimensional virtual application to generate a command to perform a graphic process according to motion of the virtual object or according to a movement of a camera, using the six degrees of freedom structured data,

wherein, when the number of touches is two, the displayed virtual manipulation medium has a spherical form which has a diameter which is determined by a distance between two fingers, and the displayed virtual medium includes display of three-dimensional (3D) coordinate axes.