APPARATUS AND METHOD FOR PROVIDING MOTION HAPTIC EFFECT USING VIDEO ANALYSIS

• US 20150109528A1
• Filed: 10/20/2014
• Published: 04/23/2015
• Est. Priority Date: 10/21/2013
• Status: Abandoned Application

• Current Assignee: Postech Academy-Industry Foundation
• Original Assignee: Postech Academy-Industry Foundation
• Inventors: CHOI, Seung Moon, LEE, Jae Bong
• Application Number:
  US14/518,238
• Publication Number:
  US 20150109528A1
• Time in Patent Office: Days
• US Class Current: 348/460
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Provided are an apparatus and method for providing a motion haptic effect using video analysis. The apparatus includes a camera viewpoint classifier configured to analyze the camera viewpoint of an input video and classify the input video as a first-person viewpoint video or a third-person viewpoint video, a first-person viewpoint video processor configured to estimate a camera egomotion for the first-person viewpoint video, and a third-person viewpoint video processor configured to estimate a global motion of a target object for tracking included in the third-person viewpoint video. Accordingly, it is possible to effectively generate multimedia content capable of providing a motion haptic effect.

Claims

1. An apparatus for providing a motion haptic effect using video analysis, the apparatus comprising:

a camera viewpoint classifier configured to analyze a camera viewpoint of an input video and classify the input video as a first-person viewpoint video or a third-person viewpoint video;

a first-person viewpoint video processor configured to estimate a camera egomotion for the first-person viewpoint video; and

a third-person viewpoint video processor configured to estimate a global motion of a target object for tracking included in the third-person viewpoint video.

2. The apparatus of claim 1, further comprising a velocity information converter configured to convert the camera egomotion for the first-person viewpoint video or the global motion of the target object into acceleration information.

3. The apparatus of claim 2, further comprising a motion haptic feedback unit configured to generate motion haptic feedback based on the acceleration information from a viewpoint of a viewer who views the input video.

4. The apparatus of claim 1, wherein the first-person viewpoint video processor calculates an optical flow of the first-person viewpoint video, and estimates the camera egomotion for the first-person viewpoint video based on the optical flow.

5. The apparatus of claim 1, wherein the third-person viewpoint video processor separates the target object from a background area.

6. The apparatus of claim 5, wherein the third-person viewpoint video processor calculates motion of the target object, and estimates a camera egomotion for the third-person viewpoint video based on the background area.

7. The apparatus of claim 6, wherein the third-person viewpoint video processor estimates the global motion of the target object in a global coordinate system by considering the camera egomotion for the third-person viewpoint video for the motion of the target object.

8. The apparatus of claim 1, wherein the input video is a two-dimensional (2D) or three-dimensional (3D) video.

9. A method of providing a motion haptic effect using video analysis, the method comprising:

estimating a camera egomotion for a first-person viewpoint video;

converting the camera egomotion for the first-person viewpoint video into acceleration information; and

generating motion haptic feedback based on the acceleration information from a viewpoint of a viewer who views the first-person viewpoint video.

10. The method of claim 9, wherein the estimating of the camera egomotion for the first-person viewpoint video comprises calculating an optical flow of the first-person viewpoint video, and estimating the camera egomotion for the first-person viewpoint video based on the optical flow.

11. The method of claim 9, wherein the first-person viewpoint video is a two-dimensional (2D) or three-dimensional (3D) video.

12. A method of providing a motion haptic effect using video analysis, the method comprising:

estimating a global motion of a target object for tracking included in a third-person viewpoint video;

converting the global motion of the target object into acceleration information; and

generating motion haptic feedback based on the acceleration information from a viewpoint of a viewer who views the third-person viewpoint video.

13. The method of claim 12, wherein the estimating of the global motion of the target object for tracking included in the third-person viewpoint video comprises:

separating the target object from a background area;

calculating motion of the target object;

estimating a camera egomotion for the third-person viewpoint video based on the background area; and

estimating the global motion of the target object in a global coordinate system by considering the camera egomotion for the third-person viewpoint video for the motion of the target object.

14. The method of claim 12, wherein the third-person viewpoint video is a two-dimensional (2D) or three-dimensional (3D) video.

15. A method of providing a motion haptic effect using video analysis, the method comprising:

analyzing a camera viewpoint of an input video and classifying the input video as a first-person viewpoint video or a third-person viewpoint video;

estimating a camera egomotion for the first-person viewpoint video when the input video is classified as the first-person viewpoint video; and

estimating a global motion of a target object for tracking included in the third-person viewpoint video when the input video is classified as the third-person viewpoint video.

16. The method of claim 15, further comprising converting the camera egomotion for the first-person viewpoint video or the global motion of the target object into acceleration information.

17. The method of claim 16, further comprising generating motion haptic feedback based on the acceleration information from a viewpoint of a viewer who views the input video.