AUGMENTED REALITY CONTENTS GENERATION AND PLAY SYSTEM AND METHOD USING THE SAME

• US 20170200314A1
• Filed: 07/09/2015
• Published: 07/13/2017
• Est. Priority Date: 07/09/2014
• Status: Active Grant

• Current Assignee: Ji-Yeon Jeong
• Original Assignee: Ji-Yeon Jeong
• Inventors: JEONG, Ji-Yeon
• Granted Patent: US 9,959,681 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

The present disclosure relates to an augmented reality contents generation and play system. The augmented reality contents generation and play system includes an image acquisition unit configured to include a preview mode in which an image which is input through a lens is displayed on a screen and a shooting mode in which, if an image to be shot is displayed in the preview mode, the image is captured by operating a shooting button and the captured image is converted into an image file; a storage unit configured to store an image which is obtained from the image acquisition unit when the shooting button is pressed and video information which is automatically obtained for a predetermined time based on a shooting point of time; a mapping unit configured to map the image which is stored in the storage unit onto a video which is stored in the storage unit; an augmented reality processing unit configured to recognize an image corresponding to a marker from the image which is obtained by the image acquisition unit, configured to call the video mapped onto the image, and configured to overlay the video on a position of the marker on a screen in the preview mode; and a control unit configured to control a process of generating and playing contents by using an augmented reality contents.

Claims

1. An augmented reality contents generation and play system comprising:

an image acquisition unit configured to include a preview mode in which an image which is input through a lens is displayed on a screen and a shooting mode in which, if an image to be shot is displayed in the preview mode, the image is captured by operating a shooting button and the captured image is converted into an image file;

a storage unit configured to store an image which is obtained from the image acquisition unit when the shooting button is pressed and video information which is automatically obtained for a predetermined time based on a shooting point of time;

a mapping unit configured to map the image which is stored in the storage unit onto a video which is stored in the storage unit;

an augmented reality processing unit configured to recognize an image corresponding to a marker from the image which is obtained by the image acquisition unit, configured to call the video mapped onto the image, and configured to overlay the video on a position of the marker on a screen in the preview mode; and

a control unit configured to control a process of generating and playing contents by using an augmented reality contents.

3. The augmented reality contents generation and play system according to claim 1, wherein, in the preview mode, if the video is stored when the shooting button is pressed and the storage of a video ends when the shooting button is released, the image acquisition unit automatically stores the image (marker) when the shooting button is pressed or when the shooting button is released, or automatically obtains the image when the storage of a video starts or ends by using a method of starting the storage of a video by pressing the shooting button and ending the storage of a video by pressing the shooting button again, or obtains the image by pressing the shooting button while the video is stored by using the method of starting the storage of a video by pressing the shooting button and ending the storage of a video by pressing the shooting button again.

4. The augmented reality contents generation and play system according to claim 1, wherein, if the image acquisition unit obtains a high resolution image for a marker, a plurality of or a predetermined number of images are automatically captured for initial predetermined time if the shooting button is pressed, and the video is automatically shot while the shooting button is pressed, the video shooting ends when the shooting button is released, or the image acquisition unit selects an image with high quality among a plurality of images which are obtained by automatically capturing an image for a marker by multiple times for a predetermined time before the shooting button is released.

5. The augmented reality contents generation and play system according to claim 1, wherein if the image acquisition unit obtains the image for a marker, the image acquisition unit sets the video and the image to have the same resolution, and thereby, the image with the same resolution as the video is obtained when a video shooting button is pressed.

6. The augmented reality contents generation and play system according to claim 1, wherein the augmented reality processing unit provides a UI which displays a plurality of specific frames of a video file that is obtained by the image acquisition unit on a screen, and controls such that an image that a user wants is selected to be used as a marker.

7. The augmented reality contents generation and play system according to claim 1, wherein the augmented reality processing unit includes:

a marker detection unit configured to extract an image corresponding to the marker from an actual image which is recognized by the image acquisition unit, and detects the corresponding image by comparing the extracted image with information on a feature point of the image which is stored in the storage unit; and

a tracking unit configured to recognize a video by extracting a type of the matched video among the videos that are stored in the storage unit from the image information which are detected by the marker detection unit.

8. The augmented reality contents generation and play system according to claim 7, wherein mapping unit adds the feature point by adding a barcode or a predetermined pattern to a border of the captured image, and thereby the marker detection unit recognizes the marker.

9. The augmented reality contents generation and play system according to claim 1, wherein the control unit includes:

an input module configured to input a shooting button signal;

a contents generation module configured to obtain image and video data by outputting a signal to the storage unit and the mapping unit if a signal is input from the input module, and configured to assign a unique key value to the data and store the data;

a contents play module configured to recognize an image corresponding to the marker through the marker detection unit if an augmented reality application is executed, and configured to call a recognized photo and a mapped video by transmitting a signal to the mapping unit; and

an image display module configured to play the video by overlaying the called video on a position of the marker of a display unit.

14. The augmented reality contents generation and play system according to claim 4, wherein the augmented reality processing unit provides a UI which displays a plurality of specific frames of a video file that is obtained by the image acquisition unit on a screen, and controls such that an image that a user wants is selected to be used as a marker.

2. An augmented reality contents generation and play system comprising:

an image acquisition unit configured to include a preview mode in which an image which is input through a lens is displayed on a screen and a shooting mode in which, if an image to be shot is displayed in the preview mode, the image is captured by operating a shooting button and the captured image is converted into an image file;

a storage unit configured to automatically store a video file and the image file which are obtained by the image acquisition unit;

a mapping unit configured to map the image which is stored in the storage unit onto a video which is stored in the storage unit;

an augmented reality processing unit configured to recognize an image corresponding to a marker from the image which is obtained by the image acquisition unit, configured to call the video mapped onto the image, and configured to overlay the video on a position of the marker on a screen in a preview mode; and

a control unit configured to control a process of generating and playing contents by using an augmented reality contents,

wherein the image acquisition unit automatically store an image (marker) when the shooting button is pressed or when the shooting button is released, if the video is stored when the shooting button is pressed and storage of the video ends when the shooting button is released, in the preview mode, and

wherein the image is automatically obtained when the storage of a video starts or ends by using a method of starting the storage of a video starts when the shooting button is pressed and ending the storage of a video when the shooting button is pressed again, and the image is obtained by pressing the shooting button while the video is stored by using the method of starting the storage of a video starts when the shooting button is pressed and ending the storage of a video when the shooting button is pressed again.

12. The augmented reality contents generation and play system according to claim 2, wherein, if the image acquisition unit obtains a high resolution image for a marker, a plurality of or a predetermined number of images are automatically captured for initial predetermined time if the shooting button is pressed, and the video is automatically shot while the shooting button is pressed, the video shooting ends when the shooting button is released, or the image acquisition unit selects an image with high quality among a plurality of images which are obtained by automatically capturing an image for a marker by multiple times for a predetermined time before the shooting button is released.

13. The augmented reality contents generation and play system according to claim 2, wherein if the image acquisition unit obtains the image for a marker, the image acquisition unit sets the video and the image to have the same resolution, and thereby, the image with the same resolution as the video is obtained when a video shooting button is pressed.

15. The augmented reality contents generation and play system according to claim 2, wherein the augmented reality processing unit includes:

a marker detection unit configured to extract an image corresponding to the marker from an actual image which is recognized by the image acquisition unit, and detects the corresponding image by comparing the extracted image with information on a feature point of the image which is stored in the storage unit; and

a tracking unit configured to recognize a video by extracting a type of the matched video among the videos that are stored in the storage unit from the image information which are detected by the marker detection unit.

16. The augmented reality contents generation and play system according to claim 2, wherein the control unit includes:

an input module configured to input a shooting button signal;

a contents generation module configured to obtain image and video data by outputting a signal to the storage unit and the mapping unit if a signal is input from the input module, and configured to assign a unique key value to the data and store the data;

a contents play module configured to recognize an image corresponding to the marker through the marker detection unit if an augmented reality application is executed, and configured to call a recognized photo and a mapped video by transmitting a signal to the mapping unit; and

an image display module configured to play the video by overlaying the called video on a position of the marker of a display unit.

10. An augmented reality contents generation and play method comprising:

shooting a video in a preview mode of the image acquisition unit;

obtaining at least one image and video data by operating a shooting button if an image to be captured is displayed in a preview mode, assigning a unique key value to the data, storing the data, also automatically storing a video in a predetermined time zone before and after a shooting point of time, and obtaining a high resolution image among the captured images;

mapping the stored image onto the video by using the mapping unit and storing mapping information;

printing the captured image;

recognizing a photo corresponding to a marker by using the marker detection unit if augmented reality application is executed, and calling a video mapped onto the recognized photo by transmitting a signal to the mapping unit; and

playing the video by overlaying the called video on a position of the marker of the image display unit.

11. The augmented reality contents generation and play method according to claim 10,

wherein the obtaining includes obtaining a high resolution image file, and

wherein, in order to obtain a high resolution image, if the image acquisition unit obtains a high resolution image for a marker, a plurality of or a predetermined number of images are automatically captured for initial predetermined time if the shooting button is pressed, and the video is automatically shot while the shooting button is pressed, the video shooting ends when the shooting button is released, or the image acquisition unit selects an image that a user wants among a plurality of images which are obtained by automatically capturing an image for a marker by multiple times for a predetermined time before the shooting button is released.