AUGMENTED REALITY SYSTEM AND CONTROL METHOD THEREOF

• US 20140168267A1
• Filed: 12/11/2013
• Published: 06/19/2014
• Est. Priority Date: 12/18/2012
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KIM, Hark-Joon, HAN, Tack-Don, KIM, Ha-Young, SEO, Jong-Hoon, CHAE, Seung-Ho
• Application Number:
  US14/103,036
• Publication Number:
  US 20140168267A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof

Abstract

A projection-based augmented reality system and a control method thereof are provided. The control method of an augmented reality system includes determining a conversion area to be converted from a work area based on a first gesture, acquiring a captured image of the determined conversion area, generating a virtual image of the determined conversion area from the acquired captured image, displaying the generated virtual image in the work area, and performing a manipulation function with respect to the displayed virtual image based on a second gesture.

Claims

1. A method of controlling an augmented reality system, the method comprising:

determining a conversion area to be converted from a work area based on a first gesture;

acquiring a captured image of the determined conversion area;

generating a virtual image of the determined conversion area from the acquired captured image;

displaying the generated virtual image in the work area; and

performing a manipulation function with respect to the displayed virtual image based on a second gesture.

2. The method of claim 1, further comprising:

displaying an area guide in the work area; and

moving, zooming in, zooming out or rotating the displayed area guide based on the first gesture,

wherein the determining of the conversion area comprises determining a part of the work area corresponding to the area guide as the conversion area.

3. The method of claim 1, wherein the first gesture comprises an operation for designating a boundary showing the conversion area from the work area, and

wherein the generating of the virtual image comprises generating the virtual image of the part of the captured image corresponding to the designated boundary.

4. The method of claim 1, wherein the manipulation function comprises at least one of moving, changing, rotating and storing the virtual image.

5. The method of claim 1, wherein the performing of the manipulation function comprises:

designating a moving path of the conversion area based on the second gesture; and

moving the virtual image along the designated moving path.

6. The method of claim 1, further comprising displaying a second virtual image in a location of at least one marker on the work area corresponding to the marker.

7. The method of claim 6, further comprising moving and displaying the second virtual image according to the movement of the marker.

8. The method of claim 6, further comprising performing the manipulation function with respect to the second virtual image based on the second gesture.

9. The method of claim 6, wherein the displaying of the second virtual image comprises:

displaying a plurality of menu items; and

displaying a virtual image with an effect corresponding to a menu item selected based on a third gesture.

10. An augmented reality system comprising:

a camera configured to acquire a captured image of a work area;

a projector configured to project an image to the work area; and

a control device configured to determine a conversion area to be converted from the work area based on a first gesture, generate a virtual image of the determined conversion area based on a captured image acquired by the camera, display the generated virtual image in the work area using the projector, and perform a manipulation function with respect to the displayed virtual image based on a second gesture.

11. The augmented reality system of claim 10, wherein the control device displays an area guide in the work area using the projector, moves, zooms in, zooms out or rotates the displayed area guide based on the first gesture, and determines a part of the work area corresponding to the area guide, as the conversion area.

12. The augmented reality system of claim 10, wherein the first gesture comprises a gesture for designating a boundary showing the conversion area from the work area, and the control device generates the virtual image of a part of the captured image corresponding to the designated boundary.

13. The augmented reality system of claim 10, wherein the manipulation function comprises at least one of movement, change, rotation and storage of the virtual image.

14. The augmented reality system of claim 10, wherein the control device designates a moving path of the conversion area based on the second gesture and moves the virtual image along the designated moving path.

15. The augmented reality system of claim 10, wherein the control device displays a second virtual image in a location of at least one marker on the work area corresponding to the marker using the projector.

16. The augmented reality system of claim 15, wherein the control device moves and displays the second virtual image according to the movement of the marker.

17. The augmented reality system of claim 15, wherein the control device performs the manipulation function with respect to the second virtual image based on the second gesture.

18. The augmented reality system of claim 15, wherein the second virtual image includes a plurality of menu items, and the control device displays a virtual image with an effect corresponding to a menu item selected based on a third gesture.

19. A method of controlling an augmented reality system, the method comprising:

displaying a first virtual image in one of a work area and a part of a user'"'"'s body located within the work area;

changing the virtual image into a second virtual image and displaying the second virtual image based on a first gesture; and

performing a manipulation function with respect to the displayed second virtual image based on a second gesture.

20. The method of claim 19, wherein the displaying of the first virtual image comprises displaying one of the first virtual image and the second virtual image in a size corresponding to the part of the user'"'"'s body.

21. The method of claim 19, further comprising selecting the second virtual image as a virtual image to which the manipulation function is performed based on a third gesture.

22. The method of claim 21, further comprising displaying the selected second virtual image in a size corresponding to the work area.

23. The method of claim 19, wherein the changing of the virtual image and the displaying of the second virtual image comprises displaying the second virtual image as a next image after the displaying of the first virtual image selected from a plurality of stored virtual images.

24. The method of claim 19, wherein the manipulation function comprises at least one of movement, change, rotation and storage of the virtual image.

25. An augmented reality system comprising:

a camera configured to acquire a captured image of a work area;

a projector configured to project an image to the work area; and

a control device configured to display a first virtual image in one of the work area and a part of a user'"'"'s body located within the work area using the projector, change the first virtual image into a second virtual image and displays the second virtual image based on a first gesture using the acquired captured image, and perform a manipulation function with respect to the displayed second virtual image based on a second gesture.

26. The augmented reality system of claim 25, wherein the control device displays one of the first virtual image and the second virtual image in a size corresponding to the part of the user'"'"'s body.

27. The augmented reality system of claim 25, wherein the control device selects the second virtual image as a virtual image to which the manipulation function is performed based on a third gesture.

28. The augmented reality system of claim 27, wherein the control device displays the selected second virtual image in a size corresponding to the work area.

29. The augmented reality system of claim 25, wherein the control device displays the second virtual image as a next image after the displaying of the first virtual image selected from a plurality of stored virtual images.

30. The augmented reality system of claim 25, wherein the manipulation function comprises at least one of movement, change, rotation and storage of the virtual image.