INTERACTIVE INPUT SYSTEM HAVING A 3D INPUT SPACE

• US 20130215148A1
• Filed: 07/19/2011
• Published: 08/22/2013
• Est. Priority Date: 07/19/2010
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Antonyuk, Viktor, Xin, Min, Tse, Edward, Hudson, Carl, Lai, Henry, Wu, Hui Xiu, Chen, Jingwen
• Granted Patent: US 9,489,040 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

An interactive input system comprises computing structure; and an input device detecting at least one physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to the computing structure, wherein the computing structure processes the output of the input device to: recognize the pattern carried by the at least one physical object in the 3D input space; and modify an image presented on a display surface by applying a transition to digital content associated with the at least one physical object based on a detected state of the at least one physical object.

Claims

1. An interactive input system comprising:

computing structure; and

an input device detecting at least one physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes the output of the input device to;

recognize the pattern carried by the at least one physical object in the 3D input space; and

modify an image presented on a display surface by applying a transition to digital content associated with the at least one physical object based on a detected state of the at least one physical object.

2. The interactive input system of claim 1, wherein the computing device processes the output of the input device to detect at least one of orientation, position and movement of the at least one physical object in the 3D input space.

3. The interactive input system of claim 1, wherein when the detected state of the at least one physical object signifies that a new recognizable physical object is positioned in the 3D input space, the image is modified to add associated digital content to the image, the added associated digital content appearing in the image using a visual effect.

4. The interactive input system of claim 1, wherein when the detected state of the at least one physical object signifies removal of a previously recognized physical object from the 3D input space, the image is modified to remove associated digital content from the image, the associated digital content disappearing from the image using a different visual effect.

5. The interactive input system of claim 3, wherein the visual effect is a fade-in visual effect or a scale-up visual effect.

6. The interactive input system of claim 4, wherein the different visual effect is a fade-out visual effect or a scale-down visual effect.

7. The interactive input system of claim 4, wherein when the detected state of the at least one physical object signifies a physical object in the 3D input space whose tracked position has been lost, the image is modified to remove the associated digital content from the image using the different visual effect.

8. The interactive input system of claim 3, wherein when the detected state of the at least one physical object signifies a lost physical object in the 3D input space whose position has been relocated, the image is modified to add associated digital content to the image using the visual effect.

9. The interactive input system of claim 1, wherein the computing structure further process the output of the input device to detect physical object gestures.

10. The interactive input system of claim 9, wherein said computing structure manipulates digital content associated with the physical object in response to detected gestures.

11. The interactive input system of claim 10, wherein said computing structure, in response to a detected gesture, one of (i) changes associated digital content, (ii) merges associated digital content, (iii) clones associated digital content, (iv) disassociates digital content, (v) associates digital content, and (vi) rotates digital content.

12. The interactive input system of claim 1, wherein the at least one physical object is a card, cube or other object having a least one surface carrying said pattern.

13. The interactive input system of claim 12, wherein the at least one physical object comprises more than one surface, at least some of said surfaces carrying different patterns.

14. The interactive input system of claim 12, wherein said at least one physical object comprises at least one display surface on which a pattern is presented.

15. The interactive input system of claim 1, wherein said input device is a camera device capturing images of said 3D input space.

16. The interactive input system of claim 1, further comprising an interactive surface on which said image is displayed.

17. An apparatus comprising:

a processing device receiving data representing a physical object carrying a recognizable pattern positioned within a three-dimensional (3D) input space; and

memory storing computer program code executable by the processing device, the processing device communicating with the memory and executing the program code causing the apparatus to;

recognize the pattern carried by the physical object in the 3D input space;

modify image data by applying a transition to digital content associated with the physical object based on a detected state of the at least one physical object; and

output the modified image data for presentation on a display surface.

18. A method comprising:

detecting a physical object carrying a unique pattern within a three-dimensional input space;

recognizing the physical object based on the unique pattern carried thereby; and

modifying image data used to present an image on a display surface based on a detected state of the physical object, wherein said modifying comprises applying a transition effect to digital content associated with the physical object.

19. An interactive input system comprising:

computing structure; and

an input device detecting at least one physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes the output of the input device to;

recognize the pattern carried by the at least one physical object in the 3D input space;

determine when the location of the pattern within the 3D input space and the location of digital content in an image presented on a display surface satisfy defined criteria; and

associate the pattern to the digital content.

20. The interactive input system of claim 19, wherein a cursor is assigned to said pattern and is presented in said image at a location corresponding to the position of said pattern within said 3D input space, said computing device determining when the pattern is moved within the 3D input space to bring said cursor into a threshold distance from said digital content.

21. The interactive input system of claim 20, wherein the computing structure further determines if the cursor is positioned from said digital content by said threshold distance for a threshold period of time before associating the pattern with the digital content.

22. The interactive input system of claim 19 wherein said input device is a camera device capturing images of said 3D input space.

23. The interactive input system of claim 19, further comprising an interactive surface on which said image is presented.

24. The interactive input system of claim 19, wherein the computing structure further process the output of the input device to detect physical object gestures.

25. The interactive input system of claim 24, wherein said computing structure manipulates digital content associated with the physical object in response to detected gestures.

26. The interactive input system of claim 25, wherein said computing structure, in response to a detected gesture, one of (i) changes associated digital content, (ii) merges associated digital content, (iii) clones associated digital content, (iv) disassociates digital content, (v) associates digital content, and (vi) rotates digital content.

27. An apparatus comprising:

a processing device receiving data representing a physical object carrying a recognizable pattern positioned within a three-dimensional (3D) input space; and

memory storing computer program code executable by the processing device, the processing device communicating with the memory and executing the program code causing the apparatus to;

recognize the pattern carried by the at least one physical object in the 3D input space;

determine when the location of the pattern within the 3D input space and the location of digital content in an image presented on a display surface satisfy defined criteria; and

associate the pattern to the digital content.

28. An apparatus according to claim 27, wherein said processing device generates an indication signifying said association.

29. An apparatus according to claim 28, wherein said indication is a visual indicator.

30. A method comprising:

detecting a physical object carrying a unique pattern within a three-dimensional input space;

recognizing the physical object based on the unique pattern carried thereby;

determining when the location of the pattern within the input space and the location of digital content within a presented image satisfy defined criteria; and

associating the pattern to the digital content so that manipulation of the physical object within the input space manipulates the digital content.

31. An interactive input system comprising:

computing structure; and

an input device detecting at least two physical objects, each carrying a recognizable pattern, within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes the output of the input device to;

recognize the patterns carried by the at least two physical objects in the 3D input space; and

associate the patterns to the same digital content.

32. The interactive input system of claim 31, wherein the computing structure further process the output of the input device to detect physical object gestures.

33. The interactive input system of claim 32, wherein said computing structure manipulates digital content associated with the physical objects in response to detected gestures.

34. An apparatus comprising:

a processing device receiving data representing at least two physical objects, each carrying a recognizable pattern, positioned within a three-dimensional (3D) input space; and

memory storing computer program code executable by the processing device, the processing device communicating with the memory and executing the program code causing the apparatus to;

recognize the patterns carried by the at least two physical objects in the 3D input space; and

associate the patterns to the same digital content.

35. An interactive input system comprising:

computing structure; and

an input device detecting a physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes said output of said input device to;

recognize said pattern carried by said physical object in said 3D input space;

determine which digital content object in a presented image is closest to a marker associated with said physical object that is in said image; and

select the digital content object closest to said marker to associate the digital content object with said marker.

36. The interactive input system of claim 35, wherein said computing device determines a distance between each digital content object and said marker.

37. The interactive input system of claim 36, wherein said computing device determines a first geometric center of said marker, and determines a second geometric center of each of said at least one of digital content objects, and measures the distance between said first geometric center and each second geometric center.

38. The interactive input system of claim 37, wherein said computing structure further determines if said marker is positioned from said digital content within a threshold distance.

39. A method comprising:

detecting a physical object carrying a unique pattern within a three-dimensional (3D) input space;

recognizing said physical object based on said unique pattern carried thereby;

determining when said physical object is positioned within said 3D input space at a location to position a corresponding marker within a threshold distance of at least one digital content object in an image;

determining the velocity of said marker; and

selecting at least one of said digital content objects to associate with said marker when said velocity is less than a velocity threshold, and when said at least one of said digital content objects is within said threshold distance from said marker, whereby manipulation of said physical object within said 3D input space manipulates at least one of said digital content objects.

40. The method of claim 39, wherein said step of determining the velocity of said marker comprises projecting said marker on to a two-dimensional plane.

41. The method of claim 39, wherein said step of determining whether said distance between at least one of said digital content objects and said marker is within said threshold distance comprises:

calculating a geometric center of said marker; and

calculating a geometric center of each of said at least one of digital content objects.

42. The method of claim 41, further comprising:

maintaining a timer; and

setting a selection time threshold for selecting at least one of said digital content objects.

43. An apparatus comprising:

a processing device receiving data representing a physical object carrying a recognizable pattern and positioned within a three-dimensional (3D) input space; and

memory storing computer program code executable by said processing device, said processing device communicating with said memory and executing said program code causing said apparatus to;

present an image including at least one digital content object and a selection marker based on said recognizable pattern;

determine when said selection marker is positioned within a threshold distance of at least one of said digital content objects within said presented image;

determine at least one of said digital content objects closest to said selection marker; and

select at least one of said digital content objects closest to said selection marker to associate with said selection marker.

44. The apparatus according to claim 43, wherein said processing device generates an indication signifying said selection.

45. The apparatus according to claim 44, wherein said selection is a visual indicator.

46. An interactive input system comprising:

computing structure;

a display coupled to said computing structure, said display presenting a three-dimensional (3D) image including at least one 3D digital content object; and

an input device detecting a physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes said output of said input device to;

recognize said pattern carried by said physical object to determine a selection marker position in said 3D image;

determine at least one of said digital content objects closest to said selection marker position in said 3D image;

select at least one of said digital content objects closest to said selection marker position to associate with said selection marker.

47. An interactive input system comprising:

computing structure; and

an input device detecting at least one physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, wherein said computing structure processes the output of the input device to;

recognize the pattern carried by the at least one physical object in the 3D input space;

detect manipulation of the at least one physical object in the 3D input space;

recognize gestures in response to detected physical object manipulations; and

manipulate digital content associated with the physical object in response to detected gestures.

48. The interactive input system of claim 47, wherein said computing structure in response to a detected gesture one of (i) changes associated digital content, (ii) merges associated digital content, (iii) clones associated digital content, (iv) disassociates digital content, (v) associates digital content, and (vi) rotates digital content.