INTERACTIVE INPUT SYSTEM HAVING A 3D INPUT SPACE

• US 20140129990A1
• Filed: 10/03/2011
• Published: 05/08/2014
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Xin, Min, Antonyuk, Viktor, Tse, Edward, Lai, Henry, Xing, Zhaohui, Benner, Erik, Banerjee, Shymmon, Wang, Luqing, Loken, Mark Cameron, Ma, Danny, Diaz-Maino, Roberto Arturo, Ginez, Esteban
• Granted Patent: US 9,619,104 B2
• Time in Patent Office: Days
• US Class Current: 715/849
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

Computerized methods and interactive input systems for manipulation of 3D objects are disclosed. An image of 3D object is displayed on a 2D interactive surface, and input is received and is interpreted for manipulating the 3D object. When the 3D object is selected, rotation control handles indicating available rotation directions are displayed. In one embodiment, the method comprises capturing images of a 3D input space, recognizing at least one object in the images, and comparing the recognized objects in the images to determine a difference therebetween based on a difference threshold. Depending on the outcome of the comparison, the recognized objects are emerged and associated with digital content, or only one of the recognized objects is maintained and associated with digital content.

Claims

1. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

displaying a two-dimensional (2D) image of a three-dimensional (3D) object on a display;

defining at least one region on said display for controlling the rotation of said 3D object about a defined axis, said axis being parallel to a display surface of said display;

receiving input applied to said at least one region;

determining, from said input, a rotation direction; and

modifying said displayed 2D image by rotating said 3D object about the defined axis along said rotation direction.

2. The method of claim 1 further comprising:

displaying at least one visual indication in said at least one region.

3. The method of claim 2 wherein said at least one region comprises a first region and a second region, for controlling the rotation of said 3D object about a defined first axis and a defined second axis, respectively, the first and second axes being parallel to said display surface and being orthogonal.

4. The method of claim 3 wherein said at least one visual indication comprises arcs, each arc indicating a respective rotation direction about a respective one of said first and second axes.

5. The method of claim 4 wherein each of said arcs comprises an arrow at each end thereof.

6. An interactive input system comprising:

a display comprising a display surface on which a two-dimensional (2D) image of a three-dimensional (3D) object and at least one region for controlling rotation of said 3D object about a defined axis are presented; and

computing structure communicating with said display, said computing structure, in response to input applied to said at least one region, determining a rotation direction and modifying said 2D image by rotating said 3D object about said defined axis in said rotation direction.

7. The interactive input system of claim 6 wherein at least one visual indication is presented in said at least one region.

8. The interactive input system of claim 7 wherein said at least one region comprises a first region and a second region, for controlling the rotation of said 3D object about a defined first axis and a defined second axis, respectively, the first and second axes being parallel to said display surface and being orthogonal.

9. The interactive input system of claim 8 wherein said at least one visual indication comprises arcs, each arc indicating a respective rotation direction about a respective one of said first and second axes.

10. The interactive input system of claim 9 wherein each of said arcs comprises an arrow at each end thereof.

11. An interactive input system comprising:

computing structure;

a display in communication with said computing structure, said display presenting an image comprising digital content; and

an input device detecting at least one physical object carrying a recognizable pattern within a three-dimensional (3D) input space and providing output to said computing structure, said computing structure being configured to process the output of the input device to;

recognize the pattern carried by the at least one physical object in the 3D input space; and

modify the image presented on the display based on at least one of the detected state of the at least one physical object and the proximity of a user to a designated location.

12. The interactive input system of claim 11 wherein the designated location is the display.

13. The interactive input system of claim 11 wherein the computing structure transitions a displayed object from a three-dimensional hover position to a position on the plane of a display surface of the display when the user is proximate the designated location.

14. The interactive input system of claim 13 wherein the image comprises interactive tools for enabling user interaction with the displayed object.

15. The interactive input system of claim 14 wherein the interactive tools comprise at least one of object manipulation tools and object annotation tools.

16. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

presenting an image comprising digital content on a display surface;

recognizing a pattern carried by a physical object placed within a three-dimensional input space; and

modifying the image based on at least one of the detected state of the at least one physical object and proximity of a user to a designated location.

17. The method of claim 16 wherein the designated location is the display.

18. The method of claim 16 further comprising transitioning a displayed object from a three-dimensional hover position to a position on the plane of said display surface when the user is proximate the designated location.

19. The method of claim 18 wherein the image comprises interactive tools for enabling user interaction with the displayed object.

20. The method of claim 19 wherein the interactive tools comprise at least one of object manipulation tools and object annotation tools.

21. A computerized method comprising:

displaying an enclosure around the periphery of a displayed graphic object upon selection of the graphic object, the enclosure having control elements associated therewith; and

manipulating the graphic object according to user interaction with at least one of said control elements.

22. The method of claim 21 wherein said control elements comprise at least one of rotation control elements and resizing control elements, and wherein said manipulating comprises at least one of rotating and resizing the displayed graphic object.

23. The method of claim 22 further comprising, upon user interaction with one of the rotation control elements, removing the enclosure while maintaining the rotation control element.

24. The method of claim 22 wherein said control elements comprise a plurality of rotation control elements each associated with a different respective axis, and wherein said manipulating comprises rotating the graphic object about the axis associated with the rotation control element interacted upon by the user.

25. The method of claim 24 wherein said control elements further comprise a free rotation tool and wherein said manipulating comprises rotating the displayed graphic object in accordance with user interaction with said free rotation tool.

26. The method of claim 25 wherein said free rotation tool comprises a snap-to-axis function.

27. The method of claim 21 further comprising changing the visual appearance of the control elements upon resizing of the graphic object or the enclosure.

28. An interactive input system comprising:

a display comprising a display surface on which at least one graphic object is presented; and

computing structure communicating with said display, said computing structure being configured to cause said display to present an enclosure having control elements associated therewith around the periphery of a selected graphic object and to manipulate the graphic object according to user interactions with at least one control element of said enclosure.

29. The interactive input system of claim 28 wherein said control elements comprise at least one of rotation control elements and resizing control elements.

30. The interactive input system of claim 29 wherein said computing structure is configured, upon user interaction with one of the rotation control elements, to remove the enclosure while maintaining the rotation control element.

31. The interactive input system of claim 28 wherein said control elements comprise a plurality of rotation control elements each associated with a different respective axis, and wherein said computing structure is configured to rotate the graphic object about the axis associated with the rotation control element interacted upon by the user.

32. The interactive input system of claim 31 wherein said control elements further comprise a free rotation tool and wherein said computing structure is configured to rotate the displayed graphic object in accordance with user interaction with said free rotation tool.

33. The interactive input system of claim 32 wherein said free rotation tool comprises a snap-to-axis function.

34. The interactive input system of claim 28 wherein said computing structure is further configured to change the visual appearance of the control elements upon resizing of the graphic object or the enclosure.

58. The method of claim 29 further comprising:

superimposing said digital content on the associated maintained recognized object or merged recognized objects.

35. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

designating one or more anchor points on a displayed graphic object;

anchoring a label box to one or more said anchor points; and

annotating said label box.

36. The method of claim 35 further comprising expanding or collapsing said label box according to the size of said graphic object.

37. An interactive input system comprising:

a display comprising a display surface; and

computing structure communicating with said display, said computing structure being configured to;

designate one or more anchor points on a graphic object presented on said display surface;

anchor a label box to one or more said anchor points; and

annotate said label box.

38. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

capturing an image of a three-dimensional (3D) input space and recognizing at least one object therein;

capturing another image of said 3D input space and recognizing at least one object therein;

comparing the captured images to determine a difference therebetween;

when said difference is greater than a difference threshold, merging the recognized objects to form a composite recognized object and storing said composite recognized object;

when said difference is not greater than said difference threshold, storing one of said recognized objects and discarding the other of said recognized objects; and

associating either said stored composite recognized object or said stored one recognized object with digital content.

39. The method of claim 38 wherein during said one recognized object storing, the recognized object having the most recognized content is stored.

40. The method of claim 38 wherein said comparing comprises comparing corresponding pixels of said images.

41. The method of claim 40 wherein during said comparing, at least one of intensity and color of said corresponding pixels is compared.

42. The method of claim 38 further comprising associating a link to the stored one recognized object or stored composite recognized object.

43. The method of claim 38 further comprising:

superimposing said digital content on the associated stored one recognized object or stored composite recognized object.

44. The method of claim 38 wherein each said at least one object is one of a textual object and a non-textual object.

45. The method of claim 42 further comprising activating said link in response to user input.

46. An interactive input system comprising:

an input device capturing images of a three-dimensional (3D) input space; and

computing structure communicating with said input device, said computing structure being configured to;

recognize at least one object in captured images;

compare captured images to determine differences therebetween;

when said differences are greater than a difference threshold, merge the recognized objects to form a composite recognized object and store said composite recognized object;

when said differences are not greater than said difference threshold, store one of said recognized objects and discard the remaining recognized objects; and

associate either said stored composite recognized object or said stored one recognized object with digital content.

47. The interactive input system of claim 46 wherein the one recognized object that is stored is the recognized object having the most recognized content.

48. The interactive input system of claim 46 wherein said computing structure is configured to compare corresponding pixels of said images.

50. The interactive input system of claim 46 wherein said computing structure is further configured to associate a link to the stored one recognized object or stored composite recognized object.

51. The interactive input system of claim 46 wherein said computing structure is further configured to superimpose said digital content on the associated stored one recognized object or stored composite recognized object.

52. The interactive input system of claim 46 wherein each said at least one object is one of a textual object and a non-textual object.

53. The interactive input system of claim 50 wherein said computing structure is further configured to activate said link in response to user input.

49. The interactive input system of 48 wherein computing structure is configured to compare at least one of intensity and color of said corresponding pixels.

54. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

capturing images of a three-dimensional (3D) input space;

recognizing at least one object in each image;

comparing a pair of said images to determine a difference therebetween;

when said difference is greater than a difference threshold, merging said recognized objects and associating digital content therewith; and

when said difference is not greater than said difference threshold, maintaining at least one recognized object from only one of said images in said pair and associating digital content therewith.

55. The method of claim 54 wherein said maintained recognized object is the recognized object having the most recognized content.

56. The method of claim 54 wherein said comparing comprises comparing corresponding pixels of said images.

57. The method of claim 54 further comprising associating a link with the maintained recognized object or the merged recognized objects.

59. The method of claim 54 wherein each said at least one object is one of a textual object and a non-textual object.

60. An interactive input system comprising:

computing structure; and

an input device capturing an image of at least one physical object carrying recognizable text within a three-dimensional (3D) input space and providing image data to said computing structure, wherein said computing structure processes the image data and is configured to;

present said image on a display coupled to said computing structure;

recognize at least one object carried by the at least one physical object in the 3D input space;

associate at least a portion of said recognized at least one object with a link to digital content;

receive user input representing activation of said link; and

superimpose said digital content on said image.

61. The interactive input system of claim 60, wherein said digital content comprises at least one 3D digital object.

62. A non-transitory computer-readable medium having computer-executable instructions embodied thereon that, when executed by a computing device, facilitate a method comprising:

presenting an image on a display coupled to computing structure;

recognizing at least one object carried by at least one physical object in a three-dimensional (3D) input space;

associating at least a portion of said recognized at least one object with a link to digital content;

receiving user input representing activation of said link; and

superimposing said digital content on said image.

63. An image capturing apparatus comprising:

computing structure;

a camera device capturing images of a three-dimensional (3D) input space coupled to said computing structure, wherein said 3D input space comprises a two-dimensional (2D) plane; and

light projection structure coupled to said computing structure, said computing structure receiving images captured by said input device, recognizing objects within said images, associating recognized objects with digital content, and projecting said digital content to said 2D plane, wherein said digital content is manipulatable on said 2D plane.

64. The image capturing apparatus of claim 63, wherein said light projection structure is a projector.

65. (canceled)