Virtualization of tangible interface objects

• US 9,354,716 B1
• Filed: 09/01/2015
• Issued: 05/31/2016
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Tangible Play, Inc. (Think & Learn Pvt Ltd.)
• Original Assignee: Tangible Play, Inc. (Think & Learn Pvt Ltd.)
• Inventors: Sharma, Pramod Kumar, Scholler, Jerome
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/842,777
• Time in Patent Office: 273 Days
• Field of Search: 345156-160, 345173-178, 345204-215
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

An example system includes a computing device located proximate to a physical activity surface, a video capture device, and a detector. The video capture device is coupled for communication with the computing device and is adapted to capture a video stream that includes an activity scene of the physical activity surface and one or more interface objects physically intractable with by a user. The detector processes the video stream to detect the one or more interface objects included in the activity scene, to identify the one or more interface objects that are detectable, to generate one or more events describing the one or more interface objects, and to provide the one or more events to an activity application configured to render virtual information on the one or more computing devices based on the one or more events.

Claims

1. A physical activity surface visualization system, comprising:

a detachable camera adapter that, when attached to a computing device, covers at least a portion of a field of view of a camera of the computing device and redirects at least a portion of the field of view of the camera to a physical activity surface located proximate to the computing device; and

a detector that receives a video stream from the camera adapted by the detachable camera adapter to capture a scene of the physical activity surface, detects one or more physical objects placed in the scene, identifies the one or more physical objects placed in the scene, and renders virtual information on a display of the computing device based on the one or more identified physical objects placed in the scene.

2. The physical activity surface visualization system of claim 1, wherein the camera is integrated with the computing device, the computing device includes a body, and the detachable camera adapter attaches to at least a portion of the body of the computing device to cover at least the portion of the field of view of the camera.

3. The physical activity surface visualization system of claim 1, wherein the computing device is a tablet computer, a smartphone, a laptop computer, or a desktop computer.

4. The physical activity surface visualization system of claim 1, wherein the detachable camera adapter includes a housing having a slot that is configured to receive an edge of a body of the computing device.

5. The physical activity surface visualization system of claim 1, wherein the detachable camera adapter includes one or more mirrors for redirecting the field of view of the camera to the physical activity surface.

6. The physical activity surface visualization system of claim 1, further comprising:

a stand placeable on the physical activity surface and configured to receive and sturdily retain the computing device and positioning the computing device for viewing and interaction with by a user.

7. The physical activity surface visualization system of claim 6, wherein the stand includes a channel configured to receive and secure the detachable camera adapter when the detachable camera adapter is detached from the computing device.

8. The physical activity surface visualization system of claim 7, wherein walls forming the channel and a housing of the detachable camera adapter both include compatible magnetic components that engage when the detachable camera adapter is placed in the channel.

9. The physical activity surface visualization system of claim 6, wherein the stand includes a slot formed in a top side, the slot having a size compatible with a portion of a body of the computing device, the slot being configured to receive and secure the portion of the body of the computing device.

10. The physical activity surface visualization system of claim 1, wherein, responsive to displaying the virtual information on the display of the computing device, a user physically interacts with the one or more physical objects in the scene, and the detector detects the physical interaction with the one or more physical objects in the scene and updates the virtual information displayed on the display based on the detected physical interaction.

11. The physical activity surface visualization system of claim 1, wherein, responsive to displaying the virtual information on the display of the computing device, a user physically interacts with the one or more physical objects in the scene, and the detector detects the physical interaction with the one or more physical objects in the scene and updates the virtual information displayed on the display based on the detected physical interaction.

12. The physical activity surface visualization system of claim 1, wherein the detachable camera adapter splits the field of view of the camera into two scenes by redirecting a first portion of the field of view of the camera toward the physical activity surface and redirecting a second portion of the field of view of the camera toward a user facing the computing device.

13. The physical activity surface visualization system of claim 1, wherein the physical activity surface is located in front of the display of the computing device or behind the display of the computing device.

14. A physical activity surface visualization system, comprising:

a detachable camera adapter that, when attached to a computing device, covers at least a portion of a field of view of a camera of the computing device and redirects at least a portion of the field of view of the camera to a physical activity surface located proximate to the computing device; and

a stand placeable on the physical activity surface and configured to receive and position a display of the computing device for viewing and interaction with by a user.

15. The physical activity surface visualization system of claim 14, wherein the camera is integrated with the computing device, the computing device includes a body, and the detachable camera adapter attaches to at least a portion of the body of the computing device to cover at least a portion of the field of view of the camera.

16. The physical activity surface visualization system of claim 14, wherein the computing device is a tablet computer, a smartphone, a laptop computer, or a desktop computer.

17. The physical activity surface visualization system of claim 14, wherein the detachable camera adapter includes a housing having a slot that is configured to receive an edge of a body of the computing device.

18. The physical activity surface visualization system of claim 14, wherein the detachable camera adapter includes one or more mirrors for redirecting the field of view of the camera to the physical activity surface.

19. The physical activity surface visualization system of claim 14, wherein, the stand is placeable on the physical activity surface and configured to receive and sturdily retain the computing device and positioning the computing device for viewing and interaction with by the user.

20. The physical activity surface visualization system of claim 14, further comprising:

a detector that receives a video stream from the camera adapted by the detachable camera adapter to capture a scene of the physical activity surface, detects one or more physical objects placed in a scene, identifies the one or more physical objects placed in the scene, and renders virtual information on the display of the computing device based on the one or more identified physical objects placed in the scene.

21. The physical activity surface visualization system of claim 20, wherein, responsive to displaying the virtual information on the display of the computing device, the user physically interacts with the one or more physical objects in the scene, and the detector detects a physical interaction with the one or more physical objects in the scene and updates the virtual information displayed on the display based on the detected physical interaction.

22. The physical activity surface visualization system of claim 14, wherein the detachable camera adapter splits the field of view of the camera into two scenes by redirecting a first portion of the field of view of the camera toward the physical activity surface and redirecting a second portion of the field of view of the camera toward the user facing the computing device.

23. The physical activity surface visualization system of claim 14, wherein the physical activity surface is located in front of the display of the computing device or behind the display of the computing device.

24. The physical activity surface visualization system of claim 14, wherein the stand includes a channel configured to receive and secure the detachable camera adapter when the detachable camera adapter is detached from the computing device.

25. The physical activity surface visualization system of claim 14, wherein walls forming the channel and a housing of the detachable camera adapter both include compatible magnetic components that engage when the adapter is placed in the channel.

26. The physical activity surface visualization system of claim 14, wherein the stand includes a slot formed in a top side, the slot having a size compatible with a portion of a body of the computing device, the slot being configured to receive and secure the portion of the body of the computing device.