System for portable tangible interaction

• US 9,262,015 B2
• Filed: 06/28/2010
• Issued: 02/16/2016
• Est. Priority Date: 06/28/2010
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Avrahami, Daniel
• Primary Examiner(s): Watko, Julie Anne
• Application Number:
  US12/824,448
• Publication Number:
  US 20110316767A1
• Time in Patent Office: 2,059 Days
• Field of Search: 345/156, 345173-184, 348/135, 348/157, 348/159, 353/52, 353/28, 463/4, 463/14, 463/48
• US Class Current: 1/1
• CPC Class Codes: A63F 2009/2435 using a video camera
• CPC Class Codes: A63F 3/00094 Ticktacktoe
• CPC Class Codes: A63F 3/00643 Electric board games; Electric features of board games electric word or number games A63F3/0421; computer chess G06F; electric raffle games A63F3/081
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Embodiments of the invention describe a system utilizing at least one camera and a display to create an object and context aware system. Embodiments of the invention may utilize the camera to sense a system'"'"'s surroundings and use recognition logic or modules to detect and recognize objects on and around the system. System applications may further act on the sensed data and use the display of the system to provide visual feedback and interactive elements as a means to interact with the system user.

Claims

1. A mobile tablet computer system comprising:

a tablet computer chassis;

a touchscreen display contained within the chassis, the touchscreen display including a surface with a surface linear polarizing filter;

a first arm and a second arm, a first end of the first arm being coupled with the chassis at a first location and a first end of the second arm being coupled with the chassis at a second location different from the first location, the first and second arms being extended upright from the first and second locations respectively relative to the touchscreen display surface;

a first camera positioned for a first camera view to capture visual data within a first target area, the first target area to include a first sub-area that includes the touchscreen display surface and a second sub-area that does not include the touchscreen display surface, the second sub-area being outside of and not on top of the touchscreen display surface, wherein the first camera is coupled with a second end of the first arm and includes a first linear polarizing filter to block view of the touchscreen display by the first camera;

a second camera positioned for a second camera view different from the first camera view to capture visual data within a second target area, the second target area to include a first sub-area that includes the touchscreen display surface and a second sub-area that does not include the touchscreen display surface, the second sub-area being outside of and not on top of the touchscreen display surface, wherein the second camera is coupled with a second end of the second arm and includes a second linear polarizing filter to block view of the touchscreen display by the second camera, the second sub-area of the first target area and the second sub-area of the second target area including an overlapping target area visible by both the first camera view and the second camera view, wherein the mobile tablet computer system is to be calibrated such that an object viewed by first camera view and the second camera view in the overlapping target area is reported as a single object;

a processor; and

software to be executed via the processor to;

generate a first graphical image to be displayed on the touchscreen display,

identify, from the captured visual data from the first camera and the second camera, a physical object other than the touchscreen display surface that is within the first target area visible to the first camera or the second target area visible to the second camera, including the overlapping target area visible to both the first camera and the second camera,

determine coordinates of a location of the detected physical object using the first view of the first camera, the second view of the second camera, or both the first view of the first camera and the second view of the second camera,

process an interaction between the physical object and the first graphical image, said interaction being based upon one or more of location, height, velocity, acceleration, gesture, and physical orientation of the physical object in the overlapping target area with respect to the first graphical image, and

generate and display a second graphical image on the touchscreen display, the second graphical image including a graphical representation of the interaction between the physical object and the first graphical image.

2. The mobile tablet computer system of claim 1, the software to further continuously monitor the first and second target areas, including the overlapping target area, to identify physical objects within the first and second target areas.

3. The mobile tablet computer system of claim 1, the software to further receive a command to monitor the first and second target areas, including the overlapping target area.

4. The mobile tablet computer system of claim 1, wherein the software is to generate the second graphical image based, at least in part, on at least one of a distance, an angle, and an orientation between the physical object and the touchscreen display surface.

5. The mobile tablet computer system of claim 1, wherein the apparatus further comprises:

a bezel surrounding the touchscreen display surface;

wherein the second sub-area of the first and second target areas includes the bezel surrounding the touchscreen display surface.

6. The mobile tablet computer system of claim 1, wherein the second sub-area of the first and second sub-areas includes an area of a supporting surface of a platform.

7. The mobile tablet computer system of claim 1, wherein the first and second arms are rotationally extendable arms.

8. The mobile tablet computer system of claim 1, wherein the representation of the interaction includes a virtual image of the physical object on the touchscreen display.

9. The mobile tablet computer system of claim 8, wherein the virtual image of the physical object includes virtual motion of the virtual image, the virtual motion representing a physical motion of the physical object.

10. The mobile tablet computer system of claim 1, wherein the first graphical image is not viewable by the first camera or the second camera because of alignment of the first camera linear polarizing filter and the second linear polarizing filter in relation to the surface linear polarizing filter.

11. The mobile tablet computer system of claim 1, wherein the first linear polarizing filter and the second linear polarizing filter are rotatable for alignment in relation to the surface linear polarizing filter.

12. An article of manufacture comprising a machine-readable non-transitory storage medium that provides instructions that, if executed by the machine, will cause the machine to perform operations comprising:

capturing visual data within an overlapping target area via a first camera and a second camera, the first camera and second camera being coupled respectively to first arm coupled with a chassis of a mobile tablet computer system at a first location and a second arm coupled with the chassis of the mobile tablet computer system at a second location different from the first location, the first and second arms being extended upright from the first and second locations respectively relative to a surface of a touchscreen display of the mobile tablet computer system, the touchscreen display including a surface linear polarizing filter, the first camera being positioned for a first view and including a first linear polarizing filter to block view of the touchscreen display by the first camera, the second camera being positioned for a second view different from the first view and including a second linear polarizing filter to block view of the touchscreen display by the second camera, each of a first target area for the first camera and a second target for the second camera to include a first sub-area that includes the touchscreen display surface of the mobile computer system and a second sub-area that does not include the touchscreen display surface, the second sub-areas of the first and second target areas being outside of and not on top of the touchscreen display surface, the overlapping target area being an area overlapping between the second sub-area of the first target area for the first camera and the second sub-area of second target area for the second camera, wherein the mobile tablet computer system is to be calibrated such that an object viewed by first camera view and the second camera view in the overlapping area is reported as a single object;

generating a first graphical image;

displaying the graphical image on the touchscreen display;

identifying, from the captured visual data from the first camera and the second camera, a physical object other than the touchscreen display surface that is within the overlapping target area visible to the first camera, the second camera, or both the first and second cameras;

determining coordinates of a location of the detected physical object in the overlapping target area using the first view of the first camera, the second view of the second camera, or both the first view of the first camera and the second view of the second camera;

processing an interaction between the physical object and the first graphical image, said interaction being based upon one or more of location, height, velocity, acceleration, gesture, and physical orientation of the physical object in the overlapping target area with respect to the first graphical image; and

generating and displaying a second graphical image on the touchscreen display, the second graphical image including a graphical representation of the interaction between the physical object and the first graphical image.

13. The article of manufacture of claim 12, the operations further comprising continuously monitoring the first and second target areas, including the overlapping target area, to identify physical objects within the target areas.

14. The article of manufacture of claim 13, the operations further comprising receiving a command to monitor the first and second target areas, including the overlapping target area.

15. The article of manufacture of claim 13, wherein generating the second graphical image is based, at least in part, on at least one of a distance, an angle, and an orientation between the physical object and the touchscreen display surface.

16. A method comprising:

capturing visual data within an overlapping target area via a first camera and a second camera, the first camera and second camera being coupled respectively to first arm coupled with a chassis of a mobile tablet computer system at a first location and a second arm coupled with the chassis of the mobile tablet computer system at a second location different from the first location, the first and second arms being extended upright from the first and second locations respectively relative to a surface of a touchscreen display of the mobile tablet computer system, the touchscreen display including a surface linear polarizing filter, the first camera being positioned for a first view and including a first linear polarizing filter to block view of the touchscreen display by the first camera, the second camera being positioned for a second view different from the first view and including a second linear polarizing filter to block view of the touchscreen display by the second camera, each of a first target area for the first camera and a second target area for the second camera to include a first sub-area that includes the touchscreen display surface of the mobile computer system and a second sub-area that does not include the touchscreen display surface, the second sub-areas of the first and second target areas being outside of and not on top of the touchscreen display surface, the overlapping target area being an area overlapping between the second sub-area of the first target area for the first camera and the second sub-area of second target area for the second camera, wherein the mobile tablet computer system is to be calibrated such that an object viewed by first camera view and the second camera view in the overlapping area is reported as a single object;

generating a first graphical image;

displaying the graphical image on the touchscreen display;

identifying, from the captured visual data from the first camera and the second camera modified by the background/foreground mask, a physical object other than the touchscreen display surface that is within the overlapping target area visible to the first camera, the second camera, or both the first and second cameras;

determining coordinates of a location of the detected physical object in the overlapping target area using the first view of the first camera, the second view of the second camera, or both the first view of the first camera and the second view of the second camera;

processing an interaction between the physical object and the first graphical image, said interaction being based upon one or more of location, height, velocity, acceleration, gesture, and physical orientation of the physical object in the overlapping target area with respect to the first graphical image; and

generating and displaying a second graphical image on the touchscreen display, the second graphical image including a graphical representation of the interaction between the physical object and the first graphical image.