Electronic Devices Including Interactive Displays Implemented Using Cameras and Related Methods and Computer Program Products

• US 20110285669A1
• Filed: 06/29/2010
• Published: 11/24/2011
• Est. Priority Date: 05/21/2010
• Status: Abandoned Application

• Current Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Lassesson, Kristian, Sassi, Jari
• Application Number:
  US12/825,545
• Publication Number:
  US 20110285669A1
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An electronic device is provided including a housing; an interactive display connected to the housing; a frame associated with the interactive display; at least one camera coupled to the interactive display and frame; and a position determination circuit coupled to the camera and the interactive display. The position determination circuit is configured to determine a position of an object in proximity to the interactive display based on images captured by the at least one camera.

Claims

1. An electronic device comprising:

a housing;

an interactive display connected to the housing;

a frame associated with the interactive display;

at least one camera coupled to the interactive display and frame; and

a position determination circuit coupled to the camera and the interactive display, the position determination circuit configured to determine a position of an object in proximity to the interactive display based on images captured by the at least one camera.

2. The electronic device of claim 1, wherein the at least one camera comprises a single camera, the electronic device further comprising:

at least two mirrors attached to the frame, the position determination circuit being further configured to determine a position of the object with respect to the interactive display based on images obtained from the single camera and the at least two mirrors.

3. The electronic device of claim 2, wherein the position determination circuit is further configured to:

capture and store a background image of the interactive display using the single camera before a user interacts with the interactive display;

obtain a plurality of images using the single camera and the at least two mirrors;

subtract the stored background image from each of the obtained plurality of images to provide a plurality of subtracted images; and

calculate the position of the object on the interactive display based on the plurality of subtracted images.

4. The electronic device of claim 3, wherein the position determination circuit is configured to calculate the position of the object by:

calculating first and second angles for each of the plurality of subtracted images, the first angle corresponding to a start position of the object and the second angle corresponding to a stop position of the object; and

calculating coordinates of the object with respect to the interactive display based on the calculated first and second angles for each of the plurality of subtracted images.

5. The electronic device of claim 1, wherein the at least one camera comprises two cameras attached to the frame, the position determination circuit being further configured to determine a position of the object with respect to the interactive display based on images obtained from the at least two cameras.

6. The electronic device of claim 5, wherein the position determination circuit is further configured to:

capture and store a background image of the interactive display using the two cameras before a user interacts with the interactive display;

obtain a plurality of images with two cameras;

subtract the stored background image from each of the obtained plurality of images to provide a plurality of subtracted images; and

calculate the position of the object with respect to the interactive display based on the plurality of subtracted images.

7. The electronic device of claim 5, wherein the position determination circuit is further configured to:

obtain a first image using a first of the two cameras and calculate first and second angles based on the obtained first image and the position of the object with respect to the interactive display;

obtain a second image using a second of the two cameras and calculate third and forth angles based on the obtained second image and the position of the object with respect to the interactive display;

compare the first and second calculated angles of the first obtained image to the third and forth angles of the second obtained image to determine an intersection point; and

determine if the intersection point is located on or above the interactive display.

8. The electronic device of claim 7, wherein the position determination circuit is further configured to:

detect contact of the object on the interactive display; and

calculate coordinates of the object on the interactive display based on the obtained first and second images, the calculated first through fourth angles and the determined intersection point.

9. The electronic device of claim 1, wherein the at least one camera comprises a single camera and wherein the interactive display has a reflective surface, the position determination circuit being further configured to determine a position of the object with respect to the interactive display based on images obtained from the single camera and a reflection of the object in the reflective surface of the interactive display as viewed by the single camera.

10. The electronic device of claim 1, wherein the at least one camera comprises a single camera positioned inside the housing of the electronic device, the position determination circuit being further configured to determine a position of the object on the interactive display based on images obtained from the single camera positioned inside the housing of the electronic device.

11. The electronic device of claim 10, wherein the position determination circuit is configured to:

obtain an image of the object using the single camera positioned inside the housing of the electronic device;

calculate a start angle and a stop angle of the image based on the position of the object with respect to the interactive display;

calculate frame angles between two known edges of the frame and the object with respect to the interactive display;

calculate a distance between the object on the interactive display and the camera using the calculated start and stop angles and frame angles; and

calculate the position and size of the object on the interactive display based on the calculated distance, start and stop angles and frame angles.

12. A method of controlling an interactive display of an electronic device, the electronic device including a housing;

an interactive display connected to the housing;

a frame associated with the interactive display; and

at least one camera coupled to the interactive display and frame, the method comprising;

determining a position of an object in proximity to the interactive display based on images captured by the at least one camera.

13. The method of claim 12, wherein the at least one camera comprises a single camera and wherein the electronic device further comprises at least two mirrors attached to the frame, the method further comprising:

determining a position of the object with respect to the interactive display based on images obtained from the single camera and the at least two mirrors.

14. The method of claim 13 further comprising:

capturing and storing a background image of the interactive display using the single camera before a user interacts with the interactive display;

obtaining a plurality of images using the single camera and the at least two mirrors;

subtracting the stored background image from each of the obtained plurality of images to provide a plurality of subtracted images; and

calculating the position of the object on the interactive display based on the plurality of subtracted images, wherein calculating the position of the object comprises;

calculating first and second angles for each of the plurality of subtracted images, the first angle corresponding to a start position of the object and the second angle corresponding to a stop position of the object; and

calculating coordinates of the object with respect to the interactive display based on the calculated first and second angles for each of the plurality of subtracted images.

15. The method of claim 12, wherein the at least one camera comprises two cameras attached to the frame, the method further comprising determining a position of the object with respect to the interactive display based on images obtained from the at least two cameras.

16. The method of claim 15 further comprising:

capturing and storing a background image of the interactive display using the two cameras before a user interacts with the interactive display;

obtaining a plurality of images using the two cameras;

subtracting the stored background image from each of the obtained plurality of images to provide a plurality of subtracted images; and

calculating the position of the object with respect to the interactive display based on the plurality of subtracted images.

17. The method of claim 15 further comprising:

obtaining a first image using a first of the two cameras and calculating first and second angles based on the obtained first image and the position of the object with respect to the interactive display;

obtaining a second image using a second of the two cameras and calculating third and forth angles based on the obtained second image and the position of the object with respect to the interactive display;

comparing the first and second calculated angles of the first obtained image to the third and forth angles of the second obtained image to determine an intersection point;

determining if the intersection point is located on or above the interactive display;

detecting contact of the object on the interactive display; and

calculating coordinates of the object on the interactive display based on the obtained first and second images, the calculated first through fourth angles and the determined intersection point.

18. The method of claim 12, wherein the at least one camera comprises a single camera and wherein the interactive display has a reflective surface, the method further comprising:

determining a position of the object with respect to the interactive display based on images obtained from the single camera and a reflection of the object in the reflective surface of the interactive display as viewed by the single camera.

19. The method of claim 12, wherein the at least one camera comprises a single camera positioned inside the housing of the electronic device, the method further comprising:

determining a position of the object on the interactive display based on images obtained from the single camera positioned inside the housing of the electronic device, wherein determining a position comprises;

obtaining an image of the object using the single camera positioned inside the housing of the electronic device;

calculating a start angle and a stop angle of the image based on the position of the object with respect to the interactive display;

calculating frame angles between two known edges of the frame and the object with respect to the interactive display;

calculating a distance between the object on the interactive display and the camera using the calculated start and stop angles and frame angles; and

calculating the position and size of the object on the interactive display based on the calculated distance, start and stop angles and frame angles.

20. A computer program product for controlling an interactive display of an electronic device, the electronic device including a housing;

an interactive display connected to the housing;

a frame associated with the interactive display; and

at least one camera coupled to the interactive display and frame, the computer program product comprising;

a computer-readable storage medium having computer-readable program code embodied in said medium, said computer-readable program code comprising;

computer-readable program code configured to determine a position of an object in proximity to the interactive display based on images captured by the at least one camera.