Methods and systems for controlling a virtual interactive surface and interactive display systems

• US 9,880,728 B2
• Filed: 12/30/2013
• Issued: 01/30/2018
• Est. Priority Date: 01/08/2013
• Status: Active Grant

• Current Assignee: FOGALE nanotech
• Original Assignee: Ayotle SAS
• Inventors: Ybanez Zepeda, Jos Alonso
• Primary Examiner(s): Ell, Matthew
• Application Number:
  US14/759,787
• Publication Number:
  US 20150355811A1
• Time in Patent Office: 1,492 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Systems and methods for controlling a virtual interactive surface in a scene include a surface for displaying information and one or several virtual sensors positioned with respect to the surface. The virtual sensor includes data representing a geometric form, data representing a position in the scene, and data representing one or several trigger conditions. A data representation of the scene is captured and a determination whether a virtual sensor trigger condition is fulfilled is made, based on an analysis of the captured data representation in an area corresponding to the geometric form and position of the virtual sensor.

Claims

1. A method for controlling a virtual interactive surface in a scene, wherein the virtual interactive surface comprises a surface for displaying information and one or several virtual sensors positioned with respect to the surface, wherein a virtual sensor comprises data representing a geometric form, data representing a position in the scene and data representing one or several trigger conditions, the method comprising:

capturing a first data representation of the scene;

generating point cloud data representing surfaces of objects of the captured scene, the point cloud data comprising data representing non-empty points of a first volume area corresponding to the captured first data representation and respective associated position in the first volume area; and

determining whether one or several of the virtual sensor trigger conditions is/are fulfilled based on an analysis of the captured first data representation in a volume area corresponding to the geometric form and position of each virtual sensor,

wherein for a first virtual sensor said analysis comprises comparing a number of non-empty points whose position falls in the volume area corresponding to the geometric form and position of said first virtual sensor is greater to a triggering threshold.

2. The method according to claim 1, wherein the first data representation of the scene includes an image data representation of the scene.

3. The method according to claim 2, further comprising:

capturing a second data representation of the scene; and

configuring the geometric form and position of each virtual sensor based on an image generated from the captured second data representation.

4. The method according to claim 1, further comprising:

capturing a second data representation of the scene; and

configuring the geometric form and position of each virtual sensor based on an image generated from the captured second data representation.

5. The method according to claim 1, comprising analyzing point cloud data representing points in the volume area which correspond to the geometric form and position of the virtual sensor, wherein the analyzing comprises:

detecting the points of the point cloud which are positioned in the volume area corresponding to the geometric form and position of the virtual sensor;

determining an intensity value representing intensity amounts for some of the detected points; and

comparing the determined intensity value to a predetermined threshold.

6. The method according to claim 1, further comprising defining and configuring one or several virtual sensors in the scene using the first data representation of the scene.

7. The method according to claim 1, further comprising:

executing one or several command actions in response to determining whether one or several of the virtual sensor trigger condition is/are fulfilled.

8. The method according to claim 7, wherein the one or several commands comprises modifying the information displayed on the surface.

9. The method according to claim 1, wherein for a second virtual sensor said analysis comprises determining a reference point from the non-empty points whose position falls in a second volume area which corresponds to the geometric form and position of said second virtual sensor, said method comprising triggering a signal for selecting a region of the surface based on the relative position of the reference point with respect to each dimension of the second volume area.

10. The method according to claim 1, wherein said one or several virtual sensors comprises a first virtual sensor for triggering a signal for launching an application on a computer, a second virtual sensor for triggering a signal for selecting a region of the surface, and a third virtual sensor for triggering a signal for launching an action on the computer.

11. The method according to claim 1, wherein said one or several virtual sensors comprises a first virtual sensor for triggering a signal for launching on a computer one of an application and an action, a second virtual sensor for triggering a signal for selecting a region of the surface, and a third virtual sensor for activating and deactivating the first and the second virtual sensors.

12. The method according to claim 1, wherein the volume areas of the virtual sensors are rectangular parallelepipeds disposed parallel to the surface.

13. The method according to claim 12, wherein the volume areas of one virtual sensor contains the volume area of another virtual sensors.

14. The method according to claim 12, wherein the volume areas of one virtual sensor are arranged separately.

19. An interactive display system comprising:

a virtual interactive surface in a scene, wherein the virtual interactive surface comprises a surface for displaying information and one or several virtual sensors positioned with respect to the surface, wherein a virtual sensor comprises data representing a geometric form, data representing a position in the scene and data representing one or several trigger conditions;

a system for controlling the virtual interactive surface according to claim 11;

a display sub-system adapted to display information on the surface, operatively connected to the system for controlling the virtual interactive surface.

20. An interactive display system according to claim 19, wherein one or several virtual sensors comprise data representing a form adapted to cover the surface for displaying information.

21. An interactive display system according to claim 19, wherein the virtual interactive surface comprises several virtual sensors, each having a different trigger condition.

15. A system for controlling a virtual interactive surface in a scene, wherein the virtual interactive surface comprises a surface for displaying information and one or several virtual sensors positioned with respect to the surface, wherein a virtual sensor comprises data representing a geometric form, data representing a position in the scene and data representing one or several trigger conditions, the system comprising:

a scene capture sub-system configured to capture a scene and generate a first data representation of the scene;

a virtual sensor sub-system in operative communication with the scene capture sub-system, the virtual sensor sub-system comprising an interface for receiving data representing the first data representation of the scene, a processor, and a memory operatively coupled to the processor, wherein the virtual sensor sub-system is configured to perform a method for detecting activation of each virtual sensor in the scene, the method comprising;

receiving point cloud data representing surfaces of objects of the captured scene and comprising data representing non-empty points of a first volume area corresponding to the captured first data representation and respective associated position in the first volume area; and

determining whether one or several of the virtual sensor trigger condition is/are fulfilled based on an analysis of the captured first data representation in an area corresponding to the geometric form and position of the virtual sensor,

wherein for a first virtual sensor said analysis comprises comparing a number of non-empty points whose position falls in the volume area corresponding to the geometric form and position of said first virtual sensor is greater to a triggering threshold.

16. A system according to claim 15, wherein the virtual sensor sub-system includes functionality to define and configure one or several virtual sensors in the scene using the first data representation of the scene.

17. A system according to claim 16, further comprising:

a command sub-sub-system in operative communication with the virtual sensor sub-system, wherein the command sub-system is configured to perform a method for;

executing one or several command actions in response to determining whether one or several of the virtual sensor trigger condition is/are fulfilled.

18. A system according to claim 15, further comprising:

a command sub-sub-system in operative communication with the virtual sensor sub-system, wherein the command sub-system is configured to perform a method for;

executing one or several command actions in response to determining whether one or several of the virtual sensor trigger condition is/are fulfilled.

22. A non-transitory computer-readable storage medium storing a computer program that, when executed, causes a system comprising a processor operatively coupled with a memory, to perform a method for controlling a virtual interactive surface in a scene, wherein the virtual interactive surface comprises a surface for displaying information and one or several virtual sensors positioned with respect to the surface, wherein a virtual sensor comprises data representing a geometric form, data representing a position in the scene and data representing one or several trigger conditions, the method comprising:

receiving point cloud data representing surfaces of objects of the captured scene and comprising data representing non-empty points of a first volume area corresponding to the captured first data representation and respective associated position in the first volume area; and

determining whether one or several of the virtual sensor trigger conditions is/are fulfilled based on an analysis of data representing a captured first data representation of the scene in a volume area corresponding to the geometric form and position of each virtual sensor,

wherein for a first virtual sensor said analysis comprises comparing a number of non-empty points whose position falls in the volume area corresponding to the geometric form and position of said first virtual sensor is greater to a triggering threshold.