System for selecting objects on display

• US 8,782,565 B2
• Filed: 01/12/2012
• Issued: 07/15/2014
• Est. Priority Date: 01/12/2012
• Status: Expired due to Fees

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Fretwell, Lisa, Grinyer, Clive
• Primary Examiner(s): Vu, Kieu
• Assistant Examiner(s): Gattew, Asteway T
• Application Number:
  US13/349,229
• Publication Number:
  US 20130185679A1
• Time in Patent Office: 915 Days
• Field of Search: 715/862, 715/856, 715/859, 345/633, 345/632
• US Class Current: 715/862
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

In one embodiment, a method of detecting a mouse click on a virtual mirror display can include displaying a selectable object on the virtual mirror display. The method can include displaying a cursor position by displaying a circle on the virtual mirror display. The cursor position can correspond to a hand position. The method can include detecting the mouse click in response to the circle hovering over at least part of the selectable object. The circle can remain steady over the selectable object on the virtual mirror display for a predetermined period of time.

Claims

1. A method of detecting a mouse click on a virtual mirror display, the method comprising:

providing, by a display device operatively connected to a processor, a live reflection on the virtual mirror display;

displaying a selectable object on the virtual mirror display;

displaying a first cursor position by displaying a first circle on the virtual mirror display, the first cursor position corresponding to a right hand position of the live reflection;

displaying a second cursor position by displaying a second circle on the virtual mirror display, the second cursor position corresponding to a left hand position of the live reflection;

detecting, by the processor, the mouse click in response to the first or second circle hovering over at least a part of the selectable object, wherein the first or second circle remains steady over the selectable object on the virtual mirror display for a predetermined period of time;

hiding the first cursor from the virtual mirror display when the right hand position is below a determined boundary and the left hand position is above the boundary, while maintaining the live reflection and second cursor on the virtual mirror display; and

hiding the second cursor from the virtual mirror display when the left hand position is below the determined boundary and the right hand position is above the boundary, while maintaining the live reflection and first cursor on the virtual mirror display.

2. The method of claim 1, wherein the predetermined period of time is displayed on the virtual mirror by incrementally filling the first or second circle with a color.

3. The method of claim 1, further comprising displaying different selectable menu options on the display screen.

4. The method of claim 1, further comprising selecting an item from a selectable list of items that is displayed on the virtual mirror display, wherein the selected item is not in the center of the selectable list of items.

5. The method of claim 1, further comprising displaying a menu option of hide me or show me on the virtual mirror display depending on whether the virtual mirror display is to display or hide the live reflection.

6. The method of claim 1, wherein the method further comprises distinguishing between a single click, a double click, a right click, a left click, or a center button click based on the left and right hand positions.

18. The method of claim 5, further comprising, when the menu option of hide me is selected, displaying on the virtual mirror display the menu option of hide me or show me and a selectable list of items as a partially transparent overlay on the live reflection so that the live reflection is at least partially visible behind the partially transparent overlay.

7. A system comprising:

a memory comprising a plurality of modules, and a processor configured to execute the plurality of modules;

an image module configured to receive a moving image of a body from an image capture device;

a hand detection module executable to determine a location of a left hand of the body in the received image, to determine a location of a right hand of the body in the received image, and to track movement of the locations of the left and right hands in the received image;

a display module executable to display a left cursor and a right cursor on a display device, a position of the left cursor on the display device corresponding to the location of the left hand of the body, a position of the right cursor on the display device corresponding to the location of the right hand of the body; and

a boundary determination module executable to determine a position of a boundary on the display device, the boundary extending at least partially horizontally and comprising a boundary point corresponding to a reference point in the received image positioned between a chest of the body and a knee of the body;

wherein the display module is executable to;

display the left cursor on the display device in response to the location of the left hand relative to the boundary and to display the right cursor on the display device in response to the location of the right hand relative to the boundary;

hide the right cursor from the display device when the location of the right hand is below the boundary and the location of the left hand is above the boundary, while maintaining the moving image of the body and left cursor on the display device; and

hide the left cursor from the display device when the location of the left hand is below the boundary and the location of the right hand is above the boundary, while maintaining the moving image of the body and right cursor on the display device.

8. The system of claim 7, further comprising an image overlay module executable to generate at least one selectable object, wherein the display module is further executable to display the selectable object on the display device, and the system further comprises an alignment recognition module executable to recognize alignment of at least one of the position of the left cursor and the position of the right cursor with the selectable object, and a notification module executable to generate a notification in response to alignment of the at least one of the position of the left cursor and the position of the right cursor with the selectable object.

9. system of claim 8, wherein the notification comprises a visual notification or an audible notification.

10. The system of claim 8, wherein the alignment recognition module is further executable to recognize misalignment of the at least one of the position of the left cursor and the position of the right cursor with the selectable object, and the notification module is further executable to reset the notification in response to misalignment of the at least one of the position of the left cursor and the position of the right cursor with the selectable object.

11. The system of claim 8, wherein the notification comprises a predetermined shape generated to surround the at least one of the left cursor and the right cursor aligned with the selectable object, the display module is further executable to display the predetermined shape on the display device, and to change a color of the predetermined shape in response to a length of time that the at least one of the left cursor and the right cursor is aligned with the selectable object.

12. The system of claim 11, wherein the display module is executable to change the color of the predetermined shape incrementally around a perimeter of the predetermined shape in response to the length of time that the at least one of the left cursor and the right cursor is aligned with the selectable object.

13. The system of claim 8, further comprising a selection module executable to select the selectable object in response to the at least one of the left cursor and the right cursor being aligned with the selectable object for a predetermined length of time.

14. The system of claim 7, wherein the display module is further executable to display the received image on the display device.

15. The system of claim 7, wherein the position of the boundary on the display device dynamically changes in response to movement of the body in the received image.

19. The system of claim 7, wherein the display module is further executable, in response to a selection of a hide me menu option, to display, on the display device, a menu option of hide me or show me and a plurality of selectable objects as a partially transparent overlay on the moving image of the body so that the moving image is at least partially visible behind the partially transparent overlay.

16. A non-transitory computer readable medium encoded with computer executable instructions, the computer executable instructions executable with a processor, the non-transitory computer readable medium comprising:

instructions executable to receive a moving image of a body;

instructions executable to determine locations of left and right hands of the body in the received image and to track movement of the locations of the left and right hands in the received image;

instructions executable to generate a selectable object;

instructions executable to display the selectable object and left and right cursors on a virtual mirror display, a position of the left cursor corresponding to the location of the left hand, and a position of the right cursor corresponding to the location of the right hand;

instructions executable to recognize alignment of at least one of the position of the left cursor and the position of the right cursor with the selectable object, and to display a circle surrounding the at least one of the left cursor and the right cursor in response to alignment with the selectable object;

instructions executable to detect a mouse click in response to the at least one of the left cursor and the right cursor remaining in alignment with the selectable object for a predetermined period of time;

instructions executable to determine a position of a boundary area on the virtual mirror display, the boundary area extending at least partially horizontally and comprising a boundary point corresponding to a reference point in the received image positioned between a chest of the body and a knee of the body;

instructions executable to display the left cursor on the virtual mirror display in response to the location of the left hand relative to the boundary area and to display the right cursor on the virtual mirror display in response to the location of the right hand relative to the boundary area;

instructions executable to omit the right cursor from the virtual mirror display in response to the location of the right hand being outside the boundary area and the location of the left hand being inside the boundary area, while the moving image of the body and the left cursor are maintained in the virtual mirror display; and

instructions executable to hide the left cursor from the virtual mirror display in response to the location of the left hand being outside the boundary area and the location of the right hand being inside the boundary area, while the moving image of the body and right cursor are maintained in the virtual mirror display.

17. The non-transitory computer readable medium of claim 16, further comprising instructions executable to fill the circle with a color in response to passage of the predetermined period of time and the cursor remaining in alignment with the selectable object.

20. The non-transitory computer readable medium of claim 16, further comprising instructions executable in response to a selection of a hide me menu option to display, on the virtual mirror display, a menu option of hide me or show me and a plurality of selectable objects as a partially transparent overlay on the moving image of the body so that the moving image is at least partially visible behind the partially transparent overlay.