System and method for manipulating an object in a three-dimensional desktop environment

• US 9,280,259 B2
• Filed: 07/26/2013
• Issued: 03/08/2016
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Blackberry Limited
• Inventors: Mankowski, Peter
• Primary Examiner(s): Hajnik, Daniel
• Application Number:
  US13/952,137
• Publication Number:
  US 20150033191A1
• Time in Patent Office: 956 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering

Abstract

An electronic device, method and interface for the device, for performing an action with a processor through a three-dimensional desktop environment is disclosed. A three-dimensional desktop environment is generated by a display and projected into a real space. At least one ultrasonic transducer propagates an ultrasonic pulse into the real space and receives a reflection of the ultrasonic pulse from a user object in the real space. A user action of the user object within the three-dimensional desktop environment is determined using the reflection of the ultrasonic pulse. The processor performs the action based on the determined user action.

Claims

1. A method of performing an action with a processor, comprising:

generating, using the processor, a three-dimensional desktop environment and projecting the three-dimensional desktop environment into a real space, wherein the three-dimensional desktop environment is segmented into a plurality of power levels spanning a range of distances from a display along a first axis, and wherein each of the power levels comprise a plurality of pixel volumes having a horizontal extent, a vertical extent, and a depth based on a resolution capability of an ultrasonic transducer;

propagating, using the ultrasonic transducer, an ultrasonic pulse into the real space;

receiving, using the ultrasonic transducer, a reflection of the ultrasonic pulse from a user object located within the real space;

determining a user action of the user object using the ultrasonic pulse and the reflected ultrasonic pulse; and

performing the action with the processor based on the determined user action.

2. The method of claim 1, further comprising:

projecting an image within the three-dimensional desktop environment;

determining the user action with respect to the projected image; and

performing the action with the processor based on the determined user action of the user object with respect to the projected image.

3. The method of claim 1, wherein determining the user action further comprises determining at least one of:

(i) a perpendicular distance of the user object from the display;

(ii) a position of the user object along a horizontal axis of the display;

(iii) a position of the user object along a vertical axis of the display;

(iv) a movement of the user object within the three-dimensional desktop environment; and

(v) a gesture made by the user object.

4. The method of claim 1, further comprising using the ultrasonic transducer to determine the user action of the user object and determining a time difference between propagating the ultrasonic pulse into the real space and receiving the reflection of the ultrasonic pulse from the user object.

5. The method of claim 4, wherein the ultrasonic transducer further comprises a transmitter, a first receiver and a second receiver, the method further comprising generating the ultrasonic pulse at the transmitter and receiving the reflection of the ultrasonic pulse via at least one of the first receiver and the second receiver.

6. The method of claim 1, further comprising assigning a user right to one or more of the pixel volumes, wherein performing the action at the processor further comprises accessing the user right for the one or more of the pixel volumes.

7. The method of claim 1, wherein performing the action further comprises at least one of:

moving a projected image within the three-dimensional desktop environment;

enlarging the projected image;

diminishing the projected image;

rotating the projected image;

selecting the action corresponding to the projected image; and

opening a computer application associated with the selected image.

8. The method of claim 1, wherein the projected image includes a plurality of projected images that are grouped according to a category, the method further comprising performing the action with respect to the group of projected images.

9. An electronic device, comprising:

a display configured to project a three-dimensional desktop environment into a real space, wherein the three-dimensional desktop environment is segmented into a plurality of power levels spanning a range of distances from the display along a first axis, and wherein each of the power levels comprises a plurality of pixel volumes having a horizontal extent, a vertical extent, and a depth based on a resolution capability of at least one ultrasonic transducer;

the at least one ultrasonic transducer configured to propagate an ultrasonic pulse into the real space and receive a reflection of the ultrasonic pulse from a user object in the real space; and

a processor configured to;

determine a user action of the user object from the reflection of the ultrasonic pulse, and

perform an action based on the determined user action of the user object.

10. The device of claim 9, wherein the display is further configured to project an image within the three-dimensional desktop environment and the processor is further configured to:

determine the user action with respect to the projected image, and

perform the action at the processor based on the determined user action of the user object with respect to the projected image.

11. The device of claim 9, wherein the processor is configured to determine the user action by determining at least one of:

(i) a perpendicular distance of the user object from the display;

(ii) a position of the user object along a horizontal axis of the display;

(iii) a position of the user object along a vertical axis of the display;

(iv) a movement of the user object within the three-dimensional desktop environment; and

(v) a gesture made by the user object.

12. The device of claim 9, wherein the processor is further configured to determine the user action by determining a time difference between propagating the ultrasonic pulse into the real space and receiving the reflection of the generated ultrasonic pulse from the user object.

13. The device of claim 12, wherein the at least one ultrasonic transducer further comprises a transmitter, a first receiver and a second receiver, the processor further configured to determine the user action from a reflection of the generated ultrasonic pulse obtained via at least one of the first receiver and the second receiver.

14. The device of claim 9, wherein the processor is further configured to:

assign a user right to one or more of the pixel volumes; and

access the user right for the one or more pixel volumes to perform the action.

15. The device of claim 9, wherein performing the action further comprises at least one of:

moving a projected image within the three-dimensional desktop environment;

enlarging the projected image;

diminishing the projected image;

rotating the projected image;

selecting the action corresponding to the projected image; and

opening a computer application associated with the selected image.

16. The device of claim 9, wherein the projected image includes a plurality of projected images that are grouped according to a category, the processor being further configured to perform the action with respect to the group of projected images.

17. An interface for providing a user interaction with an electronic device, comprising:

a display configured to project a three-dimensional desktop environment into a real space, wherein the three-dimensional desktop environment is segmented into a plurality of power levels spanning a range of distances from the display along a first axis, and wherein each of the power levels comprises a plurality of pixel volumes having a horizontal extent, a vertical extent, and a depth based on a resolution capability of at least one ultrasonic transducer;

the at least one ultrasonic transducer configured to propagate an ultrasonic pulse into the real space and receive a reflection of the ultrasonic pulse from a user object in the real space; and

a processor configured to;

determine a user action of the user object from the reflection of the ultrasonic pulse, and

perform an action based on the determined user action of the user object.

18. The interface of claim 17, wherein the display is further configured to project an image within the three-dimensional desktop environment and the processor is further configured to:

determine the user action with respect to the projected image, and

perform the action at the processor based on the determined user action of the user object with respect to the projected image.

19. The interface of claim 17, wherein the processor is further configured to determine the user action of the user object by determining a time difference between generating the ultrasonic pulse and receiving the reflection of the generated ultrasonic pulse from the user object.

20. The interface of claim 17, wherein each of the pixel volumes are associated with a minimum and maximum ultrasonic power level that indicates a spatial boundary for each of the pixel volumes.