Boundless projected interactive virtual desktop

• US 9,940,018 B2
• Filed: 08/18/2015
• Issued: 04/10/2018
• Est. Priority Date: 10/21/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brunn, Jonathan F.
• Primary Examiner(s): SHIAU, SHEN C
• Application Number:
  US14/828,563
• Publication Number:
  US 20160112688A1
• Time in Patent Office: 966 Days
• CPC Class Codes: G01B 2210/28 Beam projector and related sensors, camera, inclinometer or other active sensing or projecting device
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 1/1645 the display being suitable to be used in combination with an external overhead projector
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09B 9/307 by helmet-mounted projector or display G09B9/36 takes precedence; optical parts of head-up displays G02B27/01
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A method for creating a boundless projected interactive virtual desktop, wherein the interactive virtual desktop comprises an adjustable image of a projected portion of an area associated with at least one desktop of a computing device is provided. The method may include integrating a projector and a motion sensor into a device. The method may also include capturing at least one of a location, a change in location, a change in direction, or a change in orientation associated with the device. The method may include computing a projected image. The method may also include coordinating the computed projected image across at least one application running in the device. The method may further include projecting a view of a portion of an area associated with the coordinated projected image, wherein the projected view comprises an interactive virtual desktop. The method may additionally include adjusting the projected view based on a criteria.

Claims

1. A method for creating a boundless projected interactive virtual desktop, wherein the boundless projected interactive virtual desktop comprises an adjustable image of a projected portion of an area associated with at least one desktop of a computing device, the method comprising:

integrating a projector and a motion sensor into a device;

capturing at least one of a location, a change in location, a change in direction, or a change in orientation associated with the device from the integrated motion sensor;

computing a projected image based on the captured location, the captured change in direction, or the captured change in orientation of the device relative to a projected surface;

coordinating the computed projected image across at least one application running in the device based on the captured location and an orientation of the projector, wherein the projected image is associated with the projection surface;

projecting a view of a portion of an area associated with the coordinated projected image, wherein the projected view comprises an interactive virtual desktop including a plurality of elements comprising application images; and

adjusting the projected view of the portion of the area based on a criteria, wherein adjusting the projected view comprises projecting a smaller portion of the projected view when a battery conservation is determined to be critical for a light source associated with the projection surface, and wherein the projected smaller portion of the projected view maintains a size of a plurality of elements associated with an originally projected view.

2. The method of claim 1, wherein the criteria comprises a plurality of rules based on at least one of a power consumption;

a distance to a projection surface;

a motion;

an image fidelity;

an outside light or an ambient light;

a tilt or an angle of the device relative to the surface; and

information received from at least one other device.

3. The method of claim 1, wherein the plurality of elements are selected from a group consisting of at least one of a document;

an image;

a folder;

a video;

a web browser tab;

an application window; and

an application icon.

4. The method of claim 1, further comprising:

adjusting a strength associated with the projected view based on a distance of the projector to the projection surface.

5. The method of claim 1, wherein the motion sensor is selected from a group consisting of at least one of an embedded accelerometer;

a gyroscope;

a positioning system including an autofocus system; and

an embedded or external electromagnetic or ultrasonic motion sensor.

6. The method of claim 1, wherein the projected view preserves a position and size of at least one element within the plurality of elements in the bounded projection relative to the projection surface by making adjustments to a calculation associated with the projected view.

7. The method of claim 1, further comprising:

providing a user interface (UI) associated with the projected interactive virtual desktop, wherein the user interface (UI) allows at least two users to act on at least one element within the plurality of elements independently wherein the first user is on a first device and the second user is on a second device and wherein a combination of a projected area associated with the first device and a projected area associated with the second device reveals a contiguous image and a larger image of the portion of the area associated with the projected image than the first device or the second device could reveal in isolation of one another.

8. The method of claim 7, wherein the user interface (UI) associated with the projected interactive virtual desktop supports at least one gesture selected from a group consisting of at least one of a drag;

a pick up;

a drop;

a pin; and

a navigate.

9. The method of claim 1, wherein the adjusted projected view is based on a detected change in movement associated with the device and the adjusted projected view reveals a previously unrevealed portion of the projected image.