Kinetic Input/Output

• US 20110288964A1
• Filed: 05/24/2011
• Published: 11/24/2011
• Est. Priority Date: 05/24/2010
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Linder, Natan, Maes, Patricia
• Granted Patent: US 8,751,049 B2
• Time in Patent Office: Days
• US Class Current: 705/27.1
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/36169 Remote, host controlled, operated manual data input, keyboard
• CPC Class Codes: G05B 2219/40411 Robot assists human in non-industrial environment like home or office
• CPC Class Codes: G06Q 30/0641 Shopping interfaces

Abstract

In exemplary implementations of this invention, an input/output device (“Bulb”) is attached to on an articulated, actuated robotic arm. The robotic arm can move the Bulb by translating it along three axes and by rotating it about the arm'"'"'s base. In addition, the I/O can rotate about its own vertical axis. The Bulb comprises at least a pico-projector, two cameras, a depth sensor and an onboard computer. The onboard computer controls actuators in the robotic arm and Bulb that move the Bulb. It also processes visual data captured by the cameras in order to recognize objects or events, and to respond to them. This response may include changing the position of the Bulb or changing the parameters of an image projected by the pico-projector.

Claims

1. Apparatus comprising

an articulated arm with actuators for moving an I/O device that is attached to the articulated arm,

the I/O device, which I/O device comprises at least one sensor for capturing visual data and a projector for projecting a multi-pixel image, and

at least one processor for

analyzing the visual data to recognize an object or event,

selecting a movement of the articulated arm in response to the object or event that is recognized,

outputting movement control signals to cause the actuators to effect the movement, and

outputting image control signals to control the multi-pixel image.

2. The apparatus of claim 1, wherein the at least one processor is adapted:

to analyze the visual data to recognize an uncluttered surface,

to output movement control signals to cause the actuators to move the I/O device nearer to the uncluttered surface, and

to output image control signals to cause the multi-pixel image to be projected on an area that is entirely within the uncluttered surface.

3. The apparatus of claim 1, wherein the event is a gesture and the at least one processor is adapted:

to analyze the visual data to recognize the gesture, and

to output movement control signals to cause the actuators to effect the movement for a distance, in a direction or with a stopping point indicated by the gesture, the movement being to a new position, and

to output image control signals to cause the projector to project, from the new position, the multi-pixel image.

4. The apparatus of claim 1, wherein the robotic arm can move with at least four degrees of freedom.

5. The apparatus of claim 1, further comprising a depth sensor for detecting distance from the I/O device to an item in the vicinity of the I/O device.

6. The apparatus of claim 1, wherein the image control signals cause the projected image to correct for a keystone effect.

7. The apparatus of claim 1, wherein the image control signals control at least the orientation of the multi-pixel image.

8. The apparatus of claim 1, wherein the at least one processor is adapted:

to output movement control signals to cause the actuators to alter the distance between the I/O device and a surface on which the multi-pixel image is being or will be projected, and

to output image control signals to digitally zoom the multi-pixel image.

9. The apparatus of claim 1, wherein the at least one processor is further adapted to output the motion control signals or image control signals in response, at least in part, to direct physical manipulation of the apparatus.

10. The apparatus of claim 1, wherein the at least one processor is further adapted:

to output control signals to retrieve information from a network, which information is associated with the object or event that is recognized,

to accept data indicative of the information, and

to output signal control data to cause the multi-pixel image to display content that is based at least in part on the information.

11. The apparatus of claim 1, wherein

the apparatus is adapted to receive and transmit data to and from a network,

the multi-pixel image comprises a projected graphical user interface, and

the at least one processor is adapted to process the visual data to detect when and where a human user touches a surface on which the projected graphical user interface is projected.

12. The apparatus of claim 1, wherein the at least one processor is housed, in whole or part, onboard the I/O device.

13. The apparatus of claim 1, wherein the at least one processor performs distributed computing and comprises two or more processors, at least some of which are remote from each other.

14. The apparatus of claim 11, wherein the projected graphical user interface is for displaying an email or other electronic message and for receiving input from a human user to send an email or other electronic message.

15. The apparatus of claim 11, wherein the projected graphical user interface is for displaying information about an item for sale and for receiving input from a human user to purchase that item.

16. The apparatus of claim 1, wherein the apparatus is further adapted to transmit at least part of the visual data to at least one separate device for display or storage by that at least one separate device.

17. The apparatus of claim 16, wherein the at least one separate device is at least one other apparatus that meets the description of claim 1.

18. A process comprising:

using an I/O device to capture visual data and to recognize an object or event in the visual data,

using at least one actuator to translate the I/O device in a movement that brings the I/O device to a new position that is nearer to the object or that is indicated by the event, and

from that new position, using the I/O device to project a multi-pixel image.

19. The process of claim 18, wherein the object that is recognized is an uncluttered surface, the movement brings the I/O device closer to the uncluttered surface, and the multi-pixel image is projected on an area that falls entirely within the uncluttered surface.

20. Apparatus comprising:

actuators for causing movement of an I/O device, which movement comprises translation of the I/O device along a track, tilt of the I/O device relative to the track, and rotation of I/O device about an axis of the I/O device, and

the I/O device, which I/O device comprises a sensor for capturing visual data and a projector for projecting a multi-pixel image, and at least one processor for

analyzing the visual data to recognize an object or event, and

outputting signals, in response at least in part to the object or event that is recognized, to control the movement and to control parameters of the multi-pixel image.