System and method for 3D projection mapping with robotically controlled objects

• US 9,472,011 B2
• Filed: 11/15/2012
• Issued: 10/18/2016
• Est. Priority Date: 11/16/2011
• Status: Active Grant

• Current Assignee: X Development LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Linnell, Jeff
• Primary Examiner(s): HAROLD, JEFFEREY F
• Assistant Examiner(s): Rahman, Mustafizur
• Application Number:
  US13/678,342
• Publication Number:
  US 20130120547A1
• Time in Patent Office: 1,433 Days
• Field of Search: 348/61, 348/42, 348/44, 348/47, 348/48, 348/50, 348/51, 348/56, 348/135, 348/136, 348/333.1, 348/383, 348/744, 348/745, 345/156, 345/419, 345/652, 345/663, 345/678, 353/11, 353/48
• US Class Current: 1/1
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1682 Dual arm manipulator; Coordination of several manipulators
• CPC Class Codes: G05B 2219/39109 Dual arm, multiarm manipulation, object handled in cooperation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/393 the volume being generated by a moving, e.g. vibrating or rotating, surface
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

A system for motion control is presented. In one embodiment, a motion control 3D projection system includes a projector; and a projection surface coupled to a robotic arm, where the robotic arm moves the projection surface through a set of spatial coordinates, and a 3D projection from the projector is projected onto a set of coordinates of the projection surface and matches the 3D projection to the set of coordinates of the projection surface as the projection surface moves through the set of spatial coordinates. In additional embodiments, a master control system may integrate additional robotic arms and other devices to create a motion control scene with a master timeline.

Claims

1. A system for 3D projection with motion control comprising:

a projection surface coupled to a first device actor, wherein the first device actor moves the projection surface through a set of spatial coordinates;

a projector that is operable to project a 3D projection onto a set of coordinates of the projection surface; and

an animation computer system that matches the 3D projection to the set of coordinates of the projection surface as the projection surface moves through the set of spatial coordinates, such that movement of the 3D projection matches movement of the projection surface during at least translational movement of the projection surface from a first location in the set of spatial coordinates to a second location in the set of spatial coordinates.

2. The system of claim 1,

wherein the animation computer system is coupled to the projector that maps the 3D projection onto the projection surface and animates the 3D projection while matching the 3D projection to the set of coordinates of the projection surface as the non-planar projection surface moves through the set of spatial coordinates.

3. The system of claim 2 wherein the animation computer system is further coupled to the device actor, and the animation computer system provides the set of spatial coordinates to the device actor to instruct the device actor to move the non-planar projection surface through the set of spatial coordinates.

4. The system of claim 3 further comprising:

a master control that receives a plurality of control signals comprising control data for a plurality of device actors including the first device actor and synchronizes the plurality of control signals with a global timeline to create a plurality of synchronized signals, such that control data for each actor of the device actors is associated with a corresponding position in the global timeline; and

a master input that conveys a master input signal to the master control indicating a position in the global timeline and a rate of progression through the global timeline;

wherein the master control responds to the master input signal by communicating the plurality of synchronized signals associated with the position in the global timeline to the plurality of device actors, and the control data for each actor of the device actors is sent to respective device actors at the rate of progression through the global timeline.

5. The system of claim 4 wherein the master input comprises an analog input, a mode selector, and an analog input action control;

wherein the analog input action control returns the analog input to a first position when no force is applied to the analog input;

wherein for a first selected mode of the master input, adjustment of the analog input in a first direction increases the rate of progress through the global timeline from a predetermined standard forward rate;

wherein for the first selected mode of the master input, adjustment of the analog input in a second direction different from the first direction decreases the rate of progress through the global timeline from the predetermined standard forward rate;

wherein for a second selected mode of the master input, adjustments to the analog input in the first direction creates a variable rate of progress in the forward direction from a stationary rate; and

wherein for the second selected mode of the master input, adjustments to the analog input in the second direction creates a variable rate of progress in a reverse direction from the stationary rate.

6. The system of claim 5 wherein the animation computer system comprises a real time input for modifying the 30 projection while maintaining the 30 projection within the set of coordinates of the projection surface as the projection surface moves through the set of spatial coordinates.

7. The system of claim 4 further comprising an object detector that observes an area proximate to a first device actor of the plurality of device actors and transmits a safety shutdown signal to the first device actor when the object detector senses an object.

8. The system of claim 7 further comprising a master safety control;

wherein the master safety control comprises a programmable logic circuit; and

wherein the safety shutdown signal is transmitted to each of the plurality of device actor via the programmable logic circuit.

9. The system of claim 7 wherein the first device actor is a robotic arm, the object detector is attached to a mounting point of the robotic arm, and wherein the area proximate to the first device actor comprises an area extending a fixed distance from the object detector.

10. The system of claim 8 wherein the area proximate to the first device actor varies over time in a predetermined manner determined in association with the position in the global timeline and the rate of progress through the global timeline.

11. A method comprising:

projecting from a projection device, a 3D projection onto a set of coordinates of a projection surface, wherein the projection surface is initially located at a first location in a set of spatial coordinates;

moving the projection surface through the set of spatial coordinates using a first device actor coupled to the projection surface, wherein the movement through the set of spatial coordinates comprises a translation of the projection surface from the first location in the set of spatial coordinates to a second location in the set of spatial coordinates; and

mapping, using the animation computer, the 3D projection onto the projection surface to match movement of the 3D projection with the movement of the projection surface during the movement of the projection surface the set of spatial coordinates from the first location to the second location.

12. The method of claim 11 further comprising:

receiving control data for a plurality of device actors including the first device actor at a master control;

synchronizing a plurality of control signals, using the master control, with a global timeline to create a plurality of synchronized signals, such that the control data for each actor of the device actors is associated with a corresponding position in the global timeline;

receiving, at the master control, a master input signal indicating a position in the global timeline and a rate of progression through the global timeline; and

communicating, in response to the master input signal, the plurality of synchronized signals associated with the position in the global timeline from the master control to the plurality of device actors, wherein the plurality of synchronized signals are sent to the plurality of device actors at the rate of progression through the global timeline indicated by the master input signal.

13. The method of claim 12 further comprising:

detecting, using at least one object detector, an area proximate to the plurality of device actors; and

transmitting, from the at least one object detector, a safety shutdown signal that prevents the plurality of device actors from operating.

14. The method of claim 12 wherein the rate of progress through the global timeline comprises a reverse rate of progress through the global timeline.

15. The method of claim 12 further comprising:

receiving at the master control, during the communicating of the plurality of synchronized signals, a modifying control input for a first device actor of the plurality of device actors from an independent manual control;

synchronizing the modifying control input with the plurality of synchronized signals to create an updated plurality of synchronized signals during the communication of the plurality of synchronized signals; and

communicating the updated plurality of synchronized signals in place of the synchronized signals.

16. The method of claim 15 further comprising storing the updated plurality of synchronized signals in a database, wherein the control data is received at the master control from the database.

17. The method of claim 12 further comprising:

receiving at the master control, feedback data from the plurality of device actors describing an operating characteristic associated with the synchronized control signals; and

modifying the synchronized control signals in response to the feedback data.

18. The method of claim 11 further comprising:

adjusting in real time, the signal from the animation computer, to create a real time computer animation at the projection surface based on a user input at the animation computer.

19. A non-transitory computer readable medium storing instructions for execution by a computing system for implementing a method of creating a film using motion control, the method comprising:

modeling physical characteristics and locations, using a software control comprising a computing device;

for a plurality of device actors, at least one projector, and at least one projection surface in a motion control set;

modeling movement characteristics and an optical path from the at least one projector to the at least one projection surface, using the software control, for the plurality of device actors in the motion control set to create control data for the plurality of device actors;

analyzing the modeling of the movements, physical characteristics, and locations of the plurality of device actors, using the software control, to detect collisions, obstruction of the optical path, and to detect device actor motion that exceeds a set of predetermined operating limits; and

communicating the control data to a master control that synchronizes the control data and transmits the control data to the plurality of device actors.

20. The non-transitory computer readable medium of claim 19, wherein the method further comprises:

analyzing the modeling of the movements, physical characteristics, and locations of the plurality of device actors, using the software control, to determine a set of locations proximate to the device actors that include a collision risk; and

transmitting the set of locations to a master safety control comprising at least one object detect.