Dynamic holographic video with haptic interaction

• US 6,211,848 B1
• Filed: 02/09/1999
• Issued: 04/03/2001
• Est. Priority Date: 05/15/1998
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Plesniak, Wendy, Benton, Stephen, Pappu, Ravikanth
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): NGUYEN, KIMNHUNG T
• Application Number:
  US09/252,591
• Time in Patent Office: 784 Days
• Field of Search: 345/1, 345/173, 345/156, 359/10, 359/9, 359/11, 359/15, 359/16, 359/630
• US Class Current: 345/156
• CPC Class Codes: G03H 1/2249 Holobject properties
• CPC Class Codes: G03H 2001/0061 in haptic applications when the observer interacts with the holobject
• CPC Class Codes: G03H 2210/30 3D object
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A user is able to interact with and modify an electronic holographic image using a force-feedback (or haptic) device, which is capable of sensing and reporting the 3D position of its hand-held stylus and “displaying” appropriate forces to the user. Thus, a user can feel and modify specified shapes in the haptic workspace. The haptic workspace is precisely registered with the free-standing, spatial image displayed by a holographic video (holovideo) system. In the coincident visuo-haptic workspace, a user can see, feel, and interact with synthetic objects that exhibit many of the properties one expects of real ones, and the spatial display enables synthetic objects to become a part of the user'"'"'s manipulatory space.

Claims

1. A holographic imaging system comprising:

a. means for rendering a holographic image within a view zone;

b. means facilitating manual interaction with the holographic image, said means comprising;

i. means for sensing interaction between a user-manipulated object in the view zone and the holographic image;

ii. means for imparting a force to the object based on the interaction; and

iii. means for causing the rendering means to alter the holographic image based on the interaction.

2. The system of claim 1 wherein the interaction-sensing means comprises:

a. means for periodically sensing a three-dimensional position of the object within the view zone; and

b. means for correlating each sensed position with the holographic image to derive an apparent position of the object with respect to the holographic image.

3. The system of claim 1 wherein the object is a hand-held implement.

4. The system of claim 1 wherein the object is a user'"'"'s hand or portion thereof.

5. The system of claim 1 wherein the holographic image comprises a virtual object having surface and deformation characteristics, and further comprising means for operating the force-imparting means in accordance with a haptic model of the virtual object specifying the object characteristics.

6. The system of claim 1 wherein the rendering means comprises:

a. computational means for generating data representing a holographic fringe pattern; and

b. a spatial light modulator for generating a holographic image within the view zone from the fringe data.

7. The system of claim 6 wherein the holographic image comprises a series of hololines and exhibits parallax along a single dimension.

8. The system of claim 7 wherein the rendering means further comprises:

a. a computer memory for storing data representing a plurality of fringe patterns, each fringe pattern corresponding to a different rendering of the image; and

b. means, responsive to the altering means, for assembling from the stored fringe pattern a composite holographic image that reflects the interaction.

9. The system of claim 7 wherein the sensing means identifies hololines of the image affected by the interaction and, for each affected hololine, a specification of a new hololine that reflects the interaction, the altering means changing the identified hololines in accordance with the specifications.

10. The system of claim 2 wherein the altering means causes the holographic image to be altered only if the object penetrates at least a threshold distance into the holographic image.

11. A method of facilitating interaction with a holographic image, the method comprising the steps of:

a. rendering a holographic image within a view zone;

b. sensing interaction between a user-manipulated object in the view zone and the holographic image;

c. imparting a force to the object based on the interaction; and

d. causing the rendering means to alter the holographic image based on the interaction.

12. The method of claim 11 wherein the sensing step comprises:

a. periodically sensing a three-dimensional position of the object within the view zone; and

b. correlating each sensed position with the holographic image to derive an apparent position of the implement with respect to the holographic image.

13. The method of claim 11 wherein the holographic image comprises a virtual object having surface and deformation characteristics, the imparted force reflecting the object characteristics at the sensed position and changing as the sensed position changes.

14. The method of claim 11 wherein the object is a hand-held implement.

15. The method of claim 11 wherein the object is a user'"'"'s hand or portion thereof.

16. The method of claim 11 wherein the rendering step comprises:

a. computationally generating data representing a holographic fringe pattern; and

b. generating a holographic image within the view zone from the fringe data.

17. The method of claim 16 wherein the holographic image comprises a series of hololines and exhibits parallax along a single dimension.

18. The method of claim 17 wherein the rendering step further comprises:

a. storing data representing a plurality of fringe patterns, each fringe pattern corresponding to a different rendering of the image; and

b. assembling from the stored fringe pattern a composite holographic image that reflects the interaction.

19. The method of claim 17 wherein the sensing step comprises (i) identifing hololines of the image affected by the interaction and (ii), for each affected hololine, specifying a new hololine that reflects the interaction, the altering step comprising changing the identified hololines in accordance with the specifications.

20. The method of claim 12 wherein the holographic image is altered only if the object penetrates at least a threshold distance into the holographic image.

21. The method of claim 11 further comprising the step of causing fabrication of a physical model of the altered holographic image.