Augmented reality guided instrument positioning with modulated guiding graphics

• US 7,176,936 B2
• Filed: 03/27/2001
• Issued: 02/13/2007
• Est. Priority Date: 03/27/2001
• Status: Expired due to Term

• Current Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Bani-Hashemi, Ali, Sauer, Frank
• Primary Examiner(s): Tung; Kee M.
• Assistant Examiner(s): HARRISON, CHANTE E
• Application Number:
  US09/818,388
• Publication Number:
  US 20020140709A1
• Time in Patent Office: 2,149 Days
• Field of Search: 345/629, 345/592, 345/630, 345/632, 715/861, 715/862, 715/434, 600/861, 600/862, 600/434
• US Class Current: 345/592
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 13/339 using spatial multiplexing H04N13/337 takes precedence

Abstract

There is provided a method for augmented reality guided instrument positioning. A graphics guide is determined for positioning an instrument. The graphics guide is rendered such that an appearance of at least one portion of the graphics guide is modulated with respect to at least one of space and time.

Claims

1. A method for augmented reality guided instrument positioning, comprising the steps of:

displaying a real view of an environment including an instrument;

determining a preferred path for positioning of said instrument;

marking the preferred path with a graphics guide;

augmenting the real view with a rendering of the graphics guide such that at least one portion of the graphics guide is transparent with respect to other portions of the graphics guide to provide a substantially unobstructed view through the at least one portion of the graphics guide to at least a portion of the instrument to facilitate alignment of the real instrument with the virtual guide, the transparency of at least one portion of the guide being varied at redefined time intervals;

aligning the instrument with the graphics guide so that the instrument appears in a same location as the graphics guide in the augmented view thereby creating a mutual spatial relationship between the real instrument and the virtual guide, and when properly aligned, at least a portion of the instrument is visible through the at least one transparent portion of the graphics guide; and

inserting the instrument in the graphics guide.

2. The method according to claim 1, wherein said rendering includes a modulation of the graphics guide'"'"'s transparency along the length of the graphics guide, so that a plurality of portions of the graphics guide appear transparent with respect to other portions of the graphics guide along the length of the graphics guide to provide a substantially unobstructed view through the plurality of transparent portions of the graphics guide to a plurality of portions of the instrument once the instrument has been correctly aligned with the graphics guide.

3. The method according to claim 1, wherein said rendering step comprises the step of sequentially varying the transparency of each of a plurality of portions of the graphics guide during at least one predefined time interval to provide a substantially unobstructed view through each of the plurality of portions to at least a portion of the instrument as the transparency of each of the plurality of portions is varied during the at least one pre-defined time interval.

4. The method according to claim 3, wherein the plurality of portions are consecutive.

5. The method according to claim 1, wherein said rendering step comprises the step of varying the transparency of the at least one portion of the graphics guide such that the at least one portion repeatedly switches between transparent and less transparent.

6. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a line, and said rendering step comprises the step of modulating the transparency of the line with respect to time so that the line repeatedly fades in and out of view to provide a substantially unobstructed view of the instrument when the line is out of view.

7. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a line, and said rendering step comprises the step of modulating the transparency of portions of the line so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the line.

8. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a line, and said rendering step comprises the step of modulating the transparency of portions of the line with respect to time and space so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the line during predefined time intervals.

9. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a cylinder, and said rendering step comprises the step of modulating the transparency of the cylinder with respect to time so that the cylinder repeatedly fades in and out of view to provide a substantially unobstructed view of the instrument when the cylinder is out of view.

10. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a cylinder, and said rendering step comprises the step of modulating the transparency of portions of the cylinder so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the cylinder.

11. The method according to claim 1, wherein said determining step comprises the step of constructing the graphics guide as a cylinder, and said rendering step comprises the step of modulating the transparency of portions of the cylinder with respect to time and space so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the cylinder during pre-defined time intervals.

12. An apparatus for augmented reality guided instrument positioning, comprising:

a video camera providing a real view of an environment including an instrument;

a graphics guide generator for identifying a preferred path for positioning said instrument and generating a graphics guide for marking the preferred path; and

a rendering device for augmenting the real view with a rendering of the graphics guide such that at least one portion of the graphics guide is transparent with respect to other portions of the graphics guide to provide a substantially unobstructed view through the at least one portion of the graphics guide to at least a portion of the instrument to facilitate alignment of the real instrument with the virtual guide, the transparency of at least one portion of the guide being varied at predefined time intervals;

an alignment device for aligning the instrument to the graphics guide so that the instrument appears in a same location as the graphics guide in the augmented view thereby creating a mutual spatial relationship between the real instrument and the virtual guide, and when properly aligned, at least a portion of the instrument is visible through the at least one transparent portion of the graphics guide; and

an insertion device for inserting the instrument in the graphics guide.

13. The apparatus according to claim 12, wherein said rendering device includes a modulation of the graphics guide'"'"'s transparency along the length of the graphics guide, so that a plurality of portions of the graphics guide appear transparent with respect to other portions of the graphics guide along the length of the graphics guide to provide a substantially unobstructed view through the plurality of transparent portions of the graphics guide to a plurality of portions of the instrument once the instrument has been correctly aligned with the graphics guide.

14. The apparatus according to claim 12, wherein said rendering device sequentially varies the transparency of each of a plurality of portions of the graphics guide during at least one predefined time interval to provide a substantially unobstructed view through each of the plurality of portions to at least a portion of the instrument as the transparency of each of the plurality of portions is varied during the at least one pre-defined time interval.

15. The apparatus according to claim 12, wherein said rendering device varies the transparency of the at least one portion of the graphics guide such that the at least one portion repeatedly switches between transparent and less transparent.

16. The apparatus according to claim 12, wherein said graphics guide generator constructs the graphics guide as a line, and said rendering device modulates the transparency of the line with respect to time so that the line repeatedly fades in and out of view to provide a substantially unobstructed view of the instrument when the line is out of view.

17. The apparatus according to claim 12, wherein said determining device constructs the graphics guide as a line, and said rendering device modulates the transparency of portions of the line so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the line.

18. The apparatus according to claim 12, wherein said graphics guide generator constructs the graphics guide as a line, and said rendering device modulates the transparency of portions of the line with respect to time and space so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the line during pre-defined time intervals.

19. The apparatus according to claim 12, wherein said graphics guide generator constructs the graphics guide as a cylinder, and said rendering device modulates the transparency of the cylinder with respect to time so that the cylinder repeatedly fades in and out of view to provide a substantially unobstructed view of the instrument when the cylinder is out of view.

20. The apparatus according to claim 12, wherein said graphics guide generator constructs the graphics guide as a cylinder, and said rendering device modulates the transparency of portions of the cylinder so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the cylinder.

21. The apparatus according to claim 12, wherein said graphics guide generator constructs the graphics guide as a cylinder, and said rendering device modulates the transparency of portions of the cylinder with respect to time and space so that at least a portion of the instrument is substantially unobstructed when viewed through the portions of the cylinder during pre-defined time intervals.

22. A method for augmented reality guided instrument positioning, comprising the steps of:

displaying a real camera view of an environment including an instrument;

determining a preferred path of the instrument in the environment;

marking the preferred path with a graphics guide, the graphics guide indicating a predetermined position to which the instrument is alignable; and

augmenting the real camera view with a rendering of a virtual view of the graphics guide such that at least one portion of the graphics guide is transparent with respect to other portions of the graphics guide to provide a substantially unobstructed view through the at least one portion of the graphics guide to at least a portion of the instrument to facilitate alignment of the real instrument with the virtual guide, the transparency of at least one portion of the guide being varied at predefined time intervals;

aligning the instrument to the graphics guide so that the instrument appears in a same location as the graphics guide in the augmented view thereby creating a mutual spatial relationship between the real instrument and the virtual guide, and when properly aligned, the instrument is visible through the at least one transparent portion of the graphics guide; and

inserting the instrument in the graphics guide.