Method for presenting a virtual reality environment for an interaction

• US 7,084,876 B1
• Filed: 07/18/2003
• Issued: 08/01/2006
• Est. Priority Date: 12/07/2002
• Status: Expired due to Fees

• Current Assignee: Digenetics, Inc.
• Original Assignee: Digenetics, Inc.
• Inventors: Fogel, David B., Johnson, Douglas R., Hays, Timothy J.
• Primary Examiner(s): Chauhan, Ulka
• Assistant Examiner(s): REPKO, JASON MICHAEL
• Application Number:
  US10/623,034
• Time in Patent Office: 1,110 Days
• Field of Search: 345/473, 463/2, 463/14, 463/32, 715/758, 715/706
• US Class Current: 345/473
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 13/00 Animation

Abstract

One aspect of the invention is a method for presenting a virtual reality setting for an interaction. An example of the method includes presenting a streaming video of a real-world background scene, and presenting a series of individual video clips that are joined into the appearance of a continuous streaming image of a real-world character. The method also includes inputting an operator'"'"'s choice of action or inaction, and updating a current state based on the operator'"'"'s action or inaction. The method further includes using the current state by a decision logic to determine a response in the setting by the character, and using the current state by the decision logic to determine a selection of how to control video of the character and how to control video of the background. The method may also include simultaneously presenting a three-dimensional representation and a two-dimensional representation of a playing area.

Claims

1. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

inputting an operator'"'"'s choice of action or inaction to a situation state module;

updating a current state based on the operator'"'"'s action or inaction; and

using the current state by a decision logic to determine how to control the real world background scene.

2. The method of claim 1, further comprising the operation of sending a selection of control of the real world background scene to the display.

3. The method of claim 2, further comprising the operation of interacting with a video controller to modify the real world background scene after utilizing the decision logic.

4. The method of claim 3,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area.

5. The method of claim 4,

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.

6. The method of claim 5, further comprising the following operations:

using the current state by the decision logic to determine a response in the virtual reality setting by the image of the real person; and

modifying the image of the real person on the display.

7. The method of claim 6,

wherein the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are not contiguous on the display.

8. The method of claim 7,

further comprising presenting words on the display completely within the outside edges of the background scene on the display.

9. The method of claim 8, further comprising the operations of:

using the current state by the decision logic to determine a response by the image of the real person on the display; and

modifying the image of the real person on the display.

10. The method of claim 9,

wherein the two-dimensional representation of the playing area shows all corresponding movement in both time and space from a game shown in the three-dimensional representation of the playing area.

11. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display, wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

inputting a human operator'"'"'s choice of action or the operator'"'"'s inaction;

updating a current state based on the operator'"'"'s action or inaction;

using the current state by a decision logic to determine a response by the image of the real person presented on the display; and

modifying the image of the real person presented on the display.

12. The method of claim 11, further comprising the operation of controlling the image of the real person presented on the display in response to situations in a game.

13. The method of claim 12, further comprising the operation of controlling the image of the real person presented on the display in response to the time between moves by the human operator.

14. The method of claim 13, further comprising the operation of providing motion of the image of the real person presented on the display in a time-dependent manner based on interaction with the human operator.

15. The method of claim 14, further comprising the operation of using a series of triggers to control the image of the real person presented on the display with responses that are different depending on a game situation.

16. The method of claim 15, further comprising the operation of using a series of triggers to control the image of the real person presented on the display with responses that are different depending on a game situation, and that are continuous, wherein the responses include movements both during and between game moves of the image of the real person presented on the display and the human operator.

17. The method of claim 16, further comprising the operation of using a series of triggers for controlling the image of the real person presented on the display in response to game situations, wherein the game situations include actions and inaction by the human operator, and include the time between moves by the human operator.

18. The method of claim 17, further comprising the operation of using triggers to control responses of the image of the real person presented on the display, wherein the triggers include the human operator'"'"'s sequence of moves during a game.

19. The method of claim 18, further comprising the operation of using triggers to control responses of the image of the real person presented on the display, wherein the triggers include the human operator'"'"'s strategic position in a game.

20. The method of claim 19, wherein the operation of modifying the image of the real person presented on the display comprises interacting decision logic with a video controller.

21. The method of claim 20, wherein the operation of modifying the image of the real person presented on the display comprises making a direct change to the image of the real person presented on the display.

22. The method of claim 20, wherein the operation of modifying the image of the real person presented on the display comprises using a library of videos.

23. The method of claim 22, wherein the image of the real person presented on the display is an opponent in a game.

24. The method of claim 23,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area.

25. The method of claim 24, further comprising the following operations:

using the current state by the decision logic to determine how to control the real world background scene; and

sending a selection of control of the real world background scene to the display.

26. The method of claim 25,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area include the positions of playing pieces on a playing board representing the state of the game.

27. The method of claim 26,

wherein the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are not contiguous.

28. The method of claim 27,

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.

29. The method of claim 28,

wherein the operations further comprise, the human operator moving one of the playing pieces in the game.

30. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of animation clips of a character, wherein the series of animation clips present the appearance of a continuous and moving animated character on the display;

wherein the series of animation clips of the character are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of animation clips of the character, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of animation clips of the character and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

inputting an operator'"'"'s choice of action or the operator'"'"'s inaction;

updating a current state based on the operator'"'"'s action or inaction;

using the current state by a decision logic to determine a response in a setting by the character; and

modifying the character.

31. The method of claim 30, wherein the operation of modifying the character comprises using a library of animations.

32. The method of claim 30, wherein the animation is cell animation.

33. The method of claim 30, wherein the animation is clay animation.

34. The method of claim 31, further comprising the operation of using the current state by the decision logic to determine how to control the real world background scene.

35. The method of claim 34,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area.

61. The method of claim 30, wherein the character is an animated character, and wherein the animation is a cartoon.

36. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area;

inputting a human operator'"'"'s choice of action or inaction to a situation state module;

updating a current state based on the human operator'"'"'s action or inaction;

using the current state by a decision logic to determine how to control the real world background scene;

sending a selection of control of the real world background scene to the display;

using a series of triggers to control responses of the image of the real person presented on the display, wherein the responses are continuous and are determined by game situations, wherein the game situations include actions and inaction by the human operator, and include the time between moves by the human operator, and the human operator'"'"'s sequence of moves during a game, and the human operator'"'"'s strategic position in the game, and wherein the responses of the image of the real person include movements both during and between game moves of the image of the real person and of the human operator;

modifying the image of the real person presented on the display;

providing motion of the image of the real person presented on the display in a time-dependent manner based on interaction with the human operator;

wherein controlling the responses of the image of the real person presented on the display comprises using a library of videos and interacting decision logic with a video controller;

wherein the decision logic uses the current state to determine the response by the image of the real person presented on the display;

wherein the operation of presenting on the display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement comprises continuously streaming video;

wherein the two-dimensional representation of the playing area is a top-down view; and

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.

37. The method of claim 36,

wherein the interaction is a game, and wherein the playing area includes a playing board, and wherein the playing area further includes playing pieces.

38. The method of claim 37,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show the state of the interaction.

39. The method of claim 38,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area include the positions of the pieces on the playing board representing the state of the game.

40. The method of claim 39,

wherein the two-dimensional representation of the playing area shows all corresponding movement in both time and space from the game shown in the three-dimensional representation.

41. The method of claim 40,

wherein the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are not contiguous on the display.

42. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

inputting a human operator'"'"'s choice of action or inaction to a situation state module;

updating a current state based on the human operator'"'"'s action or inaction;

using the current state by a decision logic to determine how to control the real world background scene;

using the current state by the decision logic to determine a response by the image of the real person presented on the display; and

modifying the image of the real person presented on the display.

43. The method of claim 42,

wherein the interaction is a game.

44. The method of claim 43, image of the real person is an opponent in the game.

45. The method of claim 44,

the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are not contiguous.

46. The method of claim 45,

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.

47. The method of claim 46, further comprising the operation of controlling the image of the real person in response to actions of the human operator.

48. The method of claim 47, further comprising the operation of controlling the image of the real person in response to inaction of the human operator.

49. The method of claim 48, further comprising the operation of continuously controlling the image of the real person in response to actions and inaction of the human operator.

50. The method of claim 49, further comprising the operation of controlling the image of the real person in response to the time between moves by the human operator.

51. The method of claim 50, further comprising the operation of affecting the situation state of the setting responsive to an action or to inaction by the human operator.

52. The method of claim 51, further comprising the operation of using a series of triggers to control the image of the real person with responses that are different depending on a game situation.

53. The method of claim 52, further comprising the operation of using a series of triggers to control the image of the real person with responses that are different depending on the game situation, and wherein the responses include movements both during and between game moves of the image of the real person and a human operator.

54. The method of claim 53, further comprising the operation of using triggers to control responses of the image of the real person, wherein the triggers include the human operator'"'"'s sequence of moves during the game.

55. The method of claim 54, further comprising the operation of using triggers to control responses of the image of the real person, wherein the triggers include the human operator'"'"'s strategic position in the game.

56. The method of claim 55, further comprising the operation of using a current situation state of the interaction to determine a response in the setting by the image of the real person, wherein the image of the real person is a computer-controlled player in the game.

57. The method of claim 56, further comprising the operation of interacting the decision logic with a video controller to modify the image of the real person.

58. The method of claim 57, further comprising the operation of modifying the image of the real person by making a direct change to the image of the real person.

59. The method of claim 57, further comprising the operation of modifying the image of the real person using a library of videos.

60. The method of claim 59,

further comprising presenting words on the display completely within the outside edges of the background scene on the display.

62. The method of claim 60,

wherein the interaction is a board game.

63. The method of claim 62,

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area.

64. The method of claim 63,

wherein the real world background scene includes trees, grass, and a sky.

65. The method of claim 64, further comprising the operation of putting the human operator'"'"'s course of action in place and sending it to the display.

66. The method of claim 65,

wherein the interaction is a game of checkers.

67. The method of claim 66, further comprising the operation of putting a selection of a control of the image of the real person and a selection of a control of the real world background scene in place, and sending the selected controls to the display.

68. The method of claim 67,

wherein the operations further comprise presenting text on the display indicating a player'"'"'s turn in the game, wherein the text is presented completely within the outside edges of the real world background scene on the display, and wherein the text is presented on the display above the two-dimensional representation of the playing area.

69. The method of claim 68, further comprising the operation of relating the current state and current triggers to a course of action and to videos to determine the most appropriate update to the setting.

70. The method of claim 69, further comprising the operation of presenting courses of action to the display after utilizing the decision logic, wherein the decision logic is a computer algorithm.

71. The method of claim 70, further comprising the operation of interacting with a video controller to modify the image of the real person and the real world background scene in the environment after utilizing the decision logic.

72. The method of claim 42, wherein the operation of presenting the background comprises continuously streaming video.

73. The method of claim 71, further comprising the operation of providing information regarding a current situational state on the display.

74. The method of claim 73, wherein the interaction is a game, and wherein the playing area includes a playing board, and wherein the playing area further includes playing pieces.

75. The method of claim 74, wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area include the state of the interaction.

76. The method of claim 75, wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area include the positions of the pieces on the playing board representing the state of the game.

77. The method of claim 76, wherein the three-dimensional representation of the playing area is a view as would be seen in real-life from a player'"'"'s perspective.

78. The method of claim 77, wherein the two-dimensional representation of the playing area is a top-down view of the playing area.

79. The method of claim 78, wherein the two-dimensional representation of the playing area shows all corresponding movement in both time and space from the game shown in the three-dimensional representation.

80. A method for presenting a digital environment for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area;

inputting an operator'"'"'s choice of action or inaction;

updating a current state based on the operator'"'"'s action or inaction;

using the current state by a decision logic to determine a response in the environment by the image of the real person; and

modifying the image of the real person using a library of videos.

81. The method of claim 80, further comprising the operations of:

using the current state by the decision logic to determine how to control the real world background scene in the environment;

interacting with a video controller to modify the background after utilizing the decision logic; and

sending a selection of control of the background to the display.

82. The method of claim 81, wherein the operation of presenting the background comprises continuously streaming video.

83. The method of claim 82,

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.

84. A method for presenting a virtual reality environment for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area;

inputting an operator'"'"'s choice of action or inaction;

updating a current state based on the operator'"'"'s action or the operator'"'"'s inaction;

using the current state by a decision logic to determine a response by the image of the real person; and

modifying the image of the real person; and

wherein the interaction is a game, and wherein the playing area includes a playing board, and wherein the playing area further includes playing pieces.

85. The method of claim 84, further comprising the following operations:

using the current state by the decision logic to determine how to control the background in the environment; and

interacting with a video controller to modify the background after the decision logic uses the current state.

86. The method of claim 85,

wherein the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are not contiguous on the display.

87. A method for presenting a virtual reality setting for an interaction, the method comprising the following operations:

presenting on a display a series of concatenated pictures of the real world to provide a real world background scene having life-like movement;

presenting on the display a series of video clips of a real person, wherein the series of video clips present the appearance of a continuous and moving image of the real person on the display;

wherein the series of video clips of the real person are presented on the display contiguous with the real world background scene and completely within outside edges of the real world background scene on the display;

presenting on the display a three-dimensional representation of a playing area, wherein the three-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display;

presenting on the display a two-dimensional representation of the playing area, wherein the two-dimensional representation of the playing area is presented on the display contiguous with the real world background scene and completely within the outside edges of the real world background scene on the display, and wherein the two-dimensional representation of the playing area is partially transparent to permit viewing a portion of the real world background scene through the two-dimensional representation of the playing area;

wherein the series of video clips of the real person, and the three-dimensional representation of the playing area, and the two-dimensional representation of the playing area are presented in separate nonoverlapping areas within the outside edges of the real world background scene;

wherein the real world background scene and the series of video clips of the real person and the three-dimensional representation of the playing area and the two-dimensional representation of the playing area are visible substantially simultaneously on the display;

wherein the two-dimensional representation of the playing area and the three-dimensional representation of the playing area show a same state of the interaction, and wherein the two-dimensional representation of the playing area shows all movement shown in the three-dimensional representation of the playing area;

inputting an operator'"'"'s choice of action or inaction;

updating a current state based on the operator'"'"'s action or inaction;

using the current state by a decision logic to determine a response in the virtual reality setting by the image of the real person; and

using the current state by the decision logic to determine a selection of how to control the image of the real person on the display and how to control the real world background scene on the display; and

wherein the interaction is a board game, and wherein the playing area includes a playing board, and wherein the playing area further includes playing pieces.

88. The method of claim 87,

wherein the series of video clips of the real person are presented on the display contiguous with a portion of the three-dimensional representation of the playing area.