Apparatus, methods, and systems for viewing and manipulating a virtual environment

• US 7,403,220 B2
• Filed: 07/09/2005
• Issued: 07/22/2008
• Est. Priority Date: 08/23/2004
• Status: Expired due to Fees

• Current Assignee: Gamecaster Incorporated
• Original Assignee: Gamecaster Incorporated
• Inventors: Morelock, David, MacIntosh, David, Grand, Joseph
• Primary Examiner(s): Hannett; James M
• Application Number:
  US11/177,049
• Publication Number:
  US 20060038890A1
• Time in Patent Office: 1,109 Days
• Field of Search: 348/208.2
• US Class Current: 348/208.2
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/98 Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1062 being specially adapted to a type of game, e.g. steering wheel
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/6669 using a plurality of virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character change rooms
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

Components for generating, reviewing, processing, storing, retrieving, analyzing, and optimizing information input to or output from a virtual environment. The components may be a video camera component, a stand-alone or device mountable on a tripod to permit a camera person to select from one or more views within a virtual environment, such as a video game being executed, and to provide alternative views within the confines of the virtual environment. The component permits selection from multiple views and manipulation of the same, including a rotation leftward or rightward, a tilt upward or downward, a zooming inward or outward, a translation (for example, a track movement) leftward, rightward, forward, or backward, a rolling movement, and a camera position adjustment (for example, crane movement) upward or downward.

Claims

1. An apparatus comprising:

a camera control input device configured to provide a camera control input device signal in response to a user action;

a movement sensor configured to provide a movement sensor signal in accordance with a movement of the apparatus, the movement sensor comprising any one of;

a horizontal angular rate sensor configured to provide a horizontal movement signal in accordance with a horizontal rate of movement of the apparatus;

a vertical angular rate sensor configured to provide a vertical movement signal in accordance with a vertical rate of movement of the apparatus;

a horizontal angular acceleration sensor configured to provide the horizontal movement signal in accordance with a horizontal acceleration of the apparatus; and

a vertical angular rate sensor configured to provide the vertical movement signal in accordance with a vertical acceleration of the apparatus; and

a control signal generator configured to generate a camera control signal comprising a virtual camera control signal for controlling a virtual camera within a virtual environment that is entirely generated by a software application, the virtual environment being generated without images captured by a camera with an image sensor, and the virtual camera control signal being based on the camera control input device signal and the movement sensor signal.

2. An apparatus in accordance with claim 1, wherein the camera control signal comprises at least one of a plurality of camera commands comprising:

a crane command;

a truck command;

a zoom command;

a roll command;

a pan command; and

a tilt command.

3. An apparatus in accordance with claim 1, wherein said virtual environment comprises any one of:

a video game software application;

a digital filmmaking application;

a three-dimensional animation application;

a computer aided design (CAD) or computer aided manufacturing (CAM) application;

a holography application;

a medical analysis or surgical analysis application;

simulations of military equipment and personnel readiness war games applications;

flight simulation training applications;

modeling simulations for biological analyses applications;

modeling simulations for chemical analyses applications;

modeling simulations for molecular analyses applications; and

simulations relating to graphical rendering of building facilities applications.

4. An apparatus in accordance with claim 1, wherein a format of the virtual camera control signal is in accordance with any one of a computer mouse output, a computer keyboard output, and a video game controller output.

5. An apparatus in accordance with claim 1, wherein the controller comprises:

a processor programmed to convert the camera control input device signal and the movement sensor signal into the virtual camera control signal; and

an interface configured to generate, based on the virtual camera control signal, computer input signals emulating any one of video game controller output signals, computer keyboard output signals and computer mouse output signals.

6. An apparatus in accordance with claim 1, wherein the camera control input device comprises at least one switch.

7. An apparatus in accordance with claim 1, wherein the camera control input device comprises at least one finger operated control device.

8. An apparatus in accordance with claim 7, wherein the at least one finger operated device comprises a plurality of thumb controllers.

9. An apparatus comprising:

a camera control input device configured to provide a camera control input device signal in response to user actions;

a horizontal movement sensor configured to provide a horizontal movement signal in accordance with a horizontal movement of the apparatus, the horizontal movement sensor comprising any one of a horizontal angular acceleration sensor configured to provide the horizontal movement signal in accordance with a horizontal acceleration of the apparatus and a horizontal angular rate sensor configured to provide the horizontal movement signal in accordance with a horizontal rate of movement of the apparatus;

a vertical movement sensor configured to provide a vertical movement signal in accordance with a vertical movement of the apparatus, the vertical movement sensor comprising any one of a vertical angular rate sensor configured to provide the vertical movement signal in accordance with a vertical rate of movement of the apparatus and a vertical angular rate sensor configured to provide the vertical movement signal in accordance with a vertical acceleration of the apparatus; and

a control signal generator configured to generate a virtual camera control signal based on at least one of a plurality of signals comprising the horizontal movement signal, the vertical movement signal, and the camera control input device signal and interpretable by a software application as an instruction for changing a view within a virtual environment that is entirely generated by the software application, the virtual environment being generated without images captured by a camera with an image sensor.

10. An apparatus in accordance with claim 9, wherein the virtual camera control signal results in a movement of the virtual camera within the virtual environment emulating the movement of the apparatus.

11. An apparatus in accordance with claim 10, wherein the control signal generator is further configured to generate the virtual camera control signal comprising a pan command in response to a rotation of the apparatus about an axis orthogonal to the ground plane.

12. An apparatus in accordance with claim 10, wherein the control signal generator is further configured to generate the virtual camera control signal comprising any one of a tilt command and a roll command in response to rotations of the apparatus about axes parallel to the ground plane.

13. An apparatus in accordance with claim 9, wherein the virtual camera control signal comprises at least one of a plurality of camera commands for controlling a virtual camera within a virtual environment generated by the software application, the plurality of camera commands comprising:

a crane up command;

a crane down command;

a truck left command;

a truck right command;

a truck forward (global) command;

a truck backward (global) command;

a pan right command;

a pan left command;

a tilt up command;

a tilt down command;

a roll left command;

a roll right command;

a zoom in command; and

a zoom out command.

14. An apparatus in accordance with claim 9, wherein the software application is any one of:

a video game software application

a digital filmmaking application;

a three-dimensional animation application;

a computer aided design (CAD) or computer aided manufacturing (CAM) application;

a holography application;

a medical analysis or surgical analysis application;

simulations of military equipment and personnel readiness war games applications;

flight simulation training applications;

modeling simulations for biological analyses applications;

modeling simulations for chemical analyses applications;

modeling simulations for molecular analyses applications; and

simulations relating to graphical rendering of building facilities applications.

15. An apparatus in accordance with claim 9, further comprising a visual output device configured to present at least a portion of the virtual environment.

16. An apparatus in accordance with claim 15, wherein the at least a portion of the virtual environment is the view of the virtual environment captured by the virtual camera.

17. A camera simulating device for controlling a virtual camera capturing a view within a virtual environment that is entirely generated by a video software application, the virtual environment being generated without images captured by a camera with an image sensor, the camera simulating device comprising:

a horizontal sensor configured to provide a horizontal movement signal in accordance with a horizontal movement of the camera simulating device, the horizontal sensor comprising any one of a horizontal angular acceleration sensor configured to provide the horizontal movement signal in accordance with a horizontal acceleration of the apparatus and a horizontal angular rate sensor configured to provide the horizontal movement signal in accordance with a horizontal rate of movement of the apparatus;

a vertical sensor configured to provide a vertical movement signal in accordance with a vertical movement of the camera simulating device, the vertical sensor comprising any one of a vertical angular rate sensor configured to provide the vertical movement signal in accordance with a vertical rate of movement of the apparatus and a vertical angular rate sensor configured to provide the vertical movement signal in accordance with a vertical acceleration of the apparatus;

a camera control input device configured to provide a camera control input device signal in response to user actions;

a control signal generator configured to generate a virtual camera control signal based on at least one of a plurality of signals comprising the vertical movement signal, the horizontal movement signal, and the camera control input device signal, the virtual camera control signal recognizable by the video software application as at least one of a plurality of commands to change the view captured by the virtual camera of the virtual environment.

18. A camera simulating device in accordance with claim 17, wherein the virtual camera control signal results in a movement of the virtual camera within the virtual environment emulating the movement of the camera simulating device.

19. A camera simulating device in accordance with claim 18, wherein the control signal generator is further configured to generate the virtual camera control signal comprising a pan command in response to a rotation of the camera simulating device about an axis orthogonal to the ground plane.

20. A camera simulating device in accordance with claim 18, wherein the control signal generator is further configured to generate the virtual camera control signal comprising any one of a tilt command and a roll command in response to rotations of the camera simulating device about axes parallel to the ground plane.

21. A camera simulating device in accordance with claim 17, wherein the virtual camera control signal comprises at least one of a plurality of camera commands for controlling a virtual camera within a virtual environment generated by the video software application, the plurality of camera commands comprising:

a crane up command;

a crane down command;

a truck left command;

a truck right command;

a truck in command;

a truck out command;

a pan right command;

a pan left command;

a tilt up command;

a tilt down command;

a roll left command;

a roll right command;

a zoom in command; and

a zoom out command.

22. A camera simulating device in accordance with claim 17, further comprising a visual output device configured to present the view of the virtual environment captured by the virtual camera, and wherein the virtual environment comprises any one of:

a video game software application;

a digital filmmaking application;

a three-dimensional animation application;

a computer aided design (CAD) or computer aided manufacturing (CAM) application;

a holography application;

a medical analysis or surgical analysis application;

simulations of military equipment and personnel readiness war games applications;

flight simulation training applications;

modeling simulations for biological analyses applications;

modeling simulations for chemical analyses applications;

modeling simulations for molecular analyses applications; and

simulations relating to graphical rendering of building facilities applications.

23. A camera simulating device in accordance with claim 22, wherein the view of the virtual environment is a spectator view generated by the video game software application.

24. A method comprising:

receiving a camera control input device signal from a camera control input device;

receiving a movement sensor signal based on a movement of the user input device, the movement sensor signal comprising any one of a vertical rate signal, a vertical acceleration signal, a horizontal rate signal, and a horizontal acceleration signal; and

generating a camera control signal comprising generating a virtual camera control signal for controlling a virtual camera capturing a view within a virtual environment that is entirely generated by a software application, the virtual environment being generated without images captured by a camera with an image sensor, and the generating a virtual camera control signal being based on the camera control input device signal and the movement sensor signal.

25. A method in accordance with claim 24, wherein the camera control signal comprises at least one of a plurality of camera commands comprising:

a crane command;

a truck command;

a roll command;

a zoom command;

a pan command; and

a tilt command.

26. A method in accordance with claim 24, wherein the movement sensor signal is generated by any one of an angular rate sensor and an acceleration sensor.

27. A method in accordance with claim 26, wherein the movement sensor signal comprises:

a horizontal movement signal in accordance with a horizontal movement of the user input device; and

a vertical movement signal in accordance with a vertical movement of the user input device.

28. A method in accordance with claim 24, wherein said virtual environment comprises any one of:

a video game software application;

a digital filmmaking application;

a three-dimensional animation application;

a computer aided design (CAD) or computer aided manufacturing (CAM) application;

a holography application;

a medical analysis or surgical analysis application;

simulations of military equipment and personnel readiness war games applications;

flight simulation training applications;

modeling simulations for biological analyses applications;

modeling simulations for chemical analyses applications;

modeling simulations for molecular analyses applications; and

simulations relating to graphical rendering of building facilities applications.

29. A method in accordance with claim 24, wherein a format of the virtual camera control signal is in accordance with any one of a computer mouse output, a computer keyboard output and a video game controller output.

30. A method in accordance with claim 27, wherein generating the camera control signal further comprises:

converting the camera control input device signal and the movement sensor signal into the virtual camera control signal; and

generating, based on the camera input control device signal, computer input signals emulating any one of;

video game controller signals, computer keyboard signals and computer mouse signals.

31. A method in accordance with claim 24, wherein the camera control input device comprises at least one switch.

32. A method in accordance with claim 24, wherein the camera control input device comprises at least one finger operated control device.

33. A method in accordance with claim 32, wherein the at least one finger operated control device comprises one or more thumb controls.

38. A method in accordance with claim 24, wherein the virtual camera control signal comprises at least one of a plurality of camera commands for controlling a virtual camera within a virtual environment generated by the software application, the plurality of camera commands comprising:

a crane up command;

a crane down command;

a truck left command;

a truck right command;

a truck in command;

a truck out command;

a pan right command;

a pan left command;

a tilt up command;

a tilt down command;

a roll left command;

a roll right command;

a zoom in command; and

a zoom out command.

40. A method in accordance with claim 32, further comprising presenting at least a portion of the virtual environment through a visual output device.

41. A method in accordance with claim 40, wherein the at least a portion of the virtual environment is the view of the virtual environment captured by the virtual camera.

34. A method comprising:

receiving a horizontal movement signal in accordance with a horizontal movement of a control apparatus, the horizontal movement signal comprising any one of a horizontal rate signal and a horizontal acceleration signal;

receiving a vertical movement signal in accordance with a vertical movement of the control apparatus, the vertical movement signal comprising any one of a vertical rate signal, and a vertical acceleration signal;

receiving a camera control signal;

generating a virtual camera control signal based on at least one of a plurality of signals comprising the vertical movement signal, the horizontal movement signal, and the camera control signal, the virtual camera control signal interpretable by a software application as an instruction for changing a view within a virtual environment that is entirely generated by the software application, the virtual environment being generated without images captured by a camera with an image sensor.

35. A method in accordance with claim 34, wherein the virtual camera control signal results in a movement of the virtual camera within the virtual environment emulating the movement of the control apparatus.

36. A method in accordance with claim 35, wherein the generating step comprises generating the virtual camera control signal comprising a pan command in response to a rotation of the control apparatus about an axis orthogonal to the ground plane.

37. A method in accordance with claim 35, wherein the generating step comprises generating the virtual camera control signal comprising a tilt command or a roll command in response a rotation of the control apparatus about axes parallel to the ground plane.

39. A method in accordance with claim 34, wherein the software application is any one of:

a video game software application;

a digital filmmaking application;

a three-dimensional animation application;

a computer aided design (CAD) or computer aided manufacturing (CAM) application;

a holography application;

a medical analysis or surgical analysis application;

simulations of military equipment and personnel readiness war games applications;

flight simulation training applications;

modeling simulations for biological analyses applications;

modeling simulations for chemical analyses applications;

modeling simulations for molecular analyses applications; and

simulations relating to graphical rendering of building facilities applications.

42. A method performed in a camera simulating device for generating a virtual camera control signal for controlling a virtual camera capturing a view within a virtual environment that is entirely generated by a video software application, the virtual environment being generated without images captured by a camera with an image sensor, the method comprising:

receiving a horizontal movement signal of the camera simulating device, the horizontal movement signal comprising any one of a horizontal rate signal and a horizontal acceleration signal;

receiving a vertical movement signal in accordance with a vertical movement of the camera simulating device, the vertical movement signal comprising any one of a vertical rate signal, and a vertical acceleration signal;

receiving a camera control signal generated by a camera control input in response to user actions;

generating a virtual camera control signal based on at least one of a plurality of signals comprising the vertical movement signal, the horizontal movement signal, and the camera control signal, the virtual camera control signal recognizable by the video software application as at least one of a plurality of camera commands to change the view captured by the virtual camera within the virtual environment.

43. A method in accordance with claim 42, wherein the virtual camera control signal results in a movement of the virtual camera within the virtual environment emulating the movement camera simulating device.

44. A method in accordance with claim 43, wherein the generating comprises generating a pan command in response to a rotation of the camera simulating device about an axis orthogonal to the ground plane.

45. A method in accordance with claim 43, wherein the generating comprises generating any one of a tilt command and a roll command in response to rotations of the camera simulating device about axes parallel to the ground plane.

46. A method in accordance with claim 42, wherein the virtual camera control signal comprises at least one of a plurality of camera commands for controlling a virtual camera within a virtual environment generated by the software application, the plurality of camera commands comprising:

a crane up command;

a crane down command;

a truck left command;

a truck right command;

a truck in command;

a truck out command;

a pan right command;

a pan left command;

a tilt up command;

a tilt down command;

a roll left command;

a roll right command;

a zoom in command; and

a zoom out command.