Maintaining Multiple Views on a Shared Stable Virtual Space

• US 20110216060A1
• Filed: 11/16/2010
• Published: 09/08/2011
• Est. Priority Date: 03/05/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Miller, Thomas, Weising, George
• Granted Patent: US 8,730,156 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2009/2457 Display screens, e.g. monitors, video displays
• CPC Class Codes: A63F 3/02 Chess; Similar board games
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2320/0261 : in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 5/08 : Cursor circuits
• CPC Class Codes: H04N 23/6812 : based on additional sensors, e.g. acceleration sensors

Abstract

Methods, apparatus, and computer programs for controlling a view of a virtual scene with a portable device are presented. In one method, a signal is received and the portable device is synchronized to make the location of the portable device a reference point in a three-dimensional (3D) space. A virtual scene, which includes virtual reality elements, is generated in the 3D space around the reference point. Further, the method determines the current position in the 3D space of the portable device with respect to the reference point and creates a view of the virtual scene. The view represents the virtual scene as seen from the current position of the portable device and with a viewing angle based on the current position of the portable device. Additionally, the created view is displayed in the portable device, and the view of the virtual scene is changed as the portable device is moved by the user within the 3D space. In another method, multiple players shared the virtual reality and interact among each other view the objects in the virtual reality.

Claims

1. A method for controlling a view of a virtual scene with a portable device, the method comprising:

receiving a signal to synchronize the portable device;

synchronizing the portable device to make a location where the portable device is located a reference point in a three-dimensional (3D) space;

generating a virtual scene in the 3D space around the reference point, the virtual scene including virtual reality elements;

determining a current position in the 3D space of the portable device with respect to the reference point;

creating a view of the virtual scene, wherein the view represents the virtual scene as seen from the current position of the portable device and with a viewing angle based on the current position of the portable device;

displaying the created view in the portable device; and

changing the displayed view of the virtual scene as the portable device is moved by a user within the 3D space.

2. The method as recited in claim 1, wherein creating the view of the virtual scene further includes:

capturing an image of the 3D space with a camera in the portable device, the image including real elements in the 3D space; and

blending the real elements with the virtual reality elements, wherein the virtual elements appear in the displayed view as if the virtual elements where part of the 3D space, wherein a view of the virtual elements changes, from a geometrical perspective, in a same way as a view of the real elements changes when the portable device moves within the 3D space.

3. The method as recited in claim 2, wherein the real elements include a hand of a player, the method further including:

detecting that the hand occupies a position in the 3D space where a first virtual element is located; and

enabling, after the detecting, an interaction of the hand with the first virtual element to simulate that the hand is touching the virtual element, wherein the hand is able to manipulate the first virtual element to change a position or a property of the first virtual element as if the first virtual element were a real object.

4. The method as recited in claim 3, wherein the interaction of the hand is an action on the first virtual element selected from a group consisting of interfacing, holding, pushing, pulling, grabbing, moving, smashing, squeezing, hitting, throwing, fighting, opening, closing, turning on or off, pushing a button, firing, or eating.

5. The method as recited in claim 2, wherein creating a view further includes:

adding a shadow of the hand over the virtual elements, according to lighting conditions in the 3D space and the virtual scene.

6. The method as recited in claim 2, wherein the real elements include a table, wherein blending further includes:

placing virtual elements on top of the table as if the placed virtual elements where resting on the table.

7. The method as recited in claim 2, wherein the real elements include a wall in a room, wherein blending further includes:

adding an illustration to the wall, the illustration being one of the virtual elements.

8. The method as recited in claim 1, wherein the current position of the portable device includes geometric coordinates of the portable device and geometric coordinates of a viewing surface of a display in the portable device.

9. The method as recited in claim 8, wherein the geometric coordinates of the portable device are equal to geometric coordinates of a camera in the portable device.

10. The method as recited in claim 9, wherein the viewing angle is defined in reference to a vector with origin in a center of the viewing surface of the display and a direction perpendicular to the viewing surface of the display.

11. The method as recited in claim 1, wherein creating the view of the virtual scene further includes:

zooming in a second virtual reality element when the portable device moves closer to the second virtual reality element; and

zooming out the second virtual reality element when the portable device moves away from the second virtual reality element.

12. The method as recited in claim 1, wherein changing the displayed view further includes:

adding image stabilization to the displayed view as the portable device moves.

13. The method as recited in claim 1, further including:

receiving an input to change views; and

changing the creation of the view of the virtual scene such that the view of the virtual scene is calculated from another point different from the current position of the portable device.

14. The method as recited in claim 13, wherein the view of the virtual scene is rotated 180 degrees with respect to a vertical line intersecting the reference point.

15. The method as recited in claim 1, wherein the received signal is generated by pushing a button on the portable device or by touching a touch-sensitive display of the portable device.

16. The method as recited in claim 1, wherein boundaries of the virtual scene are defined by walls of a room where the portable device is located when the signal was received.

17. The method as recited in claim 1, wherein synchronizing the portable device further includes:

resetting position tracking modules in the portable device, the position tracking modules being selected from a group consisting of an accelerometer, a magnetometer, a GPS device, a camera, a depth camera, a compass, or a gyroscope,

wherein the determining of the current position is performed with the information from the position tracking modules.

18. A method for sharing a virtual scene among devices, the method comprising:

synchronizing a first device to a reference point in a three-dimensional (3D) space;

calculating a location of a second device relative to a location of the first device;

exchanging information between the first device and the second device to have the second device synchronized to the reference point in the 3D space, the information including the reference point and the locations of both first and second devices;

generating a virtual scene in the 3D space around the reference point, the virtual scene being common to both devices, the virtual scene changing simultaneously in both devices as both devices interact with the virtual scene;

creating a view of the virtual scene as seen from a current position of the first device with a viewing angle based on the current position of the portable device;

displaying the created view in the first device; and

changing the displayed view of the virtual scene as the portable device moves within the 3D space.

19. The method as recited in claim 18, wherein calculating the location of the second device further includes:

gathering a first relative location information between the first device and the second device, the gathering including one or more of WiFi positional tracking, audio triangulation, or image analysis obtained from a camera in the first device;

determining the relative position of the second device with respect to the first device based on the first relative location information; and

sending the relative position and coordinates of the reference point to the second device.

20. The method as recited in claim 18, further including:

gathering a first relative location information between the first device and the second device, the gathering including one or more of WiFi positional tracking, audio triangulation, or image analysis obtained from a camera in the first device or the second device;

receiving a second relative location information from the second device;

determining the relative position of the second device with respect to the first device based on the first and second relative information; and

sending the relative position and coordinates of the reference point to the second device.

21. The method as recited in claim 18, wherein the virtual scene includes a virtual board game, wherein players holding the first device and the second device, respectively, play the virtual board game.

22. The method as recited in claim 18, wherein a first player holding the first portable device controls movement of a first virtual element in the virtual scene by moving the first portable device in synchronism with the first virtual element.

23. A method for controlling a view of a virtual scene with a first device, the method comprising:

synchronizing the first device to a first reference point in a first three-dimensional (3D) space;

establishing a communications link between the first device and a second device, the second device being in a second 3D space outside the first 3D space, the second device being synchronized to a second reference point in the second 3D space;

generating a common virtual scene that includes virtual reality elements, the common virtual scene being observable by both the first and the second devices, the first device building the common virtual scene around the first reference point, the second device building the common virtual scene around the second reference point, both devices being able to interact with the virtual reality elements;

determining a current position in the first 3D space of the first device with respect to the reference point;

creating a view of the common virtual scene, wherein the view represents the common virtual scene as seen from the current position of the first device and with a viewing angle based on the current position of the first device;

displaying the created view in the first device; and

changing the displayed view of the common virtual scene as the first device moves within the first 3D space.

24. The method as recited in claim 23, wherein the communications link includes a network connection between the first device and the second device.

25. The method as recited in claim 23, further including:

assigning a virtual location in the first 3D space to the second device;

receiving, from the second device, second device interaction information corresponding to interactions between the second device and the common virtual scene; and

changing the view of the common virtual scene according to the received second device interaction information and the virtual location, wherein the second device appears to be in the first 3D space and interacting with the first device as if the second device were physically located at the first 3D space.

26. The method as recited in claim 23, wherein the view of the common virtual scene further includes an image of a second player located near the second device, the second device having a camera for capturing the image of the second player.

27. The method as recited in claim 26, further including:

updating the image of the second player periodically.

28. The method as recited in claim 26, further including:

updating the image of the second player when the second player moves.

29. The method as recited in claim 26, wherein the virtual elements include a chess board and chess pieces, wherein the first device and the second device are used to play a game of chess by manipulating the chess pieces.

30. The method as recited in claim 26, wherein the first and second devices are represented in the view of the common virtual scene as a first object and second object, respectively, wherein movement of the first object matches movement of the first device in the first 3D space and movement of the second object matches movement of the second device in the second 3D space.

31. A method for controlling a view of a virtual scene with a portable device, the method comprising:

synchronizing a portable device to a reference point in a three-dimensional (3D) space where the portable device is located, the portable device including a front camera facing the front of the portable device and a rear camera facing the rear of the portable device;

generating a virtual scene in the 3D space around the reference point, the virtual scene including virtual reality elements;

determining a current position in the 3D space of the portable device with respect to the reference point;

creating a view of the virtual scene, the view capturing a representation of the virtual scene as seen from a current eye position in the 3D space of a player holding the portable device, the capturing corresponding to what the player would see through a window into the virtual scene, the window'"'"'s position in the 3D space being equal to a position in the 3D space of a display in the portable device;

displaying the created view in the display; and

changing the displayed view of the virtual scene as the portable device or the player move within the 3D space.

32. The method as recited in claim 31, wherein images from the front camera are used to determine the current eye position and images from the rear camera are used to get a view of the 3D space.

33. The method as recited in claim 31, wherein pulling the display away from the player'"'"'s eyes causes the view to zoom in the virtual scene and pulling the display closer to the player'"'"'s eyes causes the view to zoom out of the virtual scene.

34. A method for controlling a view of a scene with a portable device, the method comprising:

receiving a signal to synchronize the portable device;

synchronizing the portable device to make a location where the portable device is located a reference point in a three-dimensional (3D) space;

generating a virtual scene in the 3D space around the reference point, the virtual scene including virtual reality elements;

creating a view of the virtual scene as the portable device is moved away from the reference point, wherein the view represents the virtual scene as seen from a current position of the portable device; and

displaying the created view in the portable device; and

changing the displayed view of the virtual scene as the portable device moves within the 3D space.

35. The method as recited in claim 34, further including:

determining a virtual element that generates sound;

emitting a sound from the portable device corresponding to the sound generated by the virtual element, wherein the sound emitted becomes louder as the portable device is moved closer to a position of the virtual element that generates sound.

36. The method as recited in claim 35, wherein the portable device has stereo speakers, wherein the sound emitted is adjusted to provide an stereo effect according to a relative location between the portable device and the virtual element that generates sound.

37. A portable device for interacting with an augmented reality, the portable device comprising:

a position module for determining a position of the portable device in a 3D space where the portable device is located, wherein the position of the portable device is set as a reference point in the 3D space at a time when the portable device receives a signal to synchronize;

a virtual reality generator that creates a virtual scene in the 3D space around the reference point, the virtual scene including virtual reality elements;

a view generator that creates a view of the virtual scene, wherein the view represents the virtual scene as seen from the position of the portable device and with a viewing angle based on the position of the portable device; and

a display for showing the view of the virtual scene, the view shown in the display changing as the portable device moves within the 3D space.

38. A computer program embedded in a computer-readable storage medium, when executed by one or more processors, for sharing a virtual scene among devices, the computer program comprising:

program instructions for synchronizing a first device to a reference point in a three-dimensional (3D) space;

program instructions for calculating a location of a second device relative to a location of the first device;

program instructions for exchanging information between the first device and the second device to have the second device synchronized to the reference point in the 3D space, the information including the reference point and the locations of both first and second devices;

program instructions for generating a virtual scene in the 3D space around the reference point, the virtual scene being common to both devices, the virtual scene changing simultaneously in both devices as both devices interact with the virtual scene;

program instructions for creating a view of the virtual scene as seen from a current position of the first device with a viewing angle based on the current position of the portable device;

program instructions for displaying the created view in the first device; and

program instructions for changing the displayed view of the virtual scene as the portable device moves within the 3D space.