Multipoint SLAM capture

• US 10,549,186 B2
• Filed: 06/26/2018
• Issued: 02/04/2020
• Est. Priority Date: 06/26/2018
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Taylor, Michael, Black, Glenn, Rico, Javier Fernandez
• Primary Examiner(s): D'Agostino, Paul A
• Application Number:
  US16/019,140
• Publication Number:
  US 20190388781A1
• Time in Patent Office: 588 Days
• Field of Search: None
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: A63F 13/843 involving concurrently two or more players on the same game device, e.g. requiring the use of a plurality of controllers or of a specific view of game data for each player
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/579 from motion

Abstract

“Feature points” in “point clouds” that are visible to multiple respective cameras (i.e., aspects of objects imaged by the cameras) are reported via wired and/or wireless communication paths to a compositing processor which can determine whether a particular feature point “moved” a certain amount relative to another image. In this way, the compositing processor can determine, e.g., using triangulation and recognition of common features, how much movement occurred and where any particular camera was positioned when a latter image from that camera is captured. Thus, “overlap” of feature points in multiple images is used so that the system can close the loop to generate a SLAM map. The compositing processor, which may be implemented by a server or other device, generates the SLAM map by merging feature point data from multiple imaging devices.

Claims

1. A storage device comprising:

at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to;

receive, from a first device, at least first feature points in a point cloud;

receive, from a second device, at least second feature points in a point cloud;

generate a simultaneous localization and mapping (SLAM) map using the first and second feature points; and

determine a distance to a first feature point at least in part based on a size of the first feature point.

2. The storage device of claim 1, wherein the instructions are executable to:

exchange device identifying information with the second device.

3. The storage device of claim 1, wherein the first feature points comprise at least portions of an image of the second device.

4. The storage device of claim 1, wherein the instructions are executable to:

receive from at least a third device third feature points; and

generate the SLAM map using the third feature points.

6. The storage device of claim 1, wherein the first and second devices are implemented as computer game headsets.

5. A storage device comprising:

at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to;

receive, from a first device, at least first feature points in a point cloud;

receive, from a second device, at least second feature points in a point cloud; and

generate a simultaneous localization and mapping (SLAM) map using the first and second feature points; and

responsive to determining that the SLAM map has a blind spot, alter presentation of a computer game on at least one of the first and second devices.

18. The storage device of claim 5, wherein the instructions are executable to:

exchange device identifying information with the second device.

19. The storage device of claim 5, wherein the first feature points comprise at least portions of an image of the second device.

20. The storage device of claim 5, wherein the instructions are executable to:

receive from at least a third device third feature points; and

generate the SLAM map using the third feature points.

21. The storage device of claim 5, wherein the first and second devices are implemented as computer game headsets.

30. The storage device of claim 5, wherein the instructions are executable to:

determine a distance to a first feature point at least in part based on a size of the first feature point.

7. An assembly comprising:

at least first and second computer game headsets, each comprising at least one respective camera;

a processor programmed with instructions to;

generate, using the respective camera of the first computer game headset, at least a first image of feature points in a point cloud;

receive from at least the second computer game headset, at least a second image of feature points in a point cloud;

generate a SLAM map using the first and second images; and

determine a distance to a computer game headset corresponding to feature points in the first image least in part based on a size of the feature points.

8. The assembly of claim 7, wherein the instructions are executable to:

exchange device identifying information with the second device.

9. The assembly of claim 7, wherein the feature points from the second device includes an image of the first computer game headset taken by the second computer game headset.

10. The assembly of claim 7, wherein the instructions are executable to:

receive from at least a third computer game headset information including an image of feature points in a point cloud; and

generate the SLAM map using the third information.

11. An assembly comprising:

at least first and second computer game headsets, each comprising at least one respective camera;

a processor programmed with instructions to;

generate, using the respective camera of the first computer game headset, at least a first image of feature points in a point cloud;

receive from at least the second computer game headset, at least a second image of feature points in a point cloud;

generate a SLAM map using the first and second images; and

responsive to determining that the SLAM map has a blind spot, alter presentation of a computer game on at least one of the first and second computer game headsets.

12. The assembly of claim 11, wherein the presentation is altered at least in part by prompting a turn toward the blind spot.

22. The assembly of claim 11, wherein the instructions are executable to:

exchange device identifying information with the second device.

23. The assembly of claim 11, wherein the feature points from the second device includes an image of the first computer game headset taken by the second computer game headset.

24. The assembly of claim 11, wherein the instructions are executable to:

receive from at least a third computer game headset information including an image of feature points in a point cloud; and

generate the SLAM map using the third information.

29. The assembly of claim 11, wherein the instructions are executable to:

determine a distance to a computer game headset corresponding to feature points in the first image least in part based on a size of the feature points.

13. A method comprising:

receiving, from a first device, at least first feature points in a point cloud;

receiving, from a second device, at least second feature points in a point cloud;

generating a simultaneous localization and mapping (SLAM) map using the first and second feature points; and

determining a distance to a first feature point at least in part based on a size of the first feature point.

14. The method of claim 13, comprising:

exchanging device identifying information with the second device.

15. The method of claim 13, wherein the first feature points comprise at least portions of an image of the second device.

16. The method of claim 13, comprising:

receiving from at least a third device third feature points; and

generating the SLAM map using the third feature points.

17. A method comprising:

receiving, from a first device, at least first feature points in a point cloud;

receiving, from a second device, at least second feature points in a point cloud;

generating a simultaneous localization and mapping (SLAM) map using the first and second feature points; and

responsive to determining that the SLAM map has a blind spot, altering presentation of a computer game on at least one of the first and second devices.

25. The method of claim 17, comprising:

exchanging device identifying information with the second device.

26. The method of claim 17, wherein the first feature points comprise at least portions of an image of the second device.

27. The method of claim 17, comprising:

receiving from at least a third device third feature points; and

generating the SLAM map using the third feature points.

28. The method of claim 17, comprising:

determining a distance to a first feature point at least in part based on a size of the first feature point.