Systems and methods for machine perception

• US 10,261,183 B2
• Filed: 12/23/2017
• Issued: 04/16/2019
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: Samsung Semiconductor Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Ratcliffe, Luke D
• Application Number:
  US15/853,783
• Publication Number:
  US 20180180733A1
• Time in Patent Office: 479 Days
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4868 Controlling received signal intensity or exposure of sensor
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system to determine a position of one or more objects includes a transmitter to emit a beam of photons to sequentially illuminate regions of one or more objects; multiple cameras that are spaced-apart with each camera having an array of pixels to detect photons; and one or more processor devices that execute stored instructions to perform actions of a method, including: directing the transmitter to sequentially illuminate regions of one or more objects with the beam of photons; for each of the regions, receiving, from the cameras, an array position of each pixel that detected photons of the beam reflected or scattered by the region of the one or more objects; and, for each of the regions detected by the cameras, determining a position of the regions using the received array positions of the pixels that detected the photons of the beam reflected or scattered by that region.

Claims

1. A system to determine a position of one or more objects, comprising:

a transmitter configured to emit a beam of photons to sequentially illuminate regions of one or more objects;

a plurality of cameras that are spaced-apart from each other, each camera comprising an array of pixels, wherein each of the pixels is configured to detect photons received by the pixel;

one or more memory devices that store instructions; and

one or more processor devices that execute the stored instructions to perform actions of a method, including;

directing the transmitter to sequentially illuminate each of a plurality of regions of one or more objects with the beam of photons for a predetermined period of time;

for each of the regions, receiving, from the plurality of cameras, an array position of each pixel in the plurality of cameras that detected photons of the beam reflected or scattered by the region of the one or more objects within a time interval that is equal to or less than the predetermined period of time; and

for each of the regions detected by a plurality of the cameras, determining a position of the regions using the received array positions of the pixels in the plurality of cameras that detected the photons of the beam reflected or scattered by that region.

2. The system of claim 1, wherein directing the transmitter comprises directing the transmitter to sequentially illuminate each region of the plurality of regions for no more than a predetermined time period.

3. The system of claim 1, wherein directing the transmitter comprises directing the transmitter to sequentially illuminate each region of the plurality of regions for no more than 50 nanoseconds.

4. The system of claim 1, wherein directing the transmitter comprises directing the transmitter to sequentially illuminate each region of the plurality of regions for no more than 10 nanoseconds.

5. The system of claim 1, wherein each of the cameras comprises a two-dimensional array of pixels and the array position comprises a two-coordinate position of the pixel in the two-dimensional array.

6. The system of claim 1, further comprising a body and a plurality of appendages, wherein each of the cameras is disposed on one of the appendages.

7. The system of claim 1, further comprising a body and a plurality of appendages, wherein the transmitter is disposed on one of the appendages.

8. The system of claim 1, further comprising a body and a plurality of appendages, wherein each of the cameras is disposed on one of the appendages, wherein the actions further comprise:

moving one or more of the appendages until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.

9. The system of claim 1, further comprising a body and a plurality of appendages, wherein the transmitter is disposed on one of the appendages and wherein the actions further comprise:

moving the appendage on which the transmitter is disposed until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.

10. The system of claim 1, wherein the actions further comprise:

identifying a plurality of points of interest (POIs) on the one or more objects;

determining first positions of the POIs;

at a later time, determining second positions of the POIs; and

predicting a trajectory of the POIs based on the first and second positions of the POIs.

11. The system of claim 1, wherein the system is mountable on a head of a user.

12. The system of claim 1, wherein the transmitter is configured to emit beams of at least three different colors and at least one of the cameras is configured to differentiate between photons of the three different colors.

13. The system of claim 1, wherein the transmitter and at least one of the cameras are part of a vehicle.

14. The system of claim 1, wherein the transmitter and all of the cameras are part of a vehicle.

15. The system of claim 1, wherein directing the transmitter comprising directing the transmitter to sequentially illuminate, according to a predetermined sequence of colors, the plurality of the regions of the one or more objects with the beam of photons.

16. The system of claim 1, directing the transmitter comprising directing the transmitter to sequentially illuminate, according to a predetermined sequence of beam intensities, the plurality of the regions of the one or more objects with the beam of photons.

17. A method for determining a position of one or more objects, comprising:

directing a transmitter to sequentially illuminate each of a plurality of regions of one or more objects with the beam of photons for a predetermined period of time;

for each of the regions, receiving, from a plurality of cameras, an array position of each pixel in the plurality of cameras that detected photons of the beam reflected or scattered by the region of the one or more objects within a time interval that is equal to or less than the predetermined period of time; and

for each of the regions detected by the plurality of the cameras, determining a position of the regions using the received array positions of the pixels in the plurality of cameras that detected the photons of the beam reflected or scattered by that region.

18. The method of claim 17, wherein directing the transmitter comprises directing the transmitter to sequentially illuminate each region of the plurality of regions for no more than a predetermined time period.

19. The method of claim 17, wherein each of the cameras is disposed on one of a plurality of appendages, wherein the method further comprises:

moving one or more of the appendages until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.

20. The method of claim 17, wherein the transmitter is disposed on one of the appendages and wherein the method further comprises:

moving the appendage on which the transmitter is disposed until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.

21. The method of claim 17, wherein the method further comprises:

identifying a plurality of points of interest (POIs) on the one or more objects;

determining first positions of the POIs;

at a later time, determining second positions of the POIs; and

predicting a trajectory of the POIs based on the first and second positions of the POIs.

22. The method of claim 17, wherein directing the transmitter comprising directing the transmitter to sequentially illuminate, according to a predetermined sequence of colors, the plurality of the regions of the one or more objects with the beam of photons.

23. The method of claim 17, wherein directing the transmitter comprising directing the transmitter to sequentially illuminate, according to a predetermined sequence of beam intensities, the plurality of the regions of the one or more objects with the beam of photons.

24. A non-transitory processor readable storage media that includes instructions for determining a position of one or more objects, wherein execution of the instructions by one or more processor devices cause the one or more processor devices to perform actions of a method, comprising:

directing a transmitter to sequentially illuminate each of a plurality of regions of one or more objects with the beam of photons for a predetermined period of time;

for each of the regions, receiving, from a plurality of cameras, an array position of each pixel in the plurality of cameras that detected photons of the beam reflected or scattered by the region of the one or more objects within a time interval that is equal to or less than the predetermined period of time; and

for each of the regions detected by the plurality of the cameras, determining a position of the regions using the received array positions of the pixels in the plurality of cameras that detected the photons of the beam reflected or scattered by that region.

25. The non-transitory processor readable storage media of claim 24, wherein directing the transmitter comprises directing the transmitter to sequentially illuminate each region of the plurality of regions for no more than a predetermined time period.

26. The non-transitory processor readable storage media of claim 24, wherein each of the cameras is disposed on one of a plurality of appendages, wherein the method further comprises:

moving one or more of the appendages until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.

27. The non-transitory processor readable storage media of claim 24, wherein the transmitter is disposed on one of the appendages and wherein the method further comprises:

moving the appendage on which the transmitter is disposed until at least three of the cameras detect photons reflected or scattered by the region of the one or more objects.