Target analysis and recommendation

• US 10,330,440 B2
• Filed: 08/11/2015
• Issued: 06/25/2019
• Est. Priority Date: 11/26/2014
• Status: Active Grant

• Current Assignee: Advanced AR Systems LLC
• Original Assignee: Philip Lyren
• Inventors: Lyren, Philip
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US14/823,559
• Publication Number:
  US 20160377381A1
• Time in Patent Office: 1,414 Days
• CPC Class Codes: F41G 3/02 using an independent line of sight
• CPC Class Codes: F41G 3/04 for dispersing fire from a battery ; for controlling spread of shots; for coordinating fire from spaced weapons
• CPC Class Codes: F41G 3/06 with rangefinder rangefinders per se G01C
• CPC Class Codes: F41G 3/165 using a TV-monitor
• CPC Class Codes: F41H 13/00 Means of attack or defence not otherwise provided for details of radar or sonar systems G01S7/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0189 Sight systems
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment

Abstract

An electronic device determines target information about a target and recommends a target based on the target information.

Claims

1. A method, comprising:

collecting, with multiple electronic glasses worn by multiple users at a geographical location, target information about people with firearms that are dispersed at the geographical location;

determining, by an electronic device, factors that influence assignment of each of the users to one of the people with firearms in which the factors include a distance from each of the users to each of the people with firearms;

determining, by the electronic device and based on the target information, a score for each of the factors with respect to each of the users for each of the people with firearms;

assigning, by the electronic device and based on the score for each of the factors with respect to each of the users, each of the users to a different one of the people with firearms; and

displaying, through the multiple electronic glasses of each user, a three dimensional (3D) image of a person that represents a person assigned to a user such that the 3D image of the person is displayed in a field of view of the user where the person assigned to the user is actually located in the geographical location, wherein the users and the user are people.

2. The method of claim 1 further comprising:

determining, by the electronic device and as one of the factors that influence assignment of each of the users to one of the people with firearms, whether or not the people with the firearms are holding the firearms;

determining, by the electronic device and as one of the factors that influence assignment of each of the users to one of the people with firearms, whether or not the people with the firearms are firing the firearms; and

determining, by the electronic device and as one of the factors that influence assignment of each of the users to one of the people with firearms, an identity of the people with the firearms.

3. The method of claim 1 further comprising:

determining, as one of the factors that influence assignment of each of the users to one of the people with firearms, a type of obstruction that exists between each of the users and each of the people with the firearms;

determining, as one of the factors that influence assignment of each of the users to one of the people with firearms, weapon penetration locations on the obstruction; and

assigning different scores to different obstructions based on the type of obstruction and the weapon penetrations locations on the obstruction.

4. The method of claim 1 further comprising:

determining that two of the users have a point of aim of a firearm on a same one of the people with firearms; and

instructing one of the two users to move a point of aim from the same one of the people with firearms to another one of the people with firearms so the two users do not have their points of aim on the same one of the people with firearms.

5. The method of claim 1 further comprising:

determining, from a facial image collected with one of the electronic glasses, an identity of one of the people with firearms;

drawing a 3D image of the one of the people with firearms to resemble the one of the identity of the one of the people with firearms; and

displaying, through the one of the electronic glasses, the 3D image of the one of the people with firearms over a location in a field of view of the one of the electronic glasses where the one of the people with firearms is located.

6. The method of claim 1 further comprising:

determining that one of the users is assigned to a first one of the people with firearms;

determining, from the target information, that a second one of the people with firearms is firing at the one of the users; and

changing an assignment of the one of the users from the first one of the people with firearms to the second one of the people with firearms based on determining that the second one of the people with firearms is firing at the one of the users.

7. The method of claim 1 further comprising:

determining that one of the users assigned a first one of the people with firearms selected to target a second one of the people with firearms in spite of being assigned the first one of the people with firearms;

executing machine learning to compare scores for factors of the first one of the people with firearms with scores for factors of the second one of the people with firearms; and

changing the scores for the factors based on the machine learning.

8. The method of claim 1 further comprising:

determining a location of one of the users;

determining, based on the target information, a safety score for the location in which the safety score represents safety of the one of the users from harm by the people with firearms;

determining, based on the target information, safety scores for alternate locations that are proximate to the location; and

recommending, through a display of electronic glasses of the one of the users, that the one of the users moves to one of the alternate locations that has an improved safety score over the safety score for the location.

9. An electronic system, comprising:

first electronic glasses that include a processor in communication with a wireless network, a display, and a camera that captures images of people and weapons in a field of view of a first user wearing the first electronic glasses;

second electronic glasses that include a processor in communication with the wireless network, a display, and a camera that captures images of the people and the weapons in a field of view of a second user wearing the second electronic glasses; and

a computer that communicates over the wireless network with the first electronic glasses and the second electronic glasses and that includes a processor and a memory with instructions that the processor of the computer executes to receive the images of the people and of the weapons from the first and second electronic glasses, to determine distances from the first user and the second user to each of the people, to determine whether each of the people is holding one of the weapons, to determine obstructions where each of the people is located, to determine assignments of the first user to one of the people and the second user to another one of the people based on the distances, the holding of the weapon, and the obstructions, and to provide the assignments to the first and second electronic glasses, wherein the first user and the second user are people.

10. The electronic system of claim 9 further comprising:

a firearm with the first user that includes a compass to determine a direction of a point of aim of the firearm, a rangefinder to determine a distance from the firearm to the one of the people, and a transceiver to wirelessly communicate the direction and the distance to the first electronic glasses.

11. The electronic system of claim 9, wherein the processor of the computer further executes the instructions to build a three-dimensional (3D) map that depicts the people holding the weapons, the obstructions where each of the people is located, the assignments, the first user, and the second user, and to provide the 3D map to the first and second electronic glasses.

12. The electronic system of claim 9, wherein the first electronic glasses include a see-thru display that displays a three-dimensional (3D) image of the one of the people at a location that coincides with where the one of the people is located, and wherein the second electronic glasses include a see-thru display that displays a 3D image of the another one of the people at a location that coincides with where the another one of the people is located.

13. The electronic system of claim 9 further comprising:

a machine-learning system that receives information that the first user fired a firearm at a person not assigned to the first user, executes an evaluation of a distance from the first user to the person, executes an evaluation of an obstruction proximate to the person, and changes priorities for how the people are assigned to the first user based on the evaluation of the distance and on the evaluation of the obstruction.

14. The electronic system of claim 9 further comprising:

a firearm with the first user that includes an electronic leveler to determine an angle with respect to ground at which the firearm is pointed when aimed at the one of the people, a rangefinder to determine a distance from the firearm to the one of the people, and a transmitter to transmit the angle and the distance to the computer; and

wherein the processor of the computer further executes the instructions to determine a height of the one of the people with respect to the first user and to map a location of the one of the people to a floor of a building based on the angle, the distance, and the height.

15. The electronic system of claim 9 further comprising:

a target recommender that evaluates the weapons of the people, locations of the weapons and the people, a location of the first user, and the obstructions where each of the people is located, that determines alternate locations where the first user can move to reduce a probability of receiving fire from the weapons of the people, that selects one of the alternate locations, and that provides the alternate location to the first electronic glasses; and

wherein the display of the first electronic glasses displays a path from a current location of the first user to the alternate location.

16. A non-transitory computer readable storage medium storing instructions that cause one or more processors to execute a method, comprising:

obtain an image of a first person holding a first firearm and a second person holding a second firearm at a geographical location that includes a user wearing electronic glasses;

evaluate distances from the user to the first person and from the user to the second person;

evaluate obstructions between the user and the first person and between the user and the second person;

determine, based on the distances and the obstructions, a recommendation as to whether the user should target the first person with a weapon or target the second person with the weapon; and

display, with the electronic glasses, the recommendation such that the recommendation visually distinguishes the first person from the second person, wherein the user is a person.

17. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

determine a material from which the obstructions are fabricated; and

determine whether a projectile fired from the weapon can penetrate the obstructions and strike the first person or the second person.

18. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

determine whether the first person is firing the first firearm at the user;

determine whether the second person is firing the second firearm at the user;

prioritize the recommendation for the user to target the first person when the first person is firing the first firearm at the user; and

prioritize the recommendation for the user to target the second person when the second person is firing the second firearm at the user.

19. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

determine whether the first person is firing the first firearm not at the user but at another user;

determine whether the second person is firing the second firearm not at the user but at the another user;

prioritize the recommendation for the user to target the first person when the first person is firing the first firearm at the another user; and

prioritize the recommendation for the user to target the second person when the second person is firing the second firearm at the another user.

20. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

display, with the electronic glasses, three dimensional (3D) weapon penetration locations that show locations at the geographical location where a projectile fired from the first person and the second person can strike the user.