Method and computer program for taking action based on determined movement path of mobile devices

• US 10,735,902 B1
• Filed: 12/17/2018
• Issued: 08/04/2020
• Est. Priority Date: 04/09/2014
• Status: Active Grant

• Current Assignee: Accuware, Inc. (Voice & Data Networks, Inc.)
• Original Assignee: Accuware, Inc. (Voice & Data Networks, Inc.)
• Inventors: Houri, Cyril, Girard, Denis
• Primary Examiner(s): Nakhjavan, Shervin K
• Application Number:
  US16/222,891
• Time in Patent Office: 596 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 64/003 locating network equipment

Abstract

Method for controlling movement of a mobile device includes obtaining an analyzeable video from an imager on the device during its movement by obtaining at least one video from the imager, and analyzing, using a processor, each video to determine presence of a fixed-in-position object in multiple sequentially obtained frames until a video is obtained including at least one fixed-in-position object in multiple sequentially obtained frames which constitutes the analyzeable video. This video is analyzed on a frame by frame basis to determine distance and direction moved by the device, which is analyzed relative to predetermined distance and direction intended for movement of the device to determine any differences, which result in changes in movement of the device. Relocation of the device is achieved by recognizing a previously imaged, fixed object in subsequent frames and comparing the position of the device at both times, with a deviation resulting in relocation.

Claims

1. A method for controlling movement of a mobile device, comprising:

obtaining an analyzeable video from an imaging device on the mobile device during movement of the mobile device by

obtaining at least one video from the imaging device on the mobile device during movement of the mobile device, and

analyzing, using a processor, each of the at least one obtained video to determine presence of a fixed-in-position object in multiple sequentially obtained frames of the video until a video is obtained including at least one fixed-in-position object in multiple sequentially obtained frames which constitutes the analyzeable video;

analyzing, using the processor and during the movement of the mobile device while obtaining the analyzeable video, the analyzeable video on a frame by frame basis to determine distance and direction moved by the mobile device while obtaining the analyzeable video and in the interval between frames;

analyzing, using the processor and during the movement of the mobile device while obtaining the analyzeable video, the determined distance and direction moved by the mobile device while obtaining the analyzeable video relative to predetermined distance and direction intended for movement of the mobile device to determine any differences; and

effecting a change in movement of the mobile device whenever any differences are determined to be present between the distance and direction moved by the mobile device while obtaining the analyzeable video and the predetermined distance and direction intended for movement of the mobile device.

2. The method of claim 1, further comprising:

transmitting the at least one video to a server that includes or is associated with the processor; and

performing the analyzing steps at the server.

3. The method of claim 1, wherein the processor is situated on or in the mobile device such that the analyzing steps are performed at the mobile device.

4. The method of claim 1, further comprising generating a diagram of movement of the mobile device including any changes in movement that have been effected.

5. The method of claim 4, further comprising displaying the diagram of movement of the mobile device on a display.

6. The method of claim 5, wherein the diagram of movement of the mobile device is displayed in two dimensions on the display.

7. The method of claim 5, wherein the diagram of movement of the mobile device is displayed in three dimensions on the display.

8. The method of claim 1, wherein the mobile device is a movable robot, and the step of effecting a change in movement of the mobile device comprises providing commands to an actuator of the robot to change its direction of movement.

9. The method of claim 1, wherein the mobile device is an airborne drone, and the step of effecting a change in movement of the mobile device comprises providing commands to an actuator of the drone to change its direction of movement.

10. The method of claim 1, wherein the mobile device is a cellular telephone, and the step of effecting a change in movement of the mobile device comprises emitting from a sound generator on the mobile device, directions to enable a holder of the mobile device to change their direction of movement.

11. The method of claim 1, wherein the mobile device is a cellular telephone, and the step of effecting a change in movement of the mobile device comprises displaying on a display of the mobile device, directions or a map including directions to enable a holder of the mobile device to change their direction of movement.

12. The method of claim 1, further comprising wirelessly transmitting the analyzeable video to a server separate and apart from the mobile device and which includes or is associated with the processor such that the analyzing steps are performed at least partly by the processor at the server.

13. The method of claim 1, further comprising:

analyzing, using the processor, frames of the analyzeable video to determine whether a previously imaged, fixed object is present in any of the frames; and

when that object is determined to be present,

analyzing, using the processor, whether a position of the mobile device relative to that object is accurate based on data about that object and the mobile device stored in a memory component accessible to the processor, and if not,

adjusting, using the processor, the position of the mobile device based on the determined position of the mobile device relative to that object.

14. The method of claim 13, further comprising:

generating a diagram of movement of the mobile device including any changes in movement that have been effected;

displaying the diagram of movement of the mobile device on a display; and

displaying the adjusted position of the mobile device on the display.

15. The method of claim 1, further comprising:

initially determining, using the processor, the presence of an object in a frame of the analyzeable video;

storing, in a memory component, identification data about the object whose presence is initially determined;

determining, using the processor, the position of the mobile device relative to that object;

then

analyzing, using the processor, subsequent frames of the analyzeable video to determine whether that object is present in any of the subsequent frames; and

when that object is determined to be present in a subsequent frame of the analyzeable video, determining, using the processor, whether a current position of the mobile device is accurate relative to that object and if not, adjusting, using the processor, the position of the mobile device based on the determined position of the mobile device relative to that object.

16. A method for controlling movement of a mobile device, comprising:

obtaining an analyzeable video from an imaging device on the mobile device during movement of the mobile device by

obtaining at least one video from the imaging device on the mobile device during movement of the mobile device,

analyzing, using a processor, each of the at least one obtained video to determine presence of a fixed-in-position object in multiple sequentially obtained frames of the video until a video is obtained including at least one fixed-in-position object in multiple sequentially obtained frames which constitutes the analyzeable video, and

storing data about a fixed object at a beginning of the analyzeable video;

analyzing, using the processor during the movement of the mobile device while obtaining the analyzeable video, the analyzeable video on a frame by frame basis to determine distance and direction moved by the mobile device while obtaining the analyzeable video and in the interval between frames;

generating a diagram of movement of the mobile device;

displaying the diagram of movement of the mobile device on a display;

analyzing, using the processor, frames of the analyzeable video to determine whether a previously imaged, fixed object is present in any of the frames; and

when that fixed object is determined to be present,

analyzing, using the processor, whether a position of the mobile device relative to that fixed object is accurate based on position information about that fixed object and the mobile device stored in a memory component accessible to the processor, and if not,

adjusting, using the processor, the position of the mobile device based on the determined position of the mobile device relative to that fixed object and displaying the adjusted position of the mobile device on the display.

17. The method of claim 16, further comprising wirelessly transmitting the analyzeable video to a server separate and apart from the mobile device, the analyzing steps being performed at least partly by a processor at the server.

18. The method of claim 16, further comprising:

analyzing, using the processor and during the movement of the mobile device while obtaining the analyzeable video, the determined distance and direction moved by the mobile device while obtaining the analyzeable video relative to predetermined distance and direction intended for movement of the mobile device to determine any differences; and

effecting a change in movement of the mobile device whenever any differences are determined to be present between the distance and direction moved by the mobile device while obtaining the analyzeable video and the predetermined distance and direction intended for movement of the mobile device.

19. A computer program embodied on non-transitory computer storage medium, and which is configured to:

receive at least one video from an imaging device on a mobile device during movement of the mobile device;

analyze each of the at least one video to determine presence of a fixed-in-position object in multiple sequentially obtained frames of the video until a video is received including at least one fixed-in-position object in multiple sequentially obtained frames which constitutes an analyzeable video;

analyze the analyzeable video on a frame by frame basis to determine distance and direction moved by the mobile device while obtaining the analyzeable video and in the interval between frames;

analyze the determined distance and direction moved by the mobile device while obtaining the analyzeable video relative to predetermined distance and direction intended for movement of the mobile device to determine any differences; and

effect a change in movement of the mobile device whenever any differences are determined to be present between the distance and direction moved by the mobile device while obtaining the analyzeable video and the predetermined distance and direction intended for movement of the mobile device.

20. The computer program of claim 19, wherein the computer program is further configured to:

analyze frames of the analyzeable video to determine whether a previously imaged, fixed object is present in any of the frames;

when that fixed object is determined to be present,

analyze whether a position of the mobile device relative to that fixed object is accurate based on data about that fixed object and the mobile device stored in a memory component accessible to the processor, and if not,

adjust the position of the mobile device based on the determined position of the mobile device relative to that fixed object;

generate a diagram of movement of the mobile device including any changes in movement that have been effected;

cause display of the diagram of movement of the mobile device on a display; and

cause display the adjusted position of the mobile device on the display.