Flow line analysis system and flow line analysis method

• US 10,567,677 B2
• Filed: 04/14/2016
• Issued: 02/18/2020
• Est. Priority Date: 04/17/2015
• Status: Active Grant

• Current Assignee: I-Pro Company Limited
• Original Assignee: Panasonic i-PRO Sensing Solutions Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Hagisu, Shinpei, Tayama, Tetsuo, Kinoshita, Hidetoshi, Yano, Koji, Nakamura, Manabu, Oguchi, Takae, Takahashi, Hideaki
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Sechser, Jill D
• Application Number:
  US15/099,352
• Publication Number:
  US 20160307049A1
• Time in Patent Office: 1,405 Days
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A camera device transmits a captured image and flow line information of moving objects to a server device for each predetermined transmission period. The server device generates a flow line analysis image in which the flow line information of the moving objects is superimposed on the captured image and displays the generated flow line analysis image on a display. The server device displays a first flow line analysis image generated based on a captured image and flow line information in a first period which is arbitrary designated, and receives an instruction by a user operation to shift the first period by an amount. The server device sets a second period by shifting the first period by the amount without changing a length in period, and displays a second flow line analysis image generated based on a captured image and flow line information in the second period.

Claims

1. A flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a captured image,

extracts flow line information relating to a staying position or a passing position of a moving object, and

transmits the generated captured image and the extracted flow line information of the moving object to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image and the extracted flow line information of the moving object for each predetermined transmission period,

causes a pass line to be displayed,

causes a number representing a count result of moving objects that cross the pass line during a target period, within a predetermined transmission period, to be displayed adjacent the pass line,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, changes the number representing the count result of moving objects that cross the pass line during the target period displayed at a first timing to a number representing a count result of moving objects that cross the pass line for the selected predefined discrete time instance that is a predetermined time period before the first timing or a predetermined time period after the first timing.

2. The flow line analysis system according to claim 1, wherein the camera device repeatedly generates the captured image.

3. The flow line analysis system of claim 1, wherein the server device generates a flow line analysis image in which the flow line information of the moving object is superimposed on the captured image and causes display of a first flow line analysis image generated based on a captured image and flow line information in a first period which is arbitrarily designated, and receives an instruction by a user operation to shift the first period by an amount, and

when the server device receives the instruction when the first flow line analysis image is displayed, the server device sets a second period by shifting the first period by the amount without changing a length in period, and causes display of a second flow line analysis image generated based on a captured image and flow line information in the second period.

4. The flow line analysis system of claim 1, wherein the number representing a count result represents a number of moving objects that cross the pass line in a first direction.

5. The flow line analysis system of claim 4, wherein the server device causes another number representing a count result of moving objects that cross the pass line in a second direction, different from the first direction, to be displayed adjacent the pass line.

6. The flow line analysis system of claim 1, wherein the server device:

when the pass line and the number representing the count result of moving objects that cross the pass line during the target period is displayed, causes a plurality of camera selection buttons, respectively corresponding to a plurality of cameras including the camera, to be displayed, each displayed camera selection button of the plurality of camera selection buttons corresponding to a camera having respective generated captured image and extracted flow line information downloaded from the camera to the server,

receives, from a user, a selection of a camera selection button from the plurality of camera selection buttons, and

in response to receiving the selection of the camera selection button from the plurality of camera selection buttons,

retrieves the generated captured image and the extracted flow line information of the camera corresponding to the selected camera selection button, and

causes the pass line and the number representing the count result of moving objects that cross the pass line of the camera corresponding to the selected camera selection button to be displayed.

7. A flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a background image of a captured image,

extracts flow line information relating to a staying position or a passing position of a moving object, and

transmits the generated background image, and the extracted flow line information of the moving object to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image and the extracted flow line information of the moving object for each predetermined transmission period,

causes a pass line to be displayed,

causes a number representing a count result of moving objects that cross the pass line during a target period within a predetermined transmission period, to be displayed adjacent the pass line,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the redetermined transmission period, changes the number representing the count result of moving objects that cross the pass line during the target period displayed at a first timing to a number representing a count result of moving objects that cross the pass line for the selected predefined discrete time instance that is a predetermined time period before the first timing or a predetermined time period after the first timing.

8. The flow line analysis system according to claim 7, wherein the camera device repeatedly generates the background image.

9. The flow line analysis system of claim 7, wherein the server device generates a flow line analysis image in which the flow line information of the moving object is superimposed on the background image and causes display of a first flow line analysis image which is generated based on a background image and flow line information in a first period, and then causes display of a second flow line analysis image which is generated based on a background image and flow line information in a second period when receiving a period changing operation.

10. A flow line analysis method in a flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a captured image,

extracts flow line information relating to a staying position or a passing position of a moving object, and

transmits the generated captured image, and the extracted flow line information of the moving object to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image and the extracted flow line information of the moving object for each predetermined transmission period,

causes a pass line to be displayed,

causes a number representing a count result of moving objects that cross the pass line during a target period, within a predetermined transmission period, to be displayed adjacent the pass line,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, changes the number representing the count result of moving objects that cross the pass line during the target period displayed at a first timing to a number representing a count result of moving objects that cross the pass line for the selected predefined discrete time instance that is a predetermined time period before the first timing or a predetermined time period after the first timing.

11. The flow line analysis method according to claim 10, wherein the camera device repeatedly generates the captured image.

12. The flow line analysis method of claim 10, wherein the server device generates a flow line analysis image in which the flow line information of the moving object is superimposed on the captured image and causes display of a first flow line analysis image generated based on a captured image and flow line information in a first period which is arbitrarily designated, and receives an instruction by a user operation to shift the first period by an amount, and

when the server device receives the instruction when the first flow line analysis image is displayed, the server device sets a second period by shifting the first period by the amount without changing a length in period, and causes display of a second flow line analysis image generated based on a captured image and flow line information in the second period.

13. A flow line analysis method in a flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a background image of a captured image,

extracts flow line information relating to a staying position or a passing position of a moving object, and

transmits the generated background image, and the extracted flow line information of the moving object to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image and the extracted flow line information of the moving object for each predetermined transmission period,

causes a pass line to be displayed,

causes a number representing a count result of moving objects that cross the pass line during a target period, within a predetermined transmission period, to be displayed adjacent the pass line,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, changes the number representing the count result of moving objects that cross the pass line during the target period displayed at a first timing to a number representing a count result of moving objects that cross the pass line for the selected predefined discrete time instance that is a predetermined time period before the first timing or a predetermined time period after the first timing.

14. The flow line analysis method according to claim 13, wherein the camera device repeatedly generates the background image.

15. The flow line analysis method of claim 13, wherein the server device generates a flow line analysis image in which the flow line information of the moving object is superimposed on the background image and causes display of a first flow line analysis image which is generated based on a background image and flow line information in a first period, and then causes display of a second flow line analysis image which is generated based on a background image and flow line information in a second period when receiving a period changing operation.

16. A flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a captured image,

extracts flow line information relating to a staying position or a passing position of moving objects,

counts a number of the moving objects passing at least one line during a target period based on the extracted flow line information, the at least one line designated by the server device or an input device with respect to the captured image, and

transmits the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line for each predetermined transmission period,

generates a count number image in which the at least one line and the count number corresponding to the at least one line is superimposed on the captured image,

causes display of the generated count number image on a display,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, switches the generated count number image, displayed at the time of operation of changing a timing of the target period, to another generated count number image for the selected predefined discrete time instance that is a predetermined time period before the generated count number image or a predetermined time period after the generated count number image.

17. The flow line analysis system according to claim 16, wherein the camera device repeatedly generates the captured image.

18. The flow line analysis system according to claim 16, wherein the camera device derives a cumulative total of moving objects which pass the at least one line in one direction as the count number.

19. The flow line analysis system according to claim 16, wherein the camera device derives a cumulative total of moving objects which pass the at least one line in one direction, and a cumulative total of moving objects which pass the at least one line in a direction opposite to the one direction as the count number.

20. The flow line analysis system of claim 16, wherein the server device causes displays of a first count number image generated based on a captured image and flow line information in a first period which is arbitrarily designated and the count number corresponding to the at least one line, and receives an instruction by a user operation to shift the first period by an amount, and

when the server device receives the instruction when the first count number image is displayed, the server device sets a second period by shifting the first period by the amount without changing a length in period, and causes display of a second count number image generated based on a captured image and flow line information in the second period and the count number corresponding to the at least one line.

21. A flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a background image of a captured image,

extracts flow line information relating to a staying position or a passing position of moving objects,

counts a number of the moving objects passing at least one line during a target period based on the extracted flow line information, the at least one line designated by the server device or an input device with respect to the captured image, and

transmits the generated background image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line for each predetermined transmission period,

generates a count number image in which the at least one line and the count number corresponding to the at least one line is superimposed on the background image,

causes display of the generated count number image on a display,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, switches the generated count number image, displayed at the time of operation of changing a timing of the target period, to another generated count number image for the selected predefined discrete time instance that is a predetermined time period before the generated count number image or a predetermined time period after the generated count number image.

22. The flow line analysis system according to claim 21, wherein the camera device repeatedly generates the background image.

23. The flow line analysis system according to claim 21, wherein the camera device derives a cumulative total of moving objects which pass the at least one line in one direction as the count number.

24. The flow line analysis system according to claim 21, wherein the camera device derives a cumulative total of moving objects which pass the at least one line in one direction, and a cumulative total of moving objects which pass the at least one line in a direction opposite to the one direction as the count number.

25. The flow line analysis system of claim 21, wherein the server device causes display of a first count number image which is generated based on a background image and flow line information in a first period and the count number corresponding to the at least one line, and then causes display of a second count number image which is generated based on a background image and flow line information in a second period and the count number corresponding to the at least one line when receiving a period changing operation.

26. A flow line analysis method in a flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a captured image,

extracts flow line information relating to a staying position or a passing position of moving objects,

counts a number of the moving objects passing at least one line during a target period based on the extracted flow line information, the at least one line designated by the server device or an input device with respect to the captured image, and

transmits the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line for each predetermined transmission period,

generates a count number image in which the at least one line and the count number corresponding to the at least one line is superimposed on the captured image,

causes display of the generated count number image on a display,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, switches the generated count number image, displayed at the time of operation of changing a timing of the target period, to another generated count number image for the selected predefined discrete time instance that is a predetermined time period before the generated count number image or a predetermined time period after the generated count number image.

27. The flow line analysis method of claim 26, wherein the server device causes display of a first count number image generated based on a captured image and flow line information in a first period which is arbitrarily designated and the count number corresponding to the at least one line, and receives an instruction by a user operation to shift the first period by an amount, and

when the server device receives the instruction when the first count number image is displayed, the server device sets a second period by shifting the first period by the amount without changing a length in period, and causes display of a second count number image generated based on a captured image and flow line information in the second period and the count number corresponding to the at least one line.

28. A flow line analysis method in a flow line analysis system in which a camera device and a server device are connected to each other,

wherein the camera device:

generates a background image of a captured image,

extracts flow line information relating to a staying position or a passing position of moving objects,

counts a number of the moving objects passing at least one line during a target period based on the extracted flow line information, the at least one line designated by the server device or an input device with respect to the captured image, and

transmits the generated background image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line to the server device for each predetermined transmission period, and

wherein the server device;

downloads and saves the generated captured image, the extracted flow line information of the moving objects, and the count number corresponding to the at least one line for each predetermined transmission period,

generates a count number image in which the at least one line and the count number corresponding to the at least one line is superimposed on the background image,

displays the generated count number image on a display,

causes a selection of a plurality of image leap commands respectively associated with a plurality of predefined discrete time instances to be displayed, a first set of the plurality of predefined discrete time instances temporally preceding the target period and a second set of the plurality of predefined discrete time instances temporally succeeding the target period,

receives, from the user via the displayed selection of the plurality of image leap commands, a selection of a predefined discrete time instance of the plurality of predefined discrete time instances, and

when the selected predefined discrete time instance is within the predetermined transmission period, switches the generated count number image, displayed at the time of operation of changing a timing of the target period, to another generated count number image for the selected predefined discrete time instance that is a predetermined time period before the generated count number image or a predetermined time period after the generated count number image.

29. The flow line analysis method of claim 28, wherein the server device causes display of a first count number image which is generated based on a background image and flow line information in a first period and the count number corresponding to the at least one line, and then causes display of a second count number image which is generated based on a background image and flow line information in a second period and the count number corresponding to the at least one line when receiving a period changing operation.