Evaluation result display method, evaluation result display apparatus, and non-transitory computer-readable recording medium storing evaluation result display program

• US 11,093,517 B2
• Filed: 03/27/2015
• Issued: 08/17/2021
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Umetani, Hideo, Ohama, Iku, Fujimura, Ryota
• Primary Examiner(s): Ruiz, Angelica
• Application Number:
  US14/671,691
• Publication Number:
  US 20150286692A1
• Time in Patent Office: 2,335 Days
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

A method is provided for reducing computer memory usage and processing time in assessing effectiveness of a prediction model. The method includes acquiring a plurality of records and sorting the records based on prediction values. The method also includes determining two of three performance data parameters, including a number of selected records and a number of correct decisions, and determining a level of effectiveness of the prediction model based on the two of three performance data parameters. The method further includes generating a graph illustrating the level of effectiveness of the prediction model based on the two of three performance parameters, and visually conveying the level of effectiveness of the prediction model utilizing the generated graph.

Claims

1. A method for reducing processor memory usage and processing time, comprising:

acquiring a plurality of records which are a collection of records each of which includes a value of the probability that each of a plurality of users of a network browses an advertisement, and a value indicating whether each user actually browsed the advertisement, wherein the value indicating whether each user actually browsed the advertisement is a first value for a user actually browsing the advertisement, and is a second value for a user who did not actually browse the advertisement;

storing the acquired plurality of records in a memory;

reading out the acquired plurality of records from the memory, sorting the plurality of records in a sorted order based on the probability values, and storing the sorted plurality of records in the memory;

selecting a number of the plurality of sorted and stored plurality of records;

identifying from the acquired plurality of records only

the number of the selected records, and

a number of the selected plurality of sorted and stored plurality of records in which users actually browsed the advertisement based on the stored value indicating whether each user actually browsed the advertisement,

refraining from displaying a number of the selected plurality of sorted and stored plurality of records in which the users did not browse the advertisement, based on the stored value indicating whether each user actually browsed the advertisement;

storing the identified number of selected records and the identified number of the selected records in which users actually browsed the advertisement in association with each other;

outputting the stored and associated identified number of selected records and identified number of the selected records in which users actually browsed the advertisement, and

controlling a display panel with a processor according to the outputted stored and associated identified number of selected records and identified number of the selected records in which users actually browsed the advertisement to display on the display panel an x-y graph displaying the identified number of the selected records in which users actually browsed the advertisement along the vertical axis of the graph, and displaying the identified number of selected records along the horizontal axis of the graph, in a coordinate system to display the graph.

2. The method according to claim 1,

wherein the sorting sorts the records based on the probability values,

wherein the sorting sorts the records in descending order of the probability values or in ascending order of the probability values.

3. The method according to claim 1,

further comprising generating the graph by plotting a line that links a point at which the number of selected records takes 0 and a number of correct decisions representing the number of users actually browsing the advertisement takes 0 with a point at which the number of selected records takes a maximum value and the number of correct decisions takes a maximum value in the coordinate system, and displaying the line.

4. The method according to claim 1, further comprising selecting a predetermined number of the records at a time and determining the number of selections of the predetermined number of the selected records and a number of correct decisions representing the number of users actually browsing the advertisement of the predetermined number of the selected records.

5. The method according to claim 1,

the sorting sorts the records based on the probability values, and

the selecting selects among the records sorted in the sorting, all the records that have the same probability value in sorted order and the identifying identifies the number of selections of the selected records having the same probability value and a number of correct decisions representing the number of users actually browsing the advertisement of the selected records having the same probability value.

6. The method according to claim 1, further comprising:

determining, with a processor, a number of selected records of a point on the graph at which a slope of a line that links a point at which the number of selected records takes 0 and a number of correct decisions representing the number of users actually browsing the advertisement takes 0 with a point at which the number of selected records takes a maximum value and the number of correct decisions takes a maximum value coincides with a slope of a tangent line of the generated graph, the number of selected records at the point on the graph being an optimum value, by using the number of selected records and the number of correct decisions output in the outputting, and

controlling the display panel with the processor to display the determined optimum value.

7. The method according to claim 1, further comprising displaying an index of a function f on the x-y graph with the display panel, wherein the displayed index is determined by using formulas (1)-(3),

wherein formula (1) is

8. The method according to claim 1,

the processor adjusts the graph so that the number of users who actually browsed the advertisement displayed along the vertical axis and the identified number of selected records displayed along the horizontal axis have a ratio of 1:

1, and

the displaying of the x-y graph displays the number of users who actually browsed the advertisement displayed along the vertical axis and the identified number of selected records displayed along the horizontal axis in a ratio of 1;

1.

9. The method according to claim 1,

wherein the displaying of the graph also displays a line representing the identified number of selected records and the number of users who actually browsed the advertisement displayed when the identified number of selected records and the number of users who actually browsed the advertisement are selected at random,

wherein the displayed line links the origin of the x-y graph with a point at which the identified number of selected records takes a maximum value and the number of users who actually browsed the advertisement takes a maximum value, and

the higher the graph of the number of users who actually browsed the advertisement displayed along the vertical axis and of the identified number of selected records displayed along the horizontal axis above the line, the better a prediction model based on the number of users who actually browsed the advertisement and the identified number of selected records.

10. The method according to claim 1, further comprising

displaying on the display panel an index of the x-y graph, wherein

the index is numerical value representing an objective evaluation of a prediction of whether users will browse the webpage based on the displayed x-y graph, and

the numerical value being a ratio of an area below the graph to an area of a rectangle whose vertices include an origin of the graph, a point along the horizontal axis representing the maximum value of the total number of selections, and a point along the vertical axis representing a maximum value of the number of correct decisions,

thereby providing a single screen displaying both the graph and the numerical value representing an evaluation of a prediction of whether users will browse the webpage based on the displayed x-y graph.

11. The method according to claim 1, wherein the first value indicating that each user actually browsed the advertisement is 1 and the second value indicating each user did not actually browse the advertisement is 0.

12. The method according to claim 1, wherein

the selecting operation comprises:

selecting a number of the plurality of sorted and stored plurality of records by selecting one record at a time,

the identifying operation of identifying from the acquired plurality of records only the number of the selected records comprises;

identifying only the total number of the selected records, which increases by one each time the selecting operation selects a record, and

the displaying of the identified number of selected records along the horizontal axis of the graph comprises;

displaying the total identified number of selected records along the horizontal axis of the graph, which increases by one when the selecting operation selects a record in a coordinate system to display the graph.

13. The method according to claim 1, wherein

the selecting operation selecting a number of the plurality of sorted and stored plurality of records comprises:

selecting a number of the plurality of sorted and stored plurality of records by selecting a plurality of record at a time,

the identifying operation identifying from the acquired plurality of records only the number of selected records comprises;

identifying only the total number of the selected records, which increases by the selected plurality of records when the selecting operation is performed, and

the displaying operation along the horizontal axis comprises;

displays the total identified number of selected records along the horizontal axis of the graph, which increases by the selected plurality of records when the selecting operation is performed in a coordinate system to display the graph.

14. The method according to claim 1, wherein

the selecting operation comprises:

selecting among the sorted plurality of records all the records having the same probability that each of the plurality of users browses the advertisement, and

the identifying operation comprises;

identifying from the acquired plurality of records having the same probability that each of the plurality of users browses the advertisement only the number of the selected records having the same probability that each of the plurality of users browses the advertisement.

15. A computing device for reducing processor memory usage and processing time comprising:

a display panel;

a memory; and

a processor,

the processor being configured to control the memory and the display panel, according to instructions stored in the memory, to

acquire a plurality of records which are a collection of records each of which includes a value of the probability that each of a plurality of users of a network browses an advertisement, and a value indicating whether each user actually browsed the advertisement, wherein the value indicating whether each user actually browsed the advertisement is a first value for a user actually browsing the advertisement, and is a second value for a user who did not actually browse the advertisement;

store the acquired plurality of records in the memory;

read out the acquired plurality of records from the memory, sort the plurality of records in a sorted order based on the probability values, and store the sorted plurality of records in the memory;

select a number of the plurality of sorted and stored plurality of records;

identify from the acquired plurality of records only

the number of the selected records, and

a number of the selected plurality of sorted and stored plurality of records in which users actually browsed the advertisement based on the stored value indicating whether each user actually browsed the advertisement,

refrain from displaying a number of the selected plurality of sorted and stored plurality of records in which the users did not browse the advertisement, based on the stored value indicating whether each user actually browsed the advertisement;

store the identified number of selected records and the identified number of the selected records in which users actually browsed the advertisement in association with each other;

output the stored and associated identified number of selected records and identified number of the selected records in which users actually browsed the advertisement, and

control the display panel according to the outputted stored and associated identified number of selected records and the identified number of the selected records in which users actually browsed the advertisement to display on the display panel an x-y graph displaying the identified number of the selected records in which users actually browsed the advertisement along the vertical axis of the graph, and the identified number of selected records along the horizontal axis of the graph, in a coordinate system to display the graph.

16. The apparatus according to claim 15, further comprising displaying an index of a function f on the x-y graph with the display panel, wherein the displayed index is determined by using formulas (1)-(3),

wherein formula (1) is

17. A non-transitory computer-readable recording medium storing a program causing a processor of a computer to reduce computer memory usage and processing time by:

acquiring a plurality of records which are a collection of records each of which includes a value of the probability that each of a plurality of users of a network browses an advertisement, and a value indicating whether each user actually browsed the advertisement, wherein the value indicating whether each user actually browsed the advertisement is a first value for a user actually browsing the advertisement, and is a second value for a user who did not actually browse the advertisement;

storing the acquired plurality of records in a memory;

reading out the acquired plurality of records from the memory, sorting the plurality of records in a sorted order based on the probability values, and storing the sorted plurality of records in the memory;

selecting a number of the plurality of sorted and stored plurality of records;

identifying from the acquired plurality of records only

the number of the selected records, and

a number of the selected plurality of sorted and stored plurality of records in which users actually browsed the advertisement based on the stored value indicating whether each user actually browsed the advertisement,

refraining from displaying a number of the selected plurality of sorted and stored plurality of records in which the users did not browse the advertisement, based on the stored value indicating whether each user actually browsed the advertisement;

storing the identified number of selected records and the identified number of the selected records in which users actually browsed the advertisement in association with each other;

outputting the stored and associated identified number of selected records and identified number of the selected records in which users actually browsed the advertisement, and

controlling a display panel with the processor according to instructions stored in the computer-readable recording medium and according to the outputted stored and associated identified number of selected records and identified number of the selected records in which users actually browsed the advertisement to display on the display panel an x-y graph, displaying the identified number of the selected records in which users actually browsed the advertisement along the vertical axis of the graph, and displaying the identified number of selected records along the horizontal axis of the graph, in a coordinate system to display the graph.

18. The non-transitory computer-readable recording medium according to claim 17, further comprising displaying an index of a function f on the x-y graph with the display panel, wherein the displayed index is determined by using formulas (1)-(3),

wherein formula (1) is

19. A method for reducing processor memory usage and processing time, comprising:

acquiring a plurality of records which are a collection of records each of which includes a value of the probability that each of a plurality of users of a network browses an advertisement, and a value indicating whether each user actually browsed the advertisement, wherein the value indicating whether each user actually browsed the advertisement is a first value for a user actually browsing the advertisement, and is a second value for a user who did not actually browse the advertisement;

storing the acquired plurality of records in a memory;

reading out the acquired plurality of records from the memory, sorting the plurality of records in a sorted order based on the probability values, and storing the sorted plurality of records in the memory;

selecting a number of the plurality of sorted and stored plurality of records;

refraining from displaying a number of the selected plurality of sorted and stored plurality of records in which the users did not browse the advertisement, based on the stored value indicating whether each user actually browsed the advertisement;

storing the number of selected records and a number of the selected records in which users actually browsed the advertisement in association with each other; and

outputting the stored and associated number of selected records and the stored number of the selected records in which users actually browsed the advertisement, and

controlling a display panel with a processor according to the outputted stored and associated number of selected records and the outputted number of the selected records in which users actually browsed the advertisement to display on the display panel an x-y graph displaying the outputted number of the selected records in which users actually browsed the advertisement along the vertical axis of the graph, and displaying the outputted number of selected records along the horizontal axis of the graph, in a coordinate system to display the graph.