METERING USER BEHAVIOUR AND ENGAGEMENT WITH USER INTERFACE IN TERMINAL DEVICES

• US 20150100887A1
• Filed: 10/04/2013
• Published: 04/09/2015
• Est. Priority Date: 10/04/2013
• Status: Active Grant

• Current Assignee: Verto Analytics Oy (Verto Analytics, Inc.)
• Original Assignee: Verto Analytics Oy (Verto Analytics, Inc.)
• Inventors: VERKASALO, Hannu
• Granted Patent: US 10,084,869 B2
• Time in Patent Office: Days
• US Class Current: 715/735
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Electronic terminal device including a processing entity for processing instructions and other data, functionally connected user interface entity, a display and user input elements, optionally a keypad or a touch-sensitive arrangement in connection with the display, a memory entity for storing the instructions and data, and a communication interface for transferring data with external entities, configured, in accordance with the instructions, to store data including a number of predetermined fingerprints, each characterizing display usage such as visual objects associated with software entities and potentially shown to the user via the display during execution of the software entity, and log data including behavioral data indicative of user engagement with the UI of the device, the logging incorporating detecting display usage, and storing indication of such detection in the memory, wherein the device is configured to determine the status of the display, and execute the logging according to the determination.

Claims

1. An electronic terminal device (104, 104a, 104b) comprising a processing

entity (250) for processing instructions (105, 258) and other data, at least functionally connected user interface (UI) entity (254) for interfacing the device with a user and comprising a display and a user input means, a memory entity

(256) for storing the instructions and other data, and a communication interface

(252) for transferring data with external entities such as a network infrastructure

(107, 108), the device being configured, in accordance with the instructions, to

store data comprising a number of fingerprints (202), each characterizing the

display usage, and log data (208, 212) comprising behavioral data indicative of user engagement with the UI of the device, said logging incorporating detecting the display usage (204, 206, 207) based on the fingerprints and storing indication of such detection in said memory, wherein the device is configured to determine (210) the status of the display, and execute said logging according to the determination.

2. The device of claim 1, wherein a number of said fingerprints characterize one or more visual objects associated with one or more software entities and potentially shown to the user via the display, and said detecting incorporates detecting the presence of any of said objects rendered on the display.

3. The device of claim 2, configured to detect said presence utilizing screen capturing (204) of display data and analyzing (207) the obtained display data against the fingerprints.

4. The device of claim 3, wherein the analysis comprises execution of a predetermined pattern recognition algorithm.

5. The device of claim 1, wherein executing the logging in accordance with the determination comprises at least one control action selected from the group consisting of:

including an indication of the determined display status in the logged data, adapting said logging, trigger logging, trigger logging of behavioral data, trigger logging of predetermined data, cease logging, cease logging of behavioral data, cease logging of predetermined data, change logging parameter such as interval, and trigger said detecting.

6. The device of claim 1, specifically configured to determine the status based on whether the display is in use or not.

7. The device of claim 6, configured to execute detecting a number of fingerprints characterizing one or more visual objects associated with one or more software entities and potentially shown to the user via the display substantially exclusively when, according to the determination, the display is in use.

8. The device of claim 1, configured to determine the status based on inspecting at least one indication thereof selected from the group consisting of:

display power or current consumption, display brightness, display lock status, user input interface or user interface lock status, user input such as touch screen input relative to a display or data shown thereon, display status-indicating API (application programming interface), and display status variable.

9. The device of claim 1, configured to observe and log data regarding at least one element selected from the group consisting of:

user online

action, HTTP activity, sensor, acceleration, accelerometer, illumination or light intensity, application usage, application launch, application termination, user id, process thread, identity of application or service used, feature used in application or service, device-internal data traffic, network traffic, source or destination entity of internal and/or network data traffic, voice call, application thread counter, process counter, message, e-mail message, short message, multimedia message, instant message, data counter, location, battery status, and network signal strength.

10. The device of claim 1, wherein the indication comprises metadata associated with the fingerprint, said metadata optionally defining the identity or category of the application, service, feature, event and/or activity underlying the fingerprint.

11. The device of claim 1, wherein the indication characterizes at least one element selected from the group consisting of:

type of display view, active window of multiple simultaneously visible windows, identity of visible windows, identity of the existing windows, identity of application or service in the active window, duration and/or time of use (e.g. start and/or end) of application or service, transaction such as payment, feature or view accessed in the displayed application or service, and user activity regarding application, service or display.

12. The device of claim 1, configured to communicate the indication and optionally other predetermined logged data towards a predetermined external entity, optionally a remote service arrangement (108), for storage, analysis, inspection or distribution.

13. The device of claim 1, configured to adapt data logging based on at least one element selected from the group consisting of:

battery status, remaining battery, remaining operating time, network status, and the current dynamically adjustable settings.

14. The device of claim 1, configured to adapt data logging through execution of at least one action selected from the group consisting of:

triggering collection of predetermined data, interrupting collection of predetermined data, triggering predetermined data processing or analysis procedure, interrupting predetermined data processing or analysis procedure, triggering data transfer with an external entity, and triggering more detailed or coarser detection of objects on the display.

15. The device of claim 1, configured to associate data logged when the display is not in use, with metadata, optionally a tag, indicative of the background nature of the data collection instant.

16. The device of claim 1, configured to detect a transaction based on at least one object rendered on the display data and fingerprint associated with the transaction, whereupon the device is further configured to log data regarding the transaction, said logged data optionally comprising at least one element selected from the group consisting of:

identity of one or more transaction parties, time of transaction, and the related application or service.

17. The device of claim 1, configured to receive, optionally wirelessly, data from an external device indicative of user engagement with the UI of the external device, such as video image data captured through camera equipped goggles.

18. The device of claim 17, configured to subject the received data to a predetermined analysis procedure, optionally image data analysis procedure, and log the results based on predetermined logic.

19. The device of claim 1, further configured to receive data from at least one external device indicative of user engagement with the UI of the external device, and configured to determine an indication of user attention spread across the electronic terminal device and said at least one external device.

20. The device of claim 1, configured to monitor whether data logging feature is functional, and if that is not the case according to predetermined criterion, further configured to activate, reactivate and/or initialize it.

21. The device of claim 1, configured to preferably periodically, optionally substantially at regular intervals, execute a number of predetermined tasks to prevent the operating system of the device from shutting down data logging activity due to passivity or lack of user interaction.

22. A server arrangement (108, 110) including one or more functionally connected servers, comprising a processing entity (250a) for processing instructions (258a) and other data, a memory entity (256a) for storing the instructions and other data, and a communication interface (252a) for transferring data, wherein the arrangement is configured, in accordance with the instructions,

to store data (218, 224) comprising a number of fingerprints (222) regarding the display usage at terminal devices, process data (218, 228) received from a plurality of terminal devices and indicative of terminal device usage and of display usage in view of the fingerprints, and construct (216) a number of aggregate usage indicators or statistics (112) reflecting user behavior and engagement with the associated terminal devices, applications, services and/or features.

23. The server arrangement of claim 22, wherein the fingerprints include a number of fingerprints regarding one or more visual objects associated with one

or more software entities and potentially shown to a user via a display of a terminal device, and wherein said data processing includes processing data indicative of any of said visual objects rendered on the display of the terminal

device during user engagement in view of the fingerprints.

24. The server arrangement of claim 22, configured to transmit a set of fingerprints to a terminal device for use locally thereat for display usage detection, preferably visual object detection in display data.

25. The server arrangement of claim 22, configured to detect (228) the presence of any of visual objects as indicated by the fingerprints (222) in the received data (224) and store indication of the findings (218).

26. The server arrangement of claim 22, configured to establish a fingerprint associated with a number of visual objects of an application or service based on a display view of the application or service.

27. The server arrangement of claim 22, configured to establish, on the basis of the data acquired from multiple terminal devices regarding a population of terminal device users, at least one indicator selected from the group consisting of:

application usage, service usage, feature usage, user profile, application usage pattern, service usage pattern, long-tail estimate of Internet, device, application, service or feature usage, duration or time of application, service or feature usage, location of application, service or feature usage, multi screen usage, multi-device usage, use of auxiliary devices, user engagement between devices, applications, services or features, churn of users relative to application or service, reach of application, performance of application or service, service or feature, Internet usage, network usage, cellular network usage, WLAN usage, short-range communications usage, monetization of application, monetization of service, mobile payment, coupon, authorization, mobile purchase, digital transaction, signal strength, quality of speech, network speed, transfer rate, average transfer rate, minimum transfer rate, maximum transfer rate, memory usage, processor load, application crash, and service down-time.

28. The server arrangement of claim 22, configured to provide a constructed aggregate usage indicator or statistics for consumption via at least one distribution channel selected from the group consisting of:

web-based UI, a native application, a widget, a HTML5 application, and/or an API.

29. The server arrangement of claim 22, configured to associate at least two data providing terminal devices with the same user based on the utilized criteria applying a number of available indicia including at least one element selected from the group consisting of:

recurring, close proximity between the terminals, common or similar identification data relating to the terminals, recurring substantially same location among the terminals, and common or similar data provided by the terminals.

30. A method (300) for tracking behavioral events to be performed by an electronic device provided with at least functionally connected display, comprising obtaining data (304) defining a number of fingerprints regarding the display usage, and logging data (306) comprising behavioral data indicative of user engagement with the user interface of the device, incorporating detecting the display usage based on the fingerprints and storing an indication of such detection, wherein the status of the display is determined (308) and said logging controlled (310, 310b) in accordance with the determination.

31. The method of claim 30, wherein a number of fingerprints characterize one

or more visual objects associated with one or more software entities and potentially shown to a user via the display, and logging data incorporates detecting the presence of any of said objects as rendered on the display based on the fingerprints.

32. The method of claim 30, wherein said determining comprises monitoring whether, based on one or more predetermined indications, the display

of the device is in use or not, and substantially exclusively when, according to the determination, the display is in use, said detecting is executed (308, 310, 310b).

35. A computer program comprising code means adapted to, when run on a computer, to execute the method items of claim 30.

36. A carrier medium comprising the computer program according to claim 35.

33. A method (300) for analyzing data collected by a plurality of electronic devices, comprising receiving data (312) collected by a plurality of electronic devices and indicative of terminal device usage and of related display usage in view of a number of fingerprints characterizing the display usage, and constructing (314, 318) a number of aggregate usage indicators or statistics reflecting user behavior and engagement with the associated devices, applications, services and/or features.

34. The method of claim 33, wherein received data is indicative of one or more visual objects rendered on the display of terminal device during user engagement

in view of a number of fingerprints, which characterize a number of visual objects associated with one or more software entities and potentially shown to a user via the display of the device during the execution of the related entity.