SYSTEM AND METHOD FOR CREATING AND IMPLEMENTING SCALABLE AND EFFECTIVE MULTI-MEDIA OBJECTS WITH HUMAN INTERACTION PROOF (HIP) CAPABILITIES

• US 20140059663A1
• Filed: 08/23/2013
• Published: 02/27/2014
• Est. Priority Date: 08/05/2011
• Status: Active Grant

• Current Assignee: [24]7.ai, Inc.
• Original Assignee: EngageClick, Inc. ([24]7.ai, Inc.)
• Inventors: RAJSHEKAR, Manoj, DEO, Shekhar Kumar
• Granted Patent: US 9,621,528 B2
• Time in Patent Office: Days
• US Class Current: 726/6
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 2221/2103 Challenge-response
• CPC Class Codes: G06F 2221/2133 Verifying human interaction, e.g., Captcha
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0258 Registration
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32

Abstract

Embodiments of the invention provide a method and apparatus (“system”) that overcome the above-mentioned problems among others and provide an innovative solution aimed at creating an interactive, dynamic and effective multi-media object with HIP capabilities which may be used in online advertising, security, and user-defined security. The system leverages the existing HIP CAPTCHA real estate to create multi-media objects that guarantee a captivated audience, especially in online advertising. Combining interactive multi-media objects with HIP capabilities helps to meet a very critical need faced by advertisers and websites today—creating an effective impression of any multi-media object on a user (a guaranteed eyeball). Embodiments of the current invention introduce a variety of formats that involve interacting with a multi-media object to provide a more natural user interaction and ease of use while maintaining security.

Claims

1. An apparatus for generating and serving scalable and effective multimedia objects with human interaction proof (HIP) capabilities on a webpage, comprising:

a webpage or application configured to send a request for a multimedia object (MMO) and a challenge (HIP) to an engagement server and receive said multimedia object and said challenge from said engagement server;

wherein the webpage or application is further configured to send to said engagement server a response to said challenge;

wherein the engagement server is configured to, based on said response, determine a success or failure and send a success message or a failure message, respectively, to said website or application;

wherein said webpage or application is further configured to;

send an acknowledgement to said engagement server when said message is said success message and send a new request for a new multimedia object and new challenge to said engagement server until a predetermined threshold is met, when said message is said failure message;

wherein the engagement server comprises;

a multimedia object server (MMOS) configured to receive said request for said multimedia object and return said corresponding MMO and challenge;

a validation server (VS) configured to determine whether said response is a success or failure by matching said response with a stored response in a challenge and response repository;

an analytics server configured to receive and store informational details about said request and receive and store details about said response from said VS;

a status server module configured to receive and record whether said response is a success or failure.

2. The apparatus of claim 1, wherein said engagement server further comprises:

a video image integration server (VIIS) configured to receive from said MMOS a request for said MMO plus filter criteria including profile information regarding user identification plus, configured to find said MMO by performing a matching process with previously stored MMOs using said filter criteria and configured to send said MMO to said MMOS when said matching process is successful; and

wherein said MMO returned to said webpage or application comprises a video that has been previously tagged by having a particular section of the video associated to said challenge.

3. The apparatus of claim 2, wherein said video is previously tagged for secret questions by:

uploading a video to be combined with one or more secret questions;

optionally selecting one or more images to be added to said MMO comprising the video;

selecting said one or more secret questions to be used when said MMO is served, wherein said one or more secret questions are to be used verify user identification;

selecting one or more frames from said video;

for each of said one or more frames;

selecting a section and tagging said section and associating said tag with any of said one or more secret questions; and

providing an answer for each of said one or more questions;

submitting said tagged video to said MMOS and said challenge and response repository, wherein said submission comprises informational data regarding said sections, video, one or more questions, answers, user identification, password, and user profile; and

said MMOS and said challenge and response repository storing said submitted data.

4. The apparatus of claim 2, wherein said video is previously tagged by:

creating a profile;

uploading a user specified video, wherein said user specified video is selected from local, public domain, or a private repository;

using video editing software, selecting one or more frames from said video;

for each of said one or more frames, selecting a section and tagging said section;

storing said video in a video repository and storing metadata of said one or more frames and associated tags in a metadata repository, wherein said video and said metadata are recombined when said MMO is served.

5. The apparatus of claim 2, wherein said MMO and said challenge are created by stitching a user specified video and user defined multiple images and using the resultant multimedia as a challenge to identify the correct user, comprising:

receiving a video file;

splitting said video file randomly into sequences pieces;

receiving user defined images;

associating challenges and responses to the pieces and the images;

stitching said images with said video pieces and creating a new video file; and

associating said new video file to a profile,

wherein said images within said new video file are used in said challenge.

6. The apparatus of claim 1, wherein said MMO and said challenge comprise secret questions used to validate a user'"'"'s identity;

wherein said secret questions are classified as belonging to particular types of advertisements (ads) and wherein when a secret question is shown, a corresponding ad is also shown and used as a challenge.

7. The apparatus of claim 1, wherein a valid response to said challenge is a combination of interaction with an image on said MMO and a response that is dependent on content displayed on said webpage or application.

8. The apparatus of claim 1, wherein said MMO and said challenge comprise one or more interactive maps by incorporating web mapping technology and wherein said web mapping technology asks a user to identify or tag important information on said one or more interactive maps which only the user may be able to identify. one or more interactive maps are used to identify a user and as HIP.

9. The apparatus of claim 1, wherein said sending said request for said MMO and said challenge further comprises:

requesting for HIP, wherein it is determined whether said request is for a text based challenge or a non-text based challenge;

when said request is for a text based challenge;

get an MMO suitable for a text based challenge;

determine whether the metadata available is sufficient, limited, or none;

when said metadata available is sufficient;

go to challenge processing data;

when said metadata available is limited;

ask a metadata processing engine whether it has data to use as a challenge and, if no, go to a metadata finder engine and if yes, go to challenge processing data;

when said metadata available is none;

go to said metadata finder engine;

wherein said metadata finder engine determines whether there is sufficient metadata and when yes, processes challenge data and when no, discards the MMO;

wherein after challenge processing data, go to validation processing engine and subsequently to HIP formation engine and send said MMO with said HIP to a client;

wherein after interaction with the HIP at the client, send a response to VS to determine whether said response is valid and when yes, said client receives a positive response and when no, make a request to said browser to resend request for HIP; and

when said request is for a non-text based challenge, get an MMO suitable for a non-text challenge and go to said step, determine whether the metadata available is sufficient, limited, or none.

10. The apparatus of claim 1, wherein said MMO and said challenge comprise hybrid challenges and wherein said hybrid challenges are created by:

said website or application requesting HIP;

said request being received at a traffic analyzer configured to send complexity information about the HIP to a hybrid challenge and response generator;

wherein said hybrid challenge and response generator obtains two or more HIPs from any of a click type challenge and response generator, a drag type challenge and response generator, and a type-in text challenge and response generator;

wherein a combined HIP generator combines said two or more HIPs;

wherein said combined HIPs are served to a client;

wherein after the combined HIPs are interacted with, a request for validation is sent to said VS;

wherein when said validation is successful, said success message is sent to said webpage or application and when said validation is a failure, send said failure message to said webpage or application when a refresh threshold is expired, otherwise request HIP again at said traffic analyzer and increment a refresh counter that is used to determine when the refresh threshold expires.

11. The apparatus of claim 1, wherein said MMO and said challenge comprise HIP;

wherein said MMO and challenge and HIP comprise known images and unknown images;

wherein the unknown images operate as noise in said HIP and cause more difficulty for bots to find valid objects in said MMO; and

wherein said known and unknown images are determined by;

an intelligent image selector recognizing and storing known images in an image repository,

said intelligent image selector executing a noise generator to generate unknown images having or being noise;

said intelligent image selector storing said unknown images in a noise repository;

said engagement server comprising a challenge and response generator which combines images from said image repository and said noise repository to create said MMO and challenge and HIP comprising known images and unknown images.

12. The apparatus of claim 1, wherein HIP is created using template based HIP building comprising:

a template selection engine for receiving an HIP request from the webpage or application;

said template selection engine obtaining a template from a template repository;

said template selection engine sending said template to an object relationship selection engine;

said object relationship engine uses intelligent classification and aggregation algorithms to populate and build relationships within elements of said template and based on said elements, picks most suitable relationships to be defined as a challenge in the HIP;

an MMO object selection engine selects an MMO object;

an HIP builder combines the MMO object and said defined challenge and sends said combined MMO object with said defined challenge to a client.

13. The apparatus of claim 12, wherein templates, relationships, objects, and types of interactions are encompassed as one set and a different combination of different or the same templates, relationships, objects, and types of interactions are encompassed as a different set and wherein two or more different combinations of said sets are used to create different HIPs.

14. The apparatus of claim 1, wherein said request for said MMO and said challenge comprises a HIP request and wherein sending said request for said MMO and said challenge further comprises:

receiving by a human computation HIP generator, the HIP request, which sends control to an active (real) challenge and response generator to obtain a part of the challenge which is active and sends control to a passive challenge and response generator to obtain a part of the challenge which is passive;

sending said active and passive parts to a HIP builder which is configured to hide or unhide active and passive parts of the challenge to create a final challenge;

wherein active HIP validation of said active parts of said challenge is performed by said VS and when validation is successful, the HIP response is sent to said website or application and when validation is not successful, control returns to said human computation HIP generator;

wherein passive HIP validation of said passive parts of said challenge is sent to a log database, which sends said validation on to said analytics server and wherein said analytics server is configured to use information about said validation in generating reports.

15. The apparatus of claim 1, wherein said request comprises information about a device from which said website or application are communicatively connected and said MMOS is further configured to parse and use said device information to choose said MMO and said challenge based on capabilities of said device.

16. The apparatus of claim 1, wherein said challenge is challenge asked against an image for a user to provide important information noticed in the picture in a local language and wherein related code translates internally the local language and matches with other responses of other languages which are stored in the system and when there is a match, the response is considered valid.

17. The apparatus of claim 1, wherein said MMO comprises in-premise or house-media, wherein in-premise or house-media comprise ads or any other branding or promotional marketing campaign objects owned by a company associated with said website or application or partners of said company.

18. The apparatus of claim 1, wherein the engagement server is further configured to detect if the request comes from a device that is a smart TV and to detect and determine interactions done in the smart TV and to understand such interactions as a response.

19. The apparatus of claim 1, wherein said MMO is configured to be turned on and off for said challenge in a workflow.

20. A computer-implemented method for generating and serving scalable and effective multimedia objects with human interaction proof (HIP) capabilities on a webpage, comprising the steps of:

providing a webpage or application configured to send a request for a multimedia object (MMO) and a challenge (HIP) to an engagement server and receive said multimedia object and said challenge from said engagement server;

wherein the webpage or application is further configured to send to said engagement server a response to said challenge;

wherein the engagement server is configured to, based on said response, determine a success or failure and send a success message or a failure message, respectively, to said website or application;

wherein said webpage or application is further configured to;

send an acknowledgement to said engagement server when said message is said success message and send a new request for a new multimedia object and new challenge to said engagement server until a predetermined threshold is met, when said message is said failure message;

wherein the engagement server comprises;

a multimedia object server (MMOS) configured to receive said request for said multimedia object and return said corresponding MMO and challenge;

a validation server (VS) configured to determine whether said response is a success or failure by matching said response with a stored response in a challenge and response repository;

an analytics server configured to receive and store informational details about said request and receive and store details about said response from said VS;

a status server module configured to receive and record whether said response is a success or failure.