Introduction

Online advertising flourishes as the ideal choice for both small and large businesses to target their marketing campaigns to the appropriate customers on the fly. An advertiser provides an advertising commissioner with its advertisements, plans a budget, and sets a commission for each customer action. The content publishers, on the other hand, make a contract with the commissioner to display advertisements on their websites. However, since publishers earn revenue based on impressions and clicks they drive to advertisers, there is an incentive for dishonest publishers to inflate the number of impressions/clicks their sites generate—a phenomenon known as click fraud. Click fraud hinders the reliability of online advertising system, and the market for online advertising will eventually contract in a long-term. Moreover, it may lead to expensive litigations from unsatisfied advertisers and bad reputation for the commissioner. It is important for the commissioner to proactively prevent click fraud so as to convince their advertisers the fairness of their accounting practices. Accordingly, a reliable click fraud detection system is needed to help identify dishonest publishers and maintain the commissioner’s credibility.

Prize

The primary entries from all participants will be ranked in decreasing order of their respective evaluation scores (see Evaluation section), computed on the test set. The top three entries will receive the following cash prizes:

• First place - SGD 4,000
• Second place - SGD 2,000
• Third place - SGD 1,000

Workshop

A workshop for the competition results will be held in conjunction with the Asian Conference on Machine Learning (ACML) 2012. Registration waivers will be given to the speakers of five selected/winning teams. For other attendees, the following registration fees apply (for a single workshop):

• Early registration fee: SGD 40 (by 28 October)
• Late registration fee: SGD 80 (after 28 October)

To register, please click the following link:
https://wis.ntu.edu.sg/pls/apexacad/f?p=406:5

More details about the conference registration fee can be found at:
http://acml12.comp.nus.edu.sg/index.php?n=Main.Registration

Publication

The best entries of the data mining competition will be assembled for a jointly submitted article to Journal of Machine Learning Research (JMLR).

Organizers

Program Chairs:

• Richard J. OENTARYO, Singapore Management University
• Ee-Peng LIM, Singapore Management University
• Feida ZHU, Singapore Management University
• David LO, Singapore Management University
• Kok Fung LAI, BuzzCity Pte. Ltd.

Web Administrators:

• Juan DU, Singapore Management University
• Philips K. PRASETYO, Singapore Management University

Overview

This competition involves advertisement data provided by BuzzCity Pte. Ltd. BuzzCity is a global mobile advertising network that has millions of consumers around the world on mobile phones and devices. In Q1 2012, over 45 billion ad banners were delivered across the BuzzCity network consisting of more than 10,000 publisher sites which reach an average of over 300 million unique users per month. The number of smartphones active on the network has also grown significantly. Smartphones now account for more than 32% phones that are served advertisements across the BuzzCity network.

Dataset

The “raw” data used in this competition has two types: publisher database and click database, both provided in CSV format. The publisher database records the publisher’s (aka partner’s) profile and comprises several fields:

• publisherid – Unique identifier of a publisher.
• bankaccount – Bank account associated with a publisher (may be empty)
• address – Mailing address of a publisher (obfuscated; may be empty)
• status – Label of a publisher, which can be the following:
  • “OK” - Publishers whom BuzzCity deems as having healthy traffic (or those who slipped their detection mechanisms)
  • “Observation” – Publishers who may have just started their traffic or their traffic statistics deviates from system wide average. BuzzCity does not have any conclusive stand with these publishers yet
  • “Fraud” – Publishers who are deemed as fraudulent with clear proof. Buzzcity suspends their accounts and their earnings will not be paid

On the other hand, the click database records the click traffics and has several fields:

• id – Unique identifier of a particular click
• numericip – Public IP address of a clicker/visitor
• deviceua – Phone model used by a clicker/visitor
• publisherid – Unique identifier of a publisher
• adscampaignid – Unique identifier of a given advertisement campaign
• usercountry – Country from which the surfer is
• clicktime – Timestamp of a given click (in YYYY-MM-DD format)
• publisherchannel – Publisher's channel type, which can be the following:
  • ad – Adult sites
  • co – Community
  • es – Entertainment and lifestyle
  • gd – Glamour and dating
  • in – Information
  • mc – Mobile content
  • pp – Premium portal
  • se – Search, portal, services
• referredurl - URL where the ad banners were clicked (obfuscated; may be empty). More details about the HTTP Referer protocol can be found in this article

Note: The raw data provided is encrypted for privacy protection, and the format of some fields (e.g., partner id, bankaccount, campaign id, referer URL) may change over time due to different encryption schemes. So predictions made should not depend on specific field format or encryption method.

Task

The goal of this competition is to build a data-driven methodology for effective detection of fraudulent publishers. In particular, the task is to detect "Fraud" publishers (positive cases) and separate them from "OK" and "Observation" publishers (negative cases), based on their click traffic and account profiles. This would help shed light on several key areas:

• What is the underlying click fraud scheme?
• What sort of concealment strategies commonly used by fraudulent parties?
• How to interpret data for patterns of dishonest publishers and websites?
• How to build effective fraud prevention/detection plans?

For training and evaluation, we provide three sets of click and publisher data:

• Training set - For building your prediction model
• Validation set - For evaluation of your model in the public leaderboard (see Evaluation). This can be treated as holdout set
• Testing set - For final evaluation of your model and thus determination of the competition winners

Note: Each dataset comes from a different time window, so there could be publishers (publisherid) who appear across different sets and whose status label may change. You can infer the time windows from the timestamps of the click dataset.

Participation

You may participate in the competition as an individual or as part of a team of up to five participants. Only two submissions per day from each team are allowed. You may only be involved in one "team" in the competition (i.e. either as an individual or as a member of a single team, but not both). For team registrations, the team must select a team leader, who will provide the team name, corresponding email address, and team affiliation during registration. Each team is required to register for a user account and log into the competition site. The e-mail addresses of the participants will not be publicly displayed during the competition.

Format

The submission format consists of a two-column CSV file. The first column contains the partner id (sorted lexicographically in ascending order). The second column contains the prediction or confidence scores for the fraud (positive) class label, typically normalized within [0, 1] or [-1, 1]. The average precision is then calculated by ranking the scores and measuring precision at different cutoffs (see Evaluation).

Submission

To submit your new results, please enter your file below (you need to login first). After uploading your file, it takes a few seconds for scoring, and then you should see your result. Your entry in the public leaderboard (see Evaluation) shall be updated if your current result beats your previous best result. Note: The order of the partner IDs (i.e. the first column) in the submitted file should match that in the sample file.

Final Submission

You can submit your final results on the test dataset below. Note that the submission deadline is on 1 Oct 04:00am UTC, and you are only allowed to submit max two times by then. Any subsequent submission shall be disregarded. Out of the two submissions, the best score will be computed automatically and used for selection of winners. You will receive a separate email showing your and other teams’ final scores after the competition ends.

Evaluation

Submissions will be evaluated using the average precision criterion. Full details of the criterion are given in this article. Briefly, suppose there are k publishers in a list ordered by their prediction score, the average precision (AP) is computed as follows:

where Precision(i) denotes the precision at cutoff i in the publisher list, i.e., the fraction of correct fraud prediction up to the position i, and m is the number of actual fraud publishers. Note that, when the ith prediction is incorrect, Precision(i) = 0.

Leaderboard

The public leaderboard below displays the best score and corresponding submission time of each team on the validation set. Please note that there will be a separate testing set to be used for final evaluation and determination of the winners. Hence, algorithm which performs best on the validation set does not always perform best on the testing set; this depends on its generalization ability on unseen data.

refresh