Recently, Revenue Performance Magazine published two Top 20 lists that sent the affiliate marketing community into a frenzy. Many contended that the Top 20 CPA and Affiliate Network Rankings failed to accurately capture the most reputable networks in business while others alleged that many networks simply bought their way into the rankings through ad sales. But never content with angry hearsay, we decided to call up Revenue Performance Editor/Publisher Chris Trayhorn and pick his brain.
Hello Chris. How are you?
I’m good, thanks. It’s a beautiful sunny day here in San Francisco and we won the World Series. I count my blessings.
Tell me a little about the Top 20 network rankings. What motivated you to start this list?
Here at Revenue Performance, we have promoted best practices in the affiliate marketing industry since 2003—when I founded the magazine. The Blue Book Top 20 rankings are an effort to provide a credible view of which networks are doing a great job for their advertisers and publishers. There are dozens of lists of ad networks available all over the Internet. Most of them are BS. I wanted to try and produce something that had some substance behind it.
Talking long-term, what kind of change are you hoping to impact by publishing a yearly list like this?
I think one of the hardest problems the industry faces in trying to grow is that new advertisers and publishers have no idea who to trust. If we can create a feedback mechanism, so that the views of the people who actually interact with networks is taken into account and given expression, then hopefully the networks will take notice.
Have you started seeing any change since the Blue Book rankings were being published?
In fact, we have already passed on to a few networks some of the comments we received in the survey we ran, and at least two of them are actively working on making improvements to their publisher interfaces as a direct result. That seems like a good thing. The networks are critically important to this industry. We need a way to evaluate how good a job they’re doing.
What kind of feedback are you getting? From networks included on the list?
Most of the ones that are included are pleased, although there are some that are angry that they aren’t ranked as highly as they think they deserve.
What about those that aren’t on the list?
Most of those that aren’t on the list are fine too–there are a lot of networks, after all. Then there are a few networks–typically those that just missed out–that are unhappy. That’s understandable. Not everybody can be a winner every time.
It seems to be ruffling a lot of feathers within the affiliate marketing world. Many are saying that there are a lot of top-tier networks absent from the Top 20 and a lot of not-so-top-tier networks ranking high in the Top 20. Is this a popularity contest? How are the rankings determined?
Yeah, I’ve been told about ruffled feathers but the funny thing is that it seems to be only happening on private discussion groups and the like. If anyone contacts me, then I’ll happily engage and explain what we did and why. Here’s the main point that people miss: it’s not intended to be a just a list of which networks are the biggest. If you want to know who gets most traffic or whatever, you can find that elsewhere. We wanted to identify the “best” networks–the ones that are well-respected, and that advertisers and publishers love to work with. There are networks like that of all sizes. Whether they’re huge or tiny, we want the Blue Book to find them and flag them.
Some are also saying that advertising clients got preferential treatment on this list.
It’s just not true. Talk to my ad sales team who–every time we produce this list–have to go round to their clients and apologize because they’re not ranked as highly as they would have liked. Doing a list like this isn’t actually a great way to make friends and influence people. Look, on the affiliate networks list, 16 of the top 20 aren’t advertisers. On the CPA list, 11 out of the top 20 aren’t advertisers. If we were just trying to plug our advertisers, we did a pretty lousy job.
In assembling a ranking like this, how do you ensure against any kind of bias?
The only way is to include as many different viewpoints as possible from as many people as possible. We have over 5,000 publishers and advertisers that responded to our survey. We filtered those responses for anything that looked automated or fake. In order to introduce an expert element to the results we recruited a Blue-Ribbon Panel from around the industry, representing large and small networks, OPMs, CPA and affiliate networks, different types of traffic, you name it. And then we filtered those results in order to discount people voting in the interests of their own network. And then we included some of our own research on social media reputation, influence and buzz. It took a lot of work and we’re proud of it. There’s nothing else like it in the industry.
What do you think worked well about the way this list was generated?
Separating the networks into CPA and affiliate categories has helped the end-users a lot, and also encouraged networks to help us with distribution of the survey. The way we use an open-text field–without any list of suggested network names–to obtain the respondents’ favorite network produces interesting results, and we weight those responses more heavily than those from the multiple choice lists later in the survey. And the Blue-Ribbon Panel added a lot of value, even though some networks seem to be suspicious of it until they learn how we managed the process.
What do you think you can tweak in your methodology for next year’s rankings?
We’re looking at how advertisers’ views might differ from those of publishers, but presentation becomes difficult and you need a lot of data. As awareness increases, we’d like to see more responses, and also to expand the scope of some of the questions.
We’d like to get the survey in the hands of as many potential respondents as possible, and I’d like to encourage any networks that are unhappy to contact me and talk about how we can get their publishers engaged. Ultimately, the end goal is to maximize value to publishers and advertisers seeking the best network partners–I’m open to anything that will help with that.