Choosing a President by Trusting Celebrities and Public Figures
I was at dinner with a few people some months ago when we got to talking politics. Rather than discuss the people and the issues though, we ended up talking about the process by which people select candidates to favor. One of the people said that his mother always picked her candidate not by reading about their position on issues or their history but instead by asking her friends who they are voting for. We joked that this person's mother is simply using collaborative filtering to get a good recommendation. Then we asked the question: Why isn't this process automated? Now it is - The Election Recommender. We originally had only joked about using celebrity endorsements but the results have been very interesting so far. If you're satisfied just trying out the site, go ahead, otherwise read on for a more detailed explanation.
Netflix uses a system called Cinematch to determine which movies you might enjoy. This system is a very specialized, highly tweaked linear collaborative filtering algorithm. It takes every rating you've done and compares that to every other user's ratings, usually using Pearson correlation or some similar algorithm. The correlation score simply translates to "How similar are you and the person compared to you". The reason pearson is used instead of just totaling up how many of the same-movie 5s, 4s.. etc you've had is because certain people tend to be biased - that is they will rate everything just a little higher or lower than you, even if you feel the exact same way about the films. Pearson takes bias into account, which is very nice.
After correlating the users, your top "blank" spots can be found in the data. Movies that several people who have similar tastes to you have rated that you have not rated can be recommended, usually using some sort of weighting scheme to give extra priority to the best-match users.
Think about it like this. Jerry, Bob and Cindy all rated the movie Alien vs Predator with 2 stars, The Incredibles with 5 stars and Iron Man with 5 stars. Jim also gave AVP 2 stars and the Incredibles 5 stars. The system sees that Jim has very strong correlation with Jerry, Bob and Cindy and so predicts that Jim will also rate Iron Man as 5 stars... And that becomes his recommendation.
Make sense?
There is actually a fantastic O'Reilly book out called "Collective Intelligence" that covers these topics very well and goes into machine learning and much more advanced material. I'll let you dive into all of that at your leisure.
Back to the matter at hand.
The Election Recommender is my third and most simple recommender system (after my netflix prize entry and a larger recommender system I developed last year). It uses linear collaborative filtering but the endorsers are static, unlike a system like netflix which uses users to collaborative with other users. It's very striaght-forward in that your preferences for the endorsers are filtered directly through to the candidates they endorsed and out pops the recommendation. It's simply based on highest score.
Some would say that it's not true CF but it really is in its simplest form, which is why I think it makes for a good example of basic recommenders. Things would get much more interesting if somehow the endorsers were to endorse each other as well as the candidates. That would just create a web of which to filter instead of the straight path it is now.
To get the election recommender more like Netflix we'd need to give the users the ability to rate the other users, which isn't exactly the interface I had in mind.
The site is written completely in Ruby on Rails, is proxied through Apache 2 and runs on a FreeBSD server. I actually hope it gets some decent traffic so I can test out some scalability solutions for rails.
Please let me know what you think of the site! Enjoy!
The Election Recommender - Recommending America's Vote
Stumble It!
Leave a comment