How Web knows what you want?

Associative collaborative filtering

We have all seen how relevant recommendations pop up on our screen while using websites or apps such as google, yahoo, amazon, Flipkart, etc. These suggestions are generated using data which they have about similar customers like us and one of the methodologies to derive suggestions is known as associative collaborative filtering. It is used to Predict ratings and create personalized recommendations for products like books, songs, or movies. It's called collaborative filtering because the filtering is done on the basis of what others have bought or what you have already bought. The recommendations are generally based on some kind of association rules.

An example of a recommendation on Amazon.com:

Common Methods:

Broadly there are 2 types of recommendation/ suggestion systems.

1. Collaborative Filtering

2. Content-Based Filtering

Collaborative filtering:

Suppose there are 2 people. One likes riding the bike and the other likes running. They both buy burgers and salad which makes these two people similar. Then the person who runs buys coke. Hence on the basis of similarity of choices we recommend coke to the person who bikes. The basic concept is to figure out people who are similar, and suggest what additional items they bought.

Content-Based Filtering:

Suppose a person buys the coke. Because Pepsi is similar to coke. So on the basis of the similarity of the item, we recommend Pepsi to this person. Based on the item, you recommend it to other users. So, if you saw romance and thrillers, and then you want to find romance and thrillers, and recommend them back to the person. That's called item-based collaborative filtering, and there are methods of combining the two techniques together as well. The hybrid algorithm uses naïve Bayes as the classifier, and it then fills in the missing values of the rating matrix. It then makes predictions over the resulting rating matrix by using a Pearson correlation algorithm, which in return gives a higher weight for the item that is rated by more users.

There is another method known as User-based collaborative filtering. So, how do you go about user-based collaborative filtering?

We target a person and its choices and create a neighborhood and then find other persons who are close to this person. For example in KNN, five neighborhoods are to be considered, and then you look at the ratings of objects which are not being seen by you, but for whom your neighbors have ratings. By considering the average ratings of users in the neighborhood, and then looking at the high ones, the prediction/ suggestion is done for you. So, basically, find the users near you, find how they have liked/ rated some other objects, the ones which you have not seen, and take an average of the items as per-users and suggest them to you based on the order of magnitude.

Similarity computation computes the similarity between items or users and is a critical step in collaborative filtering algorithms The filtration of similarity can be done on the basis of Similarity Computation, Correlation-Based Similarity, Vector Cosine-Based Similarity, conditional probability-based similarity, etc.

Fun fact:

In 2006, Netflix prized a challenge that attracted thousands of researchers and coders to compete for the million-dollar prize race for the improvement of performance for movie recommendations. The challenge was featured with a huge dataset ( 18,000 movies)

Conclusion:

Collaborative filtering methods using the neighborhood-based technique generally computes the similarity between users or items, and then uses these weighted sums of ratings/averages or some mathematical calculations to make suggestions based on the similarity values.