How to Achieve Personalization Using Hybrid Recommendation Engine


Overview:

Modern technology today means everything is available at the click of a button: you can go to youtube for entertaining videos, LinkedIn to find professional connections, e-commerce sites to shop just about anything, and much, much more.

And in every turn, we find personalization. Our taste for music, the field we work in, the stuff we usually buy- whatever it may be, it all seems to be handpicked and curated just for us. This helps us get past all the manual, hectic filtration that we’d have to do in order to find our preferences. But websites these days are already aware of that choice.

This kind of personalized experience is possible through recommendation engines. At present, we have the following ways through which we can provide such recommendations.

Content-Based Filtering:

Content-based filtering uses item features to recommend other items similar to what the user likes, based on their previous actions. It represents the user in the item profile dimension based on users’ historical action. Some of the user-related features could be explicitly provided by the user. Note that the recommendations are specific to this user, as the model did not use any information about other users. This type of recommendation does not provide a diverse range of choices. 

This type of recommendation focuses on personal choice or content and is very useful in Job Recommendations as the choice of the job for the user needs to take into account to have a perfect matching job.

Collaborative Filtering:

Collaborative filtering is a method of making automatic predictions about the interests of a user by collecting preferences or “taste” information from many users (collaborating).

This engine does take account of the explicit interaction with the item irrespective of the profile of the item. It works in an assumption that if user A likes product X, Y, Z, and user B likes product X, Y, J then as both the user likes the most similar products. Hence,  we can recommend product J to user A and, vice-versa. 

Note that these predictions are specific to the user, but use ratings gleaned from many users. This can lead to recommending a diverse set of items irrespective of personal preference. It is mostly used to recommend items on e-commerce sites.

Hybrid Filtering:

Most recommender systems now use a hybrid approach, combining collaborative filtering, content-based filtering, and other approaches. There is no reason why several different techniques of the same type could not be hybridized. Hybrid approaches can be implemented in several ways: 

  • by making content-based and collaborative-based predictions separately and then combining them,
  • by adding content-based capabilities to a collaborative-based approach (and vice versa),
  • or, by unifying the approaches into one model.

So, why use hybrid approaches?

Several studies that empirically compare the performance of the hybrid with pure collaborative and content-based methods, demonstrated that hybrid methods can provide more accurate recommendations than pure approaches. 

These methods can also be used to overcome some of the common problems in recommender systems such as the cold start and the sparsity problem, as well as the knowledge engineering bottleneck in knowledge-based approaches.

Netflix is a good example of a hybrid recommender system. It makes recommendations by comparing the watching and searching habits of similar users (i.e., collaborative filtering) as well as by offering movies that share characteristics with films that a user has rated highly (content-based filtering).

There are many types of hybrid recommendation engines, but let’s talk about one of them: CBCF (content boosted Collaborative Filtering). 

CBCF is a type of hybrid recommendation technique that uses a combination of Content-based filtering and collaborative filtering to boost performance. 

Let us take an example of music or video recommendation. When we use content-based recommendations, it recommends very specific music, which might not always be true, and it could be monotonous. Hence taking the user preferences but recommending a diverse set of music has more advantages

Benefits over CBF and CF:

  • Reduces sparsity
  • Solves cold-start problem
  • Solves first rater Problem
  • Dependent on the feature of the user and item as well as the latent feature vector of user and item.

How it works:

The main purpose of Content boosted collaborative hybrid filtering is to reduce the sparsity in the user-item sparse matrix. We predict the rating of the item not rated by the user using a content-based predictor.

The architecture of the model:

Architecture based on neighborhood/clustering:

We used this architecture and tried to predict the unrated item by the user based on the similar items rated by the user. This architecture tries to figure out a cluster of the most similar items. The user rating on those particular items is averaged to have a prediction.

Architecture Based on Deep Learning Prediction Model:

This architecture takes item profiles into account. Similarly, a user profile is created based on the historical item interaction. Those two profiles are combined into a single feature and the deep learning model is trained. After training the model, the feature is created by combining the item profile not rated by the user with the user profile, hence a prediction is generated which is used to make the sparse matrix dense.

Evaluation and Result:

From our observation, we can say that it reduces sparsity and also decreases the evaluation error.

Before:

When sparsity is 99.2785% the average RMSE error tends to be around 1.0682 with a minimum standard deviation.

After:

After filling all the unrated items by the user using a content-based predictor, which makes sparsity 0. The average RMSE tends to be around 0.1099

Conclusion:

From our experience, we have concluded that types of recommendation engines are chosen based on usage, preference, and what purpose we are using it for. Selecting the appropriate recommendation system makes a difference.

The content-based recommendations will have advantages on user-specific applications where only user preference matters. Collaborative filtering will have a better hand when there is explicit feedback given. Hybrid recommendation, content boosted collaborative filtering takes the advantages of both content and collaborative filtering, which decreases sparsity and reduces errors. 


Find Anish on Linkedin.

Anish Pandey

Anish is a Machine Learning Engineer at Leapfrog. He is proficient at natural language processing and recommendation systems. He loves to churn numbers into meaningful insights.

More in Blogs

Age of Personalized Healthcare Artificial Intelligence

Age of Personalized Healthcare

With the proliferation of technology, crucial components of healthcare are evolving. Suddenly, patients are turning into “Digital Consumers”, medicines are

Read more
Handling Cold Start Problems in Recommender Systems Artificial Intelligence

Handling Cold Start Problems in Recommender Systems

Wikipedia defines a cold start problem as a potential problem in computer-based information systems which involve a degree of automated

Read more
Optimal Vaccine Allocation Using Recommendation-as-a-Service Artificial IntelligenceInsights

Optimal Vaccine Allocation Using Recommendation-as-a-Service

Given the global nature of the Covid-19 crisis, a high degree of coordination is needed among global bodies such as

Read more