Model-based recommendation systems
Memory-based recommendation systems are not always as fast and scalable as we would like them to be, especially in the context of actual systems that generate real-time recommendations on the basis of very large datasets. To achieve these goals, model-based recommendation systems are used.
Model-based recommendation systems involve building a model based on the dataset of ratings. In other words, we extract some information from the dataset, and use that as a "model" to make recommendations without having to use the complete dataset every time. This approach potentially offers the benefits of both speed and scalability.
Although the basic idea behind model-based recommendation systems is the same, there are a number of approaches that we can take to actually build the model and use it. Some examples are:
Probability problem: From this perspective, the problem of predicting a rating for a user-item pair is seen as the problem of predicting the probability of the rating being a particular value. Bayesian networks and clustering (for example, see [1]) often use this idea. For an algorithm that we implemented and uses this idea, please see Personality Diagnosis.
Enhancement to memory-based algorithms: The main idea behind memory-based recommendation systems is to calculate and use the similarities between users and/or items and use them as "weights" to predict a rating for a user and an item. The same idea can be used in model-based algorithms: the similarities between users and/or items can be calculated and then stored as a model, and then we can use the stored similarity values to predict ratings. These models can also be built using similarities between items rather than users and in fact, sometimes it is more desirable to do so. For example, the NetflixPrize data contains slightly fewer than 5,00,000 users, but only a little over 17,000 movies. This makes it likely that the resulting model over will be items will be smaller than that for users.
A model-based system such as this also often allows trimming of the model to make the system more scalable. In particular, we can limit the number of similar entities (users or items) that we store for each entity; in other words we store only k most similar entities. Researchers (e.g., [3]) have found (as we also did with Personality Diagnosis that storing a limited number of similar entities often has little effect on the accuracy of predictions.
For an example that we implemented, see Item-based collaborative filtering.
Linear algebra problem: The problem of making recommendations is sometimes viewed by considering the matrices of users and ratings available to us and performing linear algebra operations on them. For an example that we implemented, please see Singular Value Decomposition.
Advantages
Scalability: Most models resulting from model-based algorithms are much smaller than the actual dataset, so that even for very large datasets, the model ends up being small enough to be used efficiently. This imparts scalability to the overall system.
Prediction speed: Model-based systems are also likely to be faster, at least in comparison to memory-based systems because, the time required to query the model (as opposed to the whole dataset) is usually much smaller than that required to query the whole dataset.
Avoidance of overfitting: If the dataset over which we build our model is representative enough of real-world data, it is easier to try to avoid overfitting with model-based systems.
Disadvantages
Inflexibility: Because building a model is often a time- and resource-consuming process, it is usually more difficult to add data to model-based systems, making them inflexible.
Quality of predictions: The fact that we are not using all the information (the whole dataset) available to us, it is possible that with model-based systems, we don't get predictions as accurate as with model-based systems. It should be noted, however, that the quality of predictions depends on the way the model is built. In fact, as can be seen from the results page, a model-based system performed the best among all the algorithms we tried.
References
[1] J.S. Breese, D.Heckerman, and C.Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the Fourteenth Conference on Uncertainty in Artifical Intelligence, 1998.
[2] M.Deshpande and G. Karypis. Item-based top-n recommendation algorithms. ACM Trans. Inf. Syst., 22(1):143-177, 2004.
[3] B.M. Sarwar, G. Karypis, J.A. Konstan, and J. Reidl. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International World Wide Web Conference, pages 285-295, 2001.