Description
Viral marketing, viral advertising, or marketing buzz are buzzwords referring to marketing techniques that use pre-existing social networks and other technologies to produce increases in brand awareness or to achieve other marketing objectives
The Dynamics of Viral Marketing ?
Jure Leskovec
Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA
Lada A. Adamic
School of Information, University of Michigan, Ann Arbor, MI
Bernardo A. Huberman
HP Labs, Palo Alto, CA 94304
April 20, 2007
Abstract We present an analysis of a person-to-person recommendation network, consisting of 4 million people who made 16 million recommendations on half a million products. We observe the propagation of recommendations and the cascade sizes, which we explain by a simple stochastic model. We analyze how user behavior varies within user communities de?ned by a recommendation network. Product purchases follow a ’long tail’ where a signi?cant share of purchases belongs to rarely sold items. We establish how the recommendation network grows over time and how e?ective it is from the viewpoint of the sender and receiver of the recommendations. While on average recommendations are not very e?ective at inducing purchases and do not spread very far, we present a model that successfully identi?es communities, product and pricing categories for which viral marketing seems to be very e?ective.
1
Introduction
With consumers showing increasing resistance to traditional forms of advertising such as TV or newspaper ads, marketers have turned to alternate strategies, including viral marketing. Viral marketing exploits existing social networks by encouraging customers to share product information with their friends. Previously, a few in depth studies have shown that social networks a?ect the adoption of individual innovations and products (for a review see [Rog95] or [SS98]). But until recently it has been di?cult to measure how in?uential person-to-person recommendations actually are over a wide range of products. Moreover, Subramani and Rajagopalan [SR03] noted that “there needs to be a greater understanding of the contexts in which viral marketing strategy works and the characteristics of products and services for which it is most
? This work also appears in: Leskovec, J., Adamic, L. A., and Huberman, B. A. 2007. The dynamics of viral marketing. ACM Transactions on the Web, 1, 1 (May 2007).
1
2
J. Leskovec et al.
e?ective. This is particularly important because the inappropriate use of viral marketing can be counterproductive by creating unfavorable attitudes towards products. What is missing is an analysis of viral marketing that highlights systematic patterns in the nature of knowledge-sharing and persuasion by in?uencers and responses by recipients in online social networks.” Here we were able to in detail study the above mentioned problem. We were able to directly measure and model the e?ectiveness of recommendations by studying one online retailer’s incentivised viral marketing program. The website gave discounts to customers recommending any of its products to others, and then tracked the resulting purchases and additional recommendations. Although word of mouth can be a powerful factor in?uencing purchasing decisions, it can be tricky for advertisers to tap into. Some services used by individuals to communicate are natural candidates for viral marketing, because the product can be observed or advertised as part of the communication. Email services such as Hotmail and Yahoo had very fast adoption curves because every email sent through them contained an advertisement for the service and because they were free. Hotmail spent a mere $50,000 on traditional marketing and still grew from zero to 12 million users in 18 months [Jur00]. The Hotmail user base grew faster than any media company in history – faster than CNN, faster than AOL, even faster than Seinfeld’s audience. By mid-2000, Hotmail had over 66 million users with 270,000 new accounts being established each day [Bro98]. Google’s Gmail also captured a signi?cant part of market share in spite of the fact that the only way to sign up for the service was through a referral. Most products cannot be advertised in such a direct way. At the same time the choice of products available to consumers has increased manyfold thanks to online retailers who can supply a much wider variety of products than traditional brick-andmortar stores. Not only is the variety of products larger, but one observes a ‘fat tail’ phenomenon, where a large fraction of purchases are of relatively obscure items. On Amazon.com, somewhere between 20 to 40 percent of unit sales fall outside of its top 100,000 ranked products [BHS03]. Rhapsody, a streaming-music service, streams more tracks outside than inside its top 10,000 tunes [Ano05]. Some argue that the presence of the long tail indicates that niche products with low sales are contributing signi?cantly to overall sales online. We ?nd that product purchases that result from recommendations are not far from the usual 80-20 rule. The rule states that the top twenty percent of the products account for 80 percent of the sales. In our case the top 20% of the products contribute to about half the sales. E?ectively advertising these niche products using traditional advertising approaches is impractical. Therefore using more targeted marketing approaches is advantageous both to the merchant and the consumer, who would bene?t from learning about new products. The problem is partly addressed by the advent of online product and merchant reviews, both at retail sites such as EBay and Amazon, and specialized product comparison sites such as Epinions and CNET. Of further help to the consumer are collaborative ?ltering recommendations of the form “people who bought x also bought y ” feature [LSY03]. These re?nements help consumers discover new products and receive more accurate evaluations, but they cannot completely substitute personalized
The Dynamics of Viral Marketing
3
recommendations that one receives from a friend or relative. It is human nature to be more interested in what a friend buys than what an anonymous person buys, to be more likely to trust their opinion, and to be more in?uenced by their actions. As one would expect our friends are also acquainted with our needs and tastes, and can make appropriate recommendations. A Lucid Marketing survey found that 68% of individuals consulted friends and relatives before purchasing home electronics – more than the half who used search engines to ?nd product information [Bur03]. In our study we are able to directly observe the e?ectiveness of person to person word of mouth advertising for hundreds of thousands of products for the ?rst time. We ?nd that most recommendation chains do not grow very large, often terminating with the initial purchase of a product. However, occasionally a product will propagate through a very active recommendation network. We propose a simple stochastic model that seems to explain the propagation of recommendations. Moreover, the characteristics of recommendation networks in?uence the purchase patterns of their members. For example, individuals’ likelihood of purchasing a product initially increases as they receive additional recommendations for it, but a saturation point is quickly reached. Interestingly, as more recommendations are sent between the same two individuals, the likelihood that they will be heeded decreases. We ?nd that communities (automatically found by graph theoretic community ?nding algorithm) were usually centered around a product group, such as books, music, or DVDs, but almost all of them shared recommendations for all types of products. We also ?nd patterns of homophily, the tendency of like to associate with like, with communities of customers recommending types of products re?ecting their common interests. We propose models to identify products for which viral marketing is e?ective: We ?nd that the category and price of product plays a role, with recommendations of expensive products of interest to small, well connected communities resulting in a purchase more often. We also observe patterns in the timing of recommendations and purchases corresponding to times of day when people are likely to be shopping online or reading email. We report on these and other ?ndings in the following sections. We ?rst survey the related work in section 2. We then describe the characteristics of the incentivised recommendations program and the dataset in section 3. Section 4 studies the temporal and static characteristics of the recommendation network. We investigate the propagation of recommendations and model the cascading behavior in section 5. Next we concentrate on the various aspects of the recommendation success from the viewpoint of the sender and the recipient of the recommendation in section 6. The timing and the time lag between the recommendations and purchases is studied in section 7. We study network communities, product characteristics and the purchasing behavior in section 8. Last, in section 9 we present a model that relates product characteristics and the surrounding recommendation network to predict the product recommendation success. We discuss the implications of our ?ndings and conclude in section 10.
4
J. Leskovec et al.
2
Related work
Viral marketing can be thought of as a di?usion of information about the product and its adoption over the network. Primarily in social sciences there is a long history of the research on the in?uence of social networks on innovation and product di?usion. However, such studies have been typically limited to small networks and typically a single product or service. For example, Brown and Reingen [BR87] interviewed the families of students being instructed by three piano teachers, in order to ?nd out the network of referrals. They found that strong ties, those between family or friends, were more likely to be activated for information ?ow and were also more in?uential than weak ties [Gra73] between acquaintances. Similar observations were also made by DeBruyn and Lilien in [DL04] in the context of electronic referrals. They found that characteristics of the social tie in?uenced recipients behavior but had di?erent e?ects at di?erent stages of decision making process: tie strength facilitates awareness, perceptual a?nity triggers recipients interest, and demographic similarity had a negative in?uence on each stage of the decision-making process. Social networks can be composed by using various information, i.e. geographic similarity, age, similar interests and so on. Yang and Allenby [YA03] showed that the geographically de?ned network of consumers is more useful than the demographic network for explaining consumer behavior in purchasing Japanese cars. A recent study by Hill et al. [HPV06] found that adding network information, speci?cally whether a potential customer was already “talking to” an existing customer, was predictive of the chances of adoption of a new phone service option. For the customers linked to a prior customer the adoption rate of was 3–5 times greater than the baseline. Factors that in?uence customers’ willingness to actively share the information with others via word of mouth have also been studied. Frenzen and Nakamoto [FN93] surveyed a group of people and found that the stronger the moral hazard presented by the information, the stronger the ties must be to foster information propagation. Also, the network structure and information characteristics interact when individuals form decisions about transmitting information. Bowman and Narayandas [BN01] found that self-reported loyal customers were more likely to talk to others about the products when they were dissatis?ed, but interestingly not more likely when they were satis?ed. In the context of the internet word-of-mouth advertising is not restricted to pairwise or small-group interactions between individuals. Rather, customers can share their experiences and opinions regarding a product with everyone. Quantitative marketing techniques have been proposed [Mon01] to describe product information ?ow online, and the rating of products and merchants has been shown to e?ect the likelihood of an item being bought [RZ02, CM06]. More sophisticated online recommendation systems allow users to rate others’ reviews, or directly rate other reviewers to implicitly form a trusted reviewer network that may have very little overlap with a person’s actual social circle. Richardson and Domingos [RD02] used Epinions’ trusted reviewer network to construct an algorithm to maximize viral marketing e?ciency assuming that individuals’ probability of purchasing a product depends on the opinions on the trusted peers in their network. Kempe, Kleinberg and Tardos [KKT03] have followed up on Richardson and Domingos’ challenge of maximizing viral information spread by evaluating several algorithms given various models of adoption we discuss
The Dynamics of Viral Marketing
5
next. Most of the previous research on the ?ow of information and in?uence through the networks has been done in the context of epidemiology and the spread of diseases over the network. See the works of Bailey [Bai75] and Anderson and May [AM02] for reviews of this area. The classical disease propagation models are based on the stages of a disease in a host: a person is ?rst susceptible to a disease, then if she is exposed to an infectious contact she can become infected and thus infectious. After the disease ceases the person is recovered or removed. Person is then immune for some period. The immunity can also wear o? and the person becomes again susceptible. Thus SIR (susceptible – infected – recovered) models diseases where a recovered person never again becomes susceptible, while SIRS (SIS, susceptible – infected – (recovered) – susceptible) models population in which recovered host can become susceptible again. Given a network and a set of infected nodes the epidemic threshold is studied, i.e. conditions under which the disease will either dominate or die out. In our case SIR model would correspond to the case where a set of initially infected nodes corresponds to people that purchased a product without ?rst receiving the recommendations. A node can purchase a product only once, and then tries to infect its neighbors with a purchase by sending out the recommendations. SIS model corresponds to less realistic case where a person can purchase a product multiple times as a result of multiple recommendations. The problem with these type of models is that they assume a known social network over which the diseases (product recommendations) are spreading and usually a single parameter which speci?es the infectiousness of the disease. In our context this would mean that the whole population is equally susceptible to recommendations of a particular product. There are numerous other models of in?uence spread in social networks. One of the ?rst and most in?uential di?usion models was proposed by Bass [Bas69]. The model of product di?usion predicts the number of people who will adopt an innovation over time. It does not explicitly account for the structure of the social network but it rather assumes that the rate of adoption is a function of the current proportion of the population who have already adopted (purchased a product in our case). The di?usion equation models the cumulative proportion of adopters in the population as a function of the intrinsic adoption rate, and a measure of social contagion. The model describes an S-shaped curve, where adoption is slow at ?rst, takes o? exponentially and ?attens at the end. It can e?ectively model word-of-mouth product di?usion at the aggregate level, but not at the level of an individual person, which is one of the topics we explore in this paper. Di?usion models that try to model the process of adoption of an idea or a product can generally be divided into two groups: • Threshold model [Gra78] where each node in the network has a threshold t ? [0, 1], typically drawn from some probability distribution. We also assign connection weights wu,v on the edges of the network. A node adopts the behavior if a sum of the connection weights of its neighbors that already adopted the behavior (purchased a product in our case) is greater than the threshold: t ? adopters(u) wu,v . • Cascade model [GLM01] where whenever a neighbor v of node u adopts, then node u also adopts with probability pu,v . In other words, every time a neighbor
6
J. Leskovec et al.
of u purchases a product, there is a chance that u will decide to purchase as well. In the independent cascade model, Goldenberg et al. [GLM01] simulated the spread of information on an arti?cially generated network topology that consisted both of strong ties within groups of spatially proximate nodes and weak ties between the groups. They found that weak ties were important to the rate of information di?usion. Centola and Macy [CM05] modeled product adoption on small world topologies when a person’s chance of adoption is dependent on having more than one contact who had previously adopted. Wu and Huberman [WH04] modeled opinion formation on di?erent network topologies, and found that if highly connected nodes were seeded with a particular opinion, this would proportionally e?ect the long term distribution of opinions in the network. Holme and Newman [HN06] introduced a model where individuals’ preferences are shaped by their social networks, but their choices of whom to include in their social network are also in?uenced by their preferences. While these models address the question of how in?uence spreads in a network, they are based on assumed rather than measured in?uence e?ects. In contrast, our study tracks the actual di?usion of recommendations through email, allowing us to quantify the importance of factors such as the presence of highly connected individuals, or the e?ect of receiving recommendations from multiple contacts. Compared to previous empirical studies which tracked the adoption of a single innovation or product, our data encompasses over half a million di?erent products, allowing us to model a product’s suitability for viral marketing in terms of both the properties of the network and the product itself.
3
3.1
The Recommendation Network
Recommendation program and dataset description
Our analysis focuses on the recommendation referral program run by a large retailer. The program rules were as follows. Each time a person purchases a book, music, or a movie he or she is given the option of sending emails recommending the item to friends. The ?rst person to purchase the same item through a referral link in the email gets a 10% discount. When this happens the sender of the recommendation receives a 10% credit on their purchase. The following information is recorded for each recommendation 1. Sender Customer ID (shadowed) 2. Receiver Customer ID (shadowed) 3. Date of Sending 4. Purchase ?ag (buy-bit ) 5. Purchase Date (error-prone due to asynchrony in the servers) 6. Product identi?er 7. Price
The Dynamics of Viral Marketing
7
The recommendation dataset consists of 15,646,121 recommendations made among 3,943,084 distinct users. The data was collected from June 5 2001 to May 16 2003. In total, 548,523 products were recommended, 99% of them belonging to 4 main product groups: Books, DVDs, Music and Videos. In addition to recommendation data, we also crawled the retailer’s website to obtain product categories, reviews and ratings for all products. Of the products in our data set, 5813 (1%) were discontinued (the retailer no longer provided any information about them). Although the data gives us a detailed and accurate view of recommendation dynamics, it does have its limitations. The only indication of the success of a recommendation is the observation of the recipient purchasing the product through the same vendor. We have no way of knowing if the person had decided instead to purchase elsewhere, borrow, or otherwise obtain the product. The delivery of the recommendation is also somewhat di?erent from one person simply telling another about a product they enjoy, possibly in the context of a broader discussion of similar products. The recommendation is received as a form email including information about the discount program. Someone reading the email might consider it spam, or at least deem it less important than a recommendation given in the context of a conversation. The recipient may also doubt whether the friend is recommending the product because they think the recipient might enjoy it, or are simply trying to get a discount for themselves. Finally, because the recommendation takes place before the recommender receives the product, it might not be based on a direct observation of the product. Nevertheless, we believe that these recommendation networks are re?ective of the nature of word of mouth advertising, and give us key insights into the in?uence of social networks on purchasing decisions.
3.2
Identifying successful recommendations
For each recommendation, the dataset includes information about the recommended product, sender and received or the recommendation, and most importantly, the success of recommendation. See section 3.1 for more details. We represent this data set as a directed multi graph. The nodes represent customers, and a directed edge contains all the information about the recommendation. The edge (i, j, p, t) indicates that i recommended product p to customer j at time t. Note that as there can be multiple recommendations of between the persons (even on the same product) there can be multiple edges between two nodes. The typical process generating edges in the recommendation network is as follows: a node i ?rst buys a product p at time t and then it recommends it to nodes j1 , . . . , jn . The j nodes can then buy the product and further recommend it. The only way for a node to recommend a product is to ?rst buy it. Note that even if all nodes j buy a product, only the edge to the node jk that ?rst made the purchase (within a week after the recommendation) will be marked by a buy-bit. Because the buy-bit is set only for the ?rst person who acts on a recommendation, we identify additional purchases by the presence of outgoing recommendations for a person, since all recommendations must be preceded by a purchase. We call this type of evidence of purchase a buy-edge. Note that buy-edges provide only a lower bound on the total number of purchases without discounts. It is possible for a customer to not be the ?rst to act on a recommendation and also to not recommend the product to others. Unfortunately, this
8 Group Book DVD Music Video Full network p 103,161 19,829 393,598 26,131 542,719 n 2,863,977 805,285 794,148 239,583 3,943,084 r 5,741,611 8,180,393 1,443,847 280,270 15,646,121
J. Leskovec et al.
e 2,097,809 962,341 585,738 160,683 3,153,676
bb 65,344 17,232 7,837 909 91,322
be 17,769 58,189 2,739 467 79,164
Table 1: Product group recommendation statistics. p: number of products, n: number of nodes, r : number of recommendations, e: number of edges, bb : number of buy bits, be : number of buy edges.
was not recorded in the data set. We consider, however, the buy-bits and buy-edges as proxies for the total number of purchases through recommendations. As mentioned above the ?rst buyer only gets a discount (the buy-bit is turned on) if the purchase is made within one week of the recommendation. In order to account for as many purchases as possible, we consider all purchases where the recommendation preceded the purchase (buy-edge) regardless of the time di?erence between the two events. To avoid confusion we will refer to edges in a multi graph as recommendations (or multi-edges) — there can be more than one recommendation between a pair of nodes. We will use the term edge (or unique edge) to refer to edges in the usual sense, i.e. there is only one edge between a pair of people. And, to get from recommendations to edges we create an edge between a pair of people if they exchanged at least one recommendation.
4
The recommendation network
For each product group we took recommendations on all products from the group and created a network. Table 1 shows the sizes of various product group recommendation networks with p being the total number of products in the product group, n the total number of nodes spanned by the group recommendation network, and r the number of recommendations (there can be multiple recommendations between two nodes). Column e shows the number of (unique) edges – disregarding multiple recommendations between the same source and recipient (i.e., number of pairs of people that exchanged at least one recommendation). In terms of the number of di?erent items, there are by far the most music CDs, followed by books and videos. There is a surprisingly small number of DVD titles. On the other hand, DVDs account for more half of all recommendations in the dataset. The DVD network is also the most dense, having about 10 recommendations per node, while books and music have about 2 recommendations per node and videos have only a bit more than 1 recommendation per node. Music recommendations reached about the same number of people as DVDs but used more than 5 times fewer recommendations to achieve the same coverage of the nodes. Book recommendations reached by far the most people – 2.8 million. Notice that all networks have a very small number of unique edges. For books, videos and music the number of unique edges is smaller than the number of nodes – this suggests
The Dynamics of Viral Marketing
9 ec 184,188 442,747 82,844 15,331 521,803 bbc 1,919 6,199 348 2 8,468 bec 1,921 41,744 456 74 44,195
Group Book DVD Music Video Full network
nc 53,681 39,699 22,044 4,964 100,460
rc 933,988 6,903,087 295,543 23,555 8,283,753
Table 2: Statistics for the largest connected component of each product group. nc : number of nodes in largest connected component, rc : number recommendations in the component, ec : number of edges in the component, bbc : number of buy bits, bec : number of buy edges in the largest connected component, and bbc and bec are the number of purchase through a buy-bit and a buy-edge, respectively.
that the networks are highly disconnected [ER60]. Back to table 1: given the total number of recommendations r and purchases (bb + be ) in?uenced by recommendations we can estimate how many recommendations need to be independently sent over the network to induce a new purchase. Using this metric books have the most in?uential recommendations followed by DVDs and music. For books one out of 69 recommendations resulted in a purchase. For DVDs it increases to 108 recommendations per purchase and further increases to 136 for music and 203 for video. Table 2 gives more insight into the structure of the largest connected component of each product group’s recommendation network. We performed the same measurements as in table 1 with the di?erence being that we did not use the whole network but only its largest weakly connected component. The table shows the number of nodes n, the number of recommendations rc , and the number of (unique) edges ec in the largest component. The last two columns (bbc and bec ) show the number of purchases resulting in a discount (buy-bit, bbc ) and the number of purchases through buy-edges (bec ) in the largest connected component. First, notice that the largest connected components are very small. DVDs have the largest - containing 4.9% of the nodes, books have the smallest at 1.78%. One would also expect that the fraction of the recommendations in the largest component would be proportional to its size. We notice that this is not the case. For example, the largest component in the full recommendation network contains 2.54% of the nodes and 52.9% of all recommendations, which is the result of heavy bias in DVD recommendations. Breaking this down by product categories we see that for DVDs 84.3% of the recommendations are in largest component (which contains 4.9% of all DVD nodes), vs. 16.3% for book recommendations (component size 1.79%), 20.5% for music recommendations (component size 2.77%), and 8.4% for video recommendations (component size 2.1%). This shows that the dynamic in the largest component is very much di?erent from the rest of the network. Especially for DVDs we can see that a very small fraction of users generated most of the recommendations.
4.1
Recommendation network over time
The recommendations that occurred were exchanged over an existing underlying social network. In the real world, it is estimated that any two people on the globe
10
J. Leskovec et al.
12 size of giant component 10 8 6 4 2
x 10
4
4
x 10
6
2 # nodes 0 0 1.7*106m 10 20 m (month) by month quadratic fit 1 2 number of nodes 3 x 10 4
6
0 0
Figure 1: (a) The size of the largest connected component of customers over time. The inset shows the linear growth in the number of customers n over time.
are connected via a short chain of acquaintances - popularly known as the small world phenomenon [TM69]. We examined whether the edges formed by aggregating recommendations over all products would similarly yield a small world network, even though they represent only a small fraction of a person’s complete social network. We measured the growth of the largest weakly connected component over time, shown in Figure 1. Within the weakly connected component, any node can be reached from any other node by traversing (undirected) edges. For example, if u recommended product x to v , and w recommended product y to v , then uand w are linked through one intermediary and thus belong to the same weakly connected component. Note that connected components do not necessarily correspond to communities (clusters) which we often think of as densely linked parts of the networks. Nodes belong to same component if they can reach each other via an undirected path regardless of how densely they are linked. Figure 1 shows the size of the largest connected component, as a fraction of the total network. The largest component is very small over all time. Even though we compose the network using all the recommendations in the dataset, the largest connected component contains less than 2.5% (100,420) of the nodes, and the second largest component has only 600 nodes. Still, some smaller communities, numbering in the tens of thousands of purchasers of DVDs in categories such as westerns, classics and Japanese animated ?lms (anime), had connected components spanning about 20% of their members. The insert in ?gure 1 shows the growth of the customer base over time. Surprisingly it was linear, adding on average 165,000 new users each month, which is an indication that the service itself was not spreading epidemically. Further evidence of non-viral spread is provided by the relatively high percentage (94%) of users who made their ?rst recommendation without having previously received one.
n
The Dynamics of Viral Marketing
11
10 N(x = s ) (Count) 10 10 10
4
= 8.3e3 x
?1.90
R =0.93 N(x = s ) (Count)
2
10 10 10 10
4
= 6.6e3 x
?1.96
R =0.93 N(x = s ) (Count)
2
10
3
= 2.0e3 x
?1.76
R =0.90
2
3
3
10
2
c
c
2
2
c
1
1
10
1
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
(a) LCC growth
(b) Sender in LCC
(c) Sender outside LCC
Figure 2: Growth of the largest connected component (LCC). (a) the distribution of sizes of components when they are merged into the largest connected component. (b) same as (a), but restricted to cases when a member of the LCC sends a recommendation to someone outside the largest component. (c) a sender outside the largest component sends a recommendation to a member of the component.
4.1.1
Growth of the largest connected component
Next, we examine the growth of the largest connected component (LCC). In ?gure 1 we saw that the largest component seems to grow quadratically over time, but at the end of the data collection period is still very small, i.e. only 2.5% of the nodes belong to largest weakly connected component. Here we are not interested in how fast the largest component grows over time but rather how big other components are when they get merged into the largest component. Also, since our graph is directed we are interested in determining whether smaller components become attached to the largest component by a recommendation sent from inside of the largest component. One can think of these recommendations as being tentacles reaching out of largest component to attach smaller components. The other possibility is that the recommendation comes from a node outside the component to a member of the largest component and thus the initiative to attach comes from outside the largest component. We look at whether the largest component grows gradually, adding nodes one by one as the members send out more recommendations, or whether a new recommendation might act as a bridge to a component consisting of several nodes who are already linked by their previous recommendations. To this end we measure the distribution of a component’s size when it gets merged to the largest weakly connected component. We operate under the following setting. Recommendations are arriving over time one by one creating edges between the nodes of the network. As more edges are being added the size of largest connected component grows. We keep track of the currently largest component, and measure how big the separate components are when they get attached to the largest component. Figure 2(a) shows the distribution of merged connected component (CC) sizes. On the x-axis we plot the component size (number of nodes N ) and on the y-axis the number of components of size N that were merged over time with the largest component. We see that a majority of the time a single node (component of size 1) merged with the currently largest component. On the other extreme is the case when a component of 1, 568 nodes merged with the largest component. Interestingly, out of all merged components, in 77% of the cases the source of the
12
J. Leskovec et al.
recommendation comes from inside the largest component, while in the remaining 23% of the cases it is the smaller component that attaches itself to the largest one. Figure 2(b) shows the distribution of component sizes only for the case when the sender of the recommendation was a member of the largest component, i.e. the small component was attached from the largest component. Lastly, Figure 2(c) shows the distribution for the opposite case when the sender of the recommendation was not a member of the largest component, i.e. the small component attached itself to the largest. Also notice that in all cases the distribution of merged component sizes follows a heavy-tailed distribution. We ?t a power-law distribution and note the power-law exponent of 1.90 (?g. 2(a)) when considering all merged components. Limiting the analysis to the cases where the source of the edge that attached a small component to the largest is in the largest component we obtain power-law exponent of 1.96 (?g. 2(b)), and when the edge originated from the small component to attached it to the largest, the power-law exponent is 1.76. This shows that even though in most cases the LCC absorbs the small component, we see that components that attach themselves to the LCC tend to be larger (smaller power-law exponent) than those attracted by the LCC. This means that the component sometimes grows a bit before it attaches itself to the largest component. Intuitively, an individual node can get attached to the largest component simply by passively receiving a recommendation. But if it is the outside node that sends a recommendation to someone in the giant component, it is already an active recommender and could therefore have recommended to several others previously, thus forming a slightly bigger component that is then merged. From these experiments we see that the largest component is very active, adding smaller components by generating new recommendations. Most of the time these newly merged components are quite small, but occasionally sizable components are attached.
4.2
Preliminary observations and discussion
Even with these simple counts and experiments we can already make a few observations. It seems that some people got quite heavily involved in the recommendation program, and that they tended to recommend a large number of products to the same set of friends (since the number of unique edges is so small as shown on table 1). This means that people tend to buy more DVDs and also like to recommend them to their friends, while they seem to be more conservative with books. One possible reason is that a book is a bigger time investment than a DVD: one usually needs several days to read a book, while a DVD can be viewed in a single evening. Another factor may be how informed the customer is about the product. DVDs, while fewer in number, are more heavily advertised on TV, billboards, and movie theater previews. Furthermore, it is possible that a customer has already watched a movie and is adding the DVD to their collection. This could make them more con?dent in sending recommendations before viewing the purchased DVD. One external factor which may be a?ecting the recommendation patterns for DVDs is the existence of referral websites (www.dvdtalk.com). On these websites people, who want to buy a DVD and get a discount, would ask for recommendations. This way there would be recommendations made between people who don’t really know
The Dynamics of Viral Marketing
13
938
973
(a) Medical book
(b) Japanese graphic novel
Figure 3: Examples of two product recommendation networks: (a) First aid study guide First Aid for the USMLE Step, (b) Japanese graphic novel (manga) Oh My Goddess!: Mara Strikes Back.
Group Book DVD Music Video Total
Number of nodes Purchases Forward Percent 65,391 15,769 24.2 16,459 7,336 44.6 7,843 1,824 23.3 909 250 27.6 90,602 25,179 27.8
Table 3: Fraction of people that purchase and also recommend forward. Purchases: number of nodes that purchased as a result of receiving a recommendation. Forward: nodes that purchased and then also recommended the product to others.
each other but rather have an economic incentive to cooperate. In e?ect, the viral marketing program is altering, albeit brie?y and most likely unintentionally, the structure of the social network it is spreading on. We were not able to ?nd similar referral sharing sites for books or CDs.
5
5.1
Propagation of recommendations
Forward recommendations
Not all people who accept a recommendation by making a purchase also decide to give recommendations. In estimating what fraction of people that purchase also decide to recommend forward, we can only use the nodes with purchases that resulted in a discount. Table 3 shows that only about a third of the people that purchase also recommend the product forward. The ratio of forward recommendations is much higher for DVDs than for other kinds of products. Videos also have a higher ratio of forward recommendations, while books have the lowest. This shows that people are most keen on recommending movies, possibly for the above mentioned reasons, while more conservative when recommending books and music. Figure 4 shows the cumulative out-degree distribution, that is the number of
14
J. Leskovec et al.
10 10 N(x >= k )
p
6
5
10 10 10
4
3
level 0 ? = 2.6 level 1 ? = 2.0 level 2 ? = 1.5 level 3 ? = 1.2 level 4 ? = 1.2
2
10 0 10
1
10 10 k (recommendations by a person for a product)
p
1
2
10
3
Figure 4: The number of recommendations sent by a user with each curve representing a di?erent depth of the user in the recommendation chain. A power law exponent ? is ?tted to all but the tail, which shows an exponential drop-o? at around 100 recommendations sent). This drop-o? is consistent across all depth levels, and may re?ect either a natural disinclination to send recommendation to over a hundred people, or a technical issue that might have made it more inconvenient to do so. The ?tted lines follow the order of the level number (i.e. top line corresponds to level 0 and bottom to level 4).
level 0 1 2 3 4
prob. buy & forward N/A 0.0069 0.0149 0.0115 0.0082
average out-degree 1.99 5.34 24.43 72.79 111.75
Table 4: Statistics about individuals at di?erent levels of the cascade.
people who sent out at least kp recommendations, for a product. We ?t a power-law to all but the tail of the distribution. Also, notice the exponential decay in the tail of the distribution which could be, among other reasons, attributed to the ?nite time horizon of our dataset. The ?gure 4 shows that the deeper an individual is in the cascade, if they choose to make recommendations, they tend to recommend to a greater number of people on average (the ?tted line has smaller slope ? , i.e. the distribution has higher variance). This e?ect is probably due to only very heavily recommended products producing large enough cascades to reach a certain depth. We also observe, as is shown in Table 4, that the probability of an individual making a recommendation at all (which can only occur if they make a purchase), declines after an initial increase as one gets deeper into the cascade.
The Dynamics of Viral Marketing
15
10
8
10 = 3.4e6 x
?2.30
8
R =0.96 10 Count
6
2
= 4.1e6 x
?2.49
R =0.99
2
10 Count
6
10
4
10
4
10
2
10
2
10 0 10 Number of recommendations
0
10
5
10 0 10
0
10 10 10 Number of purchases
1
2
3
10
4
(a) Recommendations
(b) Purchases
Figure 5: Distribution of the number of recommendations and number of purchases made by a customer.
5.2
Identifying cascades
As customers continue forwarding recommendations, they contribute to the formation of cascades. In order to identify cascades, i.e. the “causal” propagation of recommendations, we track successful recommendations as they in?uence purchases and further recommendations. We de?ne a recommendation to be successful if it reached a node before its ?rst purchase. We consider only the ?rst purchase of an item, because there are many cases when a person made multiple purchases of the same product, and in between those purchases she may have received new recommendations. In this case one cannot conclude that recommendations following the ?rst purchase in?uenced the later purchases. Each cascade is a network consisting of customers (nodes) who purchased the same product as a result of each other’s recommendations (edges). We delete late recommendations — all incoming recommendations that happened after the ?rst purchase of the product. This way we make the network time increasing or causal — for each node all incoming edges (recommendations) occurred before all outgoing edges. Now each connected component represents a time obeying propagation of recommendations. Figure 3 shows two typical product recommendation networks: (a) a medical study guide and (b) a Japanese graphic novel. Throughout the dataset we observe very similar patters. Most product recommendation networks consist of a large number of small disconnected components where we do not observe cascades. Then there is usually a small number of relatively small components with recommendations successfully propagating. This observation is re?ected in the heavy tailed distribution of cascade sizes (see ?gure 6), having a power-law exponent close to 1 for DVDs in particular. We determined the power-law exponent by ?tting a line on log-log scales using the least squares method. We also notice bursts of recommendations (?gure 3(b)). Some nodes recommend to many friends, forming a star like pattern. Figure 5 shows the distribution of
16
J. Leskovec et al.
10
6
= 1.8e6 x?4.98 R2=0.99 10
4
= 3.4e3 x?1.56 R2=0.83
10
4
10
2
10
2
10 0 10
0
10 (a) Book
1
10
2
10 0 10
0
10 10 (b) DVD
1
2
10
3
= 4.9e5 x?6.27 R2=0.97 10
4
= 7.8e4 x?5.87 R2=0.97 10
4
10
2
10
2
10 0 10
0
10 (c) Music
1
10
2
10 0 10
0
10 (d) Video
1
10
2
Figure 6: Size distribution of cascades (size of cascade vs. count). Bold line presents a power-?t.
the recommendations and purchases made by a single node in the recommendation network. Notice the power-law distributions and long ?at tails. The most active customer made 83,729 recommendations and purchased 4,416 di?erent items. Finally, we also sometimes observe ‘collisions’, where nodes receive recommendations from two or more sources. A detailed enumeration and analysis of observed topological cascade patterns for this dataset is made in [LSK06]. Last, we examine the number of exchanged recommendations between a pair of people in ?gure 7. Overall, 39% of pairs of people exchanged just a single recommendation. This number decreases for DVDs to 37%, and increases for books to 45%. The distribution of the number of exchanged recommendations follows a heavy tailed distribution. To get a better understanding of the distributions we show the power-law decay lines. Notice that one gets much stronger decay exponent (distribution has weaker tail) of -2.7 for books and a very shallow power-law exponent of -1.5 for DVDs. This means that even a pair of people exchanges more DVD than book recommendations.
The Dynamics of Viral Marketing
17
10 10 N(x=r ) (Count) 10 10 10 10
6
? = ?2.0 N(x=r ) (Count)
5
10 10 10 10 10
5
? = ?2.7 N(x=r ) (Count)
10 10 10 10 10
5
? = ?1.5
4
4
4
3
3
3
e
e
e
2 1
2
2
1
1
10 0 1 2 3 4 10 10 10 10 10 r (Number of exchanged recommendations)
e
0
10 0 1 2 3 10 10 10 10 r (Number of exchanged recommendations)
e
0
10 0 1 2 3 10 10 10 10 r (Number of exchanged recommendations)
e
0
(a) All
(b) Books
(c) DVD
Figure 7: Distribution of the number of exchanged recommendations between pairs of people.
5.3
The recommendation propagation model
A simple model can help explain how the wide variance we observe in the number of recommendations made by individuals can lead to power-laws in cascade sizes (?gure 6). The model assumes that each recipient of a recommendation will forward it to others if its value exceeds an arbitrary threshold that the individual sets for herself. Since exceeding this value is a probabilistic event, let’s call pt the probability that at time step t the recommendation exceeds the threshold. In that case the number of recommendations Nt+1 at time (t + 1) is given in terms of the number of recommendations at an earlier time by Nt+1 = pt Nt (1)
where the probability pt is de?ned over the unit interval. Notice that, because of the probabilistic nature of the threshold being exceeded, one can only compute the ?nal distribution of recommendation chain lengths, which we now proceed to do. Subtracting from both sides of this equation the term Nt and diving by it we obtain N(t+1) ? Nt = pt ? 1 Nt (2)
Summing both sides from the initial time to some very large time T and assuming that for long times the numerator is smaller than the denominator (a reasonable assumption) we get, up to a unit constant dN = N pt (3)
The left hand integral is just log(N ), and the right hand side is a sum of random variables, which in the limit of a very large uncorrelated number of recommendations is normally distributed (central limit theorem). This means that the logarithm of the number of messages is normally distributed, or equivalently, that the number of messages passed is log-normally distributed. In other words the probability density for N is given by
18
J. Leskovec et al.
P (N ) =
1 ?(log(N ) ? µ)2 ? exp 2? 2 N 2?? 2
(4)
which, for large variances describes a behavior whereby the typical number of recommendations is small (the mode of the distribution) but there are unlikely events of large chains of recommendations which are also observable. Furthermore, for large variances, the lognormal distribution can behave like a power law for a range of values. In order to see this, take the logarithms on both sides of the equation (equivalent to a log-log plot) and one obtains ? (log (N ) ? µ)2 (5) log(P (N )) = ? log(N ) ? log( 2?? 2 ) ? 2? 2 So, for large ? , the last term of the right hand side goes to zero, and since the second term is a constant one obtains a power law behavior with exponent value of minus one. There are other models which produce power-law distributions of cascade sizes, but we present ours for its simplicity, since it does not depend on network topology [GGLNT04] or critical thresholds in the probability of a recommendation being accepted [Wat02].
6
Success of Recommendations
So far we only looked into the aggregate statistics of the recommendation network. Next, we ask questions about the e?ectiveness of recommendations in the recommendation network itself. First, we analyze the probability of purchasing as one gets more and more recommendations. Next, we measure recommendation e?ectiveness as two people exchange more and more recommendations. Lastly, we observe the recommendation network from the perspective of the sender of the recommendation. Does a node that makes more recommendations also in?uence more purchases?
6.1
Probability of buying versus number of incoming recommendations
First, we examine how the probability of purchasing changes as one gets more and more recommendations. One would expect that a person is more likely to buy a product if she gets more recommendations. On the other had one would also think that there is a saturation point – if a person hasn’t bought a product after a number of recommendations, they are not likely to change their minds after receiving even more of them. So, how many recommendations are too many? Figure 8 shows the probability of purchasing a product as a function of the number of incoming recommendations on the product. Because we exclude late recommendations, those that were received after the purchase, an individual counts as having received three recommendations only if they did not make a purchase after the ?rst two, and either purchased or did not receive further recommendations after receiving the third one. As we move to higher numbers of incoming recommendations, the number of observations drops rapidly. For example, there were 5 million cases with 1 incoming recommendation on a book, and only 58 cases where a person got 20
The Dynamics of Viral Marketing
19
0.06 0.05 Probability of Buying Probability of Buying 2 4 6 8 Incoming Recommendations 10
0.08
0.06
0.04 0.03 0.02 0.01 0
0.04
0.02
0
10
20 30 40 50 Incoming Recommendations
60
(a) Books
0.2
0.2
(b) DVD
Probability of Buying
0.15
Probability of Buying
0.15
0.1
0.1
0.05
0.05
0 1
2
3 4 5 6 7 Incoming Recommendations
8
0
2
4 6 8 10 12 14 Incoming Recommendations
16
(c) Music
(d) Video
Figure 8: Probability of buying a book (DVD) given a number of incoming recommendations.
incoming recommendations on a particular book. The maximum was 30 incoming recommendations. For these reasons we cut-o? the plot when the number of observations becomes too small and the error bars too large. We calculate the purchase probabilities and the standard errors of the estimates which we use to plot the error bars in the following way. We regard each point as a binomial random variable. Given the number of observations n, let m be the number of successes, and k (k=n-m) the number of failures. In our case, m is the number of people that ?rst purchased a product after receiving r recommendations on it, and k is the number of people that received the total of r recommendations on a product (till the end of the dataset) but did purchase it, then the estimated probability of p(1 ? p)/n. purchasing is p ˆ = m/n and the standard error sp ˆ is sp ˆ of estimate p ˆ= Figure 8(a) shows that, overall, book recommendations are rarely followed. Even more surprisingly, as more and more recommendations are received, their success decreases. We observe a peak in probability of buying at 2 incoming recommendations and then a slow drop. This implies that if a person doesn’t buy a book after the ?rst recommendation, but receives another, they are more likely to be persuaded by the second recommendation. But thereafter, they are less likely to respond to additional
20
J. Leskovec et al.
recommendations, possibly because they perceive them as spam, are less susceptible to others’ opinions, have a strong opinion on the particular product, or have a di?erent means of accessing it. For DVDs (?gure 8(b)) we observe a saturation around 10 incoming recommendations. This means that with each additional recommendation, a person is more and more likely to be persuaded - up to a point. After a person gets 10 recommendations on a particular DVD, their probability of buying does not increase anymore. The number of observations is 2.5 million at 1 incoming recommendation and 100 at 60 incoming recommendations. The maximal number of received recommendations is 172 (and that person did not buy), but someone purchased a DVD after 169 receiving recommendations. The di?erent patterns between book and DVD recommendations may be a result of the recommendation exchange websites for DVDs. Someone receiving many DVD recommendations may have signed up to receive them for a product they intended to purchase, and hence a greater number of received recommendations corresponds to a higher likelihood of purchase (up to a point).
6.2
Success of subsequent recommendations
Next, we analyze how the e?ectiveness of recommendations changes as one received more and more recommendations from the same person. A large number of exchanged recommendations can be a sign of trust and in?uence, but a sender of too many recommendations can be perceived as a spammer. A person who recommends only a few products will have her friends’ attention, but one who ?oods her friends with all sorts of recommendations will start to loose her in?uence. We measure the e?ectiveness of recommendations as a function of the total number of previously received recommendations from a particular node. We thus measure how spending changes over time, where time is measured in the number of received recommendations. We construct the experiment in the following way. For every recommendation r on some product p between nodes u and v , we ?rst determine how many recommendations node u received from v before getting r. Then we check whether v , the recipient of recommendation, purchased p after the recommendation r arrived. If so, we count the recommendation as successful since it in?uenced the purchase. This way we can calculate the recommendation success rate as more recommendations were exchanged. For the experiment we consider only node pairs (u, v ), where there were at least a total of 10 recommendations sent from u to v . We perform the experiment using only recommendations from the same product group. We decided to set a lower limit on the number of exchanged recommendations so that we can measure how the e?ectiveness of recommendations changes as the same two people exchange more and more recommendations. Considering all pairs of people would heavily bias our ?ndings since most pairs exchange just a few or even just a single recommendation. Using the data from ?gure 7 we see that 91% of pairs of people that exchange at least 1 recommendation exchange less than 10. For books this number increases to 96%, and for DVDs it is even smaller (81%). In the DVD network there are 182 thousand pairs that exchanged more than 10 recommendations, and 70 thousand for the book network. Figure 9 shows the probability of buying as a function of the total number of
The Dynamics of Viral Marketing
21
12
x 10
?3
0.07 0.06 0.05 0.04 0.03 0.02
Probability of buying
8
6
4
5
10 15 20 25 30 35 Exchanged recommendations
40
Probability of buying
10
5
10 15 20 25 30 35 Exchanged recommendations
40
(a) Books
(b) DVD
Figure 9: The e?ectiveness of recommendations with the number of received recommendations.
received recommendations from a particular person up to that point. One can think of x-axis as measuring time where the unit is the number of received recommendations from a particular person. For books we observe that the e?ectiveness of recommendation remains about constant up to 3 exchanged recommendations. As the number of exchanged recommendations increases, the probability of buying starts to decrease to about half of the original value and then levels o?. For DVDs we observe an immediate and consistent drop. We performed the experiment also for video and music, but the number of observations was too low and the measurements were noisy. This experiment shows that recommendations start to lose e?ect after more than two or three are passed between two people. Also, notice that the e?ectiveness of book recommendations decays much more slowly than that of DVD recommendations, ?attening out at around 20 recommendations, compared to around 10 DVD exchanged recommendations. The result has important implications for viral marketing because providing too much incentive for people to recommend to one another can weaken the very social network links that the marketer is intending to exploit.
6.3
Success of outgoing recommendations
In previous sections we examined the data from the viewpoint of the receiver of the recommendation. Now we look from the viewpoint of the sender. The two interesting questions are: how does the probability of getting a 10% credit change with the number of outgoing recommendations; and given a number of outgoing recommendations, how many purchases will they in?uence? One would expect that recommendations would be the most e?ective when recommended to the right subset of friends. If one is very selective and recommends to too few friends, then the chances of success are slim. One the other hand, recommending to everyone and spamming them with recommendations may have limited returns as well.
22
5
0.5 0.4 0.3 0.2 0.1 0 10 20 30 40 50 60 70 Outgoing Recommendations 80
J. Leskovec et al.
0.2 0.25 0.2 0.15 0.1 0.05
Number of Purchases
Number of Purchases
Number of Purchases
0.15
3 2 1 0
0.1
0.05
20 40 60 80 Outgoing Recommendations
100
0
Number of Purchases 5 10 15 20 Outgoing Recommendations
4
0
2
4 6 8 10 Outgoing Recommendations
12
0.25 0.2 0.15 0.1 0.05 0
0.12 0.1 Probability of Credit 0.08 0.06 0.04 0.02
0.1 0.08 0.06 0.04 0.02 0
0.08
Probability of Credit
Probability of Credit 5 10 15 20 Outgoing Recommendations
Probability of Credit
0.06
0.04
0.02
10
20 30 40 50 60 70 Outgoing Recommendations
80
0
20 40 60 80 Outgoing Recommendations
100
0
2
4 6 8 10 Outgoing Recommendations
12
(a) Books
(b) DVD
(c) Music
(d) Video
Figure 10: Top row: Number of resulting purchases given a number of outgoing recommendations. Bottom row: Probability of getting a credit given a number of outgoing recommendations.
The top row of ?gure 10 shows how the average number of purchases changes with the number of outgoing recommendations. For books, music, and videos the number of purchases soon saturates: it grows fast up to around 10 outgoing recommendations and then the trend either slows or starts to drop. DVDs exhibit di?erent behavior, with the expected number of purchases increasing throughout. These results are even more interesting since the receiver of the recommendation does not know how many other people also received the recommendation. Thus the plots of ?gure 10 show that there are interesting dependencies between the product characteristics and the recommender that manifest through the number of recommendations sent. It could be the case that widely recommended products are not suitable for viral marketing (we ?nd something similar in section 9.2), or that the recommender did not put too much thought into who to send the recommendation to, or simply that people soon start to ignore mass recommenders. Plotting the probability of getting a 10% credit as a function of the number of outgoing recommendations, as in the bottom row of ?gure 10, we see that the success of DVD recommendations saturates as well, while books, videos and music have qualitatively similar trends. The di?erence in the curves for DVD recommendations points to the presence of collisions in the dense DVD network, which has 10 recommendations per node and around 400 per product — an order of magnitude more than other product groups. This means that many di?erent individuals are recommending to the same person, and after that person makes a purchase, even though all of them made a ‘successful recommendation’ by our de?nition, only one of them receives a credit.
6.4
Probability of buying given the total number of incoming recommendations
The collisions of recommendations are a dominant feature of the DVD recommendation network. Book recommendations have the highest chance of getting a credit, but DVD recommendations cause the most purchases. So far it seems people are
The Dynamics of Viral Marketing
23
0.1 0.08 0.06 0.04 0.02 0
Probability of Buying
Probability of Buying
0.1
0.05
2
4 6 8 10 12 Total Incomming Products
14
0
5 10 15 Total Incomming Products
20
(a) Books
0.07 0.06 Probability of Buying Probability of Buying 0.05 0.04 0.03 0.02 0.01 0 5 10 15 Total Incomming Products 20 0 0.03 0.04
(b) DVD
0.02
0.01
5 10 15 Total Incomming Products
20
(c) Music
(d) Video
Figure 11: The probability of buying a product given a number of di?erent products a node got recommendations on.
very keen on recommending various DVDs, while very conservative on recommending books. But how does the behavior of customers change as they get more involved into the recommendation network? We would expect that most of the people are not heavily involved, so their probability of buying is not high. In the extreme case we expect to ?nd people who buy almost everything they get recommendations on. There are two ways to measure the involvedness of a person in the network: by the total number of incoming recommendations (on all products) or the total number of di?erent products they were recommended. For every purchase of a book at time t, we count the number of di?erent books (DVDs, ...) the person received recommendations for before time t. As in all previous experiments we delete late recommendations, i.e. recommendations that arrived after the ?rst purchase of a product. We show the probability of buying as a function of the number of di?erent products recommended in Figure 11. Figure A-2 plots the same data but with the total number of incoming recommendations on the x-axis. We calculate the error bars as described in section 6.1. The number of observations is large enough (error bars are su?ciently small) to draw conclusions about the trends observed in the ?gures. For example, there are more than 15, 000 observations (users) that had 15 incoming DVD recommendations.
24
0.2
J. Leskovec et al.
0.08 Probability of Buying
Probability of Buying 0.15
0.06
0.04
0.1
0.02
0.05
0
10 20 30 40 50 Total Incomming Recommendations
0
5 10 15 20 25 30 35 Total Incomming Recommendations
40
(a) Books
0.08 0.04
(b) DVD
Probability of Buying
0.06
Probability of Buying 5 10 15 20 25 30 35 Total Incomming Recommendations 40
0.03
0.04
0.02
0.02
0.01
0
0
5 10 15 Total Incomming Recommendations
20
(c) Music
(d) Video
Figure 12: Probability of buying a product given a total number of incoming recommendations on all products.
Notice that trends are quite similar regardless whether we measure how involved is the user in the network by counting the number of products recommended (?gure 11) or the number of incoming recommendations (?g. A-2). We observe two distinct trends. For books and music (?gures 11 and A-2, (a) and (c)) the probability of buying is the highest when a person got recommendations on just 1 item, as the number of incoming recommended products increases to 2 or more the probability of buying quickly decreases and then ?attens. Movies (DVDs and videos) exhibit di?erent behavior (?gure 11 and A-2, (b) and (d)). A person is more likely to buy the more recommendations she gets. For DVDs the peak is at around 15 incoming products, while for videos there is no such peak – the probability remains fairly level. Interestingly for DVDs the distribution reaches its low at 2 and 3 items, while for videos it lies somewhere between 3 and 8 items. The results suggest that books and music buyers tend to be conservative and focused. On the other hand there are people who like to buy movies in general. One could hypothesize that buying a book is a larger investment of time and e?ort than buying a movie. One can ?nish a movie in an evening, while reading a book requires more e?ort. There are also many more book and music titles than movie titles. The other di?erence between the book and music recommendations in compar-
The Dynamics of Viral Marketing
25
ison to movies are the recommendation referral websites where people could go to get recommendations. One could see these websites as recommendation subscription services – posting one’s email on a list results in a higher number of incoming recommendations. For movies, people with a high number of incoming recommendations “subscribed” to them and thus expected/wanted the recommendations. On the other hand people with high numbers of incoming book or music recommendations did not “sign up” for them, so they may perceive recommendations as spam and thus the in?uence of recommendations drops. Another evidence of the existence of recommendations referral websites includes the DVD recommendation network degree distribution. The DVDs follow a power law degree distribution with an exception of a peak at out-degree 50. Other plots of DVD recommendation behavior also exhibited abnormalities at around 50 recommendations. We believe these can be attributed to the recommendation referral websites.
7
Timing of recommendations and purchases
The recommendation referral program encourages people to purchase as soon as possible after they get a recommendation, since this maximizes the probability of getting a discount. We study the time lag between the recommendation and the purchase of di?erent product groups, e?ectively how long it takes a person to receive a recommendation, consider it, and act on it. We present the histograms of the “thinking time”, i.e. the di?erence between the time of purchase and the time the last recommendation was received for the product prior to the purchase (?gure 13). We use a bin size of 1 day. Around 35%-40% of book and DVD purchases occurred within a day after the last recommendation was received. For DVDs 16% purchases occur more than a week after the last recommendation, while this drops to 10% for books. In contrast, if we consider the lag between the purchase and the ?rst recommendation, only 23% of DVD purchases are made within a day, while the proportion stays the same for books. This re?ects a greater likelihood for a person to receive multiple recommendations for a DVD than for a book. At the same time, DVD recommenders tend to send out many more recommendations, only one of which can result in a discount. Individuals then often miss their chance of a discount, which is re?ected in the high ratio (78%) of recommended DVD purchases that did not a get discount (see table 1, columns bb and be ). In contrast, for books, only 21% of purchases through recommendations did not receive a discount. We also measure the variation in intensity by time of day for three di?erent activities in the recommendation system: recommendations (?gure 14(a)), all purchases (?gure 14(b)), and ?nally just the purchases which resulted in a discount (?gure 14(c)). Each is given as a total count by hour of day. The recommendations and purchases follow the same pattern. The only small di?erence is that purchases reach a sharper peak in the afternoon (after 3pm Paci?c Time, 6pm Eastern time). This means that the willingness to recommend does not change with time, since about a constant fraction of purchases also result in recommendations sent (plots 14(a) and (b) follow the same shape). The purchases that resulted in a discount (?g. 14(c)) look like a negative image of the ?rst two ?gures. If recommendations would have no e?ect then plot (c) should follow the same shape as (a) and (b), since a fraction of people that buy would
26
J. Leskovec et al.
0.35
0.5 Proportion of Purchases 0.4 0.3 0.2 0.1 0
Proportion of Purchases
0.3 0.25 0.2 0.15 0.1 0.05 0 1 2 3 4 5 6 Lag [day] 7 >7
1
2
3
4 5 6 Lag [day]
7
>7
(a) Books
(b) DVD
Figure 13: The time between the recommendation and the actual purchase. We use all purchases.
10 8 Recommendtions
x 10
5
2
x 10
4
7000 6000 Discounted Purchases
1.5
6 4 2 0
All Purchases
5000 4000 3000 2000 1000
1
0.5
0
5
10 15 Hour of the Day
20
25
0
0
5
10 15 Hour of the Day
20
25
0
0
5
10 15 Hour of the Day
20
25
(a) Recommendations
(b) Purchases
(c) Purchases with Discount
Figure 14: Time of day for purchases and recommendations. (a) shows the distribution of recommendations over the day, (b) shows all purchases and (c) shows only purchases that resulted in a discount.
become ?rst buyers, i.e. the more recommendations sent, the more ?rst buyers and thus discounts. However, this does not seem to be the case. The number of purchases with discount is the high when the number of purchases is small. This means that most of discounted purchases happened in the morning when the tra?c (number of purchases/recommendations) on the retailer’s website was low. This makes sense since most of the recommendations happened during the day, and if the person wanted to get the discount by being the ?rst one to purchase, she had the highest chances when the tra?c on the website was the lowest. There are also other factors that come into play here. Assuming that recommendations are sent to people’s personal (non-work) email addresses, then people probably check these email accounts for new email less regularly while at work. So checking personal email while at work and reacting to a recommendation would mean higher chances of getting a discount. Second, there are also network e?ects, i.e. the more recommendations sent, the higher chance of recommendation collision, the lower
The Dynamics of Viral Marketing
27
chance of getting discount, since one competes with the larger set of people.
8
Recommendations and communities of interest
Social networks are a product of the contexts that bring people together. The context can be a shared interest in a particular topic or kind of a book. Sometimes there are circumstances, such as a speci?c job or religious a?liation, that would make people more likely to be interested in the same type of book or DVD. We ?rst apply a community discovery algorithm to automatically detect communities of individuals who exchange recommendations with one another and to identify the kinds of products each community prefers. We then compare the e?ectiveness of recommendations across book categories, showing that books on di?erent subjects have varying success rates.
8.1
Communities and purchases
In aggregating all recommendations between any two individuals in Section 4.1 we showed that the network consists of one large component, containing a little over 100,000 customers, and many smaller components, the largest of which has 634 customers. However, knowing that a hundred thousand customers are linked together in a large network does not reveal whether a product in a particular category is likely to di?use through it. Consider for example a new science ?ction book one would like to market by word-of-mouth. If science ?ction fans are scattered throughout the network, with very few recommendations shared between them, then recommendations about the new book are unlikely to di?use. If on the other hand one ?nds one or more science ?ction communities, where sci-? fans are close together in the network because they exchange recommendations with one another, then the book recommendation has a chance of spreading by word-of-mouth. In the following analysis, we use a community ?nding algorithm [CNM04] in order to discover the types of products that link customers and so de?ne a community. The algorithm breaks up the component into parts, such that the modularity Q, Q = (number of edges within communities) ? (expected number of such edges), (6) is maximized. In other words, the algorithm identi?es communities such that individuals within those communities tend to preferentially exchange recommendations with one another. The results of the community ?nding analysis, while primarily descriptive, illustrate both the presence of communities whose members are linked by their common interests, and the presence cross-cutting interests between communities. Applying the algorithm to the largest component, we identify many small communities and a few larger ones. The largest contains 21,000 nodes, 5,000 of whom are senders of a relatively modest 335,000 recommendations. More interesting than simply observing the size of communities is discovering what interests bring them together. We identify those interests by observing product categories where the number of recommendations within the community is signi?cantly higher than it is for the overall customer population. Let pc be the proportion of all recommendations that fall within a particular
28 # nodes 735 710 667 653 541 502 388 309 192 163 # senders 74 179 181 121 112 104 77 81 30 48
J. Leskovec et al.
topics books: American literature, poetry sci-? books, TV series DVDs, alternative rock music music: dance, indie discounted DVDs books: art & photography, web development, graphical design, sci-? books: sci-? and other books: Christianity and Catholicism books: business and investing, computers, Harry Potter books: parenting, women’s health, pregnancy books: comparative religion, Egypt’s history, new age, role playing games
Table 5: A sample of the medium sized communities present in the largest component
product category c. Then for a set of individuals sending xg recommendations, we would expect by chance that xg ? pc ± xg ? pc ? (1 ? pc ) would fall within category c. We note the product categories for which the observed number of recommendations in the community is many standard deviations higher than expected. For example, compared to the background population, the largest community is focused on a wide variety of books and music. In contrast, the second largest community, involving 10,412 individuals (4,205 of whom are sending over 3 million recommendations), is predominantly focused on DVDs from many di?erent genres, with no particular emphasis on anime. The anime community itself emerges as a highly unusual group of 1,874 users who exchanged over 3 million recommendations. Perhaps the most interesting are the medium sized communities, some of which are listed in Table 5, having between 100 and 1000 members and often re?ecting speci?c interests. Among the hundred or so medium communities, we found, for example, several communities focusing on Christianity. While some of the Christian communities also shared an interest in children’s books, broadway musicals, and travel to Italy, others focused on prayer and bibles, still others also enjoyed DVDs of the Simpsons TV series, and others still took an interest in Catholicism, occult spirituality and kabbalah. Communities were usually centered around a product group, such as books, music, or DVDs, but almost all of them shared recommendations for all types of products. The DVD communities ranged from bargain shoppers purchasing discounted comedy and action DVDs to smaller anime or independent movie communities, to a group of customers purchasing predominantly children’s movies. One community focused heavily on indie music, and imported dance and club music. Another seemed to center around intellectual pursuits, including reading books on sociology, politics, arti?cial intelligence, mathematics, and media culture, listening to classical music and watching neo-noir ?lm. Several communities centered around business and investment books and frequently also recommended books on computing. One business and investment community included fans of the Harry Potter ?ction series, while another enjoyed science ?ction and adventure DVDs. One of communities with the most particular interests recommended not only business and investing books to one another, but also an unusual number of books on terrorism, bacteriology, and military history. A community of what one can presume are web designers recommended books to one
The Dynamics of Viral Marketing
29
another on art and photography, web development, graphical design, and Ray Bradbury’s science ?ction novels. Several sci-? TV series such as Bu?y the Vampire Slayer and Star Trek appeared prominently in a few communities, while Stephen King and Douglas Clegg featured in a community recommending horror, sci-?, and thrillers to one another. One community focused predominantly on parenting, women’s health and pregnancy, while another recommended a variety of books but especially a collection of cookie baking recipes. Going back to components in the network that were disconnected from the largest component, we ?nd similar patterns of homophily, the tendency of like to associate with like. Two of the components recommended technical books about medicine, one focused on dance music, while some others predominantly purchased books on business and investing. Given more time, it is quite possible that one of the customers in one of these disconnected components would have received a recommendation from a customer within the largest component, and the two components would have merged. For example, a disconnected component of medical students purchasing medical textbooks might have sent or received a recommendation from the medical community within the largest component. However, the medical community may also become linked to other parts of the network through a di?erent interest of one of its members. At the very least many communities, no matter their focus, will have recommendations for children’s books or movies, since children are a focus for a great many people. The community ?nding algorithm on the other hand is able to break up the larger social network to automatically identify groups of individuals with a particular focus or a set of related interests. Now that we have shown that communities of customers recommend types of products re?ecting their interests, we will examine whether these di?erent kinds of products tend to have di?erent success rates in their recommendations.
8.2
Recommendation e?ectiveness by book category
Some contexts result in social ties that are more e?ective at conducting an action. For example, in small world experiments, where participants attempt to reach a target individual through their chain of acquaintances, profession trumped geography, which in turn was more useful in locating a target than attributes such as religion or hobbies [KB78, TM69]. In the context of product recommendations, we can ask whether a recommendation for a work of ?ction, which may be made by any friend or neighbor, is more or less in?uential than a recommendation for a technical book, which may be made by a colleague at work or school. Table 6 shows recommendation trends for all top level book categories by subject. For clarity, we group the results by 4 di?erent category types: ?ction, personal/leisure, professional/technical, and non?ction/other. Fiction encompasses categories such as Sci-Fi and Romance, as well as children’s and young adult books. Personal/Leisure encompasses everything from gardening, photography and cooking to health and religion. First, we compare the relative number of recommendations to reviews posted on the site (column cav /rp1 of table 6). Surprisingly, we ?nd that the number of people making personal recommendations was only a few times greater than the number of people posting a public review on the website. We observe that ?ction books have
30 category Books general Fiction Children Literature Mystery Science ?ction Romance Teens Comics Horror Personal Religion Health/Body History Home/Garden Entertainment Arts/Photo Travel Sports Parenting Cooking Outdoors Professional Professional Business Science Computers Medicine Engineering Law Other Non?ction Reference Biographies np 370230 46,451 41,682 10,734 10,008 6,317 5,857 3,565 2,773 43,423 33,751 28,458 19,024 18,724 17,153 12,670 10,183 8,324 7,655 6,413 41,794 29,002 25,697 18,941 16,047 10,312 5,176 55,868 26,834 18,233 n 2,860,714 390,283 502,179 123,392 175,168 60,902 81,260 46,564 48,321 441,263 572,704 28,3406 180,009 258,142 179,074 113,939 120,103 182,792 146,522 59,764 459,889 476,542 271,391 375,712 175,520 107,255 53,182 560,552 371,959 277,356 cc 1.87 2.82 3.06 6.03 6.17 5.65 5.72 11.70 9.35 1.89 1.54 2.74 2.91 3.65 3.49 3.91 1.74 0.73 3.02 2.23 1.72 1.55 2.64 2.22 1.08 1.30 2.64 2.03 1.94 2.80 rp1 5.28 6.44 13.09 20.14 19.90 12.81 20.52 4.76 21.26 3.87 4.34 4.34 1.78 3.48 1.56 2.74 3.36 4.71 3.14 1.93 1.91 3.61 2.41 4.51 1.41 1.43 1.89 3.13 2.49 7.65 vav 4.32 4.52 4.30 4.08 4.15 4.17 4.36 4.36 4.16 4.45 4.41 4.30 4.31 4.29 4.42 4.26 4.34 4.42 4.45 4.42 4.30 4.22 4.30 3.98 4.40 4.14 4.25 4.29 4.19 4.34
J. Leskovec et al.
cav / rp1 1.41 1.12 0.57 0.36 0.64 0.52 0.41 2.03 0.44 1.73 2.39 1.27 3.48 2.26 3.85 1.87 1.99 2.57 3.49 2.50 3.22 2.94 2.42 3.10 4.19 3.85 2.67 1.89 3.04 0.90
pm 14.95 8.76 11.87 9.60 10.39 6.99 9.56 10.47 9.60 9.99 13.96 18.00 15.37 13.97 20.95 13.27 13.97 11.87 13.97 15.00 32.50 20.99 28.00 34.95 39.95 59.95 24.95 18.95 17.47 14.00
b ? 100 3.12 2.06** 2.82* 2.40** 2.34** 1.78** 1.94** 2.30* 1.81** 3.13 3.04 2.84 2.26** 2.66* 2.87 2.39** 2.26** 2.81 2.38* 3.05 4.54** 3.62** 3.90** 3.61** 5.68** 4.10** 3.66* 3.28** 3.21 2.96
Table 6: Statistics by book category: np :number of products in category, n number of customers, cc percentage of customers in the largest connected component, rp1 avg. # reviews in 2001 – 2003, rp2 avg. # reviews 1st 6 months 2005, vav average star rating, cav average number of people recommending product, cav /rp1 ratio of recommenders to reviewers, pm median price, b ratio of the number of purchases resulting from a recommendation to the number of recommenders. The symbol ** denotes statistical signi?cance at the 0.01 level, * at the 0.05 level.
The Dynamics of Viral Marketing
31
relatively few recommendations compared to the number of reviews, while professional and technical books have more recommendations than reviews. This could re?ect several factors. One is that people feel more con?dent reviewing ?ction than technical books. Another is that they hesitate to recommend a work of ?ction before reading it themselves, since the recommendation must be made at the point of purchase. Yet another explanation is that the median price of a work of ?ction is lower than that of a technical book. This means that the discount received for successfully recommending a mystery novel or thriller is lower and hence people have less incentive to send recommendations. Next, we measure the per category e?cacy of recommendations by observing the ratio of the number of purchases occurring within a week following a recommendation to the number of recommenders for each book subject category (column b of table 6). On average, only 2% of the recommenders of a book received a discount because their recommendation was accepted, and another 1% made a recommendation that resulted in a purchase, but not a discount. We observe marked di?erences in the response to recommendation for di?erent categories of books. Fiction in general is not very e?ectively recommended, with only around 2% of recommenders succeeding. The e?cacy was a bit higher (around 3%) for non-?ction books dealing with personal and leisure pursuits. Perhaps people generally know what their friends’ leisure interests are, or even have gotten to know them through those shared interests. On the other hand they may not know as much about each others’ tastes in ?ction. Recommendation success is highest in the professional and technical category. Medical books have nearly double the average rate of recommendation acceptance. This could be in part attributed to the higher median price of medical books and technical books in general. As we will see in Section 9.2, a higher product price increases the chance that a recommendation will be accepted. Recommendations are also more likely to be accepted for certain religious categories: 4.3% for Christian living and theology and 4.8% for Bibles. In contrast, books not tied to organized religions, such as ones on the subject of new age (2.5%) and occult (2.2%) spirituality, have lower recommendation e?ectiveness. These results raise the interesting possibility that individuals have greater in?uence over one another in an organized context, for example through a professional contact or a religious one. There are exceptions of course. For example, Japanese anime DVDs have a strong following in the US, and this is re?ected in their frequency and success in recommendations. Another example is that of gardening. In general, recommendations for books relating to gardening have only a modest chance of being accepted, which agrees with the individual prerogative that accompanies this hobby. At the same time, orchid cultivation can be a highly organized and social activity, with frequent ‘shows’ and online communities devoted entirely to orchids. Perhaps because of this, the rate of acceptance of orchid book recommendations is twice as high as those for books on vegetable or tomato growing.
9
Products and recommendations
We have examined the properties of the recommendation network in relation to viral marketing. Now we focus on the products themselves and their characteristics that determine the success of recommendations.
32
J. Leskovec et al.
9.1
How long is the long tail?
Recently a ‘long tail’ phenomenon has been observed, where a large fraction of purchases are of relatively obscure items where each of them sells in very low numbers but there are many of those items. On Amazon.com, somewhere between 20 to 40 percent of unit sales fall outside of its top 100,000 ranked products [BHS03]. Considering that a typical brick and mortar store holds around 100,000 books, this presents a signi?cant share. A streaming-music service streams more tracks outside than inside its top 10,000 tunes [Ano05]. We performed a similar experiment using our data. Since we do not have direct sales data we used the number of successful recommendations as a proxy to the number of purchases. Figure 15 plots the distribution of the number of purchases and the number of recommendations per product. Notice that both the number of recommendations and the number of purchases per product follow a heavy-tailed distribution and that the distribution of recommendations has a heavier tail. Interestingly, ?gure 15(a) shows that just the top 100 products account for 11.4% of the all sales (purchases with discount), and the top 1000 products amount to 27% of total sales through the recommendation system. On the other hand 67% of the products have only a single purchase and they account for 30% of all sales. This shows that a signi?cant portion of sales come from products that sell very few times. Recently there has been some debate about the long tail [Gom06, And06]. Some argue that the presence of the long tail indicates that niche products with low sales are contributing signi?cantly to overall sales online. We also ?nd that the tail is a bit longer than the usual 80-20 rule, with the top 20% of the products contributing to about half the sales. It is important to note, however, that our observations do not re?ect the total sales of the products on the website, since they include only successful recommendations that resulted in a discount. This incorporates both a bias in the kind of product that is likely to be recommended, and in the probability that a recommendation for that kind of product is accepted. If we look at the distribution in the number of recommendations per product, shown in Figure 15(b), we observe an even more skewed distribution. 30% of the products have only a single recommendation and the top 56,000 most recommended products (top 10%) account for 84% of all recommendations. This is consistent with our previous observations some types of products, e.g. anime DVDs, are more heavily recommended than others. Next we examine the distribution of the product recommendation success rate. Out of more than half a million products we took all the products with at least a single purchase, of which there are 41,000 (7%). Figure 16 shows the success rate (purchases/recommendations). Notice that the distribution is not heavy tailed and has a mode at around 1.3% recommendation success rate. 55% of the products have a success rate bellow 5% and there are around 14% of the products that have a recommendation success rate higher than 20%.
9.2
Modeling the product recommendation success
So far we have seen that some products generate many recommendations and some have a better return than others on those recommendations, but one question still remains: what determines the product’s viral marketing success? We present a model
The Dynamics of Viral Marketing
33
10
4
Data = 4.0e4 x?2.49 R2=0.98
10 Count
4
Data = 9.3e5 x?1.91 R2=0.95
10 Count
3
10
2
10
2
10
1
10 0 10
0
10 10 10 Number of purchases per product
1
2
3
10
4
10
0
10
0
10 Number of recommendation per product
5
(a) Purchases
(b) Recommendations
Figure 15: Distribution of number of purchases and recommendations of a product. (a) shows the number of purchases that resulted in a discount per product, and (b) shows the distribution of the number of recommendations per product.
1200
10
Data Moving average
4
Number of products
1000 Number of products 800 600 400 200 0
Data Moving average 10
3
10
2
10
1
5 10 15 Recommendation success rate [%]
20
10 ?1 10
0
10 10 Recommendation success rate [%]
0
1
(a) Success rate (linear scale)
(b) Success rate (log scale)
Figure 16: Distribution of product recommendation success rates. Both plots show the same data: (a) on a linear (lin-lin) scale, and (b) on a logarithmic (log-log) scale. The bold line presents the moving average smoothing.
which characterizes product categories for which recommendations are more likely to be accepted. We use a regression of the following product attributes to correlate them with recommendation success: • n: number of nodes in the social network (number of unique senders and receivers) • ns : number of senders of recommendations • nr : number of recipients of recommendations • r: number of recommendations
34 log(s) 1 0.275 0.103 0.310 0.396 0.392 0.185 -0.050 -0.031 log
1 0.907 0.994 0.979 0.981 0.098 0.465 0.064 log(ns ) log(ne ) log(r) log(e)
J. Leskovec et al.
log(p)
log(v )
log(t)
log(s) log log(ns ) log(nr ) log(r) log(e) log(p) log(v ) log(t)
1 0.864 0.828 0.831 0.088 0.490 0.071
1.000 0.988 0.990 0.098 0.449 0.061
1 0.999 0.107 0.421 0.056
1 0.106 0.423 0.056
1 -0.053 -0.019
1 0.269
1
Table 7: Pairwise Correlation Matrix of the Books and DVD Product Attributes. log(s): log recommendation success rate, log: log number of nodes, log(ns ): log number of senders of recommendations, log(nr ): log number of receivers, log(r ): log number of recommendations, log(e): log number of edges, log(p): log price, log(v ): log number of reviews, log(t): log average rating.
• e: number of edges in the social network (number of unique (sender, receiver) pairs) • p: price of the product • v : number of reviews of the product • t: average product rating From the original set of the half-million products, we compute a success rate s for the 8,192 DVDs and 50,631 books that had at least 10 recommendation senders and for which a price was given. In section 8.2 we de?ned recommendation success rate s as the ratio of the total number purchases made through recommendations and the number of senders of the recommendations. We decided to use this kind of normalization, rather than normalizing by the total number of recommendations sent, in order not to penalize communities where a few individuals send out many recommendations (?gure 3(b)). Note that in general s could be greater than 1, but in practice this happens extremely rarely (there are only 107 products where s > 1 which were discarded for the purposes of this analysis). Since the variables follow a heavy tailed distribution, we use the following model: s = exp(
i
?i log(xi ) + ?i )
(7)
where xi are the product attributes (as described on previous page), and ?i is random error. We ?t the model using least squares and obtain the coe?cients ?i shown in table 8. With the exception of the average rating, they are all signi?cant, but just the number of recommendations alone accounts for 15% of the variance (taking all eight variables into consideration yields an R2 of 0.30 for books and 0.81 for DVDs). We should also note that the variables in our model are highly collinear, as can be seen
The Dynamics of Viral Marketing
35 DVD Coe?cient ?i 0.929 (0.0100) ** 0.171 (0.0124) ** -0.070 (0.0023) ** -0.360 (0.0104) ** -0.002 (0.0083) 0.251 (0.0088) ** 0.007 (0.0016) ** -0.003 (0.0006) ** 0.000 (0.0009) 0.81
Variable const n ns nr r e p v t R2
Books Coe?cient ?i 1.317 (0.0038) ** -0.579 (0.0060) ** 0.144 (0.0018) ** -0.006 (0.0064) 0.062 (0.0084) ** 0.383 (0.0106) ** 0.013 (0.0003) ** -0.003 (0.0001) ** -0.001 (0.0006) * 0.30
Table 8: Regression Using the Log of the Recommendation Success Rate log(s), as the Dependent Variable for Books and DVDs separately. For each coe?cient we provide the standard error and the statistical signi?cance level (**:0.001, *:0.1). We ?t separate models for books and DVDs.
from the pairwise correlation matrix (table 7). For example, the number of recommendations r is highly negatively correlated with the dependent variable (log(s)) but in the regression model it exhibits positive in?uence on the dependent variable. This is probably due to the fact that the number of recommendations is naturally dependent on the number of senders and number of recipients, but it is the high number of recommendations relative to the number of senders that is of importance. To illustrate the dependencies between the variables we train a Bayesian dependency network [Chi03], and show the learned structure for the combined (Books and DVDs) data in ?gure 17. In this a directed acyclic graph where nodes are variables, and directed edges indicate that the distribution of a child depends on the values taken in the parent variables. Notice that the average rating (t) is not predictive of the recommendation success rate (s). It is no surprise that the number of recommendations r is predictive of number of senders ns . Similarly, the number of edges e is predictive of number of senders ns . Interestingly, price p is only related to the number of reviews v . Number of recommendations r, number of senders ns and price p, are directly predictive of the recommendation success rate s. Returning to our regression model, we ?nd that the numbers of nodes and receivers have negative coe?cients, showing that successfully recommended products are actually more likely to be not so widely popular. The only attributes with positive coe?cients are the number of recommendations r, number of edges e, and price p. This shows that more expensive and more recommended products have a higher success rate. These recommendations should occur between a small number of senders and receivers, which suggests a very dense recommendation network where lots of recommendations are exchanged between a small community of people. These insights could be of use to marketers — personal recommendations are most e?ective in small, densely connected communities enjoying expensive products.
36
J. Leskovec et al.
v t e r s
Figure 17: A Bayesian network showing the dependencies between the variables. s: recommendation success rate, n: number of nodes, ns : number of senders of recommendations, nr : log number of receivers, r : number of recommendations, e: number of edges, p: price, v : number of reviews, t: average rating.
nr n ns p
10
Discussion and Conclusion
Although the retailer may have hoped to boost its revenues through viral marketing, the additional purchases that resulted from recommendations are just a drop in the bucket of sales that occur through the website. Nevertheless, we were able to obtain a number of interesting insights into how viral marketing works that challenge common assumptions made in epidemic and rumor propagation modeling. Firstly, it is frequently assumed in epidemic models (e.g., SIRS type of models) that individuals have equal probability of being infected every time they interact [AM02, Bai75]. Contrary to this we observe that the probability of infection decreases with repeated interaction. Marketers should take heed that providing excessive incentives for customers to recommend products could back?re by weakening the credibility of the very same links they are trying to take advantage of. Traditional epidemic and innovation di?usion models also often assume that individuals either have a constant probability of ‘converting’ every time they interact with an infected individual [GLM01], or that they convert once the fraction of their contacts who are infected exceeds a threshold [Gra78]. In both cases, an increasing number of infected contacts results in an increased likelihood of infection. Instead, we ?nd that the probability of purchasing a product increases with the number of recommendations received, but quickly saturates to a constant and relatively low probability. This means individuals are often impervious to the recommendations of their friends, and resist buying items that they do not want. In network-based epidemic models, extremely highly connected individuals play a very important role. For example, in needle sharing and sexual contact networks these nodes become the “super-spreaders” by infecting a large number of people. But these models assume that a high degree node has as much of a probability of infecting each of its neighbors as a low degree node does. In contrast, we ?nd that there are limits to how in?uential high degree nodes are in the recommendation network. As a person sends out more and more recommendations past a certain number for a
The Dynamics of Viral Marketing
37
product, the success per recommendation declines. This would seem to indicate that individuals have in?uence over a few of their friends, but not everybody they know. We also presented a simple stochastic model that allows for the presence of relatively large cascades for a few products, but re?ects well the general tendency of recommendation chains to terminate after just a short number of steps. Aggregating such cascades over all the products, we obtain a highly disconnected network, where the largest component grows over time by aggregating typically very small but occasionally fairly large components. We observed that the most popular categories of items recommended within communities in the largest component re?ect di?ering interests between these communities. We presented a model which shows that these smaller and more tightly knit groups tend to be more conducive to viral marketing. We saw that the characteristics of product reviews and e?ectiveness of recommendations vary by category and price, with more successful recommendations being made on technical or religious books, which presumably are placed in the social context of a school, workplace or place of worship. A small fraction of the products accounts for a large proportion of the recommendations. Although not quite as extreme in proportions, the number of successful recommendations also varies widely by product. Still, a sizeable portion of successful recommendations were for a product with only one such sale - hinting at a long tail phenomenon. Since viral marketing was found to be in general not as epidemic as one might have hoped, marketers hoping to develop normative strategies for word-of-mouth advertising should analyze the topology and interests of the social network of their customers. Our study has provided a number of new insights which we hope will have general applicability to marketing strategies and to future models of viral information spread.
ACKNOWLEDGEMENTS
We thank anonymous reviewers for their insightful comments. This work was partially supported by the National Science Foundation under grants SENSOR-0329549 IIS-0326322 IIS-0534205. This work is also supported in part by the Pennsylvania Infrastructure Technology Alliance (PITA). Additional funding was provided by a generous gift from Hewlett-Packard. Jure Leskovec was partially supported by a Microsoft Research Graduate Fellowship.
References
[AM02] [And06] [Ano05] [Bai75] Roy M. Anderson and Robert M. May. Infectious diseases of humans: Dynamics and control. Oxford Press, 2002. C. Anderson. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, 2006. Anonymous. Pro?ting from obscurity: What the ”long tail” means for the economics of e-commerce. Economist, 2005. Norman Bailey. The Mathematical Theory of Infectious Diseases and its Applications. Gri?n, London, 1975.
38
J. Leskovec et al.
[Bas69] [BHS03]
Frank Bass. A new product growth for model consumer durables. Management Science, 15(5):215–227, 1969. Erik Brynjolfsson, Yu Hu, and Michael D. Smith. Consumer surplus in the digital economy: Estimating the value of increased product variety at online booksellers. Management Science, 49(11):1580–1596, 2003. D. Bowman and D. Narayandas. Managing customerinitiated contacts with manufacturers: The impact on share of category requirements and word-of-mouth behavior. Journal of Marketing Research, 38(3):281 – 297, August 2001. J. J. Brown and P. H. Reingen. Social ties and word-of-mouth referral behavior. The Journal of Consumer Research, 14(3):350–362, 1987. Po Bronson. Hotmale. Wired Magazine, 6(12), 1998. Kevin Burke. As consumer attitudes shift, so must marketing strategies. 2003. David Maxwell Chickering. Optimal structure identi?cation with greedy search. JMLR, 3:507–554, 2003. Damon Centola and Michael Macy. Complex contagion and the weakness of long ties. ftp://hive.soc.cornell.edu/mwm14/webpage/WLT.pdf, 2005. Judith Chevalier and Dina Mayzlin. The e?ect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43(3):345, 2006. Aaron Clauset, M. E. J. Newman, and Cristopher Moore. Finding community structure in very large networks. Physical Review E, 70:066111, 2004. Arnaud DeBruyn and Gary Lilien. A multi-stage model of word of mouth through electronic referrals. 2004. P. Erd¨ os and A. R´ enyi. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci., 5:17–61, 1960. Jonathan Frenzen and Kent Nakamoto. Structure, cooperation, and the ?ow of market information. Journal of Consumer Research, 20(3):360– 375, December 1993.
[BN01]
[BR87] [Bro98] [Bur03] [Chi03] [CM05]
[CM06]
[CNM04]
[DL04] [ER60] [FN93]
[GGLNT04] D. Gruhl, R. Guha, D. Liben-Nowell, and A. Tomkins. Information di?usion through blogspace. In WWW ’04, 2004. [GLM01] J. Goldenberg, B. Libai, and E. Muller. Talk of the network: A complex systems look at the underlying process of word-of-mouth. Marketing Letters, 3(12):211–223, 2001.
The Dynamics of Viral Marketing
39
[Gom06] [Gra73] [Gra78] [HN06]
L. Gomes. It may be a long time before the long tail is wagging the web. The Wall Street Jounal, 2006. July 26 2006. M. S. Granovetter. The strength of weak ties. American Journal of Sociology, 78:1360–1380, 1973. M. Granovetter. Threshold models of collective behavior. American Journal of Sociology, 83(6):1420–1443, 1978. Petter Holme and M. E. J. Newman. Nonequilibrium phase transition in the coevolution of networks and opinions. Physical Review E, 74:056108, 2006. S. Hill, F. Provost, and C. Volinsky. Network-based marketing: Identifying likely adopters via consumer networks. Statistical Science, 21(2):256– 276, 2006. S. Jurvetson. What exactly is viral marketing? Red Herring, 78:110–112, 2000. P. Killworth and H. Bernard. Reverse small world experiment. Social Networks, 1:159–192, 1978. D. Kempe, J. Kleinberg, and E. Tardos. Maximizing the spread of infuence in a social network. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2003. Jure Leskovec, Lada A. Adamic, and Bernardo A. Huberman. The dynamics of viral marketing. In ACM conference on Electronic Commerce, pages 228–237, 2006. Jure Leskovec, Ajit Singh, and Jon Kleinberg. Patterns of in?uence in a recommendation network. In Paci?c-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2006. G. Linden, B. Smith, and J. York. Amazon.com recommendations: itemto-item collaborative ?ltering. IEEE Internet Computing, 7(1):76–80, 2003. Alan L. Montgomery. Applying quantitative marketing techniques to the internet. Interfaces, 30:90–108, 2001. Matt Richardson and Pedro Domingos. Mining knowledge-sharing sites for viral marketing. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2002. Everett M. Rogers. Di?usion of Innovations. Free Press, New York, fourth edition, 1995. Paul Resnick and Richard Zeckhauser. Trust among strangers in internet transactions: Empirical analysis of ebay’s reputation system. In The Economics of the Internet and E-Commerce. Elsevier Science, 2002.
[HPV06]
[Jur00] [KB78] [KKT03]
[LAH06]
[LSK06]
[LSY03]
[Mon01] [RD02]
[Rog95] [RZ02]
40
J. Leskovec et al.
[SR03]
Mani R. Subramani and Balaji Rajagopalan. Knowledge-sharing and in?uence in online social networks via viral marketing. Communications of the ACM, 46(12):300–307, 2003. David Strang and Sarah A. Soule. Di?usion in organizations and social movements: From hybrid corn to poison pills. Annual Review of Sociology, 24:265–290, 1998. J. Travers and S. Milgram. An experimental study of the small world problem. Sociometry, 32:425–443, 1969. D.J. Watts. A simple model of global cascades on random networks. PNAS, 99(9):4766–5771, Apr 2002. Fang Wu and Bernardo A. Huberman. Social structure and opinion formation. Computational Economics 0407002, EconWPA, July 2004. available athttp://ideas.repec.org/p/wpa/wuwpco/0407002.html. Sha Yang and Greg M. Allenby. Modeling interdependent consumer preferences. Journal of Marketing Research, 40(3):282 – 294, August 2003.
[SS98]
[TM69] [Wat02] [WH04]
[YA03]
The Dynamics of Viral Marketing
41
Appendix
0.35 0.3 Number of Purchases 0.25 0.2 0.15 0.1 0.05 = 0.01 x 0 2
1.09 2
3 2.5 2 1.5 1 0.5
R =0.99 12
Number of Purchases
= 0.02 x1.34 R2=0.98 0 10 20 30 40 Outgoing Recommendations
4 6 8 10 Outgoing Recommendations
0.12 0.1 Probability of credit Probability of credit
1.12 2
0.08 0.07 0.06 0.05 0.04 0.03 0.02 = 0.01 x R =0.99 10 0.01 = 0.01 x
0.49 2
0.08 0.06 0.04 0.02 0 2 4 6 8 Outgoing recommendations
R =0.73 20
5 10 15 Outgoing recommendations
(a) Books
(b) DVD
Figure A-1: Top row: Power ?t to the non-linear part of the number of resulting purchases given a number of outgoing recommendations. Bottom row: Power ?t to the probability of getting a credit given a number of outgoing recommendations.
42
J. Leskovec et al.
0.2
0.08 Probability of Buying
Probability of Buying 0.15
0.06
0.04
0.1
0.02
0.05
0
10 20 30 40 50 Total Incomming Recommendations
0
5 10 15 20 25 30 35 Total Incomming Recommendations
40
(a) Books
0.08 0.04
(b) DVD
Probability of Buying
0.06
Probability of Buying 5 10 15 20 25 30 35 Total Incomming Recommendations 40
0.03
0.04
0.02
0.02
0.01
0
0
5 10 15 Total Incomming Recommendations
20
(c) Music
(d) Video
Figure A-2: Probability of buying a product given a total number of incoming recommendations on all products.
0.4 Proportion of Purchases Proportion of Purchases 1 2 3 4 5 6 Lag [day] 7 >7
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
1
2
3
4 5 6 Lag [day]
7
>7
(a) Music
(b) Video
Figure A-3: The time between the ?rst recommendation and the purchase. The histograms show how long does it take to accumulate su?cient number of recommendations to trigger a purchase. Figure 13 plots the same quantity for Books and DVD. The bin size is 1 day. We use all purchases through recommendations.
The Dynamics of Viral Marketing
43
2500 2000 1500 1000 500 0 0
600 500 400
Count
Count
300 200 100
24
48
72 96 120 144 168 Lag [hours]
0 0
24
48
72 96 120 144 168 Lag [hours]
(a) Books
300 250
(b) DVD
40
30
200 Count
Count
150 100
20
10
50 0 0 24 48 72 96 120 144 168 Lag [hours]
0 0
24
48
72 96 120 144 168 Lag [hours]
(c) Music
(d) Video
Figure A-4: The time between the recommendation and the purchase taking only the recommendations that resulted in a 10% discount. The bin size is 3 hours. The dashed line presents a logarithmic ?t.
44
J. Leskovec et al.
category Anime and Manga Classics Animation Science Fiction & Fantasy Art House & International Television Horror Action and Adventure Mystery and Suspense Military and War Cult Movies Kids and Family Drama Comedy Musicals & Performing Arts Westerns Sports Documentary Educational Music Video and Concerts Special Interests Fitness and Yoga African American Cinema
np 1301 266 237 1410 3185 1133 1125 2058 1683 379 324 1357 3376 2455 1091 234 484 1058 89 2222 963 223 81
n 46941 24922 80092 317420 276142 195948 79744 248674 151101 69180 94049 230300 255544 312033 88665 17612 23191 53538 5532 91657 43225 17160 10609
cc 18.92 25.59 11.99 6.61 7.37 8.17 13.15 7.00 9.28 12.53 11.24 6.70 7.12 6.08 10.24 24.40 16.92 15.24 19.60 8.44 10.42 2.23 17.92
rp1 14.40 9.68 41.90 59.18 24.35 18.95 30.00 39.80 26.73 39.31 37.93 30.96 25.14 26.25 17.07 11.76 8.64 6.12 3.39 8.06 5.83 14.65 16.00
rp1 /rp2 17.17 6.66 19.17 16.66 12.97 11.68 9.10 15.11 10.45 11.14 8.45 12.81 11.02 11.14 11.11 7.30 7.89 9.08 2.63 11.16 7.45 6.66 9.06
vav 4.19 4.18 4.03 3.85 3.95 4.22 3.59 3.80 3.82 4.12 3.89 4.12 3.98 4.02 4.09 3.94 3.97 3.95 3.97 4.09 3.99 3.88 4.15
cav /rp1 2.96 4.16 3.88 2.51 2.22 5.32 1.37 1.96 2.20 2.26 3.34 3.35 2.10 3.30 2.34 2.72 2.49 3.70 5.48 2.88 3.43 2.93 3.41
pm 26.96 22.49 22.49 17.99 22.46 17.99 17.98 17.96 17.98 17.96 17.98 17.98 17.98 17.98 22.48 13.48 17.98 17.99 19.95 17.99 18.74 17.96 17.98
b 28.44** 11.22** 10.43 9.62 9.43* 8.90 8.72 8.42** 7.57 7.41 7.28 6.75 6.72* 6.01** 4.93 4.71* 4.55* 4.24 3.99 3.85 2.62 1.98 1.56
Table A-1: Statistics by DVD genre. * denotes signi?cance at the 0.05 level, ** at the 0.01 level
The Dynamics of Viral Marketing
45
category Anime and Manga Educational Fitness Animation Kids and Family Special Interests Mystery and Suspense Art House & International Science Fiction and Fantasy Documentary Television Music Video & Concerts Musicals & Performing Arts Sports Comedy Drama Military and War Westerns Classics African American Cinema Horror Action and Adventure Cult Movies
np 962 607 920 171 4736 3769 1514 2459 1583 2936 3632 1595 1621 1251 3645 4837 829 487 326 87 935 2390 401
n 5081 6569 24627 9500 84608 36862 13459 24713 29565 18884 31475 14360 22539 7987 55868 52691 10859 3743 3029 1861 6728 25921 5260
cc 9.64 1.64 0.43 4.04 1.13 1.45 9.90 3.52 2.54 1.15 0.95 4.46 3.13 0.49 2.13 1.87 1.13 1.58 0.56 0.64 1.07 1.84 0.65
rp1 13.98 1.97 8.41 61.83 14.26 3.19 30.09 17.54 51.92 3.33 5.13 8.75 13.22 4.07 22.26 21.72 28.54 9.42 8.73 15.53 36.38 33.13 32.06
rp1 /rp2 18.76 10.75 12.09 19.58 12.11 12.73 9.83 10.09 13.76 9.83 12.11 11.26 9.39 9.83 10.60 9.25 9.39 6.01 8.15 7.59 9.02 11.90 7.63
vav 4.39 4.17 4.09 4.29 4.29 4.14 4.01 4.18 4.01 4.21 4.33 4.40 4.20 4.15 4.13 4.15 4.22 4.12 4.12 4.10 3.81 3.96 3.90
cav /rp1 0.26 3.01 1.92 0.36 0.85 1.65 0.14 0.28 0.18 0.95 1.01 0.49 0.51 0.91 0.36 0.26 0.21 0.43 0.51 0.61 0.10 0.17 0.18
pm 17.99 19.95 14.95 17.99 12.98 19.95 14.95 17.99 13.99 19.95 14.95 16.99 19.95 16.99 13.99 14.95 14.95 9.99 14.94 9.99 12.99 13.99 9.99
b ? 100 1.99* 1.59 1.48 1.36 1.16 1.09 1.01 0.84 0.83 0.82 0.71 0.70 0.69 0.69 0.59 0.56 0.56 0.56 0.49 0.49 0.40 0.31 0.30
Table A-2: Statistics for videos in VHS format by genre
46
J. Leskovec et al.
category Broadway and Vocalists Country Rock Alternative Rock Soundtracks Classical Folk Pop Opera and Vocal Miscellaneous Blues Hard Rock and Metal Christian and Gospel Jazz Classic Rock Children s Music Dance and DJ New Age International Latin Music Rap and Hip Hop R&B
np 5423 5876 10717 13405 4491 14223 5244 16764 5402 5823 2987 4787 2977 11868 5711 1755 11332 4219 13139 4634 3996 5965
n 104396 98069 196852 216324 133507 116937 87580 322431 61643 80243 31199 63893 37554 113078 117255 37015 139787 60951 130499 38725 60135 85380
cc 4.25 4.67 4.10 5.12 4.81 5.34 5.33 3.30 6.08 5.71 6.62 4.96 2.02 4.49 4.74 4.89 5.16 5.90 5.02 5.06 3.67 2.78
rp1 6.03 5.50 11.00 13.20 7.92 2.65 4.40 9.55 3.32 3.54 2.76 18.23 5.41 2.91 13.62 3.96 7.14 3.92 3.54 2.57 12.23 8.49
rp1 /rp2 13.86 18.45 10.18 11.24 13.82 11.60 13.54 13.19 12.90 12.31 11.53 7.92 16.75 11.40 6.78 12.52 14.64 13.79 9.52 16.76 9.64 12.90
vav 4.49 4.56 4.40 4.41 4.38 4.52 4.60 4.43 4.48 4.35 4.59 4.33 4.67 4.59 4.29 4.53 4.38 4.54 4.57 4.60 4.38 4.48
cav /rp1 1.68 1.76 0.99 0.81 1.77 1.82 2.05 1.22 1.69 1.90 1.89 0.42 1.20 1.99 7 0.87 2.94 1.05 1.95 1.51 1.75 0.67 0.89
pm 14.49 13.99 14.99 13.99 14.99 15.49 14.99 13.99 15.99 13.98 14.99 14.99 14.99 14.99 13.99 12.32 14.99 14.99 14.99 13.99 14.99 13.98
b 2.01 1.87 1.87 1.87 1.87 1.83 1.81 1.78 1.73 1.62 1.54 1.52 1.51 1.50 1.50 1.47 1.42 1.42 1.32 1.30 1.14 1.13
Table A-3: Statistics by Music Style
doc_947057348.pdf
Viral marketing, viral advertising, or marketing buzz are buzzwords referring to marketing techniques that use pre-existing social networks and other technologies to produce increases in brand awareness or to achieve other marketing objectives
The Dynamics of Viral Marketing ?
Jure Leskovec
Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA
Lada A. Adamic
School of Information, University of Michigan, Ann Arbor, MI
Bernardo A. Huberman
HP Labs, Palo Alto, CA 94304
April 20, 2007
Abstract We present an analysis of a person-to-person recommendation network, consisting of 4 million people who made 16 million recommendations on half a million products. We observe the propagation of recommendations and the cascade sizes, which we explain by a simple stochastic model. We analyze how user behavior varies within user communities de?ned by a recommendation network. Product purchases follow a ’long tail’ where a signi?cant share of purchases belongs to rarely sold items. We establish how the recommendation network grows over time and how e?ective it is from the viewpoint of the sender and receiver of the recommendations. While on average recommendations are not very e?ective at inducing purchases and do not spread very far, we present a model that successfully identi?es communities, product and pricing categories for which viral marketing seems to be very e?ective.
1
Introduction
With consumers showing increasing resistance to traditional forms of advertising such as TV or newspaper ads, marketers have turned to alternate strategies, including viral marketing. Viral marketing exploits existing social networks by encouraging customers to share product information with their friends. Previously, a few in depth studies have shown that social networks a?ect the adoption of individual innovations and products (for a review see [Rog95] or [SS98]). But until recently it has been di?cult to measure how in?uential person-to-person recommendations actually are over a wide range of products. Moreover, Subramani and Rajagopalan [SR03] noted that “there needs to be a greater understanding of the contexts in which viral marketing strategy works and the characteristics of products and services for which it is most
? This work also appears in: Leskovec, J., Adamic, L. A., and Huberman, B. A. 2007. The dynamics of viral marketing. ACM Transactions on the Web, 1, 1 (May 2007).
1
2
J. Leskovec et al.
e?ective. This is particularly important because the inappropriate use of viral marketing can be counterproductive by creating unfavorable attitudes towards products. What is missing is an analysis of viral marketing that highlights systematic patterns in the nature of knowledge-sharing and persuasion by in?uencers and responses by recipients in online social networks.” Here we were able to in detail study the above mentioned problem. We were able to directly measure and model the e?ectiveness of recommendations by studying one online retailer’s incentivised viral marketing program. The website gave discounts to customers recommending any of its products to others, and then tracked the resulting purchases and additional recommendations. Although word of mouth can be a powerful factor in?uencing purchasing decisions, it can be tricky for advertisers to tap into. Some services used by individuals to communicate are natural candidates for viral marketing, because the product can be observed or advertised as part of the communication. Email services such as Hotmail and Yahoo had very fast adoption curves because every email sent through them contained an advertisement for the service and because they were free. Hotmail spent a mere $50,000 on traditional marketing and still grew from zero to 12 million users in 18 months [Jur00]. The Hotmail user base grew faster than any media company in history – faster than CNN, faster than AOL, even faster than Seinfeld’s audience. By mid-2000, Hotmail had over 66 million users with 270,000 new accounts being established each day [Bro98]. Google’s Gmail also captured a signi?cant part of market share in spite of the fact that the only way to sign up for the service was through a referral. Most products cannot be advertised in such a direct way. At the same time the choice of products available to consumers has increased manyfold thanks to online retailers who can supply a much wider variety of products than traditional brick-andmortar stores. Not only is the variety of products larger, but one observes a ‘fat tail’ phenomenon, where a large fraction of purchases are of relatively obscure items. On Amazon.com, somewhere between 20 to 40 percent of unit sales fall outside of its top 100,000 ranked products [BHS03]. Rhapsody, a streaming-music service, streams more tracks outside than inside its top 10,000 tunes [Ano05]. Some argue that the presence of the long tail indicates that niche products with low sales are contributing signi?cantly to overall sales online. We ?nd that product purchases that result from recommendations are not far from the usual 80-20 rule. The rule states that the top twenty percent of the products account for 80 percent of the sales. In our case the top 20% of the products contribute to about half the sales. E?ectively advertising these niche products using traditional advertising approaches is impractical. Therefore using more targeted marketing approaches is advantageous both to the merchant and the consumer, who would bene?t from learning about new products. The problem is partly addressed by the advent of online product and merchant reviews, both at retail sites such as EBay and Amazon, and specialized product comparison sites such as Epinions and CNET. Of further help to the consumer are collaborative ?ltering recommendations of the form “people who bought x also bought y ” feature [LSY03]. These re?nements help consumers discover new products and receive more accurate evaluations, but they cannot completely substitute personalized
The Dynamics of Viral Marketing
3
recommendations that one receives from a friend or relative. It is human nature to be more interested in what a friend buys than what an anonymous person buys, to be more likely to trust their opinion, and to be more in?uenced by their actions. As one would expect our friends are also acquainted with our needs and tastes, and can make appropriate recommendations. A Lucid Marketing survey found that 68% of individuals consulted friends and relatives before purchasing home electronics – more than the half who used search engines to ?nd product information [Bur03]. In our study we are able to directly observe the e?ectiveness of person to person word of mouth advertising for hundreds of thousands of products for the ?rst time. We ?nd that most recommendation chains do not grow very large, often terminating with the initial purchase of a product. However, occasionally a product will propagate through a very active recommendation network. We propose a simple stochastic model that seems to explain the propagation of recommendations. Moreover, the characteristics of recommendation networks in?uence the purchase patterns of their members. For example, individuals’ likelihood of purchasing a product initially increases as they receive additional recommendations for it, but a saturation point is quickly reached. Interestingly, as more recommendations are sent between the same two individuals, the likelihood that they will be heeded decreases. We ?nd that communities (automatically found by graph theoretic community ?nding algorithm) were usually centered around a product group, such as books, music, or DVDs, but almost all of them shared recommendations for all types of products. We also ?nd patterns of homophily, the tendency of like to associate with like, with communities of customers recommending types of products re?ecting their common interests. We propose models to identify products for which viral marketing is e?ective: We ?nd that the category and price of product plays a role, with recommendations of expensive products of interest to small, well connected communities resulting in a purchase more often. We also observe patterns in the timing of recommendations and purchases corresponding to times of day when people are likely to be shopping online or reading email. We report on these and other ?ndings in the following sections. We ?rst survey the related work in section 2. We then describe the characteristics of the incentivised recommendations program and the dataset in section 3. Section 4 studies the temporal and static characteristics of the recommendation network. We investigate the propagation of recommendations and model the cascading behavior in section 5. Next we concentrate on the various aspects of the recommendation success from the viewpoint of the sender and the recipient of the recommendation in section 6. The timing and the time lag between the recommendations and purchases is studied in section 7. We study network communities, product characteristics and the purchasing behavior in section 8. Last, in section 9 we present a model that relates product characteristics and the surrounding recommendation network to predict the product recommendation success. We discuss the implications of our ?ndings and conclude in section 10.
4
J. Leskovec et al.
2
Related work
Viral marketing can be thought of as a di?usion of information about the product and its adoption over the network. Primarily in social sciences there is a long history of the research on the in?uence of social networks on innovation and product di?usion. However, such studies have been typically limited to small networks and typically a single product or service. For example, Brown and Reingen [BR87] interviewed the families of students being instructed by three piano teachers, in order to ?nd out the network of referrals. They found that strong ties, those between family or friends, were more likely to be activated for information ?ow and were also more in?uential than weak ties [Gra73] between acquaintances. Similar observations were also made by DeBruyn and Lilien in [DL04] in the context of electronic referrals. They found that characteristics of the social tie in?uenced recipients behavior but had di?erent e?ects at di?erent stages of decision making process: tie strength facilitates awareness, perceptual a?nity triggers recipients interest, and demographic similarity had a negative in?uence on each stage of the decision-making process. Social networks can be composed by using various information, i.e. geographic similarity, age, similar interests and so on. Yang and Allenby [YA03] showed that the geographically de?ned network of consumers is more useful than the demographic network for explaining consumer behavior in purchasing Japanese cars. A recent study by Hill et al. [HPV06] found that adding network information, speci?cally whether a potential customer was already “talking to” an existing customer, was predictive of the chances of adoption of a new phone service option. For the customers linked to a prior customer the adoption rate of was 3–5 times greater than the baseline. Factors that in?uence customers’ willingness to actively share the information with others via word of mouth have also been studied. Frenzen and Nakamoto [FN93] surveyed a group of people and found that the stronger the moral hazard presented by the information, the stronger the ties must be to foster information propagation. Also, the network structure and information characteristics interact when individuals form decisions about transmitting information. Bowman and Narayandas [BN01] found that self-reported loyal customers were more likely to talk to others about the products when they were dissatis?ed, but interestingly not more likely when they were satis?ed. In the context of the internet word-of-mouth advertising is not restricted to pairwise or small-group interactions between individuals. Rather, customers can share their experiences and opinions regarding a product with everyone. Quantitative marketing techniques have been proposed [Mon01] to describe product information ?ow online, and the rating of products and merchants has been shown to e?ect the likelihood of an item being bought [RZ02, CM06]. More sophisticated online recommendation systems allow users to rate others’ reviews, or directly rate other reviewers to implicitly form a trusted reviewer network that may have very little overlap with a person’s actual social circle. Richardson and Domingos [RD02] used Epinions’ trusted reviewer network to construct an algorithm to maximize viral marketing e?ciency assuming that individuals’ probability of purchasing a product depends on the opinions on the trusted peers in their network. Kempe, Kleinberg and Tardos [KKT03] have followed up on Richardson and Domingos’ challenge of maximizing viral information spread by evaluating several algorithms given various models of adoption we discuss
The Dynamics of Viral Marketing
5
next. Most of the previous research on the ?ow of information and in?uence through the networks has been done in the context of epidemiology and the spread of diseases over the network. See the works of Bailey [Bai75] and Anderson and May [AM02] for reviews of this area. The classical disease propagation models are based on the stages of a disease in a host: a person is ?rst susceptible to a disease, then if she is exposed to an infectious contact she can become infected and thus infectious. After the disease ceases the person is recovered or removed. Person is then immune for some period. The immunity can also wear o? and the person becomes again susceptible. Thus SIR (susceptible – infected – recovered) models diseases where a recovered person never again becomes susceptible, while SIRS (SIS, susceptible – infected – (recovered) – susceptible) models population in which recovered host can become susceptible again. Given a network and a set of infected nodes the epidemic threshold is studied, i.e. conditions under which the disease will either dominate or die out. In our case SIR model would correspond to the case where a set of initially infected nodes corresponds to people that purchased a product without ?rst receiving the recommendations. A node can purchase a product only once, and then tries to infect its neighbors with a purchase by sending out the recommendations. SIS model corresponds to less realistic case where a person can purchase a product multiple times as a result of multiple recommendations. The problem with these type of models is that they assume a known social network over which the diseases (product recommendations) are spreading and usually a single parameter which speci?es the infectiousness of the disease. In our context this would mean that the whole population is equally susceptible to recommendations of a particular product. There are numerous other models of in?uence spread in social networks. One of the ?rst and most in?uential di?usion models was proposed by Bass [Bas69]. The model of product di?usion predicts the number of people who will adopt an innovation over time. It does not explicitly account for the structure of the social network but it rather assumes that the rate of adoption is a function of the current proportion of the population who have already adopted (purchased a product in our case). The di?usion equation models the cumulative proportion of adopters in the population as a function of the intrinsic adoption rate, and a measure of social contagion. The model describes an S-shaped curve, where adoption is slow at ?rst, takes o? exponentially and ?attens at the end. It can e?ectively model word-of-mouth product di?usion at the aggregate level, but not at the level of an individual person, which is one of the topics we explore in this paper. Di?usion models that try to model the process of adoption of an idea or a product can generally be divided into two groups: • Threshold model [Gra78] where each node in the network has a threshold t ? [0, 1], typically drawn from some probability distribution. We also assign connection weights wu,v on the edges of the network. A node adopts the behavior if a sum of the connection weights of its neighbors that already adopted the behavior (purchased a product in our case) is greater than the threshold: t ? adopters(u) wu,v . • Cascade model [GLM01] where whenever a neighbor v of node u adopts, then node u also adopts with probability pu,v . In other words, every time a neighbor
6
J. Leskovec et al.
of u purchases a product, there is a chance that u will decide to purchase as well. In the independent cascade model, Goldenberg et al. [GLM01] simulated the spread of information on an arti?cially generated network topology that consisted both of strong ties within groups of spatially proximate nodes and weak ties between the groups. They found that weak ties were important to the rate of information di?usion. Centola and Macy [CM05] modeled product adoption on small world topologies when a person’s chance of adoption is dependent on having more than one contact who had previously adopted. Wu and Huberman [WH04] modeled opinion formation on di?erent network topologies, and found that if highly connected nodes were seeded with a particular opinion, this would proportionally e?ect the long term distribution of opinions in the network. Holme and Newman [HN06] introduced a model where individuals’ preferences are shaped by their social networks, but their choices of whom to include in their social network are also in?uenced by their preferences. While these models address the question of how in?uence spreads in a network, they are based on assumed rather than measured in?uence e?ects. In contrast, our study tracks the actual di?usion of recommendations through email, allowing us to quantify the importance of factors such as the presence of highly connected individuals, or the e?ect of receiving recommendations from multiple contacts. Compared to previous empirical studies which tracked the adoption of a single innovation or product, our data encompasses over half a million di?erent products, allowing us to model a product’s suitability for viral marketing in terms of both the properties of the network and the product itself.
3
3.1
The Recommendation Network
Recommendation program and dataset description
Our analysis focuses on the recommendation referral program run by a large retailer. The program rules were as follows. Each time a person purchases a book, music, or a movie he or she is given the option of sending emails recommending the item to friends. The ?rst person to purchase the same item through a referral link in the email gets a 10% discount. When this happens the sender of the recommendation receives a 10% credit on their purchase. The following information is recorded for each recommendation 1. Sender Customer ID (shadowed) 2. Receiver Customer ID (shadowed) 3. Date of Sending 4. Purchase ?ag (buy-bit ) 5. Purchase Date (error-prone due to asynchrony in the servers) 6. Product identi?er 7. Price
The Dynamics of Viral Marketing
7
The recommendation dataset consists of 15,646,121 recommendations made among 3,943,084 distinct users. The data was collected from June 5 2001 to May 16 2003. In total, 548,523 products were recommended, 99% of them belonging to 4 main product groups: Books, DVDs, Music and Videos. In addition to recommendation data, we also crawled the retailer’s website to obtain product categories, reviews and ratings for all products. Of the products in our data set, 5813 (1%) were discontinued (the retailer no longer provided any information about them). Although the data gives us a detailed and accurate view of recommendation dynamics, it does have its limitations. The only indication of the success of a recommendation is the observation of the recipient purchasing the product through the same vendor. We have no way of knowing if the person had decided instead to purchase elsewhere, borrow, or otherwise obtain the product. The delivery of the recommendation is also somewhat di?erent from one person simply telling another about a product they enjoy, possibly in the context of a broader discussion of similar products. The recommendation is received as a form email including information about the discount program. Someone reading the email might consider it spam, or at least deem it less important than a recommendation given in the context of a conversation. The recipient may also doubt whether the friend is recommending the product because they think the recipient might enjoy it, or are simply trying to get a discount for themselves. Finally, because the recommendation takes place before the recommender receives the product, it might not be based on a direct observation of the product. Nevertheless, we believe that these recommendation networks are re?ective of the nature of word of mouth advertising, and give us key insights into the in?uence of social networks on purchasing decisions.
3.2
Identifying successful recommendations
For each recommendation, the dataset includes information about the recommended product, sender and received or the recommendation, and most importantly, the success of recommendation. See section 3.1 for more details. We represent this data set as a directed multi graph. The nodes represent customers, and a directed edge contains all the information about the recommendation. The edge (i, j, p, t) indicates that i recommended product p to customer j at time t. Note that as there can be multiple recommendations of between the persons (even on the same product) there can be multiple edges between two nodes. The typical process generating edges in the recommendation network is as follows: a node i ?rst buys a product p at time t and then it recommends it to nodes j1 , . . . , jn . The j nodes can then buy the product and further recommend it. The only way for a node to recommend a product is to ?rst buy it. Note that even if all nodes j buy a product, only the edge to the node jk that ?rst made the purchase (within a week after the recommendation) will be marked by a buy-bit. Because the buy-bit is set only for the ?rst person who acts on a recommendation, we identify additional purchases by the presence of outgoing recommendations for a person, since all recommendations must be preceded by a purchase. We call this type of evidence of purchase a buy-edge. Note that buy-edges provide only a lower bound on the total number of purchases without discounts. It is possible for a customer to not be the ?rst to act on a recommendation and also to not recommend the product to others. Unfortunately, this
8 Group Book DVD Music Video Full network p 103,161 19,829 393,598 26,131 542,719 n 2,863,977 805,285 794,148 239,583 3,943,084 r 5,741,611 8,180,393 1,443,847 280,270 15,646,121
J. Leskovec et al.
e 2,097,809 962,341 585,738 160,683 3,153,676
bb 65,344 17,232 7,837 909 91,322
be 17,769 58,189 2,739 467 79,164
Table 1: Product group recommendation statistics. p: number of products, n: number of nodes, r : number of recommendations, e: number of edges, bb : number of buy bits, be : number of buy edges.
was not recorded in the data set. We consider, however, the buy-bits and buy-edges as proxies for the total number of purchases through recommendations. As mentioned above the ?rst buyer only gets a discount (the buy-bit is turned on) if the purchase is made within one week of the recommendation. In order to account for as many purchases as possible, we consider all purchases where the recommendation preceded the purchase (buy-edge) regardless of the time di?erence between the two events. To avoid confusion we will refer to edges in a multi graph as recommendations (or multi-edges) — there can be more than one recommendation between a pair of nodes. We will use the term edge (or unique edge) to refer to edges in the usual sense, i.e. there is only one edge between a pair of people. And, to get from recommendations to edges we create an edge between a pair of people if they exchanged at least one recommendation.
4
The recommendation network
For each product group we took recommendations on all products from the group and created a network. Table 1 shows the sizes of various product group recommendation networks with p being the total number of products in the product group, n the total number of nodes spanned by the group recommendation network, and r the number of recommendations (there can be multiple recommendations between two nodes). Column e shows the number of (unique) edges – disregarding multiple recommendations between the same source and recipient (i.e., number of pairs of people that exchanged at least one recommendation). In terms of the number of di?erent items, there are by far the most music CDs, followed by books and videos. There is a surprisingly small number of DVD titles. On the other hand, DVDs account for more half of all recommendations in the dataset. The DVD network is also the most dense, having about 10 recommendations per node, while books and music have about 2 recommendations per node and videos have only a bit more than 1 recommendation per node. Music recommendations reached about the same number of people as DVDs but used more than 5 times fewer recommendations to achieve the same coverage of the nodes. Book recommendations reached by far the most people – 2.8 million. Notice that all networks have a very small number of unique edges. For books, videos and music the number of unique edges is smaller than the number of nodes – this suggests
The Dynamics of Viral Marketing
9 ec 184,188 442,747 82,844 15,331 521,803 bbc 1,919 6,199 348 2 8,468 bec 1,921 41,744 456 74 44,195
Group Book DVD Music Video Full network
nc 53,681 39,699 22,044 4,964 100,460
rc 933,988 6,903,087 295,543 23,555 8,283,753
Table 2: Statistics for the largest connected component of each product group. nc : number of nodes in largest connected component, rc : number recommendations in the component, ec : number of edges in the component, bbc : number of buy bits, bec : number of buy edges in the largest connected component, and bbc and bec are the number of purchase through a buy-bit and a buy-edge, respectively.
that the networks are highly disconnected [ER60]. Back to table 1: given the total number of recommendations r and purchases (bb + be ) in?uenced by recommendations we can estimate how many recommendations need to be independently sent over the network to induce a new purchase. Using this metric books have the most in?uential recommendations followed by DVDs and music. For books one out of 69 recommendations resulted in a purchase. For DVDs it increases to 108 recommendations per purchase and further increases to 136 for music and 203 for video. Table 2 gives more insight into the structure of the largest connected component of each product group’s recommendation network. We performed the same measurements as in table 1 with the di?erence being that we did not use the whole network but only its largest weakly connected component. The table shows the number of nodes n, the number of recommendations rc , and the number of (unique) edges ec in the largest component. The last two columns (bbc and bec ) show the number of purchases resulting in a discount (buy-bit, bbc ) and the number of purchases through buy-edges (bec ) in the largest connected component. First, notice that the largest connected components are very small. DVDs have the largest - containing 4.9% of the nodes, books have the smallest at 1.78%. One would also expect that the fraction of the recommendations in the largest component would be proportional to its size. We notice that this is not the case. For example, the largest component in the full recommendation network contains 2.54% of the nodes and 52.9% of all recommendations, which is the result of heavy bias in DVD recommendations. Breaking this down by product categories we see that for DVDs 84.3% of the recommendations are in largest component (which contains 4.9% of all DVD nodes), vs. 16.3% for book recommendations (component size 1.79%), 20.5% for music recommendations (component size 2.77%), and 8.4% for video recommendations (component size 2.1%). This shows that the dynamic in the largest component is very much di?erent from the rest of the network. Especially for DVDs we can see that a very small fraction of users generated most of the recommendations.
4.1
Recommendation network over time
The recommendations that occurred were exchanged over an existing underlying social network. In the real world, it is estimated that any two people on the globe
10
J. Leskovec et al.
12 size of giant component 10 8 6 4 2
x 10
4
4
x 10
6
2 # nodes 0 0 1.7*106m 10 20 m (month) by month quadratic fit 1 2 number of nodes 3 x 10 4
6
0 0
Figure 1: (a) The size of the largest connected component of customers over time. The inset shows the linear growth in the number of customers n over time.
are connected via a short chain of acquaintances - popularly known as the small world phenomenon [TM69]. We examined whether the edges formed by aggregating recommendations over all products would similarly yield a small world network, even though they represent only a small fraction of a person’s complete social network. We measured the growth of the largest weakly connected component over time, shown in Figure 1. Within the weakly connected component, any node can be reached from any other node by traversing (undirected) edges. For example, if u recommended product x to v , and w recommended product y to v , then uand w are linked through one intermediary and thus belong to the same weakly connected component. Note that connected components do not necessarily correspond to communities (clusters) which we often think of as densely linked parts of the networks. Nodes belong to same component if they can reach each other via an undirected path regardless of how densely they are linked. Figure 1 shows the size of the largest connected component, as a fraction of the total network. The largest component is very small over all time. Even though we compose the network using all the recommendations in the dataset, the largest connected component contains less than 2.5% (100,420) of the nodes, and the second largest component has only 600 nodes. Still, some smaller communities, numbering in the tens of thousands of purchasers of DVDs in categories such as westerns, classics and Japanese animated ?lms (anime), had connected components spanning about 20% of their members. The insert in ?gure 1 shows the growth of the customer base over time. Surprisingly it was linear, adding on average 165,000 new users each month, which is an indication that the service itself was not spreading epidemically. Further evidence of non-viral spread is provided by the relatively high percentage (94%) of users who made their ?rst recommendation without having previously received one.
n
The Dynamics of Viral Marketing
11
10 N(x = s ) (Count) 10 10 10
4
= 8.3e3 x
?1.90
R =0.93 N(x = s ) (Count)
2
10 10 10 10
4
= 6.6e3 x
?1.96
R =0.93 N(x = s ) (Count)
2
10
3
= 2.0e3 x
?1.76
R =0.90
2
3
3
10
2
c
c
2
2
c
1
1
10
1
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
10 0 10
0
10 10 10 sc (Size of merged component)
1
2
3
10
4
(a) LCC growth
(b) Sender in LCC
(c) Sender outside LCC
Figure 2: Growth of the largest connected component (LCC). (a) the distribution of sizes of components when they are merged into the largest connected component. (b) same as (a), but restricted to cases when a member of the LCC sends a recommendation to someone outside the largest component. (c) a sender outside the largest component sends a recommendation to a member of the component.
4.1.1
Growth of the largest connected component
Next, we examine the growth of the largest connected component (LCC). In ?gure 1 we saw that the largest component seems to grow quadratically over time, but at the end of the data collection period is still very small, i.e. only 2.5% of the nodes belong to largest weakly connected component. Here we are not interested in how fast the largest component grows over time but rather how big other components are when they get merged into the largest component. Also, since our graph is directed we are interested in determining whether smaller components become attached to the largest component by a recommendation sent from inside of the largest component. One can think of these recommendations as being tentacles reaching out of largest component to attach smaller components. The other possibility is that the recommendation comes from a node outside the component to a member of the largest component and thus the initiative to attach comes from outside the largest component. We look at whether the largest component grows gradually, adding nodes one by one as the members send out more recommendations, or whether a new recommendation might act as a bridge to a component consisting of several nodes who are already linked by their previous recommendations. To this end we measure the distribution of a component’s size when it gets merged to the largest weakly connected component. We operate under the following setting. Recommendations are arriving over time one by one creating edges between the nodes of the network. As more edges are being added the size of largest connected component grows. We keep track of the currently largest component, and measure how big the separate components are when they get attached to the largest component. Figure 2(a) shows the distribution of merged connected component (CC) sizes. On the x-axis we plot the component size (number of nodes N ) and on the y-axis the number of components of size N that were merged over time with the largest component. We see that a majority of the time a single node (component of size 1) merged with the currently largest component. On the other extreme is the case when a component of 1, 568 nodes merged with the largest component. Interestingly, out of all merged components, in 77% of the cases the source of the
12
J. Leskovec et al.
recommendation comes from inside the largest component, while in the remaining 23% of the cases it is the smaller component that attaches itself to the largest one. Figure 2(b) shows the distribution of component sizes only for the case when the sender of the recommendation was a member of the largest component, i.e. the small component was attached from the largest component. Lastly, Figure 2(c) shows the distribution for the opposite case when the sender of the recommendation was not a member of the largest component, i.e. the small component attached itself to the largest. Also notice that in all cases the distribution of merged component sizes follows a heavy-tailed distribution. We ?t a power-law distribution and note the power-law exponent of 1.90 (?g. 2(a)) when considering all merged components. Limiting the analysis to the cases where the source of the edge that attached a small component to the largest is in the largest component we obtain power-law exponent of 1.96 (?g. 2(b)), and when the edge originated from the small component to attached it to the largest, the power-law exponent is 1.76. This shows that even though in most cases the LCC absorbs the small component, we see that components that attach themselves to the LCC tend to be larger (smaller power-law exponent) than those attracted by the LCC. This means that the component sometimes grows a bit before it attaches itself to the largest component. Intuitively, an individual node can get attached to the largest component simply by passively receiving a recommendation. But if it is the outside node that sends a recommendation to someone in the giant component, it is already an active recommender and could therefore have recommended to several others previously, thus forming a slightly bigger component that is then merged. From these experiments we see that the largest component is very active, adding smaller components by generating new recommendations. Most of the time these newly merged components are quite small, but occasionally sizable components are attached.
4.2
Preliminary observations and discussion
Even with these simple counts and experiments we can already make a few observations. It seems that some people got quite heavily involved in the recommendation program, and that they tended to recommend a large number of products to the same set of friends (since the number of unique edges is so small as shown on table 1). This means that people tend to buy more DVDs and also like to recommend them to their friends, while they seem to be more conservative with books. One possible reason is that a book is a bigger time investment than a DVD: one usually needs several days to read a book, while a DVD can be viewed in a single evening. Another factor may be how informed the customer is about the product. DVDs, while fewer in number, are more heavily advertised on TV, billboards, and movie theater previews. Furthermore, it is possible that a customer has already watched a movie and is adding the DVD to their collection. This could make them more con?dent in sending recommendations before viewing the purchased DVD. One external factor which may be a?ecting the recommendation patterns for DVDs is the existence of referral websites (www.dvdtalk.com). On these websites people, who want to buy a DVD and get a discount, would ask for recommendations. This way there would be recommendations made between people who don’t really know
The Dynamics of Viral Marketing
13
938
973
(a) Medical book
(b) Japanese graphic novel
Figure 3: Examples of two product recommendation networks: (a) First aid study guide First Aid for the USMLE Step, (b) Japanese graphic novel (manga) Oh My Goddess!: Mara Strikes Back.
Group Book DVD Music Video Total
Number of nodes Purchases Forward Percent 65,391 15,769 24.2 16,459 7,336 44.6 7,843 1,824 23.3 909 250 27.6 90,602 25,179 27.8
Table 3: Fraction of people that purchase and also recommend forward. Purchases: number of nodes that purchased as a result of receiving a recommendation. Forward: nodes that purchased and then also recommended the product to others.
each other but rather have an economic incentive to cooperate. In e?ect, the viral marketing program is altering, albeit brie?y and most likely unintentionally, the structure of the social network it is spreading on. We were not able to ?nd similar referral sharing sites for books or CDs.
5
5.1
Propagation of recommendations
Forward recommendations
Not all people who accept a recommendation by making a purchase also decide to give recommendations. In estimating what fraction of people that purchase also decide to recommend forward, we can only use the nodes with purchases that resulted in a discount. Table 3 shows that only about a third of the people that purchase also recommend the product forward. The ratio of forward recommendations is much higher for DVDs than for other kinds of products. Videos also have a higher ratio of forward recommendations, while books have the lowest. This shows that people are most keen on recommending movies, possibly for the above mentioned reasons, while more conservative when recommending books and music. Figure 4 shows the cumulative out-degree distribution, that is the number of
14
J. Leskovec et al.
10 10 N(x >= k )
p
6
5
10 10 10
4
3
level 0 ? = 2.6 level 1 ? = 2.0 level 2 ? = 1.5 level 3 ? = 1.2 level 4 ? = 1.2
2
10 0 10
1
10 10 k (recommendations by a person for a product)
p
1
2
10
3
Figure 4: The number of recommendations sent by a user with each curve representing a di?erent depth of the user in the recommendation chain. A power law exponent ? is ?tted to all but the tail, which shows an exponential drop-o? at around 100 recommendations sent). This drop-o? is consistent across all depth levels, and may re?ect either a natural disinclination to send recommendation to over a hundred people, or a technical issue that might have made it more inconvenient to do so. The ?tted lines follow the order of the level number (i.e. top line corresponds to level 0 and bottom to level 4).
level 0 1 2 3 4
prob. buy & forward N/A 0.0069 0.0149 0.0115 0.0082
average out-degree 1.99 5.34 24.43 72.79 111.75
Table 4: Statistics about individuals at di?erent levels of the cascade.
people who sent out at least kp recommendations, for a product. We ?t a power-law to all but the tail of the distribution. Also, notice the exponential decay in the tail of the distribution which could be, among other reasons, attributed to the ?nite time horizon of our dataset. The ?gure 4 shows that the deeper an individual is in the cascade, if they choose to make recommendations, they tend to recommend to a greater number of people on average (the ?tted line has smaller slope ? , i.e. the distribution has higher variance). This e?ect is probably due to only very heavily recommended products producing large enough cascades to reach a certain depth. We also observe, as is shown in Table 4, that the probability of an individual making a recommendation at all (which can only occur if they make a purchase), declines after an initial increase as one gets deeper into the cascade.
The Dynamics of Viral Marketing
15
10
8
10 = 3.4e6 x
?2.30
8
R =0.96 10 Count
6
2
= 4.1e6 x
?2.49
R =0.99
2
10 Count
6
10
4
10
4
10
2
10
2
10 0 10 Number of recommendations
0
10
5
10 0 10
0
10 10 10 Number of purchases
1
2
3
10
4
(a) Recommendations
(b) Purchases
Figure 5: Distribution of the number of recommendations and number of purchases made by a customer.
5.2
Identifying cascades
As customers continue forwarding recommendations, they contribute to the formation of cascades. In order to identify cascades, i.e. the “causal” propagation of recommendations, we track successful recommendations as they in?uence purchases and further recommendations. We de?ne a recommendation to be successful if it reached a node before its ?rst purchase. We consider only the ?rst purchase of an item, because there are many cases when a person made multiple purchases of the same product, and in between those purchases she may have received new recommendations. In this case one cannot conclude that recommendations following the ?rst purchase in?uenced the later purchases. Each cascade is a network consisting of customers (nodes) who purchased the same product as a result of each other’s recommendations (edges). We delete late recommendations — all incoming recommendations that happened after the ?rst purchase of the product. This way we make the network time increasing or causal — for each node all incoming edges (recommendations) occurred before all outgoing edges. Now each connected component represents a time obeying propagation of recommendations. Figure 3 shows two typical product recommendation networks: (a) a medical study guide and (b) a Japanese graphic novel. Throughout the dataset we observe very similar patters. Most product recommendation networks consist of a large number of small disconnected components where we do not observe cascades. Then there is usually a small number of relatively small components with recommendations successfully propagating. This observation is re?ected in the heavy tailed distribution of cascade sizes (see ?gure 6), having a power-law exponent close to 1 for DVDs in particular. We determined the power-law exponent by ?tting a line on log-log scales using the least squares method. We also notice bursts of recommendations (?gure 3(b)). Some nodes recommend to many friends, forming a star like pattern. Figure 5 shows the distribution of
16
J. Leskovec et al.
10
6
= 1.8e6 x?4.98 R2=0.99 10
4
= 3.4e3 x?1.56 R2=0.83
10
4
10
2
10
2
10 0 10
0
10 (a) Book
1
10
2
10 0 10
0
10 10 (b) DVD
1
2
10
3
= 4.9e5 x?6.27 R2=0.97 10
4
= 7.8e4 x?5.87 R2=0.97 10
4
10
2
10
2
10 0 10
0
10 (c) Music
1
10
2
10 0 10
0
10 (d) Video
1
10
2
Figure 6: Size distribution of cascades (size of cascade vs. count). Bold line presents a power-?t.
the recommendations and purchases made by a single node in the recommendation network. Notice the power-law distributions and long ?at tails. The most active customer made 83,729 recommendations and purchased 4,416 di?erent items. Finally, we also sometimes observe ‘collisions’, where nodes receive recommendations from two or more sources. A detailed enumeration and analysis of observed topological cascade patterns for this dataset is made in [LSK06]. Last, we examine the number of exchanged recommendations between a pair of people in ?gure 7. Overall, 39% of pairs of people exchanged just a single recommendation. This number decreases for DVDs to 37%, and increases for books to 45%. The distribution of the number of exchanged recommendations follows a heavy tailed distribution. To get a better understanding of the distributions we show the power-law decay lines. Notice that one gets much stronger decay exponent (distribution has weaker tail) of -2.7 for books and a very shallow power-law exponent of -1.5 for DVDs. This means that even a pair of people exchanges more DVD than book recommendations.
The Dynamics of Viral Marketing
17
10 10 N(x=r ) (Count) 10 10 10 10
6
? = ?2.0 N(x=r ) (Count)
5
10 10 10 10 10
5
? = ?2.7 N(x=r ) (Count)
10 10 10 10 10
5
? = ?1.5
4
4
4
3
3
3
e
e
e
2 1
2
2
1
1
10 0 1 2 3 4 10 10 10 10 10 r (Number of exchanged recommendations)
e
0
10 0 1 2 3 10 10 10 10 r (Number of exchanged recommendations)
e
0
10 0 1 2 3 10 10 10 10 r (Number of exchanged recommendations)
e
0
(a) All
(b) Books
(c) DVD
Figure 7: Distribution of the number of exchanged recommendations between pairs of people.
5.3
The recommendation propagation model
A simple model can help explain how the wide variance we observe in the number of recommendations made by individuals can lead to power-laws in cascade sizes (?gure 6). The model assumes that each recipient of a recommendation will forward it to others if its value exceeds an arbitrary threshold that the individual sets for herself. Since exceeding this value is a probabilistic event, let’s call pt the probability that at time step t the recommendation exceeds the threshold. In that case the number of recommendations Nt+1 at time (t + 1) is given in terms of the number of recommendations at an earlier time by Nt+1 = pt Nt (1)
where the probability pt is de?ned over the unit interval. Notice that, because of the probabilistic nature of the threshold being exceeded, one can only compute the ?nal distribution of recommendation chain lengths, which we now proceed to do. Subtracting from both sides of this equation the term Nt and diving by it we obtain N(t+1) ? Nt = pt ? 1 Nt (2)
Summing both sides from the initial time to some very large time T and assuming that for long times the numerator is smaller than the denominator (a reasonable assumption) we get, up to a unit constant dN = N pt (3)
The left hand integral is just log(N ), and the right hand side is a sum of random variables, which in the limit of a very large uncorrelated number of recommendations is normally distributed (central limit theorem). This means that the logarithm of the number of messages is normally distributed, or equivalently, that the number of messages passed is log-normally distributed. In other words the probability density for N is given by
18
J. Leskovec et al.
P (N ) =
1 ?(log(N ) ? µ)2 ? exp 2? 2 N 2?? 2
(4)
which, for large variances describes a behavior whereby the typical number of recommendations is small (the mode of the distribution) but there are unlikely events of large chains of recommendations which are also observable. Furthermore, for large variances, the lognormal distribution can behave like a power law for a range of values. In order to see this, take the logarithms on both sides of the equation (equivalent to a log-log plot) and one obtains ? (log (N ) ? µ)2 (5) log(P (N )) = ? log(N ) ? log( 2?? 2 ) ? 2? 2 So, for large ? , the last term of the right hand side goes to zero, and since the second term is a constant one obtains a power law behavior with exponent value of minus one. There are other models which produce power-law distributions of cascade sizes, but we present ours for its simplicity, since it does not depend on network topology [GGLNT04] or critical thresholds in the probability of a recommendation being accepted [Wat02].
6
Success of Recommendations
So far we only looked into the aggregate statistics of the recommendation network. Next, we ask questions about the e?ectiveness of recommendations in the recommendation network itself. First, we analyze the probability of purchasing as one gets more and more recommendations. Next, we measure recommendation e?ectiveness as two people exchange more and more recommendations. Lastly, we observe the recommendation network from the perspective of the sender of the recommendation. Does a node that makes more recommendations also in?uence more purchases?
6.1
Probability of buying versus number of incoming recommendations
First, we examine how the probability of purchasing changes as one gets more and more recommendations. One would expect that a person is more likely to buy a product if she gets more recommendations. On the other had one would also think that there is a saturation point – if a person hasn’t bought a product after a number of recommendations, they are not likely to change their minds after receiving even more of them. So, how many recommendations are too many? Figure 8 shows the probability of purchasing a product as a function of the number of incoming recommendations on the product. Because we exclude late recommendations, those that were received after the purchase, an individual counts as having received three recommendations only if they did not make a purchase after the ?rst two, and either purchased or did not receive further recommendations after receiving the third one. As we move to higher numbers of incoming recommendations, the number of observations drops rapidly. For example, there were 5 million cases with 1 incoming recommendation on a book, and only 58 cases where a person got 20
The Dynamics of Viral Marketing
19
0.06 0.05 Probability of Buying Probability of Buying 2 4 6 8 Incoming Recommendations 10
0.08
0.06
0.04 0.03 0.02 0.01 0
0.04
0.02
0
10
20 30 40 50 Incoming Recommendations
60
(a) Books
0.2
0.2
(b) DVD
Probability of Buying
0.15
Probability of Buying
0.15
0.1
0.1
0.05
0.05
0 1
2
3 4 5 6 7 Incoming Recommendations
8
0
2
4 6 8 10 12 14 Incoming Recommendations
16
(c) Music
(d) Video
Figure 8: Probability of buying a book (DVD) given a number of incoming recommendations.
incoming recommendations on a particular book. The maximum was 30 incoming recommendations. For these reasons we cut-o? the plot when the number of observations becomes too small and the error bars too large. We calculate the purchase probabilities and the standard errors of the estimates which we use to plot the error bars in the following way. We regard each point as a binomial random variable. Given the number of observations n, let m be the number of successes, and k (k=n-m) the number of failures. In our case, m is the number of people that ?rst purchased a product after receiving r recommendations on it, and k is the number of people that received the total of r recommendations on a product (till the end of the dataset) but did purchase it, then the estimated probability of p(1 ? p)/n. purchasing is p ˆ = m/n and the standard error sp ˆ is sp ˆ of estimate p ˆ= Figure 8(a) shows that, overall, book recommendations are rarely followed. Even more surprisingly, as more and more recommendations are received, their success decreases. We observe a peak in probability of buying at 2 incoming recommendations and then a slow drop. This implies that if a person doesn’t buy a book after the ?rst recommendation, but receives another, they are more likely to be persuaded by the second recommendation. But thereafter, they are less likely to respond to additional
20
J. Leskovec et al.
recommendations, possibly because they perceive them as spam, are less susceptible to others’ opinions, have a strong opinion on the particular product, or have a di?erent means of accessing it. For DVDs (?gure 8(b)) we observe a saturation around 10 incoming recommendations. This means that with each additional recommendation, a person is more and more likely to be persuaded - up to a point. After a person gets 10 recommendations on a particular DVD, their probability of buying does not increase anymore. The number of observations is 2.5 million at 1 incoming recommendation and 100 at 60 incoming recommendations. The maximal number of received recommendations is 172 (and that person did not buy), but someone purchased a DVD after 169 receiving recommendations. The di?erent patterns between book and DVD recommendations may be a result of the recommendation exchange websites for DVDs. Someone receiving many DVD recommendations may have signed up to receive them for a product they intended to purchase, and hence a greater number of received recommendations corresponds to a higher likelihood of purchase (up to a point).
6.2
Success of subsequent recommendations
Next, we analyze how the e?ectiveness of recommendations changes as one received more and more recommendations from the same person. A large number of exchanged recommendations can be a sign of trust and in?uence, but a sender of too many recommendations can be perceived as a spammer. A person who recommends only a few products will have her friends’ attention, but one who ?oods her friends with all sorts of recommendations will start to loose her in?uence. We measure the e?ectiveness of recommendations as a function of the total number of previously received recommendations from a particular node. We thus measure how spending changes over time, where time is measured in the number of received recommendations. We construct the experiment in the following way. For every recommendation r on some product p between nodes u and v , we ?rst determine how many recommendations node u received from v before getting r. Then we check whether v , the recipient of recommendation, purchased p after the recommendation r arrived. If so, we count the recommendation as successful since it in?uenced the purchase. This way we can calculate the recommendation success rate as more recommendations were exchanged. For the experiment we consider only node pairs (u, v ), where there were at least a total of 10 recommendations sent from u to v . We perform the experiment using only recommendations from the same product group. We decided to set a lower limit on the number of exchanged recommendations so that we can measure how the e?ectiveness of recommendations changes as the same two people exchange more and more recommendations. Considering all pairs of people would heavily bias our ?ndings since most pairs exchange just a few or even just a single recommendation. Using the data from ?gure 7 we see that 91% of pairs of people that exchange at least 1 recommendation exchange less than 10. For books this number increases to 96%, and for DVDs it is even smaller (81%). In the DVD network there are 182 thousand pairs that exchanged more than 10 recommendations, and 70 thousand for the book network. Figure 9 shows the probability of buying as a function of the total number of
The Dynamics of Viral Marketing
21
12
x 10
?3
0.07 0.06 0.05 0.04 0.03 0.02
Probability of buying
8
6
4
5
10 15 20 25 30 35 Exchanged recommendations
40
Probability of buying
10
5
10 15 20 25 30 35 Exchanged recommendations
40
(a) Books
(b) DVD
Figure 9: The e?ectiveness of recommendations with the number of received recommendations.
received recommendations from a particular person up to that point. One can think of x-axis as measuring time where the unit is the number of received recommendations from a particular person. For books we observe that the e?ectiveness of recommendation remains about constant up to 3 exchanged recommendations. As the number of exchanged recommendations increases, the probability of buying starts to decrease to about half of the original value and then levels o?. For DVDs we observe an immediate and consistent drop. We performed the experiment also for video and music, but the number of observations was too low and the measurements were noisy. This experiment shows that recommendations start to lose e?ect after more than two or three are passed between two people. Also, notice that the e?ectiveness of book recommendations decays much more slowly than that of DVD recommendations, ?attening out at around 20 recommendations, compared to around 10 DVD exchanged recommendations. The result has important implications for viral marketing because providing too much incentive for people to recommend to one another can weaken the very social network links that the marketer is intending to exploit.
6.3
Success of outgoing recommendations
In previous sections we examined the data from the viewpoint of the receiver of the recommendation. Now we look from the viewpoint of the sender. The two interesting questions are: how does the probability of getting a 10% credit change with the number of outgoing recommendations; and given a number of outgoing recommendations, how many purchases will they in?uence? One would expect that recommendations would be the most e?ective when recommended to the right subset of friends. If one is very selective and recommends to too few friends, then the chances of success are slim. One the other hand, recommending to everyone and spamming them with recommendations may have limited returns as well.
22
5
0.5 0.4 0.3 0.2 0.1 0 10 20 30 40 50 60 70 Outgoing Recommendations 80
J. Leskovec et al.
0.2 0.25 0.2 0.15 0.1 0.05
Number of Purchases
Number of Purchases
Number of Purchases
0.15
3 2 1 0
0.1
0.05
20 40 60 80 Outgoing Recommendations
100
0
Number of Purchases 5 10 15 20 Outgoing Recommendations
4
0
2
4 6 8 10 Outgoing Recommendations
12
0.25 0.2 0.15 0.1 0.05 0
0.12 0.1 Probability of Credit 0.08 0.06 0.04 0.02
0.1 0.08 0.06 0.04 0.02 0
0.08
Probability of Credit
Probability of Credit 5 10 15 20 Outgoing Recommendations
Probability of Credit
0.06
0.04
0.02
10
20 30 40 50 60 70 Outgoing Recommendations
80
0
20 40 60 80 Outgoing Recommendations
100
0
2
4 6 8 10 Outgoing Recommendations
12
(a) Books
(b) DVD
(c) Music
(d) Video
Figure 10: Top row: Number of resulting purchases given a number of outgoing recommendations. Bottom row: Probability of getting a credit given a number of outgoing recommendations.
The top row of ?gure 10 shows how the average number of purchases changes with the number of outgoing recommendations. For books, music, and videos the number of purchases soon saturates: it grows fast up to around 10 outgoing recommendations and then the trend either slows or starts to drop. DVDs exhibit di?erent behavior, with the expected number of purchases increasing throughout. These results are even more interesting since the receiver of the recommendation does not know how many other people also received the recommendation. Thus the plots of ?gure 10 show that there are interesting dependencies between the product characteristics and the recommender that manifest through the number of recommendations sent. It could be the case that widely recommended products are not suitable for viral marketing (we ?nd something similar in section 9.2), or that the recommender did not put too much thought into who to send the recommendation to, or simply that people soon start to ignore mass recommenders. Plotting the probability of getting a 10% credit as a function of the number of outgoing recommendations, as in the bottom row of ?gure 10, we see that the success of DVD recommendations saturates as well, while books, videos and music have qualitatively similar trends. The di?erence in the curves for DVD recommendations points to the presence of collisions in the dense DVD network, which has 10 recommendations per node and around 400 per product — an order of magnitude more than other product groups. This means that many di?erent individuals are recommending to the same person, and after that person makes a purchase, even though all of them made a ‘successful recommendation’ by our de?nition, only one of them receives a credit.
6.4
Probability of buying given the total number of incoming recommendations
The collisions of recommendations are a dominant feature of the DVD recommendation network. Book recommendations have the highest chance of getting a credit, but DVD recommendations cause the most purchases. So far it seems people are
The Dynamics of Viral Marketing
23
0.1 0.08 0.06 0.04 0.02 0
Probability of Buying
Probability of Buying
0.1
0.05
2
4 6 8 10 12 Total Incomming Products
14
0
5 10 15 Total Incomming Products
20
(a) Books
0.07 0.06 Probability of Buying Probability of Buying 0.05 0.04 0.03 0.02 0.01 0 5 10 15 Total Incomming Products 20 0 0.03 0.04
(b) DVD
0.02
0.01
5 10 15 Total Incomming Products
20
(c) Music
(d) Video
Figure 11: The probability of buying a product given a number of di?erent products a node got recommendations on.
very keen on recommending various DVDs, while very conservative on recommending books. But how does the behavior of customers change as they get more involved into the recommendation network? We would expect that most of the people are not heavily involved, so their probability of buying is not high. In the extreme case we expect to ?nd people who buy almost everything they get recommendations on. There are two ways to measure the involvedness of a person in the network: by the total number of incoming recommendations (on all products) or the total number of di?erent products they were recommended. For every purchase of a book at time t, we count the number of di?erent books (DVDs, ...) the person received recommendations for before time t. As in all previous experiments we delete late recommendations, i.e. recommendations that arrived after the ?rst purchase of a product. We show the probability of buying as a function of the number of di?erent products recommended in Figure 11. Figure A-2 plots the same data but with the total number of incoming recommendations on the x-axis. We calculate the error bars as described in section 6.1. The number of observations is large enough (error bars are su?ciently small) to draw conclusions about the trends observed in the ?gures. For example, there are more than 15, 000 observations (users) that had 15 incoming DVD recommendations.
24
0.2
J. Leskovec et al.
0.08 Probability of Buying
Probability of Buying 0.15
0.06
0.04
0.1
0.02
0.05
0
10 20 30 40 50 Total Incomming Recommendations
0
5 10 15 20 25 30 35 Total Incomming Recommendations
40
(a) Books
0.08 0.04
(b) DVD
Probability of Buying
0.06
Probability of Buying 5 10 15 20 25 30 35 Total Incomming Recommendations 40
0.03
0.04
0.02
0.02
0.01
0
0
5 10 15 Total Incomming Recommendations
20
(c) Music
(d) Video
Figure 12: Probability of buying a product given a total number of incoming recommendations on all products.
Notice that trends are quite similar regardless whether we measure how involved is the user in the network by counting the number of products recommended (?gure 11) or the number of incoming recommendations (?g. A-2). We observe two distinct trends. For books and music (?gures 11 and A-2, (a) and (c)) the probability of buying is the highest when a person got recommendations on just 1 item, as the number of incoming recommended products increases to 2 or more the probability of buying quickly decreases and then ?attens. Movies (DVDs and videos) exhibit di?erent behavior (?gure 11 and A-2, (b) and (d)). A person is more likely to buy the more recommendations she gets. For DVDs the peak is at around 15 incoming products, while for videos there is no such peak – the probability remains fairly level. Interestingly for DVDs the distribution reaches its low at 2 and 3 items, while for videos it lies somewhere between 3 and 8 items. The results suggest that books and music buyers tend to be conservative and focused. On the other hand there are people who like to buy movies in general. One could hypothesize that buying a book is a larger investment of time and e?ort than buying a movie. One can ?nish a movie in an evening, while reading a book requires more e?ort. There are also many more book and music titles than movie titles. The other di?erence between the book and music recommendations in compar-
The Dynamics of Viral Marketing
25
ison to movies are the recommendation referral websites where people could go to get recommendations. One could see these websites as recommendation subscription services – posting one’s email on a list results in a higher number of incoming recommendations. For movies, people with a high number of incoming recommendations “subscribed” to them and thus expected/wanted the recommendations. On the other hand people with high numbers of incoming book or music recommendations did not “sign up” for them, so they may perceive recommendations as spam and thus the in?uence of recommendations drops. Another evidence of the existence of recommendations referral websites includes the DVD recommendation network degree distribution. The DVDs follow a power law degree distribution with an exception of a peak at out-degree 50. Other plots of DVD recommendation behavior also exhibited abnormalities at around 50 recommendations. We believe these can be attributed to the recommendation referral websites.
7
Timing of recommendations and purchases
The recommendation referral program encourages people to purchase as soon as possible after they get a recommendation, since this maximizes the probability of getting a discount. We study the time lag between the recommendation and the purchase of di?erent product groups, e?ectively how long it takes a person to receive a recommendation, consider it, and act on it. We present the histograms of the “thinking time”, i.e. the di?erence between the time of purchase and the time the last recommendation was received for the product prior to the purchase (?gure 13). We use a bin size of 1 day. Around 35%-40% of book and DVD purchases occurred within a day after the last recommendation was received. For DVDs 16% purchases occur more than a week after the last recommendation, while this drops to 10% for books. In contrast, if we consider the lag between the purchase and the ?rst recommendation, only 23% of DVD purchases are made within a day, while the proportion stays the same for books. This re?ects a greater likelihood for a person to receive multiple recommendations for a DVD than for a book. At the same time, DVD recommenders tend to send out many more recommendations, only one of which can result in a discount. Individuals then often miss their chance of a discount, which is re?ected in the high ratio (78%) of recommended DVD purchases that did not a get discount (see table 1, columns bb and be ). In contrast, for books, only 21% of purchases through recommendations did not receive a discount. We also measure the variation in intensity by time of day for three di?erent activities in the recommendation system: recommendations (?gure 14(a)), all purchases (?gure 14(b)), and ?nally just the purchases which resulted in a discount (?gure 14(c)). Each is given as a total count by hour of day. The recommendations and purchases follow the same pattern. The only small di?erence is that purchases reach a sharper peak in the afternoon (after 3pm Paci?c Time, 6pm Eastern time). This means that the willingness to recommend does not change with time, since about a constant fraction of purchases also result in recommendations sent (plots 14(a) and (b) follow the same shape). The purchases that resulted in a discount (?g. 14(c)) look like a negative image of the ?rst two ?gures. If recommendations would have no e?ect then plot (c) should follow the same shape as (a) and (b), since a fraction of people that buy would
26
J. Leskovec et al.
0.35
0.5 Proportion of Purchases 0.4 0.3 0.2 0.1 0
Proportion of Purchases
0.3 0.25 0.2 0.15 0.1 0.05 0 1 2 3 4 5 6 Lag [day] 7 >7
1
2
3
4 5 6 Lag [day]
7
>7
(a) Books
(b) DVD
Figure 13: The time between the recommendation and the actual purchase. We use all purchases.
10 8 Recommendtions
x 10
5
2
x 10
4
7000 6000 Discounted Purchases
1.5
6 4 2 0
All Purchases
5000 4000 3000 2000 1000
1
0.5
0
5
10 15 Hour of the Day
20
25
0
0
5
10 15 Hour of the Day
20
25
0
0
5
10 15 Hour of the Day
20
25
(a) Recommendations
(b) Purchases
(c) Purchases with Discount
Figure 14: Time of day for purchases and recommendations. (a) shows the distribution of recommendations over the day, (b) shows all purchases and (c) shows only purchases that resulted in a discount.
become ?rst buyers, i.e. the more recommendations sent, the more ?rst buyers and thus discounts. However, this does not seem to be the case. The number of purchases with discount is the high when the number of purchases is small. This means that most of discounted purchases happened in the morning when the tra?c (number of purchases/recommendations) on the retailer’s website was low. This makes sense since most of the recommendations happened during the day, and if the person wanted to get the discount by being the ?rst one to purchase, she had the highest chances when the tra?c on the website was the lowest. There are also other factors that come into play here. Assuming that recommendations are sent to people’s personal (non-work) email addresses, then people probably check these email accounts for new email less regularly while at work. So checking personal email while at work and reacting to a recommendation would mean higher chances of getting a discount. Second, there are also network e?ects, i.e. the more recommendations sent, the higher chance of recommendation collision, the lower
The Dynamics of Viral Marketing
27
chance of getting discount, since one competes with the larger set of people.
8
Recommendations and communities of interest
Social networks are a product of the contexts that bring people together. The context can be a shared interest in a particular topic or kind of a book. Sometimes there are circumstances, such as a speci?c job or religious a?liation, that would make people more likely to be interested in the same type of book or DVD. We ?rst apply a community discovery algorithm to automatically detect communities of individuals who exchange recommendations with one another and to identify the kinds of products each community prefers. We then compare the e?ectiveness of recommendations across book categories, showing that books on di?erent subjects have varying success rates.
8.1
Communities and purchases
In aggregating all recommendations between any two individuals in Section 4.1 we showed that the network consists of one large component, containing a little over 100,000 customers, and many smaller components, the largest of which has 634 customers. However, knowing that a hundred thousand customers are linked together in a large network does not reveal whether a product in a particular category is likely to di?use through it. Consider for example a new science ?ction book one would like to market by word-of-mouth. If science ?ction fans are scattered throughout the network, with very few recommendations shared between them, then recommendations about the new book are unlikely to di?use. If on the other hand one ?nds one or more science ?ction communities, where sci-? fans are close together in the network because they exchange recommendations with one another, then the book recommendation has a chance of spreading by word-of-mouth. In the following analysis, we use a community ?nding algorithm [CNM04] in order to discover the types of products that link customers and so de?ne a community. The algorithm breaks up the component into parts, such that the modularity Q, Q = (number of edges within communities) ? (expected number of such edges), (6) is maximized. In other words, the algorithm identi?es communities such that individuals within those communities tend to preferentially exchange recommendations with one another. The results of the community ?nding analysis, while primarily descriptive, illustrate both the presence of communities whose members are linked by their common interests, and the presence cross-cutting interests between communities. Applying the algorithm to the largest component, we identify many small communities and a few larger ones. The largest contains 21,000 nodes, 5,000 of whom are senders of a relatively modest 335,000 recommendations. More interesting than simply observing the size of communities is discovering what interests bring them together. We identify those interests by observing product categories where the number of recommendations within the community is signi?cantly higher than it is for the overall customer population. Let pc be the proportion of all recommendations that fall within a particular
28 # nodes 735 710 667 653 541 502 388 309 192 163 # senders 74 179 181 121 112 104 77 81 30 48
J. Leskovec et al.
topics books: American literature, poetry sci-? books, TV series DVDs, alternative rock music music: dance, indie discounted DVDs books: art & photography, web development, graphical design, sci-? books: sci-? and other books: Christianity and Catholicism books: business and investing, computers, Harry Potter books: parenting, women’s health, pregnancy books: comparative religion, Egypt’s history, new age, role playing games
Table 5: A sample of the medium sized communities present in the largest component
product category c. Then for a set of individuals sending xg recommendations, we would expect by chance that xg ? pc ± xg ? pc ? (1 ? pc ) would fall within category c. We note the product categories for which the observed number of recommendations in the community is many standard deviations higher than expected. For example, compared to the background population, the largest community is focused on a wide variety of books and music. In contrast, the second largest community, involving 10,412 individuals (4,205 of whom are sending over 3 million recommendations), is predominantly focused on DVDs from many di?erent genres, with no particular emphasis on anime. The anime community itself emerges as a highly unusual group of 1,874 users who exchanged over 3 million recommendations. Perhaps the most interesting are the medium sized communities, some of which are listed in Table 5, having between 100 and 1000 members and often re?ecting speci?c interests. Among the hundred or so medium communities, we found, for example, several communities focusing on Christianity. While some of the Christian communities also shared an interest in children’s books, broadway musicals, and travel to Italy, others focused on prayer and bibles, still others also enjoyed DVDs of the Simpsons TV series, and others still took an interest in Catholicism, occult spirituality and kabbalah. Communities were usually centered around a product group, such as books, music, or DVDs, but almost all of them shared recommendations for all types of products. The DVD communities ranged from bargain shoppers purchasing discounted comedy and action DVDs to smaller anime or independent movie communities, to a group of customers purchasing predominantly children’s movies. One community focused heavily on indie music, and imported dance and club music. Another seemed to center around intellectual pursuits, including reading books on sociology, politics, arti?cial intelligence, mathematics, and media culture, listening to classical music and watching neo-noir ?lm. Several communities centered around business and investment books and frequently also recommended books on computing. One business and investment community included fans of the Harry Potter ?ction series, while another enjoyed science ?ction and adventure DVDs. One of communities with the most particular interests recommended not only business and investing books to one another, but also an unusual number of books on terrorism, bacteriology, and military history. A community of what one can presume are web designers recommended books to one
The Dynamics of Viral Marketing
29
another on art and photography, web development, graphical design, and Ray Bradbury’s science ?ction novels. Several sci-? TV series such as Bu?y the Vampire Slayer and Star Trek appeared prominently in a few communities, while Stephen King and Douglas Clegg featured in a community recommending horror, sci-?, and thrillers to one another. One community focused predominantly on parenting, women’s health and pregnancy, while another recommended a variety of books but especially a collection of cookie baking recipes. Going back to components in the network that were disconnected from the largest component, we ?nd similar patterns of homophily, the tendency of like to associate with like. Two of the components recommended technical books about medicine, one focused on dance music, while some others predominantly purchased books on business and investing. Given more time, it is quite possible that one of the customers in one of these disconnected components would have received a recommendation from a customer within the largest component, and the two components would have merged. For example, a disconnected component of medical students purchasing medical textbooks might have sent or received a recommendation from the medical community within the largest component. However, the medical community may also become linked to other parts of the network through a di?erent interest of one of its members. At the very least many communities, no matter their focus, will have recommendations for children’s books or movies, since children are a focus for a great many people. The community ?nding algorithm on the other hand is able to break up the larger social network to automatically identify groups of individuals with a particular focus or a set of related interests. Now that we have shown that communities of customers recommend types of products re?ecting their interests, we will examine whether these di?erent kinds of products tend to have di?erent success rates in their recommendations.
8.2
Recommendation e?ectiveness by book category
Some contexts result in social ties that are more e?ective at conducting an action. For example, in small world experiments, where participants attempt to reach a target individual through their chain of acquaintances, profession trumped geography, which in turn was more useful in locating a target than attributes such as religion or hobbies [KB78, TM69]. In the context of product recommendations, we can ask whether a recommendation for a work of ?ction, which may be made by any friend or neighbor, is more or less in?uential than a recommendation for a technical book, which may be made by a colleague at work or school. Table 6 shows recommendation trends for all top level book categories by subject. For clarity, we group the results by 4 di?erent category types: ?ction, personal/leisure, professional/technical, and non?ction/other. Fiction encompasses categories such as Sci-Fi and Romance, as well as children’s and young adult books. Personal/Leisure encompasses everything from gardening, photography and cooking to health and religion. First, we compare the relative number of recommendations to reviews posted on the site (column cav /rp1 of table 6). Surprisingly, we ?nd that the number of people making personal recommendations was only a few times greater than the number of people posting a public review on the website. We observe that ?ction books have
30 category Books general Fiction Children Literature Mystery Science ?ction Romance Teens Comics Horror Personal Religion Health/Body History Home/Garden Entertainment Arts/Photo Travel Sports Parenting Cooking Outdoors Professional Professional Business Science Computers Medicine Engineering Law Other Non?ction Reference Biographies np 370230 46,451 41,682 10,734 10,008 6,317 5,857 3,565 2,773 43,423 33,751 28,458 19,024 18,724 17,153 12,670 10,183 8,324 7,655 6,413 41,794 29,002 25,697 18,941 16,047 10,312 5,176 55,868 26,834 18,233 n 2,860,714 390,283 502,179 123,392 175,168 60,902 81,260 46,564 48,321 441,263 572,704 28,3406 180,009 258,142 179,074 113,939 120,103 182,792 146,522 59,764 459,889 476,542 271,391 375,712 175,520 107,255 53,182 560,552 371,959 277,356 cc 1.87 2.82 3.06 6.03 6.17 5.65 5.72 11.70 9.35 1.89 1.54 2.74 2.91 3.65 3.49 3.91 1.74 0.73 3.02 2.23 1.72 1.55 2.64 2.22 1.08 1.30 2.64 2.03 1.94 2.80 rp1 5.28 6.44 13.09 20.14 19.90 12.81 20.52 4.76 21.26 3.87 4.34 4.34 1.78 3.48 1.56 2.74 3.36 4.71 3.14 1.93 1.91 3.61 2.41 4.51 1.41 1.43 1.89 3.13 2.49 7.65 vav 4.32 4.52 4.30 4.08 4.15 4.17 4.36 4.36 4.16 4.45 4.41 4.30 4.31 4.29 4.42 4.26 4.34 4.42 4.45 4.42 4.30 4.22 4.30 3.98 4.40 4.14 4.25 4.29 4.19 4.34
J. Leskovec et al.
cav / rp1 1.41 1.12 0.57 0.36 0.64 0.52 0.41 2.03 0.44 1.73 2.39 1.27 3.48 2.26 3.85 1.87 1.99 2.57 3.49 2.50 3.22 2.94 2.42 3.10 4.19 3.85 2.67 1.89 3.04 0.90
pm 14.95 8.76 11.87 9.60 10.39 6.99 9.56 10.47 9.60 9.99 13.96 18.00 15.37 13.97 20.95 13.27 13.97 11.87 13.97 15.00 32.50 20.99 28.00 34.95 39.95 59.95 24.95 18.95 17.47 14.00
b ? 100 3.12 2.06** 2.82* 2.40** 2.34** 1.78** 1.94** 2.30* 1.81** 3.13 3.04 2.84 2.26** 2.66* 2.87 2.39** 2.26** 2.81 2.38* 3.05 4.54** 3.62** 3.90** 3.61** 5.68** 4.10** 3.66* 3.28** 3.21 2.96
Table 6: Statistics by book category: np :number of products in category, n number of customers, cc percentage of customers in the largest connected component, rp1 avg. # reviews in 2001 – 2003, rp2 avg. # reviews 1st 6 months 2005, vav average star rating, cav average number of people recommending product, cav /rp1 ratio of recommenders to reviewers, pm median price, b ratio of the number of purchases resulting from a recommendation to the number of recommenders. The symbol ** denotes statistical signi?cance at the 0.01 level, * at the 0.05 level.
The Dynamics of Viral Marketing
31
relatively few recommendations compared to the number of reviews, while professional and technical books have more recommendations than reviews. This could re?ect several factors. One is that people feel more con?dent reviewing ?ction than technical books. Another is that they hesitate to recommend a work of ?ction before reading it themselves, since the recommendation must be made at the point of purchase. Yet another explanation is that the median price of a work of ?ction is lower than that of a technical book. This means that the discount received for successfully recommending a mystery novel or thriller is lower and hence people have less incentive to send recommendations. Next, we measure the per category e?cacy of recommendations by observing the ratio of the number of purchases occurring within a week following a recommendation to the number of recommenders for each book subject category (column b of table 6). On average, only 2% of the recommenders of a book received a discount because their recommendation was accepted, and another 1% made a recommendation that resulted in a purchase, but not a discount. We observe marked di?erences in the response to recommendation for di?erent categories of books. Fiction in general is not very e?ectively recommended, with only around 2% of recommenders succeeding. The e?cacy was a bit higher (around 3%) for non-?ction books dealing with personal and leisure pursuits. Perhaps people generally know what their friends’ leisure interests are, or even have gotten to know them through those shared interests. On the other hand they may not know as much about each others’ tastes in ?ction. Recommendation success is highest in the professional and technical category. Medical books have nearly double the average rate of recommendation acceptance. This could be in part attributed to the higher median price of medical books and technical books in general. As we will see in Section 9.2, a higher product price increases the chance that a recommendation will be accepted. Recommendations are also more likely to be accepted for certain religious categories: 4.3% for Christian living and theology and 4.8% for Bibles. In contrast, books not tied to organized religions, such as ones on the subject of new age (2.5%) and occult (2.2%) spirituality, have lower recommendation e?ectiveness. These results raise the interesting possibility that individuals have greater in?uence over one another in an organized context, for example through a professional contact or a religious one. There are exceptions of course. For example, Japanese anime DVDs have a strong following in the US, and this is re?ected in their frequency and success in recommendations. Another example is that of gardening. In general, recommendations for books relating to gardening have only a modest chance of being accepted, which agrees with the individual prerogative that accompanies this hobby. At the same time, orchid cultivation can be a highly organized and social activity, with frequent ‘shows’ and online communities devoted entirely to orchids. Perhaps because of this, the rate of acceptance of orchid book recommendations is twice as high as those for books on vegetable or tomato growing.
9
Products and recommendations
We have examined the properties of the recommendation network in relation to viral marketing. Now we focus on the products themselves and their characteristics that determine the success of recommendations.
32
J. Leskovec et al.
9.1
How long is the long tail?
Recently a ‘long tail’ phenomenon has been observed, where a large fraction of purchases are of relatively obscure items where each of them sells in very low numbers but there are many of those items. On Amazon.com, somewhere between 20 to 40 percent of unit sales fall outside of its top 100,000 ranked products [BHS03]. Considering that a typical brick and mortar store holds around 100,000 books, this presents a signi?cant share. A streaming-music service streams more tracks outside than inside its top 10,000 tunes [Ano05]. We performed a similar experiment using our data. Since we do not have direct sales data we used the number of successful recommendations as a proxy to the number of purchases. Figure 15 plots the distribution of the number of purchases and the number of recommendations per product. Notice that both the number of recommendations and the number of purchases per product follow a heavy-tailed distribution and that the distribution of recommendations has a heavier tail. Interestingly, ?gure 15(a) shows that just the top 100 products account for 11.4% of the all sales (purchases with discount), and the top 1000 products amount to 27% of total sales through the recommendation system. On the other hand 67% of the products have only a single purchase and they account for 30% of all sales. This shows that a signi?cant portion of sales come from products that sell very few times. Recently there has been some debate about the long tail [Gom06, And06]. Some argue that the presence of the long tail indicates that niche products with low sales are contributing signi?cantly to overall sales online. We also ?nd that the tail is a bit longer than the usual 80-20 rule, with the top 20% of the products contributing to about half the sales. It is important to note, however, that our observations do not re?ect the total sales of the products on the website, since they include only successful recommendations that resulted in a discount. This incorporates both a bias in the kind of product that is likely to be recommended, and in the probability that a recommendation for that kind of product is accepted. If we look at the distribution in the number of recommendations per product, shown in Figure 15(b), we observe an even more skewed distribution. 30% of the products have only a single recommendation and the top 56,000 most recommended products (top 10%) account for 84% of all recommendations. This is consistent with our previous observations some types of products, e.g. anime DVDs, are more heavily recommended than others. Next we examine the distribution of the product recommendation success rate. Out of more than half a million products we took all the products with at least a single purchase, of which there are 41,000 (7%). Figure 16 shows the success rate (purchases/recommendations). Notice that the distribution is not heavy tailed and has a mode at around 1.3% recommendation success rate. 55% of the products have a success rate bellow 5% and there are around 14% of the products that have a recommendation success rate higher than 20%.
9.2
Modeling the product recommendation success
So far we have seen that some products generate many recommendations and some have a better return than others on those recommendations, but one question still remains: what determines the product’s viral marketing success? We present a model
The Dynamics of Viral Marketing
33
10
4
Data = 4.0e4 x?2.49 R2=0.98
10 Count
4
Data = 9.3e5 x?1.91 R2=0.95
10 Count
3
10
2
10
2
10
1
10 0 10
0
10 10 10 Number of purchases per product
1
2
3
10
4
10
0
10
0
10 Number of recommendation per product
5
(a) Purchases
(b) Recommendations
Figure 15: Distribution of number of purchases and recommendations of a product. (a) shows the number of purchases that resulted in a discount per product, and (b) shows the distribution of the number of recommendations per product.
1200
10
Data Moving average
4
Number of products
1000 Number of products 800 600 400 200 0
Data Moving average 10
3
10
2
10
1
5 10 15 Recommendation success rate [%]
20
10 ?1 10
0
10 10 Recommendation success rate [%]
0
1
(a) Success rate (linear scale)
(b) Success rate (log scale)
Figure 16: Distribution of product recommendation success rates. Both plots show the same data: (a) on a linear (lin-lin) scale, and (b) on a logarithmic (log-log) scale. The bold line presents the moving average smoothing.
which characterizes product categories for which recommendations are more likely to be accepted. We use a regression of the following product attributes to correlate them with recommendation success: • n: number of nodes in the social network (number of unique senders and receivers) • ns : number of senders of recommendations • nr : number of recipients of recommendations • r: number of recommendations
34 log(s) 1 0.275 0.103 0.310 0.396 0.392 0.185 -0.050 -0.031 log
1 0.907 0.994 0.979 0.981 0.098 0.465 0.064 log(ns ) log(ne ) log(r) log(e)J. Leskovec et al.
log(p)
log(v )
log(t)
log(s) log
log(ns ) log(nr ) log(r) log(e) log(p) log(v ) log(t)1 0.864 0.828 0.831 0.088 0.490 0.071
1.000 0.988 0.990 0.098 0.449 0.061
1 0.999 0.107 0.421 0.056
1 0.106 0.423 0.056
1 -0.053 -0.019
1 0.269
1
Table 7: Pairwise Correlation Matrix of the Books and DVD Product Attributes. log(s): log recommendation success rate, log
: log number of nodes, log(ns ): log number of senders of recommendations, log(nr ): log number of receivers, log(r ): log number of recommendations, log(e): log number of edges, log(p): log price, log(v ): log number of reviews, log(t): log average rating.• e: number of edges in the social network (number of unique (sender, receiver) pairs) • p: price of the product • v : number of reviews of the product • t: average product rating From the original set of the half-million products, we compute a success rate s for the 8,192 DVDs and 50,631 books that had at least 10 recommendation senders and for which a price was given. In section 8.2 we de?ned recommendation success rate s as the ratio of the total number purchases made through recommendations and the number of senders of the recommendations. We decided to use this kind of normalization, rather than normalizing by the total number of recommendations sent, in order not to penalize communities where a few individuals send out many recommendations (?gure 3(b)). Note that in general s could be greater than 1, but in practice this happens extremely rarely (there are only 107 products where s > 1 which were discarded for the purposes of this analysis). Since the variables follow a heavy tailed distribution, we use the following model: s = exp(
i
?i log(xi ) + ?i )
(7)
where xi are the product attributes (as described on previous page), and ?i is random error. We ?t the model using least squares and obtain the coe?cients ?i shown in table 8. With the exception of the average rating, they are all signi?cant, but just the number of recommendations alone accounts for 15% of the variance (taking all eight variables into consideration yields an R2 of 0.30 for books and 0.81 for DVDs). We should also note that the variables in our model are highly collinear, as can be seen
The Dynamics of Viral Marketing
35 DVD Coe?cient ?i 0.929 (0.0100) ** 0.171 (0.0124) ** -0.070 (0.0023) ** -0.360 (0.0104) ** -0.002 (0.0083) 0.251 (0.0088) ** 0.007 (0.0016) ** -0.003 (0.0006) ** 0.000 (0.0009) 0.81
Variable const n ns nr r e p v t R2
Books Coe?cient ?i 1.317 (0.0038) ** -0.579 (0.0060) ** 0.144 (0.0018) ** -0.006 (0.0064) 0.062 (0.0084) ** 0.383 (0.0106) ** 0.013 (0.0003) ** -0.003 (0.0001) ** -0.001 (0.0006) * 0.30
Table 8: Regression Using the Log of the Recommendation Success Rate log(s), as the Dependent Variable for Books and DVDs separately. For each coe?cient we provide the standard error and the statistical signi?cance level (**:0.001, *:0.1). We ?t separate models for books and DVDs.
from the pairwise correlation matrix (table 7). For example, the number of recommendations r is highly negatively correlated with the dependent variable (log(s)) but in the regression model it exhibits positive in?uence on the dependent variable. This is probably due to the fact that the number of recommendations is naturally dependent on the number of senders and number of recipients, but it is the high number of recommendations relative to the number of senders that is of importance. To illustrate the dependencies between the variables we train a Bayesian dependency network [Chi03], and show the learned structure for the combined (Books and DVDs) data in ?gure 17. In this a directed acyclic graph where nodes are variables, and directed edges indicate that the distribution of a child depends on the values taken in the parent variables. Notice that the average rating (t) is not predictive of the recommendation success rate (s). It is no surprise that the number of recommendations r is predictive of number of senders ns . Similarly, the number of edges e is predictive of number of senders ns . Interestingly, price p is only related to the number of reviews v . Number of recommendations r, number of senders ns and price p, are directly predictive of the recommendation success rate s. Returning to our regression model, we ?nd that the numbers of nodes and receivers have negative coe?cients, showing that successfully recommended products are actually more likely to be not so widely popular. The only attributes with positive coe?cients are the number of recommendations r, number of edges e, and price p. This shows that more expensive and more recommended products have a higher success rate. These recommendations should occur between a small number of senders and receivers, which suggests a very dense recommendation network where lots of recommendations are exchanged between a small community of people. These insights could be of use to marketers — personal recommendations are most e?ective in small, densely connected communities enjoying expensive products.
36
J. Leskovec et al.
v t e r s
Figure 17: A Bayesian network showing the dependencies between the variables. s: recommendation success rate, n: number of nodes, ns : number of senders of recommendations, nr : log number of receivers, r : number of recommendations, e: number of edges, p: price, v : number of reviews, t: average rating.
nr n ns p
10
Discussion and Conclusion
Although the retailer may have hoped to boost its revenues through viral marketing, the additional purchases that resulted from recommendations are just a drop in the bucket of sales that occur through the website. Nevertheless, we were able to obtain a number of interesting insights into how viral marketing works that challenge common assumptions made in epidemic and rumor propagation modeling. Firstly, it is frequently assumed in epidemic models (e.g., SIRS type of models) that individuals have equal probability of being infected every time they interact [AM02, Bai75]. Contrary to this we observe that the probability of infection decreases with repeated interaction. Marketers should take heed that providing excessive incentives for customers to recommend products could back?re by weakening the credibility of the very same links they are trying to take advantage of. Traditional epidemic and innovation di?usion models also often assume that individuals either have a constant probability of ‘converting’ every time they interact with an infected individual [GLM01], or that they convert once the fraction of their contacts who are infected exceeds a threshold [Gra78]. In both cases, an increasing number of infected contacts results in an increased likelihood of infection. Instead, we ?nd that the probability of purchasing a product increases with the number of recommendations received, but quickly saturates to a constant and relatively low probability. This means individuals are often impervious to the recommendations of their friends, and resist buying items that they do not want. In network-based epidemic models, extremely highly connected individuals play a very important role. For example, in needle sharing and sexual contact networks these nodes become the “super-spreaders” by infecting a large number of people. But these models assume that a high degree node has as much of a probability of infecting each of its neighbors as a low degree node does. In contrast, we ?nd that there are limits to how in?uential high degree nodes are in the recommendation network. As a person sends out more and more recommendations past a certain number for a
The Dynamics of Viral Marketing
37
product, the success per recommendation declines. This would seem to indicate that individuals have in?uence over a few of their friends, but not everybody they know. We also presented a simple stochastic model that allows for the presence of relatively large cascades for a few products, but re?ects well the general tendency of recommendation chains to terminate after just a short number of steps. Aggregating such cascades over all the products, we obtain a highly disconnected network, where the largest component grows over time by aggregating typically very small but occasionally fairly large components. We observed that the most popular categories of items recommended within communities in the largest component re?ect di?ering interests between these communities. We presented a model which shows that these smaller and more tightly knit groups tend to be more conducive to viral marketing. We saw that the characteristics of product reviews and e?ectiveness of recommendations vary by category and price, with more successful recommendations being made on technical or religious books, which presumably are placed in the social context of a school, workplace or place of worship. A small fraction of the products accounts for a large proportion of the recommendations. Although not quite as extreme in proportions, the number of successful recommendations also varies widely by product. Still, a sizeable portion of successful recommendations were for a product with only one such sale - hinting at a long tail phenomenon. Since viral marketing was found to be in general not as epidemic as one might have hoped, marketers hoping to develop normative strategies for word-of-mouth advertising should analyze the topology and interests of the social network of their customers. Our study has provided a number of new insights which we hope will have general applicability to marketing strategies and to future models of viral information spread.
ACKNOWLEDGEMENTS
We thank anonymous reviewers for their insightful comments. This work was partially supported by the National Science Foundation under grants SENSOR-0329549 IIS-0326322 IIS-0534205. This work is also supported in part by the Pennsylvania Infrastructure Technology Alliance (PITA). Additional funding was provided by a generous gift from Hewlett-Packard. Jure Leskovec was partially supported by a Microsoft Research Graduate Fellowship.
References
[AM02] [And06] [Ano05] [Bai75] Roy M. Anderson and Robert M. May. Infectious diseases of humans: Dynamics and control. Oxford Press, 2002. C. Anderson. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, 2006. Anonymous. Pro?ting from obscurity: What the ”long tail” means for the economics of e-commerce. Economist, 2005. Norman Bailey. The Mathematical Theory of Infectious Diseases and its Applications. Gri?n, London, 1975.
38
J. Leskovec et al.
[Bas69] [BHS03]
Frank Bass. A new product growth for model consumer durables. Management Science, 15(5):215–227, 1969. Erik Brynjolfsson, Yu Hu, and Michael D. Smith. Consumer surplus in the digital economy: Estimating the value of increased product variety at online booksellers. Management Science, 49(11):1580–1596, 2003. D. Bowman and D. Narayandas. Managing customerinitiated contacts with manufacturers: The impact on share of category requirements and word-of-mouth behavior. Journal of Marketing Research, 38(3):281 – 297, August 2001. J. J. Brown and P. H. Reingen. Social ties and word-of-mouth referral behavior. The Journal of Consumer Research, 14(3):350–362, 1987. Po Bronson. Hotmale. Wired Magazine, 6(12), 1998. Kevin Burke. As consumer attitudes shift, so must marketing strategies. 2003. David Maxwell Chickering. Optimal structure identi?cation with greedy search. JMLR, 3:507–554, 2003. Damon Centola and Michael Macy. Complex contagion and the weakness of long ties. ftp://hive.soc.cornell.edu/mwm14/webpage/WLT.pdf, 2005. Judith Chevalier and Dina Mayzlin. The e?ect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43(3):345, 2006. Aaron Clauset, M. E. J. Newman, and Cristopher Moore. Finding community structure in very large networks. Physical Review E, 70:066111, 2004. Arnaud DeBruyn and Gary Lilien. A multi-stage model of word of mouth through electronic referrals. 2004. P. Erd¨ os and A. R´ enyi. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci., 5:17–61, 1960. Jonathan Frenzen and Kent Nakamoto. Structure, cooperation, and the ?ow of market information. Journal of Consumer Research, 20(3):360– 375, December 1993.
[BN01]
[BR87] [Bro98] [Bur03] [Chi03] [CM05]
[CM06]
[CNM04]
[DL04] [ER60] [FN93]
[GGLNT04] D. Gruhl, R. Guha, D. Liben-Nowell, and A. Tomkins. Information di?usion through blogspace. In WWW ’04, 2004. [GLM01] J. Goldenberg, B. Libai, and E. Muller. Talk of the network: A complex systems look at the underlying process of word-of-mouth. Marketing Letters, 3(12):211–223, 2001.
The Dynamics of Viral Marketing
39
[Gom06] [Gra73] [Gra78] [HN06]
L. Gomes. It may be a long time before the long tail is wagging the web. The Wall Street Jounal, 2006. July 26 2006. M. S. Granovetter. The strength of weak ties. American Journal of Sociology, 78:1360–1380, 1973. M. Granovetter. Threshold models of collective behavior. American Journal of Sociology, 83(6):1420–1443, 1978. Petter Holme and M. E. J. Newman. Nonequilibrium phase transition in the coevolution of networks and opinions. Physical Review E, 74:056108, 2006. S. Hill, F. Provost, and C. Volinsky. Network-based marketing: Identifying likely adopters via consumer networks. Statistical Science, 21(2):256– 276, 2006. S. Jurvetson. What exactly is viral marketing? Red Herring, 78:110–112, 2000. P. Killworth and H. Bernard. Reverse small world experiment. Social Networks, 1:159–192, 1978. D. Kempe, J. Kleinberg, and E. Tardos. Maximizing the spread of infuence in a social network. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2003. Jure Leskovec, Lada A. Adamic, and Bernardo A. Huberman. The dynamics of viral marketing. In ACM conference on Electronic Commerce, pages 228–237, 2006. Jure Leskovec, Ajit Singh, and Jon Kleinberg. Patterns of in?uence in a recommendation network. In Paci?c-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2006. G. Linden, B. Smith, and J. York. Amazon.com recommendations: itemto-item collaborative ?ltering. IEEE Internet Computing, 7(1):76–80, 2003. Alan L. Montgomery. Applying quantitative marketing techniques to the internet. Interfaces, 30:90–108, 2001. Matt Richardson and Pedro Domingos. Mining knowledge-sharing sites for viral marketing. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2002. Everett M. Rogers. Di?usion of Innovations. Free Press, New York, fourth edition, 1995. Paul Resnick and Richard Zeckhauser. Trust among strangers in internet transactions: Empirical analysis of ebay’s reputation system. In The Economics of the Internet and E-Commerce. Elsevier Science, 2002.
[HPV06]
[Jur00] [KB78] [KKT03]
[LAH06]
[LSK06]
[LSY03]
[Mon01] [RD02]
[Rog95] [RZ02]
40
J. Leskovec et al.
[SR03]
Mani R. Subramani and Balaji Rajagopalan. Knowledge-sharing and in?uence in online social networks via viral marketing. Communications of the ACM, 46(12):300–307, 2003. David Strang and Sarah A. Soule. Di?usion in organizations and social movements: From hybrid corn to poison pills. Annual Review of Sociology, 24:265–290, 1998. J. Travers and S. Milgram. An experimental study of the small world problem. Sociometry, 32:425–443, 1969. D.J. Watts. A simple model of global cascades on random networks. PNAS, 99(9):4766–5771, Apr 2002. Fang Wu and Bernardo A. Huberman. Social structure and opinion formation. Computational Economics 0407002, EconWPA, July 2004. available athttp://ideas.repec.org/p/wpa/wuwpco/0407002.html. Sha Yang and Greg M. Allenby. Modeling interdependent consumer preferences. Journal of Marketing Research, 40(3):282 – 294, August 2003.
[SS98]
[TM69] [Wat02] [WH04]
[YA03]
The Dynamics of Viral Marketing
41
Appendix
0.35 0.3 Number of Purchases 0.25 0.2 0.15 0.1 0.05 = 0.01 x 0 2
1.09 2
3 2.5 2 1.5 1 0.5
R =0.99 12
Number of Purchases
= 0.02 x1.34 R2=0.98 0 10 20 30 40 Outgoing Recommendations
4 6 8 10 Outgoing Recommendations
0.12 0.1 Probability of credit Probability of credit
1.12 2
0.08 0.07 0.06 0.05 0.04 0.03 0.02 = 0.01 x R =0.99 10 0.01 = 0.01 x
0.49 2
0.08 0.06 0.04 0.02 0 2 4 6 8 Outgoing recommendations
R =0.73 20
5 10 15 Outgoing recommendations
(a) Books
(b) DVD
Figure A-1: Top row: Power ?t to the non-linear part of the number of resulting purchases given a number of outgoing recommendations. Bottom row: Power ?t to the probability of getting a credit given a number of outgoing recommendations.
42
J. Leskovec et al.
0.2
0.08 Probability of Buying
Probability of Buying 0.15
0.06
0.04
0.1
0.02
0.05
0
10 20 30 40 50 Total Incomming Recommendations
0
5 10 15 20 25 30 35 Total Incomming Recommendations
40
(a) Books
0.08 0.04
(b) DVD
Probability of Buying
0.06
Probability of Buying 5 10 15 20 25 30 35 Total Incomming Recommendations 40
0.03
0.04
0.02
0.02
0.01
0
0
5 10 15 Total Incomming Recommendations
20
(c) Music
(d) Video
Figure A-2: Probability of buying a product given a total number of incoming recommendations on all products.
0.4 Proportion of Purchases Proportion of Purchases 1 2 3 4 5 6 Lag [day] 7 >7
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
1
2
3
4 5 6 Lag [day]
7
>7
(a) Music
(b) Video
Figure A-3: The time between the ?rst recommendation and the purchase. The histograms show how long does it take to accumulate su?cient number of recommendations to trigger a purchase. Figure 13 plots the same quantity for Books and DVD. The bin size is 1 day. We use all purchases through recommendations.
The Dynamics of Viral Marketing
43
2500 2000 1500 1000 500 0 0
600 500 400
Count
Count
300 200 100
24
48
72 96 120 144 168 Lag [hours]
0 0
24
48
72 96 120 144 168 Lag [hours]
(a) Books
300 250
(b) DVD
40
30
200 Count
Count
150 100
20
10
50 0 0 24 48 72 96 120 144 168 Lag [hours]
0 0
24
48
72 96 120 144 168 Lag [hours]
(c) Music
(d) Video
Figure A-4: The time between the recommendation and the purchase taking only the recommendations that resulted in a 10% discount. The bin size is 3 hours. The dashed line presents a logarithmic ?t.
44
J. Leskovec et al.
category Anime and Manga Classics Animation Science Fiction & Fantasy Art House & International Television Horror Action and Adventure Mystery and Suspense Military and War Cult Movies Kids and Family Drama Comedy Musicals & Performing Arts Westerns Sports Documentary Educational Music Video and Concerts Special Interests Fitness and Yoga African American Cinema
np 1301 266 237 1410 3185 1133 1125 2058 1683 379 324 1357 3376 2455 1091 234 484 1058 89 2222 963 223 81
n 46941 24922 80092 317420 276142 195948 79744 248674 151101 69180 94049 230300 255544 312033 88665 17612 23191 53538 5532 91657 43225 17160 10609
cc 18.92 25.59 11.99 6.61 7.37 8.17 13.15 7.00 9.28 12.53 11.24 6.70 7.12 6.08 10.24 24.40 16.92 15.24 19.60 8.44 10.42 2.23 17.92
rp1 14.40 9.68 41.90 59.18 24.35 18.95 30.00 39.80 26.73 39.31 37.93 30.96 25.14 26.25 17.07 11.76 8.64 6.12 3.39 8.06 5.83 14.65 16.00
rp1 /rp2 17.17 6.66 19.17 16.66 12.97 11.68 9.10 15.11 10.45 11.14 8.45 12.81 11.02 11.14 11.11 7.30 7.89 9.08 2.63 11.16 7.45 6.66 9.06
vav 4.19 4.18 4.03 3.85 3.95 4.22 3.59 3.80 3.82 4.12 3.89 4.12 3.98 4.02 4.09 3.94 3.97 3.95 3.97 4.09 3.99 3.88 4.15
cav /rp1 2.96 4.16 3.88 2.51 2.22 5.32 1.37 1.96 2.20 2.26 3.34 3.35 2.10 3.30 2.34 2.72 2.49 3.70 5.48 2.88 3.43 2.93 3.41
pm 26.96 22.49 22.49 17.99 22.46 17.99 17.98 17.96 17.98 17.96 17.98 17.98 17.98 17.98 22.48 13.48 17.98 17.99 19.95 17.99 18.74 17.96 17.98
b 28.44** 11.22** 10.43 9.62 9.43* 8.90 8.72 8.42** 7.57 7.41 7.28 6.75 6.72* 6.01** 4.93 4.71* 4.55* 4.24 3.99 3.85 2.62 1.98 1.56
Table A-1: Statistics by DVD genre. * denotes signi?cance at the 0.05 level, ** at the 0.01 level
The Dynamics of Viral Marketing
45
category Anime and Manga Educational Fitness Animation Kids and Family Special Interests Mystery and Suspense Art House & International Science Fiction and Fantasy Documentary Television Music Video & Concerts Musicals & Performing Arts Sports Comedy Drama Military and War Westerns Classics African American Cinema Horror Action and Adventure Cult Movies
np 962 607 920 171 4736 3769 1514 2459 1583 2936 3632 1595 1621 1251 3645 4837 829 487 326 87 935 2390 401
n 5081 6569 24627 9500 84608 36862 13459 24713 29565 18884 31475 14360 22539 7987 55868 52691 10859 3743 3029 1861 6728 25921 5260
cc 9.64 1.64 0.43 4.04 1.13 1.45 9.90 3.52 2.54 1.15 0.95 4.46 3.13 0.49 2.13 1.87 1.13 1.58 0.56 0.64 1.07 1.84 0.65
rp1 13.98 1.97 8.41 61.83 14.26 3.19 30.09 17.54 51.92 3.33 5.13 8.75 13.22 4.07 22.26 21.72 28.54 9.42 8.73 15.53 36.38 33.13 32.06
rp1 /rp2 18.76 10.75 12.09 19.58 12.11 12.73 9.83 10.09 13.76 9.83 12.11 11.26 9.39 9.83 10.60 9.25 9.39 6.01 8.15 7.59 9.02 11.90 7.63
vav 4.39 4.17 4.09 4.29 4.29 4.14 4.01 4.18 4.01 4.21 4.33 4.40 4.20 4.15 4.13 4.15 4.22 4.12 4.12 4.10 3.81 3.96 3.90
cav /rp1 0.26 3.01 1.92 0.36 0.85 1.65 0.14 0.28 0.18 0.95 1.01 0.49 0.51 0.91 0.36 0.26 0.21 0.43 0.51 0.61 0.10 0.17 0.18
pm 17.99 19.95 14.95 17.99 12.98 19.95 14.95 17.99 13.99 19.95 14.95 16.99 19.95 16.99 13.99 14.95 14.95 9.99 14.94 9.99 12.99 13.99 9.99
b ? 100 1.99* 1.59 1.48 1.36 1.16 1.09 1.01 0.84 0.83 0.82 0.71 0.70 0.69 0.69 0.59 0.56 0.56 0.56 0.49 0.49 0.40 0.31 0.30
Table A-2: Statistics for videos in VHS format by genre
46
J. Leskovec et al.
category Broadway and Vocalists Country Rock Alternative Rock Soundtracks Classical Folk Pop Opera and Vocal Miscellaneous Blues Hard Rock and Metal Christian and Gospel Jazz Classic Rock Children s Music Dance and DJ New Age International Latin Music Rap and Hip Hop R&B
np 5423 5876 10717 13405 4491 14223 5244 16764 5402 5823 2987 4787 2977 11868 5711 1755 11332 4219 13139 4634 3996 5965
n 104396 98069 196852 216324 133507 116937 87580 322431 61643 80243 31199 63893 37554 113078 117255 37015 139787 60951 130499 38725 60135 85380
cc 4.25 4.67 4.10 5.12 4.81 5.34 5.33 3.30 6.08 5.71 6.62 4.96 2.02 4.49 4.74 4.89 5.16 5.90 5.02 5.06 3.67 2.78
rp1 6.03 5.50 11.00 13.20 7.92 2.65 4.40 9.55 3.32 3.54 2.76 18.23 5.41 2.91 13.62 3.96 7.14 3.92 3.54 2.57 12.23 8.49
rp1 /rp2 13.86 18.45 10.18 11.24 13.82 11.60 13.54 13.19 12.90 12.31 11.53 7.92 16.75 11.40 6.78 12.52 14.64 13.79 9.52 16.76 9.64 12.90
vav 4.49 4.56 4.40 4.41 4.38 4.52 4.60 4.43 4.48 4.35 4.59 4.33 4.67 4.59 4.29 4.53 4.38 4.54 4.57 4.60 4.38 4.48
cav /rp1 1.68 1.76 0.99 0.81 1.77 1.82 2.05 1.22 1.69 1.90 1.89 0.42 1.20 1.99 7 0.87 2.94 1.05 1.95 1.51 1.75 0.67 0.89
pm 14.49 13.99 14.99 13.99 14.99 15.49 14.99 13.99 15.99 13.98 14.99 14.99 14.99 14.99 13.99 12.32 14.99 14.99 14.99 13.99 14.99 13.98
b 2.01 1.87 1.87 1.87 1.87 1.83 1.81 1.78 1.73 1.62 1.54 1.52 1.51 1.50 1.50 1.47 1.42 1.42 1.32 1.30 1.14 1.13
Table A-3: Statistics by Music Style
doc_947057348.pdf