WO2009021198A9 - Method and apparatus for context-based content recommendation - Google Patents

Method and apparatus for context-based content recommendation Download PDF

Info

Publication number
WO2009021198A9
WO2009021198A9 PCT/US2008/072660 US2008072660W WO2009021198A9 WO 2009021198 A9 WO2009021198 A9 WO 2009021198A9 US 2008072660 W US2008072660 W US 2008072660W WO 2009021198 A9 WO2009021198 A9 WO 2009021198A9
Authority
WO
WIPO (PCT)
Prior art keywords
user
context
current
vector
terms
Prior art date
Application number
PCT/US2008/072660
Other languages
French (fr)
Other versions
WO2009021198A1 (en
Inventor
Scott Brave
Jack Jia
Original Assignee
Baynote Inc
Scott Brave
Jack Jia
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Baynote Inc, Scott Brave, Jack Jia filed Critical Baynote Inc
Priority to JP2010520330A priority Critical patent/JP2010536102A/en
Priority to EP08782688A priority patent/EP2176730A4/en
Priority to CN200880102339.5A priority patent/CN101779180B/en
Publication of WO2009021198A1 publication Critical patent/WO2009021198A1/en
Publication of WO2009021198A9 publication Critical patent/WO2009021198A9/en
Priority to US13/312,665 priority patent/US20120311139A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation

Definitions

  • the invention relates to electronic access to information. More particularly, the invention relates to a method and apparatus for context-based content recommendation.
  • Web site design is a manual attempt to solve the problem of information discovery: to organize information in a way that the designer imagines helps a user find what they are looking for. While effective in some cases, trying to find information in this way often is slow and ineffective as users resort to poking around a site looking for the information they need. Most users actually abandon a site if they do not find what they are looking for within three clicks.
  • One problem is that the site is static.
  • Web analytics has emerged as an attempt to alleviate this problem.
  • Designers can see all of the actions that happen on their site and collect it into reports that aim to provide some guidance on how the site can be redesigned or reconfigured more effectively. While providing some benefit, the information provided is often ambiguous and provides only hints rather than concrete suggestions for improvement. At best, the process is tedious, requires a great deal of manual effort as designers redesign the site in line with learnings, and takes a long time. The feedback loop is thus slow and ineffective.
  • Automatic content recommendation is a completely different strategy that emerged very early in the life of the Web.
  • Search engines such as Google, Yahoo, and Ask, are the common manifestation of such techniques.
  • the basic idea is that the user explicitly describes what they are looking for in the form of a search query, and an automatic process attempts to identify the piece of content, most often a Web page, that best matches their query.
  • the approach for doing this amounts to looking at all possible documents and recommending those where the target query occurs within the text with highest frequency, i.e. keyword match.
  • Modern adaptations of this basic technique add layers of sophistication, e.g. natural language processing, but the key in these approaches is still to use properties of the content itself, e.g. words within the document, to determine the ultimate relevancy ranking.
  • Many variations to this approach exist including, most notably, meta-tagging.
  • the content creator selects a small number of terms to describe the content. These terms are embedded within the content, often as HTML meta-tags, but are not necessarily made visible to the consumer of the content. This is one way to allow search engines to search content that is not text-based, such as video clips. This approach was very common in the late 1990's, but has since fallen out of favor due to the enormous effort required to keep the meta- tags up to date and in-synch with changes to the content.
  • this first content-centric approach on the surface make a lot of sense, i.e. if you want to recommend content, consider the content itself.
  • A-key_ problem with this approach is that it often brings back lots of documents that may be relevant but not useful. Many documents may exhibit a strong keyword match, but are outdated or not truly relevant to the user's current interest. If users do not find a useful result within the first few results, they are most likely going to abandon the search.
  • Keyword match does not really reflect how we find information most efficiently in the real world.
  • the best way to find the information/products/services we are looking for is to ask someone who knows to point us in the right direct.
  • the second phase of content recommendation thus shifts the focus from content to users (see Figure 1).
  • Google's "PageRank” algorithm though we place it in phase 1 , was really a transitional technology that harkened the coming of phase 2.
  • the page rank algorithm's break-through was to consider not only the content of the page itself, but how it had been linked to from other pages by other Web site designers. This represented a form of voting on the importance of Web pages. Thus, pages that were linked to more often were seen as more valuable.
  • Phase 2 of content recommendation is all about the users.
  • the three most well known approaches that fall into phase 2 are: folksonomy, profiling/behavioral targeting, and collaborative filtering.
  • the first, folksonomy represents the most straight-forward addition to phase 1.
  • users are allowed to tag content themselves. So, rather than the Web site designers, or a single designer, being responsible for coming up with the best set of keywords to describe the content, folksonomy lets the community do it. Once this is done, those community created tags essentially become part of the content and can be searched using traditional information retrieval/search techniques developed in phase 1.
  • a big assumption in this approach is that the subset of the community who takes the time to tag the pages explicitly, ultimately produce a description that is valid and representative of the larger community's opinion. This is often not the case.
  • Profiling/Behavioral targeting in its common form also borrows heavily from phase 1 techniques.
  • a profile is built for that user. This profile may, in the simple case, be based on a collection of pages clicked or products purchased.
  • the profile may also make use of the content itself or meta-tags to attempt to discern the user's historical topics of interest. For example, if a user purchased many films tagged as "horror" by content providers in the past, then a behavioral targeting system would tend to recommend more "horror” films to the user.
  • a major assumption here is that a user's historical behaviors are a good predictor of future interest.
  • Collaborative filtering is another user-centric approach which is arguably the most strictly user-centric.
  • users are compared to one another based on common purchases, click histories, or explicit ratings. For example, based on a person's previous ratings of movies on a movie site, find other people who most agree with that person's ratings and recommend other movies that he liked. Standard "people who bought this also bought that" approaches are actually a variation on the collaborative filtering approach, where a user's most recent action serves as the sole basis for identifying similar users.
  • This approach was made popular by Amazon's recommendation engine.
  • a big assumption in this approach is that some global similarity measure between users based on past behavior is a useful way to predict future interest. This is a flawed assumption, however.
  • Explicit measures include asking the user to indicate their liking of a particular piece of content, e.g. on a 1-5 scale.
  • Such approaches are almost always biased because they represent a very small percentage of the population.
  • the people who are taking the time to do these ratings are not representative of the community as a whole. They tend to be very opinionated or reflect a specific personality type that is willing to spend the time to voice their opinion.
  • An embodiment of the invention represents Phase 3 in the evolution of content recommendation (see Figure 1).
  • the idea is to start by understanding the current user's context, i.e.: What is their intent? What are they looking for? Based on this understanding, then find the appropriate peer group representing other users who are most like the current user in the context of this identified interest. From there, find the content that that peer group identifies as most relevant to the current context.
  • Figure 1 is a schematic overview of the evolution of content recommendation leading up to the invention
  • Figure 2 is an architectural schematic diagram showing a method and apparatus for context-based content recommendation according to the invention
  • FIG. 3 is a schematic architectural diagram showing affinity engine integration according to the invention.
  • Figure 4 is a schematic architectural diagram showing a dynamic, adaptive, realtime platform of community wisdom and community guided Web according to the invention
  • Figure 5 is an architectural schematic diagram showing a community wisdom platform according to the invention.
  • Figure 6 is schematic flow diagram showing the process of translating context to recommendations
  • Figure 7 is schematic flow diagram showing the process for achieving a virtual folksonomy according to the invention
  • Figure 8 is a schematic diagram showing topic attraction, topic match, and activeness according to the invention.
  • Figure 9 is a schematic flow diagram showing the behavioral fingerprinting process according to the invention.
  • Figure 10 is a graph showing a long tail marketing model with an architectural overlay according to the invention.
  • Figure 11 is a "Wisdom of Crowds" pseudo-equation
  • Figure 12 is a schematic flow diagram showing an affinity engine memory prediction machine according to the invention.
  • Figure 13 is a screen shot of the welcome screen of the customer portal according to the invention.
  • Figure 14 is a screenshot of a lift report within the customer portal according to the invention.
  • Figure 15 is a screenshot showing community-guided e-commerce according to the invention, including recommendations for competing and complementary products according to the invention
  • Figure 16 is a screenshot showing community-guided e-travel according to the invention.
  • Figure 17 is a screenshot showing community-guided marketing according to the invention.
  • Figure 18 is a screenshot showing social search and community information popup/overlay according to the invention;
  • Figure 19 is a screenshot showing community-guided online support according to the invention;
  • Figure 20 is a screenshot showing a community intranet/knowledge portal according to the invention.
  • Figure 21 is a screenshot showing community-guided media according to the invention.
  • Figure 22 is a screenshot showing community-guided social search for media according to the invention.
  • Figure 23 is a screenshot and schematic diagram showing community-guided cross-site recommendations for media according to the invention.
  • Figure 24 is a screenshot showing community-guided video recommendations according to the invention.
  • Figure 25 is a screenshot showing community-guided topic recommendations according to the invention.
  • Figure 26 is a schematic flow diagram showing the community-guided context- relevant Ad recommendations according to the invention.
  • Figure 27 is a schematic flow diagram showing a live connection according to the invention.
  • Figure 28 is a block schematic diagram showing the system architecture of a preferred embodiment of the invention.
  • Figure 29 is a block schematic diagram showing code snippets of the client integration through JavaScript tags and REST in a preferred embodiment of the invention.
  • Figure 30 is a block schematic diagram showing an AJAX platform according to the invention.
  • the invention represents Phase 3 in the evolution of content recommendation (see Figure 1).
  • the idea is to start by understanding the current user's context, i.e.: What is their intent? What are they looking for? Based on this understanding, then find the appropriate peer group representing other users who are most like the current user in the context of this identified interest. From there, find the content that that peer group identifies as most relevant to the current context.
  • FIG. 2 is an architectural schematic diagram showing a method and apparatus for context-based content recommendation according to the invention.
  • Full-spectrum behavioral fingerprints provide a significant advancement over current state-of-the-art implicit ratings, which essentially amount to click analysis and purchase behavior.
  • They take into account a wide variety of user behaviors including, but not limited to, clicks, time spent on a page, scrolling and mouse movement, explicit actions such as print, email, bookmark, links used, frequency of return visits, and searches performed.
  • Second, all of these behaviors can be cross-correlated with the current user's behaviors on other pages, and also with the rest of the community's and identified peers' behaviors on the current page. From this analysis, a probability that the user has found value in a particular piece of content can be discerned and fed into the learning system.
  • a further aspect of the invention concerns the seamless integration of existing strategies for automatic content recommendation (see Figure 3), including search engines 301 and profiling systems 302, and ad servers 303, as well as systems for manual recommendations including merchandizing rules and systems 304.
  • the invention also seamlessly integrates with other information sources related to the content, such as product catalogs 305, which can be used for purposes of display, filtering, or learning.
  • the system represents a general wisdom platform 501 on top of which many applications can be built (see Figure 4 and Figure 5), such as SocialSearch 502, Content 503 and Product 504 Recommendations, Insights 505, eMail 506, and Live connect 507 which are tailored to various applications, e.g. eCommerce, eMarketing, etc.
  • Additional applications include reports and integrations with bid management systems for search engine optimization and search engine marketing 508, mobile and IPTV 509, and custom applications or mashups 510. Some of these applications are described later.
  • a core advancement in the evolution of content recommendation embodied by invention is the process for identifying and representing the users current topic of interest and converting that interest into a set of useful recommendations and information. The steps in this process are shown in Figure 6 and outlined below:
  • Step 1 When a user comes to a Web site, they immediately begin to establish their current context. They might do this by entering a query into a search box on the site, navigating to a particular section, or may even have established some context before arriving at the site by doing a search on an external search engine, such as Goggle or Yahoo, that led to this site. All of this information is captured by an Observer Tag, i.e. a piece of HTML/JavaScript embedded in the Web site. As the user continues to move through the site, they may also show interest in a particular page or piece of content, based on their implicit actions. Interest is determined based on various behaviors collected by the Observer Tag and analyzed using the invention's Full Spectrum Behavioral Fingerprint technology 601 (described in greater detail below). These pages of interest further contribute to the user's context.
  • an Observer Tag i.e. a piece of HTML/JavaScript embedded in the Web site.
  • Interest is determined based on various behaviors collected by the Observer Tag and analyzed using the invention'
  • the user's current context vector 602 is a hybrid vector of terms and documents with weights on each entry reflecting how strongly that term or document reflects the user's current context.
  • the vector entries corresponding to the terms and phrases entered or clicked are incremented to capture expressed interest.
  • the corresponding entries are decremented or decayed.
  • documents that a user clicks on or indicates interest in, as determined based on their implicit actions increment the corresponding vector entry to a degree based on the level of interest and level of certainty determined by the invention.
  • the result is a representation of the user's current context as a context vector.
  • Step 2 Expand and refine the context vector into an intent vector 603, based on affinities/associations learned from the aggregated wisdom 604 collected from observations on the community as a whole over a longer term.
  • the user may have entered a query about digital SLR cameras and expressed implicit interest in a Nikon page.
  • the entries corresponding to "digital SLR camera” as well as “digital,” “SLR,” and “camera” to a lesser degree are incremented, as is vector entry corresponding to the particular Nikon page of interest.
  • the system looks at affinities between the terms and documents in the context vector to other terms.
  • the community wisdom may have discovered that the term "high- resolution” may be highly associated with both the term “Nikon,” as well as with the specific Nikon page of interest.
  • the intent vector is thus incremented at the entry corresponding to "high-resolution.”
  • other documents that are associated to the terms in the context vector may be discovered based on the community wisdom and are incremented in the intent vector.
  • a Canon camera page may have strong affinities to "SLR camera” and become part of the intent vector to some degree.
  • the affinities 604 that allow the expansion of the context vector into an intent vector are determined by the affinity engine which is described below and in detail in U.S. patent application serial no. 11/319,928, filed 12/27/2005.
  • the affinity engine learns connections between documents and terms, documents and documents, terms and terms, as well as users to other users, documents, and terms by watching all of the implicit behaviors of the user community on the site and by applying behavioral fingerprinting and the use rank algorithm to determine interest and associations.
  • the ability to translate context into intent effectively is a key aspect of the invention.
  • Step 3 Identify the group of users who share affinity to the current intent, as well as those users who exhibit behavior most like the current user within the context of that intent.
  • This peer group is represented by a user vector 605 and each user entry in the vector may have a weight indicating how strong of a peer he is to the current user in this context.
  • the system is also capable of limiting the peer group to those users who match the current user based on a set of predefined attributes, such as age, gender, location, or other demographic variables.
  • the peer group can be limited to those users who visit the site at the same time of day, e.g. morning, afternoon, evening.
  • these attributes can be used to influence the peer weights for users, such as giving slightly more peer weight to those users who also best match the user along these predefined variables.
  • the invention can also learn these weights by comparing behavioral patterns, e.g. documents found useful in the simplest case, within and across each of the predefined attribute groups. Higher intra-group similarity in behavior compared to inter-group similarity indicates the group is differentiated and thus a higher weight of influence for groupness is warranted when influencing the peer group. Similarity can be measured by looking at similarity of documents and terms used through one of many similarity calculations, e.g. cosine similarity, on users' aggregated interest vectors.
  • Step 4 Look at which documents have highest affinity to the identified peers within the current intent 606. To do this the affinity engine looks at a combination of factors, including the overall usefulness of a piece of content, as represented by the activeness vector, learned affinities between terms and content, as represented by term-doc matrices, and predicted navigational patterns, as represented by next-step matrices, associated with those peers the factors are weighted according to their peer weight, and used in aggregate to compute those documents with highest affinity to the current intent. These documents become the unfiltered recommendations 607. Filtering, described in greater detail below, may now be applied to limit or augment those community recommendations.
  • Step 5 Now that the recommendations have been identified and appropriately filtered, a final optional step is to ask the affinity engine for community information on each of the recommendations 608. This information can be combined with other asset information, such as title, size, or price, and displayed to the user to help them understand the community wisdom underlying a recommendation. Many aspects of the community wisdom associated with a document in the affinity engine can be exposed, but in common implementations we expose the number of users in total who found value in the document, the number of peers who associated the document with the current context/intent, and the terms/phrases the peer community has associated with the document.
  • the system runs a trend detection system across all content on a periodic basis, e.g. once a day, or every five minutes for very time-sensitive sites.
  • Trend detection can be done in a number of ways, but one way is to use the Mann-Kendall algorithm. Other ways include various regression techniques that employ least-squares fit. If a strong negative trend is detected for a given piece of content, the information associated with that content in the affinity engine is decayed at a more rapid rate. The result is that the likelihood of the affinity engine recommending this piece of content is reduced.
  • Another algorithm for fad detection that is run on a periodic basis is cyclical fad detection. If a piece of content shows a strong positive trend and that same trend can be found at regular intervals in the past, then that piece of content can be automatically boosted in importance in anticipation of the coming trend.
  • a piece of content shows a strong positive trend and that same trend can be found at regular intervals in the past, then that piece of content can be automatically boosted in importance in anticipation of the coming trend.
  • techniques that are typically used survey the customer and analyze the customer responses to produce a report. By the time the report is analyzed, the results of the report are out of phase with the actual situation at the time the report is being reviewed. For example, in a traditional system, a commerce site might collect feedback during the Christmas season and redesign their site in February in response to that information. In effect, the commerce site is trying to sell Christmas products in February.
  • analytics reports may be created offline and analyzed by a team of specialists to infer trends and determine appropriate actions; a process which may take weeks or month and again lead to out-of-date site modifications.
  • the automatic nature of the invention herein allows a Web site to adapt in real-time to discovered trends and fads, providing recommendations to users that fit the current context and time.
  • the trend and fad detection algorithms used by the invention can also be exposed to owners of the content system, e.g. the Web site, through reports within a customer portal.
  • reports on community trends and fads can, for example, be used by merchants to promote certain products or content at the right time, making such promotions more effective.
  • the proprietor of the commerce site would be able to follow the sales curve based on community interaction with the Web site. If, on the other hand, an emergent news story drives demand for a product quickly, the invention allows the merchant to watch the curve of demand and respond in real-time.
  • the proprietor of the Web site can provide that content or product more quickly. If the demand dies out quickly, for example the interest was based on a fad, then the proprietor of the site can adjust to that fact as well.
  • the affinity engine learns connections/affinities between terms-documents, terms-terms, terms-users, documents-documents, documents-users, and users-users.
  • terms document, content, and asset are used interchangeably throughout this document. In all cases, these terms refer to any type of medium that provides information or services including, for example but not limited to, webpages, word documents, pdfs, video, audio files, widgets, and/or products).
  • all affinities are stored as sparse matrices and vectors. However, there are many alternative ways of storing such information known to those skilled in the art.
  • FIG. 8 is an architectural diagram showing the aspects of the invention that involve topic attraction and topic match, the two dimensions that make up the bidirectional virtual folksonomy connection between terms and documents, as well as general activeness.
  • a query is provided 801 and a percentage of respondents who ultimate find value in one of three example documents are indicated.
  • the query involves a particular Nikon camera and it can be seen that 20% of the respondents find value in a Web page focused on high-resolution cameras 802, 78% found value in the overview page for the specific camera 803, and only 2% of users making this query found value in the detailed specification sheet for the camera 804.
  • value is determined by the full spectrum of behaviors exhibited on a given page by the user (described in detail later).
  • Topic attraction starts from the query and considers the proportion of users who searched for "Nikon 580X" and subsequently found value in each document.
  • the Overview page 803 has highest topic attraction, with 78% of users finding value there, followed by the High-resolution camera page 802 with 20%, and then the spec sheet 804 with only 2%. Note that this example is simplified for the purposes of illustration. In reality, we also break down the query into its component parts and consider the affinities of these to each document, as well as consider affinities between other terms that the affinity engine has learned have similar meaning to this one.
  • topic attraction can be thought of as a problem of predicting the probability that a user finds value in a given document, given the intent represented by the query.
  • arriving at this probability is accomplished using a combination of probabilistic techniques, including Bayesian inference.
  • Topic match looks at it from the other direction starting from each document and considering all of the other terms that have been associated with it through the behaviors of other users. So, for example, the High-resolution camera page 802 may have many other camera names connected to it and, thus, the degree of focus on the particular query entered is lower than both of the other two pages, whose term-document connections are more highly concentrated around "Nikon 580X” specifically. In fact, the Spec-sheet 804 may turn out to be the document with term connections most focused around "Nikon 580X" and thus have the highest topic match for this query, even though it has the lowest topic attraction. As with topic attraction, in reality, topic match also breaks down the query into its component parts and takes into account affinities between other terms not in the query.
  • topic match can be seen as finding the document with the best overall match in topic to the intent represented by the query.
  • a vector space model and modified cosine similarity technique similar to that used by traditional full-text search, is used to determine the degree of matching.
  • one or more thresholds are applied to each dimension, including absolute and relative thresholds before they are combined together based on hybrid arithmetic- geometric weighted average.
  • Each resulting sub-affinity is then similarity subjected to a thresholding before being combined with other sub-affinities again based on hybrid arithmetic-geometric weighted average.
  • the values for the thresholds are generally fixed, however, the individual weights for the weighted averages can also be adjusted and learned based on the success of the result set returned by the affinity engine.
  • a key feature of the invention is the processing and analysis of implicit observations that are made during the actual use of a Web site, for example, versus traditional approaches in technology which use explicit feedback.
  • One key advantage of the invention is that humans are very bad at giving feedback, particularly if the feedback must be given explicitly. If a person is being surveyed, the person does not have an incentive to give actual accuracy in the form of feedback.
  • One aspect of the invention eliminates such survey bias by using the implicit behaviors observed during use of particular materials on the Web by individuals within a community. Thus, the invention trusts what people do but does not watch what they say.
  • the invention watches people's behavior through their actual action, implicit behaviors, and can accurately interpret what their true intent is and whether or not they dislike or like something. Thus, the invention observes behavior versus click actions.
  • the invention recognizes that it is the action of the user after the click that matters and not the click itself.
  • the invention tracks behaviors beyond the click to determine whether a page is good or bad. Thus, if a person backs out of the page, it is considered negative feedback, i.e. the person did not like it. In this way, clicks can identify a very negative reaction in the invention. If a person goes to a link, follows the link down, spends time there, and does other things, that behavior is tracked as well. If a peer group validates the behavior as consistent, e.g. a significant number of the group members exhibit the same behavior when reacting with the page, then the page is considered to be a good page.
  • An embodiment of the invention goes one step farther. Not only does the invention determine which assets are useful based on behavioral fingerprinting, but it also learns the context associated with the usefulness. For example, the invention can learn that a particular camera page is very useful for users that show interest and intent in high definition cameras, but not if the intent is compact cameras. In this way the affinity engine is able to distinguish the usefulness of assets based on the context and intent expressed by a user through their implicit actions.
  • the primary input to the affinity engine, which drives all of the learned associations (affinities) is the behaviors of users on the site.
  • all behaviors are captured by the ObserverTag, i.e. a piece of HTML/JavaScript embedded within the Web site, typically in a header or footer template.
  • the ObserverTag i.e. a piece of HTML/JavaScript embedded within the Web site, typically in a header or footer template.
  • FIG 9 shows the process for analyzing these behaviors or other implicit data captured from various user interface devices.
  • the input to this process is the user trail 901 which includes all assets visited and the implicit (and explicit) actions observed on those assets.
  • the first step is determining whether a user is finding value (usefulness) in a particular piece of content.
  • think mode can be approximated by non- scrolling time on a page, where some scrolling or mouse-movement has been detected with a specified time range.
  • a second normalization is to normalize based on the content. This is done based on inherent or specified properties of the content. For example, 30 seconds spent on a one paragraph document likely has a different meaning than 30 seconds spent on a ten page document. Dwell time can thus be normalized based on page length or number of words. Similarly 30 seconds spent viewing a 30 second video has a different meaning than 30 seconds spent viewing a five minute video. In many cases the Observer Tag is capable of capturing these page characteristics through information within the DOM (Document Object Model). However, when not available in the DOM, the invention provides other mechanisms for collecting the needed information from offline catalogs, e.g. a product or media catalog, or allowing the Web site designer to add explicit information in the page itself, e.g.
  • a predefined set of rules are applied to determine whether this behavior vector represents liking/usefulness.
  • Each of the normalized features is considered to determine whether it meets the pre-specified thresholds for indicating usefulness.
  • Each passing feature increases the probability of usefulness 905.
  • the thresholds for a given feature are pre-specified and tuned by the person implementing the system based on previous experience.
  • these thresholds are dynamically determined by the system. For example, certain features are known to exhibit a bimodal distribution and the threshold can be dynamically determined to lie between the two modes.
  • the invention also provides a mechanism for plugging- in various machine learning techniques to learn which features are most important, and thus dynamically learn the rules for converting from the behavior vector to the probability of usefulness.
  • the second step in the analysis of behaviors is to understand the entire context surrounding the use of that piece of content, including searches done prior to the use, links clicked prior, and pages used prior. All of this information combines with information about the user to influence affinities 906 (affinities can be learned and stored in a number of ways, described in more detail in U.S. patent application serial no. 11/319,928, filed 12/27/2005). Before doing so, however, there is one final step: validation. Validation is a form of noise filtering wherein an affinity connection is established, e.g. between a document and term, if and only if enough similar users have, through their behaviors, confirmed this connection. One user making the connection is not enough for an affinity to emerge. A minimum number of users in the same context (peers) must have exhibited similar behaviors and connections for that affinity to be validated. We call this peer-validated behavior.
  • UseRankTM the ultimate ranking of the usefulness of content based on a user's context and intent, based on learned affinities and full-spectrum behavioral fingerprinting, can be compared to the very popular PageRank strategy made famous by Google.
  • PageRank each Web page is given a value based on the number of other pages linking to it.
  • links coming from pages that are themselves of high value raise the PageRank even more.
  • This democratic strategy is quite effective in getting users to useful Web sites based on their Google query, but generally breaks down once the user begins to look for further information within the Web site itself. The main reason for this is that pages within a Web site, particularly those down into the long tail of content, are not heavily linked to either externally or internally.
  • linkages are determined solely by the structure of the Web site and more links does not necessarily mean more value.
  • many of the pages on such sites are in formats other than HTML, e.g. PDFs, Word Docs, or videos, and do not link to other content at all.
  • the UseRank methodology based on user behaviors on the Web site alleviates all of these problems. Instead of relying on the linking of documents by Web designers, it relies on usage of documents by Web users, who are a rich source of information on any Web site. Usage is a truly democratic way of learning what content is most valuable. In fact, the designers of PageRank recognized their approach as an approximation of user activity. Google's choice to approximate user value based on PageRank makes sense given the privacy concerns associated with tracking a user's behavior across the entire Web. In the preferred embodiment of UseRank, we track only the behaviors of user's on sites instrumented with the Observer Tag, and always in an anonymous fashion unless otherwise configured by the Web site deploying the Observer Tag.
  • the invention provides a context-centric approach that allows the affinity engine to narrow its focus to the subset of users (peers) and content that match the current user's context/intent, even if that intent is not particularly popular in the grand scheme of things. This is what is known as the long tail of the products, Ae. those products that may not be the most popular overall, but are extremely important because they strongly meet the need of a small but important subset of the community (see C.
  • the invention is capable of identifying that long-tail interest along with the associated peer group and related content and can thus recommend those important long- tail products/content at the appropriate time (see Figure 10). This is critical for product/content providers because it is often these long-tail products that are most useful to the community and often lead to the highest margins or benefits for the company itself.
  • the invention uses like-minded peers as an enabler to target small target segments along the long tail, instead of using individual problem personalization and historical interest as known in the prior art.
  • the invention provides contextual targeting. People's needs within a given context are typically similar among like- minded people. An insight of the invention is that people have hundreds of profiles, if not thousands of profiles. These profiles have no cross connections to reality, so it is difficult to predict an individual's use level without context. Once the individual is in a particular context, i.e., among like-minded peers, the individual behaves similarly to everybody else within the group context. Thus, if a person is visiting a golf equipment site to buy a golf driver or clubs, the person behaves in a manner similar to other golfers. It does not matter what the person's political bias is, or their cultural background. These aspects of the individual have no relevancy because the person's present context, which is based upon group membership, is more relevant.
  • the invention represents a new approach to leveraging implicit community wisdom to create adaptive Web sites and other information portals.
  • the invention represents a new approach to leveraging implicit community wisdom to create adaptive Web sites and other information portals.
  • Figure 11 provides a wisdom of Crowds pseudo-equation showing how context/intent can be incorporated in a generalized way to extend the wisdom of crowds concept, as exemplified by an embodiment of the invention.
  • each user's implicit vote is weighted according to the similarity between the current user's context and the context in which a user made that vote.
  • This equation could be further extended to incorporate similarity between users. When all contexts are identical, this equation correctly reduces to a simple average.
  • the actions of the Affinity Engine can also be considered as a form of Memory Prediction Machine.
  • Recent learnings from cognitive science teach that the brain is structured to capture and encode associations between and among objects and concepts in targeted regions of memory, and organize those associations in a hierarchy moving from abstract to detailed. New stimuli are responded to and their consequences predicted, based off of the memories created from previous stimuli and learned associations thus encoded.
  • the Affinity Engine similarly learns associations between users, objects, and contexts by observing interactions between them in the environment of Web sites and remembers those patterns in its memory, stored hierarchically. When a user exhibits a context that has been previously learned by the Affinity Engine, it lights up the appropriate associations within memory which triggers the prediction of those objects that best meet the needs of the current user and context.
  • Figure 12 is an architectural schematic of the Affinity Engine in its function as a memory prediction machine. Seamless Integration with External Systems
  • An important aspect of the invention is its ability to integrate seamlessly with other recommendation systems, policies, and information sources. Some examples are shown in Figure 3.
  • the search terms are sent to the affinity engine for processing.
  • the affinity engine determines the set of content that is most useful given the context of that search, based on learned affinities from past community behaviors. However, the affinity can also take into account opinions from external sources, such as a search engine.
  • the search engine 301 has its own set of recommendations which the affinity engine can accept. The relevance ratings from the search engine can be combined with the information embodied in the affinity engine to produce a single unified set of recommendations (discussed more fully in U.S. patent application serial no. 11/319,928, filed 12/27/2005).
  • An XML feed is a typical way for the system to interface with a search engine.
  • the invention also provides a mechanism for automatically importing product or media catalogs 305.
  • the preferred means for doing so is using an XML feed that is accessible to the system.
  • Information from the catalog may be displayed along with the recommendations, e.g. summary or price; 2) Attributes from the catalog may be used to filter the result set, e.g. the Web site designer may want to restrict recommendations to products only or PDFs only a particular section of the Web site;
  • Content categorizations can be used to group content for the purposes of normalization in the behavioral fingerprinting process described above.
  • Ad servers 303 can provide functionality similar to both search engines and catalog systems as described above.
  • the invention interfaces with ad servers in the same way as it does to these, typically through XML.
  • Merchandizing systems 304 in the general case provide a mechanism for the Web site owners to influence what products/content are recommended to users.
  • the community be the primary driver of recommendations. However, there are times when the owners have specific needs that must be met independent from the community's expressed interest.
  • the owner may choose always to recommend a particular product or piece of content first in a particular context, for example, there may be a promotion going for a product. Owners may also wish to influence the community to purchase products with higher margins, or prevent a certain piece of content from being recommended. There are two ways in which the invention can honor such influence.
  • the first is through a custom rule interface that is one component of the invention.
  • Web site owners can log in and choose from a variety of predefined rule types including pinning, i.e. forcing a particular recommendation to show up; blacklisting, i.e. preventing a given recommendation or class of recommendations from showing up; and boosting, i.e. increasing the chances that a recommendation or class of recommendations show up, but still honor the community wisdom.
  • All of these rule types can be applied globally or only in a given context, e.g. on a particular page, for a given search term, for a given class of users, in a particular time range.
  • Another type of rule allows owners to honor fully the community wisdom, but influence the goal of the recommendations. For example, rather than focusing on recommending useful products, the affinity engine can be told to recommend products that a user is most likely to purchase. As another example, the affinity engine can be asked to recommend content that most likely leads users to a particular set of target pages.
  • the invention provides a mechanism for importing rules in XML form from an external merchandizing application. When rule types exist in the external system that do not already exist within the invention, a custom rule plug-in can be designed for the invented system which matches the desired behavior of the external rule type.
  • a further type of system that the affinity engine can integrate with is user profiling systems 302.
  • a profiling system is a set of attributes, e.g. demographic attributes, associated with a user. These attributes can be passed along to the affinity engine on a recommendation request. As discussed earlier, the affinity engine can take these attributes into account when identifying a peer group, which then influences the recommendations that are returned. More sophisticated profiling systems require XML integration similar to other external components. Here, the affinity engine may have to dynamically contact the external profiling system during the peer identification process.
  • a Recommendation System is implemented in a Software as a Service (SaaS) Model to provide automatic suggestions that help Web site visitors find products or content they like or need.
  • Figure 2 is an architectural schematic diagram of a method and apparatus for context-based content recommendation.
  • interaction of users with a Web site produces implicit emergent behaviors. Such user interaction includes page referrals, links, entry trails, queries, page sizes, mouse movements, peers, negative experiences, virtual bookmarks, time spent, virtual printing, exit trails, and the like.
  • Such behaviors are processed in a recommendation/affinity engine according to the invention, resulting in automatic content and product recommendations in the form of social search and navigation guides, as well as providing real-time feedback to a merchant such as regarding visitor clubs, and identifying content gaps (each of which is discussed in greater detail below).
  • the following key processes comprise the recommendation system:
  • Implicit Behavior The implicit Web site behaviors 201 that serve as input to the recommendation system are captured client-side by small snippets of JavaScript code embedded within Web site pages, e.g. through a Web site template. The behaviors are then sent to the remote recommendation engine where they are processed using the full-spectrum behavior fingerprinting technology described herein.
  • the recommendation/affinity engine 202 processes all incoming information to identify emerging intent among Web site visitors and captures the collective wisdom of the crowd.
  • the recommendation engine identifies and learns affinities between and among users, content, and terms, identifying consistent patterns of behavior and removing noise.
  • the affinity engine also recognizes and adjusts in real time to changing fads and trends, as well as cyclical patterns of behavior, such as seasonal patterns, and other shifts in user interest.
  • Delivering Recommendations and Search Results When a visitor arrives at a page, the recommendation engine is passed information about the users current context, either by snippets of JavaScript on the client, called Recommendation Tags, or by the Web site server, and a request is made for appropriate recommendations.
  • the recommendation engine translates the user's context into intent using the technology described in this application, identifies the appropriate group of peers, and ultimate returns with a set of content recommendations 203 ordered based on the their computed UseRankTM. These recommendations are then displayed within the Webpage on the user's browser. These recommendations may take the form of navigation links or enhanced search results. When delivering recommendation results, the recommendation system may additionally contact other external services, such as the Web site's full-text search engine, product catalog, or an ad server (see Figure 3).
  • Customer Portal In the preferred embodiment, owners of the Web site also have access to information and patterns learned by the recommendation engine through a customer portal called Insights 204.
  • the customer portal provides the ability to configure the recommendation system and add merchandizing rules, as well as view a set of reports providing information on the usage of the Web site by the community and the patterns of behavior and affinities learned by the system.
  • Figure 13 shows a screenshot of the Customer Portal homepage in the preferred embodiment. Many different reports and configurations are accessible using links within this portal.
  • the invention allows for product/content gap detection, where gaps are detected based on usage patterns on the site and site owners are helped to introduce new products or content in those areas where the gaps are detected.
  • the Customer Portal also provides the Web site owner with a set of reports to track the ongoing value that the Recommendation Engine is brining to the Web site.
  • Figure 14 shows a screenshot of an A/B report that tracks the revenue lift due to the presence of recommendations on the Web site.
  • the inventors have consistently seen revenue improvements of 20-30% or more.
  • Other lift reports available within the portal include improvements in page views, engagement time, site stickiness, average order value, and user conversion to specified "business targets" within the Web site.
  • Figure 15 is a screenshot showing community-guided e-commerce. In Figure 15, five different capabilities of product recommendations are presented:
  • Comparative Products also known as Similar or Competitive Products
  • Comparative Products are shown for purpose of comparison shopping and up-selling based on affinity of like-minded peers on their product considerations, i.e. observed full-spectrum behavioral finger prints discussed earlier.
  • Consideration-based peer recommendations promote higher end / niche products, and yield more revenues and profits for a site.
  • affiliated Products are also shown for cross-selling related / non-competitive products such as accessories to increase overall order size.
  • the invention also provides a showing of most popular products, such as products with greatest appeal across the whole site or within a category.
  • the invention also generates intent-driven landing pages, also known as AdGuide and Site Concierge, with recommendations based on what a visitor searched on standard Web search engines such as Google, Yahoo or MSN. If a visitor search "Viking Cooktop" on Google and landed on a customer site, AdGuide serves the best Viking cooktop recommendations to the visitor on the landing page dynamically instead showing him something irrelevant.
  • the invention also provides a community filtered search on products based on the foregoing.
  • the invention automatically identifies small and large population segments having a unique interest and guides individuals to popular, competing, and accessory products.
  • the invention provides a user intent and product mapping that translates shoppers' intent into peer-validated products and brands.
  • the inventors have found a 700 percent conversion power based on independent studies.
  • the invention provides fad and trend detection and recognizes and adapts to seasonal, promotional, and other shifts in shopper interest in real-time.
  • the invention also allows for product gap detection, where product gaps are detected and site owners are helped to introduce new products or content in those areas where the gaps are detected.
  • the invention assists in merchandising with real-time customer feedback. It works with existing merchandising, promotions, segmentation and search, and magnifies products or product families.
  • the invention provides built-in concurrent A/B measurement and reports in real-time the net revenue generated by the invisible crowds.
  • Figure 16 is a screenshot showing community-guided product search with a travel example.
  • the user has queried for hotels in New York City.
  • Figure 17 is a screenshot showing community-guided marketing and online lead generation.
  • Five different capabilities are provided to a marketing Web site.
  • 1) Aspects of this feature of the invention include a social search which involves UseRankTM and implicit learning. This approach is intent-driven, adaptive, and makes use of an implicit folksonomy, community terms such as link-text they use and search queries they enter, as discussed above.
  • the invention supports audio, video, binary code, and all other content types because it does not have to parse the content itself, whereas a typical search engine has to, therefore is limited to text content and metadata only.
  • the invention also provides Related or Similar Content based on the implicit behavioral feedback of like-minded users.
  • the invention ranks the recommendations by usefulness and on-target-ness, based on the context and intent.
  • the invention also provides a most popular category, which provides content with the greatest value across the whole site or sub portion of the site. This promotion of information leads to business target conversion increases such as trial/download/registration conversions and may be provided at a site or topic level.
  • next step may be any next step in a business process or a natural next set of content to read in connection with the process.
  • Figure 18 is a screenshot showing social search according to the invention.
  • the search is made for storage devices. Users are provided information with regard to whether the information is on target and for how many visitors, and popularity rating and a community rank, i.e. UseRankTM. An on- target value is also provided along with the results. Further, the results are sorted by usefulness, based on information within the affinity engine. The original keyword rank from IR or full-text search engine is also provided as a before-and- after comparison. In this example, if the UseRankTM is not used, this good piece of primary storage product information would have been buried as rank 87, i.e. the seventh result on the ninth search result page, and no one would be able to find and use it.
  • the UseRankTM has moved it to the third position on the first page in the search results.
  • a Raw Results link is provided to give the users the original results if this invention were not used there. This is another way to see before-and-after impact of the UseRankTM.
  • Figure 19 is a screenshot showing community-guided online support to promote self-service and satisfaction among the customer base.
  • the page shows top issue resolutions, including most helpful articles, a top FAQ, and help forum, and blog articles.
  • the page also shows related help, including in-depth articles on the subject, alternative solutions and, finally, reminders.
  • common next steps are provided, including natural next steps for resolution, common downloads, and additional support information and contact information.
  • social search is a key aspect of this embodiment of the invention, based on UseRankTM and implicit learning.
  • social search is intent-driven and adaptive, makes use of the implicit folksonomy, and supports all content types.
  • Other good support applications for this use include knowledge management (KM) portals, community forums and discussion threads, chats, wikis and developer's networks, etc.
  • KM knowledge management
  • Figure 20 shows a community-guided intranet and knowledge portal.
  • social search is used within an intranet.
  • the search returns related pages, which include similar pages on the topic, the pages are proven by the past experience of peers, and serendipitous discovery is enabled. Further, the search filters out low value content.
  • the invention also provides most popular search results, which includes content with the greatest value, top applications within the company, and information specific to a department or cross-company information. Finally, the invention provides next steps, including where people go from this page to meet their goals, a next step in the business process, and the next set of content to read on the topic.
  • Extranets such as physician-patient portals, dealer net, customer portals, are other applications examples.
  • Figure 21 shows recommendations implemented on a media site
  • Figure 22 shows social search implemented on a media site
  • Figure 23 is a screenshot showing community-guided media and cross-site contextual recommendations. Similar Articles and Blogs are recommended based on similar contexts. If you are reading a Tiger Woods golf story, three other related Tiger news, based on other similar peer interests are promoted to the same visitors. The visitor gets more content and satisfied. The site get higher CPM and CPA, both of which are current industry measures of visitor conversion rates and site value from advertising perspective.
  • UseRankTM also known by Baynote as BrandRankTM, can also be used to re-rank the ads and product on the media, instead of placing ads or products randomly or based on keyword match.
  • Ads validated by peers have significant higher CPM and CPA values, and therefore increase revenue for the media sites.
  • the information contained in these various sources is applied to the affinity engine, which discovers cross-site affinities among the various sites. The result is to provide targeted content and products.
  • Figure 24 is an example of the invention being used to recommend video content.
  • the invention is content agnostic and is able to learn affinities between term, users, and content based entirely on implicit user actions.
  • Figure 25 shows a screenshot of an Implicit Topic Cloud.
  • Another application of the invention is to collect and display the community vocabulary or terms.
  • the Topic Cloud may appear to be similar to explicit tagged cloud such as Deli-cio-us, social bookmarking, etc. But the key difference is the implicitness.
  • Implicit Topic Cloud is built based on the actual user's activities on the site, i.e. the queries they entered and link-text they used. Actions speak louder than words.
  • the implicit Topic Cloud reflects in the interests of 100% of the site visitors, not the few loud visitors who are willing to tag things. The silent majority is the key to collect true wisdom of the community.
  • Figure 26 shows an architectural view of an integration between the invention and an AdServer.
  • the integrated system To serve the best ad to a user, the integrated system first contacts the existing Ad Server to get a list of acceptable ads to display to the user given demographic or other information. This list is then sent to the Affinity
  • FIG 4 is an architectural diagram showing the invention's role as a community wisdom platform.
  • the community-wisdom platform can also include such items as contextual email marketing, mobile applications, IPTV systems, SEO/SEM applications, and mashups in custom applications.
  • input from the Recommendation Engine can be used to dynamically determine the most appropriate information and/or organization of information to present to users.
  • Such a platform is built upon Web services that include such features as AJAX and REST (discussed below).
  • the invention is built upon new Web 2.0 fundamentals, which include a true understanding of invisible crowds, gathering information from like-minded peers, making content or product recommendations, providing onsite social search, implicit community-based reports, and the like.
  • Web 2.0 fundamentals include a true understanding of invisible crowds, gathering information from like-minded peers, making content or product recommendations, providing onsite social search, implicit community-based reports, and the like.
  • the following descriptions give details on how they are deployed:
  • Email Recommendations Marketing uses massive emails to communicate with customers and prospects. But the content of the email campaign is typically determined by the marketing staff. The invention enhances the quality of email campaigns by crowd-sourcing the like-minded peers. The content of the emails are determined by what's popular or similar to what the like-minded readers of the emails are interested in. Marketers simply facilitate the process of letting one customer promoting content to another customer implicitly.
  • Mobile Recommendations The quality demand of mobile recommendations is even higher than the Web due to the small footprint of mobile devices, such as cell phones and personal devices.
  • the invention's like-minded peer and intent driven approach is better than traditional approaches based on collaborative filtering, page views, and demographic or purchase data.
  • Intent/context driven peers predict people's need far better than the competing approaches.
  • IPTV Recommendations Similar recommendations are effective for using Digital TV/HDTV to watch internet delivered movies and video programs. With thousands of movies and TV program selections, features like "You may also like these movies" are extremely important for up-sell and cross-sell movies and TV programs. Context-driven peer recommendations are the most effective way for the viewers to like the recommendations.
  • FIG 27 shows an architectural view of the Live Connect system. Because the invention can identity like-minded peers, in addition to harvest their collective wisdom implicitly and to make peer recommendations to individuals, the invention can connect the individuals with the like-minded peer group and have them exchange information, knowledge, and experience. This is similar to the real world experience of like-minded peers who may not know each other. Imagine you are in Best Buy shopping for a TV, you see other like-minded peers also shopping for TVs, you can ask them questions for their experience of certain products.
  • the invention is also used for guiding Web site visitors at the moment of landing on the site.
  • the features are called “AdGuide,” “Site Concierge”, and “Site Maitre'd.”
  • This application examines the queries that people have searched on Google, Yahoo, MSN, ASK, and other major Web search engines, then uses them as the proxy of user intent to display the right product or content for visitors when they land on the site, either on the predesigned landing pages (SEM), or natural search result pages (SEO).
  • AdGuide serves the best Viking cooktop recommendations based on collective interests of like-minded peers to the visitor on the landing page dynamically instead showing him something irrelevant or less important by simply matching keywords.
  • the invention is also able to provide a Keyword Recommendation system that provides suggestions to marketers on which keywords should be purchased on external search engines as part of their SEO/SEM efforts. For example, the invention can suggest those terms that the community uses most often to describe content on a Web site, as well as those terms which are most likely to lead to useful content within the site. Also, given a particular collection of keyword already purchased, the invention can suggest other words used by the community that have high affinity to the existing set. These recommendations can be provided to SEO/SEM merchandisers either through reports within the Customer Portal or can be directly integrated with external SEO/SEM systems to bid on the recommend keywords dynamically and automatically.
  • the invention is used by community generated content such as blogs, forums, discussions, pod casts, video content, etc. Because of the volume of content involved, surfacing the right content in the right order is even more important than expert- generated Web sites. Using like-minded peers for recommendations and social search is extremely important and effective. These Web sites can be public facing blog sites, or login-required sites such as partner portals or patient- physician portals, or developer's networks, or intranets. Features such as "Similar content,” “Accessory products,” “Next steps,” “Most popular” and social search are very useful and effective.
  • Insights (Visitor Clubs, Content Gaps, efc): Insights, the Customer Portal, has already been discussed in part earlier in this application.
  • a primary use of the invention is as an automatic recommendation system, there are times when site administrators may want to modify the behavior of the recommendation system or see reports based on the knowledge learned by the affinity engine.
  • Insights currently offers three main categories of functionality though its Ul: configuration, management, and reports.
  • Configuration enables site administrators to configure fully all aspects of the recommendation engine required for full functionality and deployment, including integration with external sources of information such as full-text search engines and ad servers.
  • Management enables site administrators to modify the behaviors of recommendations and social search in a holistic manner or specific to defined contexts e.g.
  • Reports offer various views on the community wisdom distilled by the affinity engine. Although the reports include a subset of the click-based and purchase-based information found in traditional analytics, they go beyond these in providing many details on the community wisdom distilled through full- spectrum behavioral fingerprinting and the connections learned by the affinity engine. Example reports are discussed below.
  • the Content Gap report uses the affinity engine to analyze those user interests that are not being met by the existing content on the site. Using the full- spectrum behavioral fingerprinting technology, and by analyzing community behaviors as discussed earlier in this application, the affinity engine can identify community interests. The affinity engine can then analyze all cases in which a particular interest was expressed by a user and the subsequent behaviors of that user, including use of content and level of value extracted, as well as subsequent searches or exits. If a large portion of the user population with a particular interest is unable to find useful content, then that interest is designated a content gap. The degree to which the gap is suspected by the affinity engine can also be reported. The administrator can then take necessary actions to remedy the gap by adding new content and then continue to use the Content Gap report to see the effect on the presence of the gap.
  • the Visitor Clubs report provides the most comprehensive view on the affinities learned by the affinity engine. Visitors to the site are implicitly grouped based on shared interest. Each interest group, or visitor club is displayed in the interface along with the number of visitors in the club, their overall activity level, the virtually folksonomy of terms that describe the club, and the content/products which are most useful to that club in the context of the club's interest. In addition, all club characteristics can be charted over time to gain an understanding of how the interest is trending over time. Clubs can also be compared to one another for overlap in membership. This report provides administrator with the ability to explore the learned associations between users, content/products, and terms/contexts. Such associations can then drive promotions or other merchandizing activities within the Management section of Insights. The visitor club information can also be used to drive various business decisions outside of the context of the invented system.
  • FIG 3 is an architectural schematic diagram showing affinity engine integration according to the invention.
  • a current user exhibiting an observable behavior within a particular content site, such as current location and/or search, based on previous action, or user properties, accesses the affinity engine, which includes assets/products, users, and terms, all arranged in groupings based on affinities.
  • the user is provided with asset recommendations in connection with products, categories, information, media, and the like.
  • the affinity engine is populated by site owners who provide administrative input and exert administrative influence.
  • the information provided by XML/direct input.
  • Asset information is also provided in the form of XML information, for example, a product catalogue or a full-text search facility.
  • the affinity engine collects implicit observations from the visitor community to make such asset recommendations to a current user. These implicit observations include asset actions, such as time spent on a page, scrolling, mouse movements or clicks, page interaction, and virtual printing or bookmaking; asset navigation and asset navigation patterns, including, for example, entrance path, exit path, links used, repeated visits, and frequency visits, use of a back button, and recommendation use; and visitor-community searching, including terms entered and/or reentered, results clicked, results used, and subsequent navigation.
  • Figure 28 is a block schematic diagram showing the system architecture of a preferred embodiment of the invention. A more detailed discussion of various aspects of this architecture is provided below. To understand Figure 28 in greater detail, reference may be had to Applicant's copending U.S. patent application serial no. 11/319,928, filed 12/27/2005, which is incorporated herein in its entirety by this reference thereto, and which describes, in greater detail, a preferred system for use in connection with the invention herein.
  • the architecture consists of a server farm 20, a customer enterprise 22, and a user browser 21.
  • the user browser is instrumented with an extension 23 and accesses both customer servers 25 at the customer enterprise 22 and the server farm 20 via a load balancer 27. Communication with the server farm is currently effected using the HTTPS protocol. User access to the customer server is in accordance with the enterprise protocols.
  • the browser extension 23 is discussed in greater detail below.
  • the extension 23, as well as the enterprise extension 24, are constructed such that, if the server farm 20 does not respond in a successful fashion, the extension is shut down and the enterprise and browser interact in a normal manner.
  • the features of the invention are only provided in the event that the server is active and performing its operations correctly. Therefore, failure of the server does not in any way impair operation of the enterprise for users of the enterprise.
  • an extension 24 is also provided for the enterprise which communicates with the load balancer 27 at the server farm 20 via the HTTPS protocol.
  • the enterprise also includes a helper 26 which communicates with the server farm via an agency 31 using the HTTPS protocol.
  • the agency retrieves log information from the enterprise and provides it to log analyzers 28, which produce a result that is presented to the usage repository 29.
  • the agency also communicates with any data sources accessible to the enterprise for the purposes of loading information into the affinity engine 32.
  • Such information includes catalog database information, use profile data, authentication data, as well as any other form of data that may be useful for the purposes of determining or filtering recommendation and search results.
  • a key feature of the invention is the affinity engine 32 which comprises a plurality of processors 33/34 and a configuration administration facility 35. During operation of the invention, a form of information, also referred to as wisdom, is collected in a wisdom database 36.
  • the preferred embodiment of the Baynote extension 23 uses a custom-build tag- based AJAX-architecture for collecting Website behaviors from the client as well as serving recommendations to the client.
  • Figure 29 provides code snippets for JavaScript integration with a client.
  • Figure 30 is a block schematic diagram showing an AJAX tag platform according to the invention.
  • a Web page is displayed on a client and includes a particular script.
  • all files and results from the system are dynamically injected into the DOMS as ⁇ script> elements.
  • a coordinator on a trusted Web server provides such things as a failsafe mechanism, common code, a policy, handlers, and communication facilities.
  • a system server provides such elements as a heartbeat, common JavaScript, policies, handlers, such as an observer and a guide, and the affinity engine.
  • the coordinator is a piece of code that sits on the trusted Web site.
  • the coordinator resides on a customer's Web site.
  • This aspect of the invention concerns the notion of a trusted Web server for failsafe reasons. It is important to be certain that if the invented system crashes that it does not crash or hold up the customer Web site.
  • a coordinator code is provided as a failsafe procedure. It goes through a failsafe procedure to ensure proper recovery after a crash of the server's Web site.
  • the architecture of the code is broken down, such that common code is used. Common code or a piece of the code is used for all types of recommendations.
  • a tag might require common code.
  • the policy may be customer specific or it can be user specific. This concerns how recommendations may be made.
  • the policy information is loaded separately.
  • the code needed for specific tags is loaded as well. For example, if there is a tag for the most popular recommendation, and another one which is the recommendation, then these tags are loaded.
  • the system then loads the code that is necessary for performing the particular tagging operations.
  • a further type of tag or code is the observer that observes what users do, as discussed above. Users do not see the tag, but it is still a tag.
  • the tags must also be able to communicate back to the system. Each of these functions is performed by a piece of code, for example JavaScript, is loaded into the Web page. The JavaScript is loaded into the page through a dynamic script injection.
  • This aspect of the invention recognizes that a Web page can become more dynamic. For example, there are things such as dynamic HTML and JavaScript which cause things to be added to a page dynamically. If a script tag needs a particular piece of information, then that code is injected into the page. For example, if a policy is needed, then a JavaScript tag is injected into the page that accesses the policy. The policy may require further injection of information, such as handlers. Therefore, the handlers are now dynamically injected as pieces of JavaScript into the page. Communications are handled in a similar matter.
  • the main Web page that the customer accesses may be the Web site of a merchant.
  • the merchant communicates with the system to get the information with regard to recommendations and the like.
  • the merchant is the first party and the system is a third party.
  • the invention allows the setting of cookies in both the first-party domain and in the third-party domain.
  • the ability to set first and third-party cookies also allows the invention to operate in a cross domain.

Abstract

Starting with the people in and around enterprises, the expertise and work patterns stored in people's brains as exhibited in their daily behavior is detected and captured. A behavioral based knowledge index is thus created that is used to produce expert-guided, personalized information.

Description

METHOD AND APPARATUS FOR CONTEXT-BASED CONTENT RECOMMENDATION
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. patent application serial no. 12/188,850 filed August 8, 2008 and U.S. provisional patent application serial no. 60/954,677, filed August 8, 2007, both of which are incorporated herein in their entirety by this reference thereto.
BACKGROUND OF THE INVENTION
TECHNICAL FIELD
The invention relates to electronic access to information. More particularly, the invention relates to a method and apparatus for context-based content recommendation.
DESCRIPTION OF THE PRIOR ART
One problem with finding information in an electronic network concerns how people are connected as quickly and effectively as possible with the information/products/services that meet their needs. This has been one of the main goals of Web pages and search engines since the beginning of the World
Wide Web. Failure to do so leads to lost business in the case of eCommerce eTravel, and eMarketing sites, frustrated customers on eSupport sites who likely then call customer support, thus wasting a lot of the company's money, disinterested viewers/reader on eMedia sites who quickly abandon the site, thus losing opportunities for advertising revenue, and unproductive employees on intranets.
Web site design is a manual attempt to solve the problem of information discovery: to organize information in a way that the designer imagines helps a user find what they are looking for. While effective in some cases, trying to find information in this way often is slow and ineffective as users resort to poking around a site looking for the information they need. Most users actually abandon a site if they do not find what they are looking for within three clicks. One problem is that the site is static. In more recent years, Web analytics has emerged as an attempt to alleviate this problem. Designers can see all of the actions that happen on their site and collect it into reports that aim to provide some guidance on how the site can be redesigned or reconfigured more effectively. While providing some benefit, the information provided is often ambiguous and provides only hints rather than concrete suggestions for improvement. At best, the process is tedious, requires a great deal of manual effort as designers redesign the site in line with learnings, and takes a long time. The feedback loop is thus slow and ineffective.
Automatic content recommendation is a completely different strategy that emerged very early in the life of the Web. Search engines, such as Google, Yahoo, and Ask, are the common manifestation of such techniques. The basic idea is that the user explicitly describes what they are looking for in the form of a search query, and an automatic process attempts to identify the piece of content, most often a Web page, that best matches their query. The approach for doing this amounts to looking at all possible documents and recommending those where the target query occurs within the text with highest frequency, i.e. keyword match. Modern adaptations of this basic technique add layers of sophistication, e.g. natural language processing, but the key in these approaches is still to use properties of the content itself, e.g. words within the document, to determine the ultimate relevancy ranking. This represents the first content-centric phase of content recommendation (see Figure 1). Many variations to this approach exist including, most notably, meta-tagging. In this approach, the content creator selects a small number of terms to describe the content. These terms are embedded within the content, often as HTML meta-tags, but are not necessarily made visible to the consumer of the content. This is one way to allow search engines to search content that is not text-based, such as video clips. This approach was very common in the late 1990's, but has since fallen out of favor due to the enormous effort required to keep the meta- tags up to date and in-synch with changes to the content.
In many ways, this first content-centric approach on the surface make a lot of sense, i.e. if you want to recommend content, consider the content itself. A-key_ problem with this approach is that it often brings back lots of documents that may be relevant but not useful. Many documents may exhibit a strong keyword match, but are outdated or not truly relevant to the user's current interest. If users do not find a useful result within the first few results, they are most likely going to abandon the search.
Keyword match does not really reflect how we find information most efficiently in the real world. In day-to-day life, the best way to find the information/products/services we are looking for is to ask someone who knows to point us in the right direct. The second phase of content recommendation thus shifts the focus from content to users (see Figure 1). Google's "PageRank" algorithm, though we place it in phase 1 , was really a transitional technology that harkened the coming of phase 2. The page rank algorithm's break-through was to consider not only the content of the page itself, but how it had been linked to from other pages by other Web site designers. This represented a form of voting on the importance of Web pages. Thus, pages that were linked to more often were seen as more valuable. While bringing people into the equation, the people who were voting were Web designers rather than the consumers of the content, i.e. the users. Phase 2 of content recommendation is all about the users. The three most well known approaches that fall into phase 2 are: folksonomy, profiling/behavioral targeting, and collaborative filtering.
Folksonomy
The first, folksonomy, represents the most straight-forward addition to phase 1. Here, users are allowed to tag content themselves. So, rather than the Web site designers, or a single designer, being responsible for coming up with the best set of keywords to describe the content, folksonomy lets the community do it. Once this is done, those community created tags essentially become part of the content and can be searched using traditional information retrieval/search techniques developed in phase 1. A big assumption in this approach is that the subset of the community who takes the time to tag the pages explicitly, ultimately produce a description that is valid and representative of the larger community's opinion. This is often not the case.
Profiling/Behavioral targeting
Profiling/Behavioral targeting in its common form also borrows heavily from phase 1 techniques. Here, based on a user's prior behavior on a site, e.g. the pages clicked or products purchased, a profile is built for that user. This profile may, in the simple case, be based on a collection of pages clicked or products purchased. The profile may also make use of the content itself or meta-tags to attempt to discern the user's historical topics of interest. For example, if a user purchased many films tagged as "horror" by content providers in the past, then a behavioral targeting system would tend to recommend more "horror" films to the user. A major assumption here is that a user's historical behaviors are a good predictor of future interest. While sometimes true, this assumption tends to fail at least as often as it works. The reason for failure is that people exhibit a variety of behaviors depending on their current interests, context, and goals. For example, someone who bought a few books on guitar as a one-time gift for his wife a few weeks ago, might continue to be recommended guitar books by a behavioral targeting approach, even though he may no longer have interest in that topic. Profiling approaches often also take into account demographic data of users, such as age, gender, and geographic location. The core belief underlying such approaches is: If I only knew enough about a user I could predict exactly what they want. However, some basic introspection uncovers the fallacy underlying this approach. For example, I may know more about my wife than any person or machine. I am in this way the ideal profiling system for her. However, I am unable to predict what she might be currently looking for online without some context.
Collaborative filtering
Collaborative filtering is another user-centric approach which is arguably the most strictly user-centric. Here, users are compared to one another based on common purchases, click histories, or explicit ratings. For example, based on a person's previous ratings of movies on a movie site, find other people who most agree with that person's ratings and recommend other movies that he liked. Standard "people who bought this also bought that" approaches are actually a variation on the collaborative filtering approach, where a user's most recent action serves as the sole basis for identifying similar users. This approach was made popular by Amazon's recommendation engine. A big assumption in this approach is that some global similarity measure between users based on past behavior is a useful way to predict future interest. This is a flawed assumption, however. One may be very similar to some of his co-workers in a work context, e.g. they are all Java engineers, with similar interests regarding programming, but quite different from these co-workers when outside of the office, on the golf course for instance. In the context of golf, one likely has a very different peer group. Grouping users at a global level is more often misleading than helpful.
Another weakness in all of the user-centric approaches in phase 2 is the reliance on either explicit measures of liking or overly-simplistic implicit measures. Explicit measures include asking the user to indicate their liking of a particular piece of content, e.g. on a 1-5 scale. Such approaches are almost always biased because they represent a very small percentage of the population. Further, the people who are taking the time to do these ratings are not representative of the community as a whole. They tend to be very opinionated or reflect a specific personality type that is willing to spend the time to voice their opinion.
Those approaches that leverage implicit observations as a rule either look at clicks or purchases. Clicks are a flawed way to assess liking because getting someone to click on a result has a lot more to do with an intriguing, perhaps even ambiguous, title and location on page. It tells one nothing about how a user felt about the content once it is selected for viewing. At the other extreme, many systems use purchases as a measure of liking. While purchases are a reasonable way to assess this, they are too limited. For example, when buying a camera, one may seriously consider a number of products before making a decision. All of that information could be valuable to others interested in cameras above and beyond to the one ultimately purchased.
SUMMARY OF THE INVENTION
An embodiment of the invention represents Phase 3 in the evolution of content recommendation (see Figure 1). Here, the idea is to start by understanding the current user's context, i.e.: What is their intent? What are they looking for? Based on this understanding, then find the appropriate peer group representing other users who are most like the current user in the context of this identified interest. From there, find the content that that peer group identifies as most relevant to the current context.
The approach taken in the invention is context-centric or, put another way, intent- centric. The techniques used to achieve this approach are described later and are fundamentally based on the UseRank technology and affinity engine described, in part, in U.S. patent application serial no. 11/319,928, filed December 27, 2005, which is incorporated herein in its entirety by this reference thereto. It should be noted that all previous approaches, including the content- centric and user-centric approaches, are subsumed and improved by the Phase 3 approach manifested in the invention. Because the invention adds the dimension of context to the picture on top of users and content, it is always possible to choose to ignore context and use the system to provide phase 2 functionality, such as collaborative filtering or behavioral targeting/profiling. However, even these previously known approaches are significantly improved in their functionality based on a critical aspect of the invention which provides full- spectrum behavioral fingerprints.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a schematic overview of the evolution of content recommendation leading up to the invention;
Figure 2 is an architectural schematic diagram showing a method and apparatus for context-based content recommendation according to the invention;
Figure 3 is a schematic architectural diagram showing affinity engine integration according to the invention;
Figure 4 is a schematic architectural diagram showing a dynamic, adaptive, realtime platform of community wisdom and community guided Web according to the invention;
Figure 5 is an architectural schematic diagram showing a community wisdom platform according to the invention;
Figure 6 is schematic flow diagram showing the process of translating context to recommendations; Figure 7 is schematic flow diagram showing the process for achieving a virtual folksonomy according to the invention;
Figure 8 is a schematic diagram showing topic attraction, topic match, and activeness according to the invention;
Figure 9 is a schematic flow diagram showing the behavioral fingerprinting process according to the invention;
Figure 10 is a graph showing a long tail marketing model with an architectural overlay according to the invention;
Figure 11 is a "Wisdom of Crowds" pseudo-equation;
Figure 12 is a schematic flow diagram showing an affinity engine memory prediction machine according to the invention;
Figure 13 is a screen shot of the welcome screen of the customer portal according to the invention;
Figure 14 is a screenshot of a lift report within the customer portal according to the invention;
Figure 15 is a screenshot showing community-guided e-commerce according to the invention, including recommendations for competing and complementary products according to the invention;
Figure 16 is a screenshot showing community-guided e-travel according to the invention;
Figure 17 is a screenshot showing community-guided marketing according to the invention; Figure 18 is a screenshot showing social search and community information popup/overlay according to the invention; Figure 19 is a screenshot showing community-guided online support according to the invention;
Figure 20 is a screenshot showing a community intranet/knowledge portal according to the invention;
Figure 21 is a screenshot showing community-guided media according to the invention;
Figure 22 is a screenshot showing community-guided social search for media according to the invention;
Figure 23 is a screenshot and schematic diagram showing community-guided cross-site recommendations for media according to the invention; Figure 24 is a screenshot showing community-guided video recommendations according to the invention;
Figure 25 is a screenshot showing community-guided topic recommendations according to the invention;
Figure 26 is a schematic flow diagram showing the community-guided context- relevant Ad recommendations according to the invention;
Figure 27 is a schematic flow diagram showing a live connection according to the invention; Figure 28 is a block schematic diagram showing the system architecture of a preferred embodiment of the invention;
Figure 29 is a block schematic diagram showing code snippets of the client integration through JavaScript tags and REST in a preferred embodiment of the invention; and
Figure 30 is a block schematic diagram showing an AJAX platform according to the invention.
DETAILED DESCRIPTION OF THE INVENTION
The invention represents Phase 3 in the evolution of content recommendation (see Figure 1). Here, the idea is to start by understanding the current user's context, i.e.: What is their intent? What are they looking for? Based on this understanding, then find the appropriate peer group representing other users who are most like the current user in the context of this identified interest. From there, find the content that that peer group identifies as most relevant to the current context.
The approach taken in the invention is context-centric or, put another way, intent- centric. The techniques used to achieve this approach are described later and are fundamentally based on the UseRank technology and affinity engine described, in part, in U.S. patent application serial no. 11/319,928, filed December 27, 2005. It should be noted that all previous approaches, including the content-centric and user-centric approaches, are subsumed and improved by the Phase 3 approach manifested in the invention. Because the invention adds the dimension of context to the picture, on top of users and content, it is always possible to choose to ignore context and use the system to provide collaborative filter, profiling. However, even these previously known approaches are significantly improved in their functionality based on a critical aspect of the invention which provides full-spectrum behavioral fingerprints. Figure 2 is an architectural schematic diagram showing a method and apparatus for context-based content recommendation according to the invention.
Full-spectrum behavioral fingerprints provide a significant advancement over current state-of-the-art implicit ratings, which essentially amount to click analysis and purchase behavior. First, they take into account a wide variety of user behaviors including, but not limited to, clicks, time spent on a page, scrolling and mouse movement, explicit actions such as print, email, bookmark, links used, frequency of return visits, and searches performed. Second, all of these behaviors can be cross-correlated with the current user's behaviors on other pages, and also with the rest of the community's and identified peers' behaviors on the current page. From this analysis, a probability that the user has found value in a particular piece of content can be discerned and fed into the learning system.
A further aspect of the invention concerns the seamless integration of existing strategies for automatic content recommendation (see Figure 3), including search engines 301 and profiling systems 302, and ad servers 303, as well as systems for manual recommendations including merchandizing rules and systems 304. The invention also seamlessly integrates with other information sources related to the content, such as product catalogs 305, which can be used for purposes of display, filtering, or learning.
Finally, because of the comprehensive nature of the information collected and the learned affinities, the system represents a general wisdom platform 501 on top of which many applications can be built (see Figure 4 and Figure 5), such as SocialSearch 502, Content 503 and Product 504 Recommendations, Insights 505, eMail 506, and Live connect 507 which are tailored to various applications, e.g. eCommerce, eMarketing, etc. Additional applications include reports and integrations with bid management systems for search engine optimization and search engine marketing 508, mobile and IPTV 509, and custom applications or mashups 510. Some of these applications are described later. Each of these critical aspects are now described in greater detail
Context- and Intent-centric
A core advancement in the evolution of content recommendation embodied by invention is the process for identifying and representing the users current topic of interest and converting that interest into a set of useful recommendations and information. The steps in this process are shown in Figure 6 and outlined below:
Step 1 : When a user comes to a Web site, they immediately begin to establish their current context. They might do this by entering a query into a search box on the site, navigating to a particular section, or may even have established some context before arriving at the site by doing a search on an external search engine, such as Goggle or Yahoo, that led to this site. All of this information is captured by an Observer Tag, i.e. a piece of HTML/JavaScript embedded in the Web site. As the user continues to move through the site, they may also show interest in a particular page or piece of content, based on their implicit actions. Interest is determined based on various behaviors collected by the Observer Tag and analyzed using the invention's Full Spectrum Behavioral Fingerprint technology 601 (described in greater detail below). These pages of interest further contribute to the user's context.
All of this information is stored as the user's current context vector 602, which is a hybrid vector of terms and documents with weights on each entry reflecting how strongly that term or document reflects the user's current context. As a user enters search terms and/or clicks navigation links, the vector entries corresponding to the terms and phrases entered or clicked are incremented to capture expressed interest. As these actions move further into the past, the corresponding entries are decremented or decayed. Similarly, documents that a user clicks on or indicates interest in, as determined based on their implicit actions, increment the corresponding vector entry to a degree based on the level of interest and level of certainty determined by the invention. The result is a representation of the user's current context as a context vector. It is also possible to increment the context vector further, based on historical actions and historical contexts of interest of the user. Although this is not generally done in the preferred embodiment because such information can be misleading, it is possible to increment the context vector to a lesser degree based on these historical contexts in applications where historical behavior is considered to be relevant.
Step 2: Expand and refine the context vector into an intent vector 603, based on affinities/associations learned from the aggregated wisdom 604 collected from observations on the community as a whole over a longer term. For example, the user may have entered a query about digital SLR cameras and expressed implicit interest in a Nikon page. In the context vector, the entries corresponding to "digital SLR camera" as well as "digital," "SLR," and "camera" to a lesser degree are incremented, as is vector entry corresponding to the particular Nikon page of interest. To create the intent vector, the system looks at affinities between the terms and documents in the context vector to other terms. For example, the community wisdom may have discovered that the term "high- resolution" may be highly associated with both the term "Nikon," as well as with the specific Nikon page of interest. The intent vector is thus incremented at the entry corresponding to "high-resolution." Similarly, other documents that are associated to the terms in the context vector may be discovered based on the community wisdom and are incremented in the intent vector. For example, a Canon camera page may have strong affinities to "SLR camera" and become part of the intent vector to some degree. The affinities 604 that allow the expansion of the context vector into an intent vector are determined by the affinity engine which is described below and in detail in U.S. patent application serial no. 11/319,928, filed 12/27/2005. In summary, the affinity engine learns connections between documents and terms, documents and documents, terms and terms, as well as users to other users, documents, and terms by watching all of the implicit behaviors of the user community on the site and by applying behavioral fingerprinting and the use rank algorithm to determine interest and associations. The ability to translate context into intent effectively is a key aspect of the invention.
Step 3: Identify the group of users who share affinity to the current intent, as well as those users who exhibit behavior most like the current user within the context of that intent. In U.S. patent application serial no. 11/319,928, filed 12/27/2005, these are called "experts" and "peers" respectively, but here we combine them both under the name "peers." This peer group is represented by a user vector 605 and each user entry in the vector may have a weight indicating how strong of a peer he is to the current user in this context. Although not generally recommended, the system is also capable of limiting the peer group to those users who match the current user based on a set of predefined attributes, such as age, gender, location, or other demographic variables. Similarly, the peer group can be limited to those users who visit the site at the same time of day, e.g. morning, afternoon, evening. In some cases these attributes can be used to influence the peer weights for users, such as giving slightly more peer weight to those users who also best match the user along these predefined variables. The invention can also learn these weights by comparing behavioral patterns, e.g. documents found useful in the simplest case, within and across each of the predefined attribute groups. Higher intra-group similarity in behavior compared to inter-group similarity indicates the group is differentiated and thus a higher weight of influence for groupness is warranted when influencing the peer group. Similarity can be measured by looking at similarity of documents and terms used through one of many similarity calculations, e.g. cosine similarity, on users' aggregated interest vectors.
Step 4: Look at which documents have highest affinity to the identified peers within the current intent 606. To do this the affinity engine looks at a combination of factors, including the overall usefulness of a piece of content, as represented by the activeness vector, learned affinities between terms and content, as represented by term-doc matrices, and predicted navigational patterns, as represented by next-step matrices, associated with those peers the factors are weighted according to their peer weight, and used in aggregate to compute those documents with highest affinity to the current intent. These documents become the unfiltered recommendations 607. Filtering, described in greater detail below, may now be applied to limit or augment those community recommendations.
Step 5: Now that the recommendations have been identified and appropriately filtered, a final optional step is to ask the affinity engine for community information on each of the recommendations 608. This information can be combined with other asset information, such as title, size, or price, and displayed to the user to help them understand the community wisdom underlying a recommendation. Many aspects of the community wisdom associated with a document in the affinity engine can be exposed, but in common implementations we expose the number of users in total who found value in the document, the number of peers who associated the document with the current context/intent, and the terms/phrases the peer community has associated with the document.
This latter piece, i.e. terms associated to the document, is called a virtual folksonomy because it represents terms that the community has associated to the document, but unlike a traditional folksonomy, discussed above, where users must explicitly make the associates, the virtual folksonomy is created automatically by the affinity engine based on the implicit actions of the user community. Terms that are searched or clicked as navigation links and lead, within one or more steps, to useful content, as determined by behavioral fingerprinting, become automatically associated. These term-doc connections are both fundamental to the process of providing recommendations, as well as providing useful feedback to users on the topics associated with each recommendation. When these terms are displayed to a user, they can be made clickable so that the user can click them to provide further input regarding their current intent. This information then becomes part of their context vector, the whole process repeats, and new recommendations may be provided based on this new information. Figure 7 provides a conceptual diagram of a virtual folksonomy, contrasted against traditional approaches for connecting terms to content.
Time (decay, trend, and fad detection)
Time factors in heavily to all computations in moving from context to recommendation. First, all information collected by the affinity engine is subject to time decay. This means that, for example, information from last month has less of an influence on the calculations than information from today. This is important because the site may change, e.g. new content added or removed, people may individually change, e.g. their interests may change, and the community as whole may shift interest, e.g. new fads. Although there is a default decay rate for all information, some information may decay away more rapidly if it is determined that this is necessary. For example fad behavior, such as interest in Christmas products that come and go quickly, may need to be decayed more quickly to prevent recommending Christmas products too long after Christmas has passed. To prevent this, the system runs a trend detection system across all content on a periodic basis, e.g. once a day, or every five minutes for very time-sensitive sites. Trend detection can be done in a number of ways, but one way is to use the Mann-Kendall algorithm. Other ways include various regression techniques that employ least-squares fit. If a strong negative trend is detected for a given piece of content, the information associated with that content in the affinity engine is decayed at a more rapid rate. The result is that the likelihood of the affinity engine recommending this piece of content is reduced.
Another algorithm for fad detection that is run on a periodic basis is cyclical fad detection. If a piece of content shows a strong positive trend and that same trend can be found at regular intervals in the past, then that piece of content can be automatically boosted in importance in anticipation of the coming trend. In a traditional Web site, even where the site proprietor is trying to understand customer behaviors, techniques that are typically used survey the customer and analyze the customer responses to produce a report. By the time the report is analyzed, the results of the report are out of phase with the actual situation at the time the report is being reviewed. For example, in a traditional system, a commerce site might collect feedback during the Christmas season and redesign their site in February in response to that information. In effect, the commerce site is trying to sell Christmas products in February. Alternatively, analytics reports may be created offline and analyzed by a team of specialists to infer trends and determine appropriate actions; a process which may take weeks or month and again lead to out-of-date site modifications. In this way, the automatic nature of the invention herein allows a Web site to adapt in real-time to discovered trends and fads, providing recommendations to users that fit the current context and time.
In addition to contributing to the process of automatic content recommendation, however, the trend and fad detection algorithms used by the invention can also be exposed to owners of the content system, e.g. the Web site, through reports within a customer portal. In the preferred embodiment, such reports on community trends and fads can, for example, be used by merchants to promote certain products or content at the right time, making such promotions more effective. Given the example of products that are sold during the Christmas season, if such sales were to die out after Christmas, then the proprietor of the commerce site would be able to follow the sales curve based on community interaction with the Web site. If, on the other hand, an emergent news story drives demand for a product quickly, the invention allows the merchant to watch the curve of demand and respond in real-time. Thus, if people are all of a sudden dramatically interested in a particular piece of content or product, the proprietor of the Web site can provide that content or product more quickly. If the demand dies out quickly, for example the interest was based on a fad, then the proprietor of the site can adjust to that fact as well. Affinity Engine
As already mentioned, and as discussed in greater detail in U.S. patent application serial no. 11/319,928, filed 12/27/2005, the affinity engine learns connections/affinities between terms-documents, terms-terms, terms-users, documents-documents, documents-users, and users-users. (Note: the terms document, content, and asset are used interchangeably throughout this document. In all cases, these terms refer to any type of medium that provides information or services including, for example but not limited to, webpages, word documents, pdfs, video, audio files, widgets, and/or products). In the preferred embodiment, all affinities are stored as sparse matrices and vectors. However, there are many alternative ways of storing such information known to those skilled in the art. Although there is ultimately a single number that can be calculated to represent the affinity between any two entities, e.g. a document and term, there are usually several sub-affinities that are combined in a weighted sum to arrive at that single number. The weights on that sum may be dependent on context. For example, documents have at least three kinds of affinities to other documents: similarity in virtual folksonomy, i.e. terms the community has associated to the documents; similarity in user groups, i.e. how many users have used both documents; and similarity in navigational patterns usage, e.g. do users show a pattern of finding value in one document after using the other. A variety of mathematical techniques are available to be employed as appropriate to each kind of data, although a combination of vector space models and custom probabilistic techniques are currently used by the preferred embodiment of the invention. See U.S. patent application serial no. 11/319,928, filed 12/27/2005 for more discussion.
One important aspect of the invention is that all sub-affinities are always computed bi-directionally. To illustrate this bidirectionality, we describe its application to the term-document sub-affinity derived from the virtual folksonomy. Figure 8 is an architectural diagram showing the aspects of the invention that involve topic attraction and topic match, the two dimensions that make up the bidirectional virtual folksonomy connection between terms and documents, as well as general activeness. In the example of Figure 8, a query is provided 801 and a percentage of respondents who ultimate find value in one of three example documents are indicated. The query involves a particular Nikon camera and it can be seen that 20% of the respondents find value in a Web page focused on high-resolution cameras 802, 78% found value in the overview page for the specific camera 803, and only 2% of users making this query found value in the detailed specification sheet for the camera 804. As a side note, remember that value is determined by the full spectrum of behaviors exhibited on a given page by the user (described in detail later).
There are two ways to consider the virtual folksonomy connection between the query and each example document. Topic attraction starts from the query and considers the proportion of users who searched for "Nikon 580X" and subsequently found value in each document. In this example, the Overview page 803 has highest topic attraction, with 78% of users finding value there, followed by the High-resolution camera page 802 with 20%, and then the spec sheet 804 with only 2%. Note that this example is simplified for the purposes of illustration. In reality, we also break down the query into its component parts and consider the affinities of these to each document, as well as consider affinities between other terms that the affinity engine has learned have similar meaning to this one. In this way, topic attraction can be thought of as a problem of predicting the probability that a user finds value in a given document, given the intent represented by the query. In the preferred embodiment arriving at this probability is accomplished using a combination of probabilistic techniques, including Bayesian inference.
Topic match looks at it from the other direction starting from each document and considering all of the other terms that have been associated with it through the behaviors of other users. So, for example, the High-resolution camera page 802 may have many other camera names connected to it and, thus, the degree of focus on the particular query entered is lower than both of the other two pages, whose term-document connections are more highly concentrated around "Nikon 580X" specifically. In fact, the Spec-sheet 804 may turn out to be the document with term connections most focused around "Nikon 580X" and thus have the highest topic match for this query, even though it has the lowest topic attraction. As with topic attraction, in reality, topic match also breaks down the query into its component parts and takes into account affinities between other terms not in the query. In this way, topic match can be seen as finding the document with the best overall match in topic to the intent represented by the query. In the preferred embodiment, a vector space model and modified cosine similarity technique, similar to that used by traditional full-text search, is used to determine the degree of matching.
These two directions of considering the affinity of each document to the query, i.e. topic attraction and topic match, are combined in a non-linear weighted sum along with the final factor of overall activeness i.e., usefulness, to arrive at a value for the virtual folksonomy sub-affinity. This sub-affinity is then combined with other sub-affinities, such as those based on navigational patterns, and filtered through the lens of peer groups, to arrive at an ultimate ranking of documents against the query, called UseRank. Similar bidirectional techniques are used to compute UseRank when providing recommendations on a particular Web page, and when considering the full context and intent vectors.
There are several methods for combining the various sub-affinities and the bidirectional dimensions therein to arrive at the ultimate UseRank of documents for a particular user and context. In the preferred embodiment, one or more thresholds are applied to each dimension, including absolute and relative thresholds before they are combined together based on hybrid arithmetic- geometric weighted average. Each resulting sub-affinity is then similarity subjected to a thresholding before being combined with other sub-affinities again based on hybrid arithmetic-geometric weighted average. The values for the thresholds are generally fixed, however, the individual weights for the weighted averages can also be adjusted and learned based on the success of the result set returned by the affinity engine.
Full-spectrum behavioral fingerprints
As discussed herein, a key feature of the invention is the processing and analysis of implicit observations that are made during the actual use of a Web site, for example, versus traditional approaches in technology which use explicit feedback. One key advantage of the invention, as confirmed through scientific studies, and as understood from human psychology and sociology, is that humans are very bad at giving feedback, particularly if the feedback must be given explicitly. If a person is being surveyed, the person does not have an incentive to give actual accuracy in the form of feedback. One aspect of the invention eliminates such survey bias by using the implicit behaviors observed during use of particular materials on the Web by individuals within a community. Thus, the invention trusts what people do but does not watch what they say. The invention watches people's behavior through their actual action, implicit behaviors, and can accurately interpret what their true intent is and whether or not they dislike or like something. Thus, the invention observes behavior versus click actions.
Current technologies focus on clicks, such as Web link clicks. If a person clicks on a link, that click is reported. The click may be reported as having resulted in a viewed Web page, even though not much time is spent on the page on which the person clicked. Thus, this known approach is not a good indication one way or another if the page is good or bad. If a link is put in a prominent position on a Web page, then people are likely to click on it. However, when people get to the location indicated by the link, they may immediately leave the site. This is why the number one used button on the browser is the BACK button. The use of the BACK button could indicate like or dislike of a site. Accordingly, the invention recognizes that it is the action of the user after the click that matters and not the click itself. The invention tracks behaviors beyond the click to determine whether a page is good or bad. Thus, if a person backs out of the page, it is considered negative feedback, i.e. the person did not like it. In this way, clicks can identify a very negative reaction in the invention. If a person goes to a link, follows the link down, spends time there, and does other things, that behavior is tracked as well. If a peer group validates the behavior as consistent, e.g. a significant number of the group members exhibit the same behavior when reacting with the page, then the page is considered to be a good page.
An embodiment of the invention goes one step farther. Not only does the invention determine which assets are useful based on behavioral fingerprinting, but it also learns the context associated with the usefulness. For example, the invention can learn that a particular camera page is very useful for users that show interest and intent in high definition cameras, but not if the intent is compact cameras. In this way the affinity engine is able to distinguish the usefulness of assets based on the context and intent expressed by a user through their implicit actions.
The primary input to the affinity engine, which drives all of the learned associations (affinities) is the behaviors of users on the site. In the preferred embodiment, all behaviors are captured by the ObserverTag, i.e. a piece of HTML/JavaScript embedded within the Web site, typically in a header or footer template. Although this is the preferred method for capturing user behaviors, it is also possible to capture user behaviors using a browser plug-in or lower-level network traffic analyzer. The behaviors captured include:
• Pages visited and in what order
• Time spent on each page
• Links clicked
• Searches performed • Time spent scrolling on a page
• Portion of page visible in the browser window, and for how long
• Page sub-elements opened/closed • Media launched, time spend viewing that media, and explicit action taken on the media
• Use of the back-button
• Repeated visits to a particular piece of content • Mouse movement while on the page
• Ads viewed and Ads clicked
• Explicit actions, e.g. Add to Cart, Purchase, Email, Save, Print, Print Preview
• Virtual Print: content returned to frequently over the course of several hours or days
• Virtual Bookmark: content left open over multiple hours or days with intermittent periods of activity
• Entrance and Exit paths
The captured information is send back to the affinity engine for processing. Figure 9 shows the process for analyzing these behaviors or other implicit data captured from various user interface devices. The input to this process is the user trail 901 which includes all assets visited and the implicit (and explicit) actions observed on those assets. There are two main steps in processing the behaviors that combine to provide an understanding of what content is useful and in what contexts. The first step is determining whether a user is finding value (usefulness) in a particular piece of content. Conceptually, the more time spent on the page in think mode, i.e. user is processing the information, the higher the likelihood that it is useful. Think mode can be approximated by non- scrolling time on a page, where some scrolling or mouse-movement has been detected with a specified time range. Repeat visits and percentage of page seen are also generally good indicators of liking or usefulness. For each user and piece of content, we can create a behavior vector 902 with each entry in the vector representing one of the features listed above or a predefined combination of features. However, not all content is created equal nor are all users created equal. We normalize the behavior vectors in a several ways. First, we normalize by user to make behavior vectors comparable to the rest of the user population 903. For example, some users may read slower than others, affecting their mean time spent on a page. Some users may use the mouse more than others. One way to accomplish this normalization is by translating all entries into z- scores which adjusts for means and standard deviations specific to that user.
A second normalization is to normalize based on the content. This is done based on inherent or specified properties of the content. For example, 30 seconds spent on a one paragraph document likely has a different meaning than 30 seconds spent on a ten page document. Dwell time can thus be normalized based on page length or number of words. Similarly 30 seconds spent viewing a 30 second video has a different meaning than 30 seconds spent viewing a five minute video. In many cases the Observer Tag is capable of capturing these page characteristics through information within the DOM (Document Object Model). However, when not available in the DOM, the invention provides other mechanisms for collecting the needed information from offline catalogs, e.g. a product or media catalog, or allowing the Web site designer to add explicit information in the page itself, e.g. as meta-tags or added JavaScript variables. For example, on an e-commerce site, certain pages may be defined as information versus product pages. The system can normalize the behavior vectors for each of these content groups independently. In this way, the dwell time necessary to indicate liking can be adaptable to the type of content. All behaviors (features) can be made adaptable in this way. In the current implementation, these normalization strategies are hard-coded. However, the invention allows the possibility for plugging-in various machine learning techniques to learn the appropriate normalizations.
We now have a behavior vector normalized to the user and the content 904. In the current system, a predefined set of rules are applied to determine whether this behavior vector represents liking/usefulness. Each of the normalized features, in turn, is considered to determine whether it meets the pre-specified thresholds for indicating usefulness. Each passing feature increases the probability of usefulness 905. In some cases, the thresholds for a given feature are pre-specified and tuned by the person implementing the system based on previous experience. In other cases, these thresholds are dynamically determined by the system. For example, certain features are known to exhibit a bimodal distribution and the threshold can be dynamically determined to lie between the two modes. The invention also provides a mechanism for plugging- in various machine learning techniques to learn which features are most important, and thus dynamically learn the rules for converting from the behavior vector to the probability of usefulness.
Once usefulness, or probability of usefulness, has been determined, the second step in the analysis of behaviors is to understand the entire context surrounding the use of that piece of content, including searches done prior to the use, links clicked prior, and pages used prior. All of this information combines with information about the user to influence affinities 906 (affinities can be learned and stored in a number of ways, described in more detail in U.S. patent application serial no. 11/319,928, filed 12/27/2005). Before doing so, however, there is one final step: validation. Validation is a form of noise filtering wherein an affinity connection is established, e.g. between a document and term, if and only if enough similar users have, through their behaviors, confirmed this connection. One user making the connection is not enough for an affinity to emerge. A minimum number of users in the same context (peers) must have exhibited similar behaviors and connections for that affinity to be validated. We call this peer-validated behavior.
UseRank™
UseRank™, the ultimate ranking of the usefulness of content based on a user's context and intent, based on learned affinities and full-spectrum behavioral fingerprinting, can be compared to the very popular PageRank strategy made famous by Google. In PageRank, each Web page is given a value based on the number of other pages linking to it. In addition, links coming from pages that are themselves of high value raise the PageRank even more. This democratic strategy is quite effective in getting users to useful Web sites based on their Google query, but generally breaks down once the user begins to look for further information within the Web site itself. The main reason for this is that pages within a Web site, particularly those down into the long tail of content, are not heavily linked to either externally or internally. Internally, the linkages are determined solely by the structure of the Web site and more links does not necessarily mean more value. In addition, many of the pages on such sites are in formats other than HTML, e.g. PDFs, Word Docs, or videos, and do not link to other content at all.
The UseRank methodology, based on user behaviors on the Web site alleviates all of these problems. Instead of relying on the linking of documents by Web designers, it relies on usage of documents by Web users, who are a rich source of information on any Web site. Usage is a truly democratic way of learning what content is most valuable. In fact, the designers of PageRank recognized their approach as an approximation of user activity. Google's choice to approximate user value based on PageRank makes sense given the privacy concerns associated with tracking a user's behavior across the entire Web. In the preferred embodiment of UseRank, we track only the behaviors of user's on sites instrumented with the Observer Tag, and always in an anonymous fashion unless otherwise configured by the Web site deploying the Observer Tag.
Long-Tail
Another important aspect of the invention emerges from the context-driven approach underlying the affinity engine. Many previous systems that are content-centric or user-centric suffer from a problem where only the most popular product/content is ever recommended. Because these systems lack a deep understanding of the user's current context/intent, this is the best that they can do. The invention provides a context-centric approach that allows the affinity engine to narrow its focus to the subset of users (peers) and content that match the current user's context/intent, even if that intent is not particularly popular in the grand scheme of things. This is what is known as the long tail of the products, Ae. those products that may not be the most popular overall, but are extremely important because they strongly meet the need of a small but important subset of the community (see C. Anderson, The Long Tail: Why the Future of Business Is Selling Less of More, Hyperion Press (2006)). The invention is capable of identifying that long-tail interest along with the associated peer group and related content and can thus recommend those important long- tail products/content at the appropriate time (see Figure 10). This is critical for product/content providers because it is often these long-tail products that are most useful to the community and often lead to the highest margins or benefits for the company itself.
Thus, the invention uses like-minded peers as an enabler to target small target segments along the long tail, instead of using individual problem personalization and historical interest as known in the prior art. The invention provides contextual targeting. People's needs within a given context are typically similar among like- minded people. An insight of the invention is that people have hundreds of profiles, if not thousands of profiles. These profiles have no cross connections to reality, so it is difficult to predict an individual's use level without context. Once the individual is in a particular context, i.e., among like-minded peers, the individual behaves similarly to everybody else within the group context. Thus, if a person is visiting a golf equipment site to buy a golf driver or clubs, the person behaves in a manner similar to other golfers. It does not matter what the person's political bias is, or their cultural background. These aspects of the individual have no relevancy because the person's present context, which is based upon group membership, is more relevant.
Wisdom of the Crowd
The invention represents a new approach to leveraging implicit community wisdom to create adaptive Web sites and other information portals. In his book,
The Wisdom of Crowds, James Surowiecki explains how the collective intelligence of a large group of average individuals almost always outweighs the intelligence of experts. To illustrate the concept, he uses an example from a county fair, where a group of fair attendees attempted to guess the weight of a cow. A group of so-called experts, e.g. butchers, dairy farmers, etc., also made their guess. In the end, the experts were all off on their guesses by a large amount. The average weight guessed by the crowd of non-experts, however, came within one pound of the actual weight of the cow. Surowiecki goes on to show how a similar phenomenon can be seen in everything from stock market prediction to democratic governance. This notion that groups of individual actors can collectively exhibit a level of collective intelligence going beyond even the sum of the individual actors themselves has been known in the fields of biology and artificial intelligence as emergent behavior and collective intelligence. Numerous examples exist in nature where very simplistic individual animals, such as ants or bees, in collection exhibit extraordinary intelligence and resourcefulness in meeting the needs of group, such as finding food or building nests.
In the context of Web site design, Surowiecki's experts are the Web designers. These experts attempt to make correct decisions on which Web pages should be linked to what others, such that visitors are best able to find what they are looking for. Web designers may also spend a lot of time an effort tuning search results to match the expected needs of their visitors. The crowd in this context is the large group of Web site visitors who come to the site. Generally, the crowd has no direct impact on the organization of the Web site. They remain silent. The invention gives this crowd a voice, enabling their collective actions, i.e. a form of expressed opinions, to be collected and automatically drive decisions on Web site organization through their impact on recommended content. The invention thus taps into the wisdom of the crowd for the purposes of creating useful Web sites. Although in the preferred embodiment, the impact of crowd wisdom is sectioned off into specific regions of recommendations and social search within the Web site, it is easy to imagine an extended implementation where the entire organization of the Web site, all of its links and menus, for example, are ultimately driven by and adapted to the behaviors of the Web site visitors. In this way, the entire Internet made up of multiple interconnected sites could begin to evolve and adapt into a form that best meets the needs of Internet users.
Figure 11 provides a Wisdom of Crowds pseudo-equation showing how context/intent can be incorporated in a generalized way to extend the wisdom of crowds concept, as exemplified by an embodiment of the invention. In this pseudo-equation, each user's implicit vote is weighted according to the similarity between the current user's context and the context in which a user made that vote. This equation could be further extended to incorporate similarity between users. When all contexts are identical, this equation correctly reduces to a simple average.
Memory Prediction Machine
The actions of the Affinity Engine can also be considered as a form of Memory Prediction Machine. Recent learnings from cognitive science teach that the brain is structured to capture and encode associations between and among objects and concepts in targeted regions of memory, and organize those associations in a hierarchy moving from abstract to detailed. New stimuli are responded to and their consequences predicted, based off of the memories created from previous stimuli and learned associations thus encoded. The Affinity Engine similarly learns associations between users, objects, and contexts by observing interactions between them in the environment of Web sites and remembers those patterns in its memory, stored hierarchically. When a user exhibits a context that has been previously learned by the Affinity Engine, it lights up the appropriate associations within memory which triggers the prediction of those objects that best meet the needs of the current user and context. Figure 12 is an architectural schematic of the Affinity Engine in its function as a memory prediction machine. Seamless Integration with External Systems
An important aspect of the invention is its ability to integrate seamlessly with other recommendation systems, policies, and information sources. Some examples are shown in Figure 3.
Search Engine
When a user performs a search on a site implemented with the invention, the search terms are sent to the affinity engine for processing. The affinity engine determines the set of content that is most useful given the context of that search, based on learned affinities from past community behaviors. However, the affinity can also take into account opinions from external sources, such as a search engine. The search engine 301 has its own set of recommendations which the affinity engine can accept. The relevance ratings from the search engine can be combined with the information embodied in the affinity engine to produce a single unified set of recommendations (discussed more fully in U.S. patent application serial no. 11/319,928, filed 12/27/2005). An XML feed is a typical way for the system to interface with a search engine.
Product Catalog
The invention also provides a mechanism for automatically importing product or media catalogs 305. The preferred means for doing so is using an XML feed that is accessible to the system. There are three ways in which the catalog can be used:
1) Information from the catalog may be displayed along with the recommendations, e.g. summary or price; 2) Attributes from the catalog may be used to filter the result set, e.g. the Web site designer may want to restrict recommendations to products only or PDFs only a particular section of the Web site;
3) Content categorizations can be used to group content for the purposes of normalization in the behavioral fingerprinting process described above.
Ad Servers
Another aspect of the invention is that it is content-agnostic. The system can observe behaviors on and recommend Web pages, pictures, videos, documents, blogs, downloads, and even ads. In the case of ad recommendations, it is often necessary to integrate with an ad server. Ad servers 303 can provide functionality similar to both search engines and catalog systems as described above. The invention interfaces with ad servers in the same way as it does to these, typically through XML.
Merchandizing Systems
Merchandizing systems 304 in the general case provide a mechanism for the Web site owners to influence what products/content are recommended to users.
In the invention, it is preferred that the community be the primary driver of recommendations. However, there are times when the owners have specific needs that must be met independent from the community's expressed interest.
For example, the owner may choose always to recommend a particular product or piece of content first in a particular context, for example, there may be a promotion going for a product. Owners may also wish to influence the community to purchase products with higher margins, or prevent a certain piece of content from being recommended. There are two ways in which the invention can honor such influence. The first is through a custom rule interface that is one component of the invention. Here, Web site owners can log in and choose from a variety of predefined rule types including pinning, i.e. forcing a particular recommendation to show up; blacklisting, i.e. preventing a given recommendation or class of recommendations from showing up; and boosting, i.e. increasing the chances that a recommendation or class of recommendations show up, but still honor the community wisdom. All of these rule types can be applied globally or only in a given context, e.g. on a particular page, for a given search term, for a given class of users, in a particular time range. Another type of rule allows owners to honor fully the community wisdom, but influence the goal of the recommendations. For example, rather than focusing on recommending useful products, the affinity engine can be told to recommend products that a user is most likely to purchase. As another example, the affinity engine can be asked to recommend content that most likely leads users to a particular set of target pages. In addition to the custom rule interface, the invention provides a mechanism for importing rules in XML form from an external merchandizing application. When rule types exist in the external system that do not already exist within the invention, a custom rule plug-in can be designed for the invented system which matches the desired behavior of the external rule type.
User Profiling
A further type of system that the affinity engine can integrate with is user profiling systems 302. In its simplest form, a profiling system is a set of attributes, e.g. demographic attributes, associated with a user. These attributes can be passed along to the affinity engine on a recommendation request. As discussed earlier, the affinity engine can take these attributes into account when identifying a peer group, which then influences the recommendations that are returned. More sophisticated profiling systems require XML integration similar to other external components. Here, the affinity engine may have to dynamically contact the external profiling system during the peer identification process.
Preferred Embodiment In the preferred embodiment of the invention, a Recommendation System is implemented in a Software as a Service (SaaS) Model to provide automatic suggestions that help Web site visitors find products or content they like or need. Figure 2 is an architectural schematic diagram of a method and apparatus for context-based content recommendation. In Figure 2, interaction of users with a Web site produces implicit emergent behaviors. Such user interaction includes page referrals, links, entry trails, queries, page sizes, mouse movements, peers, negative experiences, virtual bookmarks, time spent, virtual printing, exit trails, and the like. Such behaviors are processed in a recommendation/affinity engine according to the invention, resulting in automatic content and product recommendations in the form of social search and navigation guides, as well as providing real-time feedback to a merchant such as regarding visitor clubs, and identifying content gaps (each of which is discussed in greater detail below). The following key processes comprise the recommendation system:
• Capturing Implicit Behavior — The implicit Web site behaviors 201 that serve as input to the recommendation system are captured client-side by small snippets of JavaScript code embedded within Web site pages, e.g. through a Web site template. The behaviors are then sent to the remote recommendation engine where they are processed using the full-spectrum behavior fingerprinting technology described herein.
• Distilling Collective Wisdom — The recommendation/affinity engine 202 processes all incoming information to identify emerging intent among Web site visitors and captures the collective wisdom of the crowd. The recommendation engine identifies and learns affinities between and among users, content, and terms, identifying consistent patterns of behavior and removing noise. The affinity engine also recognizes and adjusts in real time to changing fads and trends, as well as cyclical patterns of behavior, such as seasonal patterns, and other shifts in user interest. • Delivering Recommendations and Search Results — When a visitor arrives at a page, the recommendation engine is passed information about the users current context, either by snippets of JavaScript on the client, called Recommendation Tags, or by the Web site server, and a request is made for appropriate recommendations. The recommendation engine translates the user's context into intent using the technology described in this application, identifies the appropriate group of peers, and ultimate returns with a set of content recommendations 203 ordered based on the their computed UseRank™. These recommendations are then displayed within the Webpage on the user's browser. These recommendations may take the form of navigation links or enhanced search results. When delivering recommendation results, the recommendation system may additionally contact other external services, such as the Web site's full-text search engine, product catalog, or an ad server (see Figure 3).
• Customer Portal — In the preferred embodiment, owners of the Web site also have access to information and patterns learned by the recommendation engine through a customer portal called Insights 204. The customer portal provides the ability to configure the recommendation system and add merchandizing rules, as well as view a set of reports providing information on the usage of the Web site by the community and the patterns of behavior and affinities learned by the system. Figure 13 shows a screenshot of the Customer Portal homepage in the preferred embodiment. Many different reports and configurations are accessible using links within this portal. For example, the invention allows for product/content gap detection, where gaps are detected based on usage patterns on the site and site owners are helped to introduce new products or content in those areas where the gaps are detected. As another example, the Customer Portal also provides the Web site owner with a set of reports to track the ongoing value that the Recommendation Engine is brining to the Web site. Figure 14 shows a screenshot of an A/B report that tracks the revenue lift due to the presence of recommendations on the Web site. The inventors have consistently seen revenue improvements of 20-30% or more. Other lift reports available within the portal include improvements in page views, engagement time, site stickiness, average order value, and user conversion to specified "business targets" within the Web site.
Applications
The following discussion involves different applications of the foregoing technology:
Figure 15 is a screenshot showing community-guided e-commerce. In Figure 15, five different capabilities of product recommendations are presented:
1) Comparative Products, also known as Similar or Competitive Products, are shown for purpose of comparison shopping and up-selling based on affinity of like-minded peers on their product considerations, i.e. observed full-spectrum behavioral finger prints discussed earlier. Consideration-based peer recommendations promote higher end / niche products, and yield more revenues and profits for a site.
2) Affiliated Products are also shown for cross-selling related / non-competitive products such as accessories to increase overall order size.
3) The invention also provides a showing of most popular products, such as products with greatest appeal across the whole site or within a category.
4) The invention also generates intent-driven landing pages, also known as AdGuide and Site Concierge, with recommendations based on what a visitor searched on standard Web search engines such as Google, Yahoo or MSN. If a visitor search "Viking Cooktop" on Google and landed on a customer site, AdGuide serves the best Viking cooktop recommendations to the visitor on the landing page dynamically instead showing him something irrelevant. 5) The invention also provides a community filtered search on products based on the foregoing.
For peer-driven recommendations, the invention automatically identifies small and large population segments having a unique interest and guides individuals to popular, competing, and accessory products. The invention provides a user intent and product mapping that translates shoppers' intent into peer-validated products and brands. The inventors have found a 700 percent conversion power based on independent studies. The invention provides fad and trend detection and recognizes and adapts to seasonal, promotional, and other shifts in shopper interest in real-time. The invention also allows for product gap detection, where product gaps are detected and site owners are helped to introduce new products or content in those areas where the gaps are detected. The invention assists in merchandising with real-time customer feedback. It works with existing merchandising, promotions, segmentation and search, and magnifies products or product families. The invention provides built-in concurrent A/B measurement and reports in real-time the net revenue generated by the invisible crowds.
Figure 16 is a screenshot showing community-guided product search with a travel example. In Figure 16, the user has queried for hotels in New York City.
As a result of similar users of this Web site for similar purposes, and by observing how the users have made use of the information returned, recommendations are made to the person with regard to hotels to select within
New York City. The search results are far more effective and relevant based on like-minded peer interests and intent instead of showing many irrelevant results by simply matching keywords or metadata, in this case "New York Hotels".
Figure 17 is a screenshot showing community-guided marketing and online lead generation. As with the e-commerce application, Five different capabilities are provided to a marketing Web site. 1) Aspects of this feature of the invention include a social search which involves UseRank™ and implicit learning. This approach is intent-driven, adaptive, and makes use of an implicit folksonomy, community terms such as link-text they use and search queries they enter, as discussed above. The invention supports audio, video, binary code, and all other content types because it does not have to parse the content itself, whereas a typical search engine has to, therefore is limited to text content and metadata only.
2) The invention also provides Related or Similar Content based on the implicit behavioral feedback of like-minded users. The invention ranks the recommendations by usefulness and on-target-ness, based on the context and intent.
3) The invention also provides a most popular category, which provides content with the greatest value across the whole site or sub portion of the site. This promotion of information leads to business target conversion increases such as trial/download/registration conversions and may be provided at a site or topic level.
4) A Next Step feature is provided that concerns common next steps that lead to customer conversions. The next step may be any next step in a business process or a natural next set of content to read in connection with the process.
5) A similar Intent-Driven Landing Page, as with the e-commerce application discussed above, is available for a marketing Web site.
Figure 18 is a screenshot showing social search according to the invention. In the example, the search is made for storage devices. Users are provided information with regard to whether the information is on target and for how many visitors, and popularity rating and a community rank, i.e. UseRank™. An on- target value is also provided along with the results. Further, the results are sorted by usefulness, based on information within the affinity engine. The original keyword rank from IR or full-text search engine is also provided as a before-and- after comparison. In this example, if the UseRank™ is not used, this good piece of primary storage product information would have been buried as rank 87, i.e. the seventh result on the ninth search result page, and no one would be able to find and use it. The UseRank™ has moved it to the third position on the first page in the search results. At the interface level, a Raw Results link is provided to give the users the original results if this invention were not used there. This is another way to see before-and-after impact of the UseRank™.
Figure 19 is a screenshot showing community-guided online support to promote self-service and satisfaction among the customer base. The page shows top issue resolutions, including most helpful articles, a top FAQ, and help forum, and blog articles. The page also shows related help, including in-depth articles on the subject, alternative solutions and, finally, reminders. Further, common next steps are provided, including natural next steps for resolution, common downloads, and additional support information and contact information. Finally, the invention shows social search, which is a key aspect of this embodiment of the invention, based on UseRank™ and implicit learning. Thus, social search is intent-driven and adaptive, makes use of the implicit folksonomy, and supports all content types. Other good support applications for this use include knowledge management (KM) portals, community forums and discussion threads, chats, wikis and developer's networks, etc.
Figure 20 shows a community-guided intranet and knowledge portal. In this example, social search is used within an intranet. The search returns related pages, which include similar pages on the topic, the pages are proven by the past experience of peers, and serendipitous discovery is enabled. Further, the search filters out low value content. The invention also provides most popular search results, which includes content with the greatest value, top applications within the company, and information specific to a department or cross-company information. Finally, the invention provides next steps, including where people go from this page to meet their goals, a next step in the business process, and the next set of content to read on the topic. Extranets, such as physician-patient portals, dealer net, customer portals, are other applications examples.
Figure 21 shows recommendations implemented on a media site, and Figure 22 shows social search implemented on a media site. Figure 23 is a screenshot showing community-guided media and cross-site contextual recommendations. Similar Articles and Blogs are recommended based on similar contexts. If you are reading a Tiger Woods golf story, three other related Tiger news, based on other similar peer interests are promoted to the same visitors. The visitor gets more content and satisfied. The site get higher CPM and CPA, both of which are current industry measures of visitor conversion rates and site value from advertising perspective. UseRank™, also known by Baynote as BrandRank™, can also be used to re-rank the ads and product on the media, instead of placing ads or products randomly or based on keyword match. Ads validated by peers have significant higher CPM and CPA values, and therefore increase revenue for the media sites. In addition, the information contained in these various sources is applied to the affinity engine, which discovers cross-site affinities among the various sites. The result is to provide targeted content and products.
Figure 24 is an example of the invention being used to recommend video content. As discussed in this application, the invention is content agnostic and is able to learn affinities between term, users, and content based entirely on implicit user actions.
Figure 25 shows a screenshot of an Implicit Topic Cloud. Another application of the invention is to collect and display the community vocabulary or terms. On the surface, the Topic Cloud may appear to be similar to explicit tagged cloud such as Deli-cio-us, social bookmarking, etc. But the key difference is the implicitness. Implicit Topic Cloud is built based on the actual user's activities on the site, i.e. the queries they entered and link-text they used. Actions speak louder than words. The implicit Topic Cloud reflects in the interests of 100% of the site visitors, not the few loud visitors who are willing to tag things. The silent majority is the key to collect true wisdom of the community.
Figure 26 shows an architectural view of an integration between the invention and an AdServer. To serve the best ad to a user, the integrated system first contacts the existing Ad Server to get a list of acceptable ads to display to the user given demographic or other information. This list is then sent to the Affinity
Engine which, based on learned affinities between the Ads, users, and context, chooses the best ad to display to the current user. This allows the ads served to be appropriate both to the user, as well as the user's current context and intent.
There are many ways to leverage the community wisdom distilled by the Recommendation Engine, in addition to social search and onsite content/product recommendations. Many applications can be built or modified to directly tap into this wisdom through APIs to the Recommendation Engine. Figure 4 is an architectural diagram showing the invention's role as a community wisdom platform. The community-wisdom platform can also include such items as contextual email marketing, mobile applications, IPTV systems, SEO/SEM applications, and mashups in custom applications. Within these systems, input from the Recommendation Engine can be used to dynamically determine the most appropriate information and/or organization of information to present to users. Such a platform is built upon Web services that include such features as AJAX and REST (discussed below). Thus, the invention is built upon new Web 2.0 fundamentals, which include a true understanding of invisible crowds, gathering information from like-minded peers, making content or product recommendations, providing onsite social search, implicit community-based reports, and the like. The following descriptions give details on how they are deployed:
Email Recommendations: Marketing uses massive emails to communicate with customers and prospects. But the content of the email campaign is typically determined by the marketing staff. The invention enhances the quality of email campaigns by crowd-sourcing the like-minded peers. The content of the emails are determined by what's popular or similar to what the like-minded readers of the emails are interested in. Marketers simply facilitate the process of letting one customer promoting content to another customer implicitly.
Mobile Recommendations: The quality demand of mobile recommendations is even higher than the Web due to the small footprint of mobile devices, such as cell phones and personal devices. The invention's like-minded peer and intent driven approach is better than traditional approaches based on collaborative filtering, page views, and demographic or purchase data. Intent/context driven peers predict people's need far better than the competing approaches.
IPTV Recommendations: Similar recommendations are effective for using Digital TV/HDTV to watch internet delivered movies and video programs. With thousands of movies and TV program selections, features like "You may also like these movies" are extremely important for up-sell and cross-sell movies and TV programs. Context-driven peer recommendations are the most effective way for the viewers to like the recommendations.
Live Connect: Figure 27 shows an architectural view of the Live Connect system. Because the invention can identity like-minded peers, in addition to harvest their collective wisdom implicitly and to make peer recommendations to individuals, the invention can connect the individuals with the like-minded peer group and have them exchange information, knowledge, and experience. This is similar to the real world experience of like-minded peers who may not know each other. Imagine you are in Best Buy shopping for a TV, you see other like-minded peers also shopping for TVs, you can ask them questions for their experience of certain products.
SEO/SEM Recommendations: The invention is also used for guiding Web site visitors at the moment of landing on the site. The features are called "AdGuide," "Site Concierge", and "Site Maitre'd." This application examines the queries that people have searched on Google, Yahoo, MSN, ASK, and other major Web search engines, then uses them as the proxy of user intent to display the right product or content for visitors when they land on the site, either on the predesigned landing pages (SEM), or natural search result pages (SEO). For example, if a visitor search "Viking Cooktop" on Google and landed on a Baynote customer site, AdGuide serves the best Viking cooktop recommendations based on collective interests of like-minded peers to the visitor on the landing page dynamically instead showing him something irrelevant or less important by simply matching keywords.
Based on the affinities learned by the affinity engine, the invention is also able to provide a Keyword Recommendation system that provides suggestions to marketers on which keywords should be purchased on external search engines as part of their SEO/SEM efforts. For example, the invention can suggest those terms that the community uses most often to describe content on a Web site, as well as those terms which are most likely to lead to useful content within the site. Also, given a particular collection of keyword already purchased, the invention can suggest other words used by the community that have high affinity to the existing set. These recommendations can be provided to SEO/SEM merchandisers either through reports within the Customer Portal or can be directly integrated with external SEO/SEM systems to bid on the recommend keywords dynamically and automatically.
Blogs, Forums, and Discussion Threads Recommendations: The invention is used by community generated content such as blogs, forums, discussions, pod casts, video content, etc. Because of the volume of content involved, surfacing the right content in the right order is even more important than expert- generated Web sites. Using like-minded peers for recommendations and social search is extremely important and effective. These Web sites can be public facing blog sites, or login-required sites such as partner portals or patient- physician portals, or developer's networks, or intranets. Features such as "Similar content," "Accessory products," "Next steps," "Most popular" and social search are very useful and effective.
Insights (Visitor Clubs, Content Gaps, efc): Insights, the Customer Portal, has already been discussed in part earlier in this application. Although a primary use of the invention is as an automatic recommendation system, there are times when site administrators may want to modify the behavior of the recommendation system or see reports based on the knowledge learned by the affinity engine. Insights currently offers three main categories of functionality though its Ul: configuration, management, and reports. Configuration enables site administrators to configure fully all aspects of the recommendation engine required for full functionality and deployment, including integration with external sources of information such as full-text search engines and ad servers. Management enables site administrators to modify the behaviors of recommendations and social search in a holistic manner or specific to defined contexts e.g. a particular search term or page location or user. For example, administrators can create product/content promotions to override community wisdom for set periods of time, blacklist particular products/content from being recommended, artificially magnify or boost defined classes of products/content, or set up business rules such as limiting the classes of products/recommendations acceptable for recommendation in particular contexts. Reports offer various views on the community wisdom distilled by the affinity engine. Although the reports include a subset of the click-based and purchase-based information found in traditional analytics, they go beyond these in providing many details on the community wisdom distilled through full- spectrum behavioral fingerprinting and the connections learned by the affinity engine. Example reports are discussed below.
The Content Gap report uses the affinity engine to analyze those user interests that are not being met by the existing content on the site. Using the full- spectrum behavioral fingerprinting technology, and by analyzing community behaviors as discussed earlier in this application, the affinity engine can identify community interests. The affinity engine can then analyze all cases in which a particular interest was expressed by a user and the subsequent behaviors of that user, including use of content and level of value extracted, as well as subsequent searches or exits. If a large portion of the user population with a particular interest is unable to find useful content, then that interest is designated a content gap. The degree to which the gap is suspected by the affinity engine can also be reported. The administrator can then take necessary actions to remedy the gap by adding new content and then continue to use the Content Gap report to see the effect on the presence of the gap.
The Visitor Clubs report provides the most comprehensive view on the affinities learned by the affinity engine. Visitors to the site are implicitly grouped based on shared interest. Each interest group, or visitor club is displayed in the interface along with the number of visitors in the club, their overall activity level, the virtually folksonomy of terms that describe the club, and the content/products which are most useful to that club in the context of the club's interest. In addition, all club characteristics can be charted over time to gain an understanding of how the interest is trending over time. Clubs can also be compared to one another for overlap in membership. This report provides administrator with the ability to explore the learned associations between users, content/products, and terms/contexts. Such associations can then drive promotions or other merchandizing activities within the Management section of Insights. The visitor club information can also be used to drive various business decisions outside of the context of the invented system.
Architecture
Recommendation Server
Figure 3 is an architectural schematic diagram showing affinity engine integration according to the invention. In Figure 3, a current user exhibiting an observable behavior within a particular content site, such as current location and/or search, based on previous action, or user properties, accesses the affinity engine, which includes assets/products, users, and terms, all arranged in groupings based on affinities. The user is provided with asset recommendations in connection with products, categories, information, media, and the like. The affinity engine is populated by site owners who provide administrative input and exert administrative influence. The information provided by XML/direct input. The administrative input and influence filters, constrains and seeds the affinity engine, applies merchandising rules, and expresses goals, for example, likely use or likely purchase of a particular product. Asset information is also provided in the form of XML information, for example, a product catalogue or a full-text search facility. The affinity engine collects implicit observations from the visitor community to make such asset recommendations to a current user. These implicit observations include asset actions, such as time spent on a page, scrolling, mouse movements or clicks, page interaction, and virtual printing or bookmaking; asset navigation and asset navigation patterns, including, for example, entrance path, exit path, links used, repeated visits, and frequency visits, use of a back button, and recommendation use; and visitor-community searching, including terms entered and/or reentered, results clicked, results used, and subsequent navigation.
Figure 28 is a block schematic diagram showing the system architecture of a preferred embodiment of the invention. A more detailed discussion of various aspects of this architecture is provided below. To understand Figure 28 in greater detail, reference may be had to Applicant's copending U.S. patent application serial no. 11/319,928, filed 12/27/2005, which is incorporated herein in its entirety by this reference thereto, and which describes, in greater detail, a preferred system for use in connection with the invention herein.
The architecture consists of a server farm 20, a customer enterprise 22, and a user browser 21. The user browser is instrumented with an extension 23 and accesses both customer servers 25 at the customer enterprise 22 and the server farm 20 via a load balancer 27. Communication with the server farm is currently effected using the HTTPS protocol. User access to the customer server is in accordance with the enterprise protocols. The browser extension 23 is discussed in greater detail below.
Of note in connection with the invention is the provision of a failsafe. The extension 23, as well as the enterprise extension 24, are constructed such that, if the server farm 20 does not respond in a successful fashion, the extension is shut down and the enterprise and browser interact in a normal manner. The features of the invention are only provided in the event that the server is active and performing its operations correctly. Therefore, failure of the server does not in any way impair operation of the enterprise for users of the enterprise.
As discussed above, an extension 24 is also provided for the enterprise which communicates with the load balancer 27 at the server farm 20 via the HTTPS protocol.
The enterprise also includes a helper 26 which communicates with the server farm via an agency 31 using the HTTPS protocol. The agency retrieves log information from the enterprise and provides it to log analyzers 28, which produce a result that is presented to the usage repository 29. The agency also communicates with any data sources accessible to the enterprise for the purposes of loading information into the affinity engine 32. Such information includes catalog database information, use profile data, authentication data, as well as any other form of data that may be useful for the purposes of determining or filtering recommendation and search results.
Information is exchanged between the affinity engine 32 and the browser and enterprise via various dispatchers 30. The browser itself provides observations to the server and receives displays in response to search or recommendation requests therefrom. Alternatively, the customer server 25 can directly request search or recommendation results directly from the affinity engine 32 through the Baynote extension 24 using REST. A key feature of the invention is the affinity engine 32 which comprises a plurality of processors 33/34 and a configuration administration facility 35. During operation of the invention, a form of information, also referred to as wisdom, is collected in a wisdom database 36.
Client-side integration
The preferred embodiment of the Baynote extension 23 uses a custom-build tag- based AJAX-architecture for collecting Website behaviors from the client as well as serving recommendations to the client. Figure 29 provides code snippets for JavaScript integration with a client. Figure 30 is a block schematic diagram showing an AJAX tag platform according to the invention. In Figure 30, a Web page is displayed on a client and includes a particular script. In this example, all files and results from the system are dynamically injected into the DOMS as <script> elements. A coordinator on a trusted Web server provides such things as a failsafe mechanism, common code, a policy, handlers, and communication facilities. A system server provides such elements as a heartbeat, common JavaScript, policies, handlers, such as an observer and a guide, and the affinity engine. In Figure 30, the coordinator is a piece of code that sits on the trusted Web site. In the presently preferred embodiment, the coordinator resides on a customer's Web site. Thus, there is a file that serves as coordinator role and that resides on a particular customer Web site, although it could reside on any other trusted Web site. This aspect of the invention concerns the notion of a trusted Web server for failsafe reasons. It is important to be certain that if the invented system crashes that it does not crash or hold up the customer Web site. Thus, a coordinator code is provided as a failsafe procedure. It goes through a failsafe procedure to ensure proper recovery after a crash of the server's Web site.
The architecture of the code is broken down, such that common code is used. Common code or a piece of the code is used for all types of recommendations.
Thus, a tag might require common code. The policy may be customer specific or it can be user specific. This concerns how recommendations may be made. The policy information is loaded separately. Then, the code needed for specific tags is loaded as well. For example, if there is a tag for the most popular recommendation, and another one which is the recommendation, then these tags are loaded. The system then loads the code that is necessary for performing the particular tagging operations. A further type of tag or code is the observer that observes what users do, as discussed above. Users do not see the tag, but it is still a tag. The tags must also be able to communicate back to the system. Each of these functions is performed by a piece of code, for example JavaScript, is loaded into the Web page. The JavaScript is loaded into the page through a dynamic script injection.
This aspect of the invention recognizes that a Web page can become more dynamic. For example, there are things such as dynamic HTML and JavaScript which cause things to be added to a page dynamically. If a script tag needs a particular piece of information, then that code is injected into the page. For example, if a policy is needed, then a JavaScript tag is injected into the page that accesses the policy. The policy may require further injection of information, such as handlers. Therefore, the handlers are now dynamically injected as pieces of JavaScript into the page. Communications are handled in a similar matter.
One advantage of the foregoing approach is to provide the option of using first- party cookies or third-party cookies. The main Web page that the customer accesses may be the Web site of a merchant. The merchant communicates with the system to get the information with regard to recommendations and the like. In this case, the merchant is the first party and the system is a third party. As a result of dynamic script injection, the invention allows the setting of cookies in both the first-party domain and in the third-party domain. The ability to set first and third-party cookies also allows the invention to operate in a cross domain.
Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the Claims included below.

Claims

1. A computer implemented method for content recommendation, comprising the steps of: said computer receiving information as a result of user actions and identifying a current user's context; based on said identification, said computer finding an appropriate peer group that represents other users who are most like the current user in said identified context; said computer finding content that said peer group identifies as most relevant to said current context; and said computer generating an output comprising recommended content for display to said current user.
2. The method of Claim 1 , further comprising the step of: said computer generating full-spectrum behavioral fingerprints by: taking into account inputs comprising a wide variety of user behaviors; cross-correlating all of these user behaviors with stored information that corresponds to the current user's behavior's on other pages and also with the rest of community and identified peers behaviors on a current page; from this analysis, said computer generating a probability that the current user has found value in a particular piece of content; and said computer storing said probability for use in a learning system.
3. A computer implemented context-centric content recommendation method, comprising the steps of: said computer establishing a current context for a user at a Web site based upon inputs that are representative of user actions at said Web site; said computer capturing said user's current context information with an observer tag that is embedded in the Web site; said computer determining said user's interest based on various user behaviors collected by the observer tag; said computer using full spectrum behavioral fingerprint technology to analyze said behaviors; said computer storing all resulting information in a memory as the user's current context vector, said context vector comprising a hybrid vector of terms and documents with weights on each entry reflecting how strongly that term or document reflects the user's current context; said computer incrementing said context vector entries corresponding to terms and phrases entered or clicked as a user enters search terms and/or clicks navigation links to capture expressed interest; said computer decrementing or decaying corresponding entries as user actions move further into the past; said computer incrementing corresponding vector entry documents that a user clicks on or indicates interest in, as determined based on a user's implicit actions; and from the foregoing, said computer generating a representation of the user's current context as a context vector for use in making recommendations of content to said user.
4. The method of Claim 3, further comprising the step of: said computer expanding and refining the context vector into an intent vector, based on affinities and associations learned from aggregated wisdom collected from observations on a community as a whole over a long term; wherein said intent vector is created by looking at affinities between terms and documents in the context vector to other terms.
5. The method of Claim 4, further comprising the step of: said computer determining affinities that allow expansion of the context vector into an intent vector with an affinity engine; wherein said affinity engine learns connections between documents and terms, documents and documents, terms and terms, as well as users to other users, documents, and terms by watching all implicit behaviors of a user community on a Web site and by applying behavioral fingerprinting and a use rank algorithm to determine interest and associations.
6. The method of Claim 5, further comprising the step of: said computer identifying a peer group of users who share affinity to a current intent, as well as those users who exhibit behavior most like the current user within the context of that intent, said peer group being represented by a user vector, wherein each user entry in said user vector may have a weight indicating how strong of a peer he is to the current user in a current context.
7. The method of Claim 6, further comprising the step of: said computer identifying which documents have highest affinity to identified peers within a current intent; wherein said affinity engine looks at an activeness vector, term-doc matrices, and next-step matrices associated with those peers, weighted according to their peer weight, and uses these in aggregate to computes those documents with highest affinity to the current intent; and wherein said documents are unfiltered recommendations.
8. The method of Claim 7, further comprising the step of: said computer asking said affinity engine for community information on each of said recommendations; wherein said information is combined with other asset information and displayed to the user to help him understand the community wisdom underlying a recommendation.
9. The method of Claim 8, wherein said displayed information comprises any of a number of users in total who found value in a document, a number of peers who associated a document with a current context and/or intent, and terms and/or phrases the peer community has associated with a document.
10. The method of Claim 9, wherein said terms and/or phrases comprise a virtual folksonomy that represents terms that the community has associated to a document; wherein said virtual folksonomy is created automatically by said affinity engine based on implicit actions of the user community.
11. An apparatus for content recommendation, comprising: means for receiving input information for use in identifying a current user's context; means for finding an appropriate peer group representing other users who are most like the current user in said identified context, based on said identification; means for finding content that said peer group identifies as most relevant to said current context; and means for generating an output comprising content recommended to said current user.
12. The apparatus of Claim 11, further comprising: means for generating full-spectrum behavioral fingerprints by: taking into account inputs comprising a wide variety of user behaviors; cross-correlating all of these user behaviors with the current user's behavior's on other pages and also with the rest of community and identified peers behaviors on a current page; from this analysis, generating a probability that the current user has found value in a particular piece of content; and storing said probability for use by a learning system.
13. A context-centric content recommendation apparatus, comprising: means for receiving inputs representative of user actions and for establishing a current user context at a Web site therefrom; an observer tag embedded in the Web site for capturing said user's current context information; means for determining said user's interest based on various behaviors collected by the observer tag; means for using full spectrum behavioral fingerprint technology to analyze said behaviors; means for storing all resulting information in a memory as the user's current context vector, said context vector comprising a hybrid vector of terms and documents with weights on each entry reflecting how strongly that term or document reflects the user's current context; means for incrementing said context vector entries corresponding to terms and phrases entered or clicked are incremented as a user enters search terms and/or clicks navigation links to capture expressed interest; means for decrementing or decaying corresponding entries as user actions move further into the past; means for incrementing corresponding vector entry documents that a user clicks on or indicates interest in, as determined based on a user's implicit actions; and means for generating a representation of the user's current context as a context vector from the foregoing for use in making recommendations to the user.
14. The apparatus of Claim 13, further comprising: means for expanding and refining the context vector into an intent vector, based on affinities and associations learned from aggregated wisdom collected from observations on a community as a whole over a long term; wherein said intent vector is created by looking at affinities between terms and documents in the context vector to other terms.
15. The apparatus of Claim 14, further comprising: means for determining affinities that allow expansion of the context vector into an intent vector with an affinity engine; wherein said affinity engine learns connections between documents and terms, documents and documents, terms and terms, as well as users to other users, documents, and terms by watching all implicit behaviors of a user community on a Web site and by applying behavioral fingerprinting and a use rank algorithm to determine interest and associations.
16. The apparatus of Claim 15, further comprising: means for identifying a peer group of users who share affinity to a current intent, as well as those users who exhibit behavior most like the current user within the context of that intent, said peer group being represented by a user vector, wherein each user entry in said user vector may have a weight indicating how strong of a peer he is to the current user in a current context.
17. The apparatus of Claim 16, further comprising: means for looking at which documents have highest affinity to identified peers within a current intent; wherein said affinity engine looks at an activeness vector, term-doc matrices, and next-step matrices associated with those peers, weighted according to their peer weight, and uses these in aggregate to computes those documents with highest affinity to the current intent; and wherein said documents are unfiltered recommendations.
18. The apparatus of Claim 17, further comprising: means for asking said affinity engine for community information on each of said recommendations; wherein said information is combined with other asset information and displayed to the user to help him understand the community wisdom underlying a recommendation.
19. The apparatus of Claim 18, wherein said displayed information comprises any of a number of users in total who found value in a document, a number of peers who associated a document with a current context and/or intent, and terms and/or phrases the peer community has associated with a document.
20. The apparatus of Claim 19, wherein said terms and/or phrases comprise a virtual folksonomy that represents terms that the community has associated to a document; wherein said virtual folksonomy is created automatically by said affinity engine based on implicit actions of the user community.
PCT/US2008/072660 2004-12-29 2008-08-08 Method and apparatus for context-based content recommendation WO2009021198A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2010520330A JP2010536102A (en) 2007-08-08 2008-08-08 Context-based content recommendation method and apparatus
EP08782688A EP2176730A4 (en) 2007-08-08 2008-08-08 Method and apparatus for context-based content recommendation
CN200880102339.5A CN101779180B (en) 2007-08-08 2008-08-08 Method and apparatus for context-based content recommendation
US13/312,665 US20120311139A1 (en) 2004-12-29 2011-12-06 Method and Apparatus for Context-Based Content Recommendation

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US95467707P 2007-08-08 2007-08-08
US60/954,677 2007-08-08
US12/188,850 US8095523B2 (en) 2004-12-29 2008-08-08 Method and apparatus for context-based content recommendation
US12/188,850 2008-08-08

Publications (2)

Publication Number Publication Date
WO2009021198A1 WO2009021198A1 (en) 2009-02-12
WO2009021198A9 true WO2009021198A9 (en) 2009-04-02

Family

ID=40341772

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/072660 WO2009021198A1 (en) 2004-12-29 2008-08-08 Method and apparatus for context-based content recommendation

Country Status (5)

Country Link
US (2) US8095523B2 (en)
EP (1) EP2176730A4 (en)
JP (1) JP2010536102A (en)
CN (1) CN101779180B (en)
WO (1) WO2009021198A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10936824B2 (en) 2013-12-03 2021-03-02 International Business Machines Corporation Detecting literary elements in literature and their importance through semantic analysis and literary correlation

Families Citing this family (326)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9819561B2 (en) 2000-10-26 2017-11-14 Liveperson, Inc. System and methods for facilitating object assignments
US7231419B1 (en) * 2001-10-19 2007-06-12 Outlooksoft Corporation System and method for adaptively selecting and delivering recommendations to a requester
US20050131837A1 (en) 2003-12-15 2005-06-16 Sanctis Jeanne D. Method, system and program product for communicating e-commerce content over-the-air to mobile devices
US7698270B2 (en) * 2004-12-29 2010-04-13 Baynote, Inc. Method and apparatus for identifying, extracting, capturing, and leveraging expertise and knowledge
WO2009021198A1 (en) * 2007-08-08 2009-02-12 Baynote, Inc. Method and apparatus for context-based content recommendation
CN1825837B (en) * 2005-02-23 2012-05-23 朗迅科技公司 Personal information subscribing for and transmitting by instant message transmission
US9065727B1 (en) 2012-08-31 2015-06-23 Google Inc. Device identifier similarity models derived from online event signals
US7797299B2 (en) * 2005-07-02 2010-09-14 Steven Thrasher Searching data storage systems and devices
US9432468B2 (en) 2005-09-14 2016-08-30 Liveperson, Inc. System and method for design and dynamic generation of a web page
US8738732B2 (en) 2005-09-14 2014-05-27 Liveperson, Inc. System and method for performing follow up based on user interactions
US20070088720A1 (en) * 2005-10-17 2007-04-19 Siemens Aktiengesellschaft Method for detecting discrepancies between a user's perception of web sites and an author's intention of these web sites
US8219457B2 (en) * 2005-10-28 2012-07-10 Adobe Systems Incorporated Custom user definable keyword bidding system and method
JP5143013B2 (en) * 2005-12-08 2013-02-13 マイバイズ インコーポレイテッド Apparatus and method for providing a marketing service
US7856446B2 (en) * 2005-12-27 2010-12-21 Baynote, Inc. Method and apparatus for determining usefulness of a digital asset
US8054756B2 (en) * 2006-09-18 2011-11-08 Yahoo! Inc. Path discovery and analytics for network data
GB2450473A (en) * 2007-06-04 2008-12-31 Sony Comp Entertainment Europe A Server in a Peer to Peer system selecting and notifying a device that it is to become a member of a peer group
US8275764B2 (en) * 2007-08-24 2012-09-25 Google Inc. Recommending media programs based on media program popularity
US8751479B2 (en) * 2007-09-07 2014-06-10 Brand Affinity Technologies, Inc. Search and storage engine having variable indexing for information associations
US20090132368A1 (en) * 2007-10-19 2009-05-21 Paul Cotter Systems and Methods for Providing Personalized Advertisement
US8494978B2 (en) 2007-11-02 2013-07-23 Ebay Inc. Inferring user preferences from an internet based social interactive construct
US11263543B2 (en) 2007-11-02 2022-03-01 Ebay Inc. Node bootstrapping in a social graph
US8666909B2 (en) * 2007-11-02 2014-03-04 Ebay, Inc. Interestingness recommendations in a computing advice facility
US8126881B1 (en) 2007-12-12 2012-02-28 Vast.com, Inc. Predictive conversion systems and methods
US8682960B2 (en) * 2008-03-14 2014-03-25 Nokia Corporation Methods, apparatuses, and computer program products for providing filtered services and content based on user context
US10867123B2 (en) * 2009-11-09 2020-12-15 Microsoft Technology Licensing, Llc Social browsing
US8019642B2 (en) * 2008-05-06 2011-09-13 Richrelevance, Inc. System and process for receiving boosting recommendations for use in providing personalized advertisements to retail customers
US8583524B2 (en) * 2008-05-06 2013-11-12 Richrelevance, Inc. System and process for improving recommendations for use in providing personalized advertisements to retail customers
US8364528B2 (en) * 2008-05-06 2013-01-29 Richrelevance, Inc. System and process for improving product recommendations for use in providing personalized advertisements to retail customers
US8108329B2 (en) * 2008-05-06 2012-01-31 Richrelevance, Inc. System and process for boosting recommendations for use in providing personalized advertisements to retail customers
US20090287547A1 (en) * 2008-05-13 2009-11-19 Scanlon Robert T Sales benchmarking and coaching tool
US8554767B2 (en) * 2008-12-23 2013-10-08 Samsung Electronics Co., Ltd Context-based interests in computing environments and systems
US8762313B2 (en) 2008-07-25 2014-06-24 Liveperson, Inc. Method and system for creating a predictive model for targeting web-page to a surfer
US8805844B2 (en) 2008-08-04 2014-08-12 Liveperson, Inc. Expert search
GB0815440D0 (en) * 2008-08-26 2008-10-01 Allen Matthew E An internet-based location intelligence system
US8407216B2 (en) * 2008-09-25 2013-03-26 Yahoo! Inc. Automated tagging of objects in databases
US9892417B2 (en) 2008-10-29 2018-02-13 Liveperson, Inc. System and method for applying tracing tools for network locations
US8606725B1 (en) * 2008-10-29 2013-12-10 Emory University Automatic client-side user-behavior analysis for inferring user intent
US8214375B2 (en) * 2008-11-26 2012-07-03 Autodesk, Inc. Manual and automatic techniques for finding similar users
US20100138414A1 (en) * 2008-12-01 2010-06-03 Andrew Newman Methods and systems for associative search
US8751927B2 (en) * 2008-12-02 2014-06-10 International Business Machines Corporation System and method for dynamic multi-content cards
US9477672B2 (en) 2009-12-02 2016-10-25 Gartner, Inc. Implicit profile for use with recommendation engine and/or question router
US8918391B2 (en) * 2009-12-02 2014-12-23 Gartner, Inc. Interactive peer directory with question router
US20100161380A1 (en) * 2008-12-23 2010-06-24 Samsung Electronics Co., Ltd. Rating-based interests in computing environments and systems
US8175902B2 (en) * 2008-12-23 2012-05-08 Samsung Electronics Co., Ltd. Semantics-based interests in computing environments and systems
US20100198604A1 (en) * 2009-01-30 2010-08-05 Samsung Electronics Co., Ltd. Generation of concept relations
US8208943B2 (en) * 2009-02-02 2012-06-26 Waldeck Technology, Llc Anonymous crowd tracking
US8434126B1 (en) * 2009-02-02 2013-04-30 Symantec Corporation Methods and systems for aiding parental control policy decisions
WO2010129108A1 (en) * 2009-03-26 2010-11-11 Scott Jones Method and system for improving targeting of advertising
US8301624B2 (en) * 2009-03-31 2012-10-30 Yahoo! Inc. Determining user preference of items based on user ratings and user features
US20100257028A1 (en) * 2009-04-02 2010-10-07 Talk3, Inc. Methods and systems for extracting and managing latent social networks for use in commercial activities
US20140108156A1 (en) * 2009-04-02 2014-04-17 Talk3, Inc. Methods and systems for extracting and managing latent social networks for use in commercial activities
US20100269042A1 (en) * 2009-04-21 2010-10-21 Ami Entertainment Network, Inc. Jukebox menu navigation system
US20120046995A1 (en) 2009-04-29 2012-02-23 Waldeck Technology, Llc Anonymous crowd comparison
JP5231328B2 (en) * 2009-05-22 2013-07-10 ヤフー株式会社 Advertisement information providing device
US8495103B2 (en) * 2009-05-29 2013-07-23 Red Hat, Inc. Method and apparatus for determining how to transform applications into transactional applications
US8429606B2 (en) 2009-05-29 2013-04-23 Red Hat, Inc. Transactional object container
US8495490B2 (en) * 2009-06-08 2013-07-23 Xerox Corporation Systems and methods of summarizing documents for archival, retrival and analysis
US8737961B2 (en) * 2009-09-23 2014-05-27 Nokia Corporation Method and apparatus for incrementally determining location context
US20110093515A1 (en) * 2009-10-15 2011-04-21 Mary Elizabeth Albanese Mobile local search platform
CN102053973A (en) 2009-10-30 2011-05-11 国际商业机器公司 Device and method for providing page navigation in multi-character supporting network application
US8473512B2 (en) 2009-11-06 2013-06-25 Waldeck Technology, Llc Dynamic profile slice
US20110131077A1 (en) * 2009-12-01 2011-06-02 Microsoft Corporation Context-Aware Recommendation Module Using Multiple Models
US20110137726A1 (en) * 2009-12-03 2011-06-09 Telefonica, S.A. Recommender system based on expert opinions
WO2011069131A1 (en) * 2009-12-04 2011-06-09 Google Inc. Presenting real time search results
US9043319B1 (en) 2009-12-07 2015-05-26 Google Inc. Generating real-time search results
US20120063367A1 (en) 2009-12-22 2012-03-15 Waldeck Technology, Llc Crowd and profile based communication addresses
US9148706B1 (en) * 2010-01-08 2015-09-29 Visible Measures Corp. Identification of user segments based on video viewership activity and analysis
US20110191691A1 (en) * 2010-01-29 2011-08-04 Spears Joseph L Systems and Methods for Dynamic Generation and Management of Ancillary Media Content Alternatives in Content Management Systems
US11157919B2 (en) * 2010-01-29 2021-10-26 Ipar, Llc Systems and methods for dynamic management of geo-fenced and geo-targeted media content and content alternatives in content management systems
US20110191287A1 (en) * 2010-01-29 2011-08-04 Spears Joseph L Systems and Methods for Dynamic Generation of Multiple Content Alternatives for Content Management Systems
US20110191246A1 (en) * 2010-01-29 2011-08-04 Brandstetter Jeffrey D Systems and Methods Enabling Marketing and Distribution of Media Content by Content Creators and Content Providers
US20110191288A1 (en) * 2010-01-29 2011-08-04 Spears Joseph L Systems and Methods for Generation of Content Alternatives for Content Management Systems Using Globally Aggregated Data and Metadata
US10102278B2 (en) 2010-02-03 2018-10-16 Gartner, Inc. Methods and systems for modifying a user profile for a recommendation algorithm and making recommendations based on user interactions with items
CN102169566A (en) * 2010-02-26 2011-08-31 国际商业机器公司 Method and device for creating recommended items in unfamiliar fields
US20110218946A1 (en) * 2010-03-03 2011-09-08 Microsoft Corporation Presenting content items using topical relevance and trending popularity
US10621608B2 (en) * 2010-03-05 2020-04-14 Ethan Fieldman Systems and methods for tracking referrals among a plurality of members of a social network
US20110238608A1 (en) 2010-03-25 2011-09-29 Nokia Corporation Method and apparatus for providing personalized information resource recommendation based on group behaviors
EP2556449A1 (en) * 2010-04-07 2013-02-13 Liveperson Inc. System and method for dynamically enabling customized web content and applications
US10204163B2 (en) * 2010-04-19 2019-02-12 Microsoft Technology Licensing, Llc Active prediction of diverse search intent based upon user browsing behavior
JP2012003359A (en) * 2010-06-15 2012-01-05 Sony Corp Item recommendation system, item recommendation method, and program
US9002924B2 (en) 2010-06-17 2015-04-07 Microsoft Technology Licensing, Llc Contextual based information aggregation system
US8230062B2 (en) * 2010-06-21 2012-07-24 Salesforce.Com, Inc. Referred internet traffic analysis system and method
US8650191B2 (en) * 2010-08-23 2014-02-11 Vistaprint Schweiz Gmbh Search engine optimization assistant
US9432746B2 (en) 2010-08-25 2016-08-30 Ipar, Llc Method and system for delivery of immersive content over communication networks
US20120054277A1 (en) * 2010-08-31 2012-03-01 Gedikian Steve S Classification and status of users of networking and social activity systems
WO2012039011A1 (en) * 2010-09-24 2012-03-29 株式会社日立製作所 Computer and method of making recommendations
US9679060B2 (en) * 2010-10-13 2017-06-13 Microsoft Technology Licensing, Llc Following online social behavior to enhance search experience
US20120095862A1 (en) 2010-10-15 2012-04-19 Ness Computing, Inc. (a Delaware Corportaion) Computer system and method for analyzing data sets and generating personalized recommendations
US8818981B2 (en) * 2010-10-15 2014-08-26 Microsoft Corporation Providing information to users based on context
CN102542474B (en) 2010-12-07 2015-10-21 阿里巴巴集团控股有限公司 Result ranking method and device
US9350598B2 (en) 2010-12-14 2016-05-24 Liveperson, Inc. Authentication of service requests using a communications initiation feature
US8918465B2 (en) 2010-12-14 2014-12-23 Liveperson, Inc. Authentication of service requests initiated from a social networking site
US8972895B2 (en) 2010-12-20 2015-03-03 Target Brands Inc. Actively and passively customizable navigation bars
US8606643B2 (en) 2010-12-20 2013-12-10 Target Brands, Inc. Linking a retail user profile to a social network user profile
US8630913B1 (en) 2010-12-20 2014-01-14 Target Brands, Inc. Online registry splash page
US8606652B2 (en) * 2010-12-20 2013-12-10 Target Brands, Inc. Topical page layout
US8589242B2 (en) 2010-12-20 2013-11-19 Target Brands, Inc. Retail interface
US8781304B2 (en) 2011-01-18 2014-07-15 Ipar, Llc System and method for augmenting rich media content using multiple content repositories
US8756121B2 (en) 2011-01-21 2014-06-17 Target Brands, Inc. Retail website user interface
US20120191515A1 (en) * 2011-01-21 2012-07-26 Charles Katz Method For Connecting Consumers For Providing Shopping Advice
US20120209907A1 (en) * 2011-02-14 2012-08-16 Andrews Anton O A Providing contextual content based on another user
JP5962926B2 (en) * 2011-03-03 2016-08-03 日本電気株式会社 Recommender system, recommendation method, and program
CN102693223A (en) * 2011-03-21 2012-09-26 潘燕辉 Search method
US9361624B2 (en) 2011-03-23 2016-06-07 Ipar, Llc Method and system for predicting association item affinities using second order user item associations
US10168413B2 (en) 2011-03-25 2019-01-01 T-Mobile Usa, Inc. Service enhancements using near field communication
CN102750288B (en) * 2011-04-21 2015-11-11 中国移动通信集团广东有限公司 A kind of internet content recommend method and device
CN102831124B (en) * 2011-06-16 2015-08-05 北京亿赞普网络技术有限公司 Based on the user behavior association analysis method of outgoing telecommunication network message
US9326033B2 (en) 2011-06-17 2016-04-26 Microsoft Technology Licensing, Llc Movie discovery system
US8965788B2 (en) 2011-07-06 2015-02-24 Target Brands, Inc. Search page topology
JP5039844B1 (en) * 2011-07-29 2012-10-03 楽天株式会社 Information providing apparatus, information providing method, information providing program, and computer-readable recording medium storing the program
CN102956009B (en) * 2011-08-16 2017-03-01 阿里巴巴集团控股有限公司 A kind of electronic commerce information based on user behavior recommends method and apparatus
US20130054433A1 (en) * 2011-08-25 2013-02-28 T-Mobile Usa, Inc. Multi-Factor Identity Fingerprinting with User Behavior
US9824199B2 (en) 2011-08-25 2017-11-21 T-Mobile Usa, Inc. Multi-factor profile and security fingerprint analysis
US8671068B2 (en) * 2011-09-22 2014-03-11 Toyota Jidosha Kabushiki Kaisha Content recommendation system
US9047606B2 (en) 2011-09-29 2015-06-02 Hewlett-Packard Development Company, L.P. Social and contextual recommendations
JP5745388B2 (en) * 2011-11-17 2015-07-08 日本電信電話株式会社 Content recommendation apparatus, method, and program
JP5662303B2 (en) * 2011-11-17 2015-01-28 日本電信電話株式会社 Content recommendation apparatus, method, and program
US20130132308A1 (en) 2011-11-22 2013-05-23 Gregory Jensen Boss Enhanced DeepQA in a Medical Environment
US9134969B2 (en) 2011-12-13 2015-09-15 Ipar, Llc Computer-implemented systems and methods for providing consistent application generation
US9024954B2 (en) 2011-12-28 2015-05-05 Target Brands, Inc. Displaying partial logos
USD711400S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
USD706794S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
USD705790S1 (en) 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD706793S1 (en) 2011-12-28 2014-06-10 Target Brands, Inc. Display screen with graphical user interface
USD703687S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD705792S1 (en) 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD712417S1 (en) 2011-12-28 2014-09-02 Target Brands, Inc. Display screen with graphical user interface
USD715818S1 (en) 2011-12-28 2014-10-21 Target Brands, Inc. Display screen with graphical user interface
USD711399S1 (en) 2011-12-28 2014-08-19 Target Brands, Inc. Display screen with graphical user interface
USD705791S1 (en) 2011-12-28 2014-05-27 Target Brands, Inc. Display screen with graphical user interface
USD703686S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD703685S1 (en) 2011-12-28 2014-04-29 Target Brands, Inc. Display screen with graphical user interface
USD701224S1 (en) 2011-12-28 2014-03-18 Target Brands, Inc. Display screen with graphical user interface
KR101869133B1 (en) 2011-12-31 2018-06-19 톰슨 라이센싱 Method and apparatus for providing web pages
US8965908B1 (en) 2012-01-24 2015-02-24 Arrabon Management Services Llc Methods and systems for identifying and accessing multimedia content
US9098510B2 (en) 2012-01-24 2015-08-04 Arrabon Management Services, LLC Methods and systems for identifying and accessing multimedia content
US10324916B2 (en) * 2012-02-22 2019-06-18 Nokia Technologies Oy Predictive service access
US9195944B1 (en) * 2012-02-24 2015-11-24 Google Inc. Scoring site quality
US8521735B1 (en) * 2012-02-27 2013-08-27 Google Inc. Anonymous personalized recommendation method
US9390243B2 (en) * 2012-02-28 2016-07-12 Disney Enterprises, Inc. Dynamic trust score for evaluating ongoing online relationships
US8805941B2 (en) 2012-03-06 2014-08-12 Liveperson, Inc. Occasionally-connected computing interface
CN103312733B (en) * 2012-03-08 2016-03-02 腾讯科技(深圳)有限公司 Information processing method and device
US9043260B2 (en) * 2012-03-16 2015-05-26 Nokia Technologies Oy Method and apparatus for contextual content suggestion
US10866692B2 (en) * 2012-03-18 2020-12-15 Adobe Inc. Methods and apparatus for creating overlays according to trending information
US20140019317A1 (en) * 2012-04-05 2014-01-16 Lemon, Inc. Transaction managing system
US9563336B2 (en) 2012-04-26 2017-02-07 Liveperson, Inc. Dynamic user interface customization
US9053185B1 (en) 2012-04-30 2015-06-09 Google Inc. Generating a representative model for a plurality of models identified by similar feature data
US8527526B1 (en) 2012-05-02 2013-09-03 Google Inc. Selecting a list of network user identifiers based on long-term and short-term history data
US9672196B2 (en) 2012-05-15 2017-06-06 Liveperson, Inc. Methods and systems for presenting specialized content using campaign metrics
US8914500B1 (en) 2012-05-21 2014-12-16 Google Inc. Creating a classifier model to determine whether a network user should be added to a list
US8886575B1 (en) 2012-06-27 2014-11-11 Google Inc. Selecting an algorithm for identifying similar user identifiers based on predicted click-through-rate
US8874589B1 (en) 2012-07-16 2014-10-28 Google Inc. Adjust similar users identification based on performance feedback
US8782197B1 (en) 2012-07-17 2014-07-15 Google, Inc. Determining a model refresh rate
US9141707B2 (en) * 2012-07-19 2015-09-22 Facebook, Inc. Context-based object retrieval in a social networking system
WO2014014473A1 (en) * 2012-07-20 2014-01-23 Ipar, Llc Method and system for predicting association item affinities using second order user item associations
WO2014018319A2 (en) * 2012-07-21 2014-01-30 Trulia, Inc. Automated landing page generation and promotion for real estate listings
US8803712B2 (en) * 2012-07-23 2014-08-12 Telenav, Inc. Navigation system with mobile engagement mechanism and method of operation thereof
US9772979B1 (en) * 2012-08-08 2017-09-26 Amazon Technologies, Inc. Reproducing user browsing sessions
US8886799B1 (en) 2012-08-29 2014-11-11 Google Inc. Identifying a similar user identifier
CN104704863A (en) * 2012-10-04 2015-06-10 华为技术有限公司 User behavior modeling for intelligent mobile companions
US9436765B2 (en) * 2012-10-15 2016-09-06 Wix.Com Ltd. System for deep linking and search engine support for web sites integrating third party application and components
US9219668B2 (en) 2012-10-19 2015-12-22 Facebook, Inc. Predicting the future state of a mobile device user
US9519867B1 (en) * 2012-10-31 2016-12-13 Sprint Communications Company L.P. Optimizing a user experience
US10467237B1 (en) * 2012-11-12 2019-11-05 Pinterest, Inc. Object relationships and similarities based on user context
JP6113463B2 (en) * 2012-11-15 2017-04-12 任天堂株式会社 Information processing program, link management apparatus, display system, and display method
US10679259B2 (en) 2012-11-27 2020-06-09 Synqy Corporation Method and system for dynamic online digital brand assets
US9348886B2 (en) * 2012-12-19 2016-05-24 Facebook, Inc. Formation and description of user subgroups
US9798788B1 (en) * 2012-12-27 2017-10-24 EMC IP Holding Company LLC Holistic methodology for big data analytics
US9086825B2 (en) 2013-01-23 2015-07-21 Hewlett-Packard Development Company, L.P. Providing supplemental content based on a selected file
US10572926B1 (en) * 2013-01-31 2020-02-25 Amazon Technologies, Inc. Using artificial intelligence to efficiently identify significant items in a database
US9038106B2 (en) 2013-02-04 2015-05-19 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Display tag cloud viewer interactive interface for enabling a viewer to quickly and effectively access a desired television program for viewing
US20140229449A1 (en) * 2013-02-11 2014-08-14 Abu Shaher Sanaullah Realtime identification of context mismatch
US20140229488A1 (en) * 2013-02-11 2014-08-14 Telefonaktiebolaget L M Ericsson (Publ) Apparatus, Method, and Computer Program Product For Ranking Data Objects
US9535899B2 (en) 2013-02-20 2017-01-03 International Business Machines Corporation Automatic semantic rating and abstraction of literature
US9152709B2 (en) 2013-02-25 2015-10-06 Microsoft Technology Licensing, Llc Cross-domain topic space
US10600011B2 (en) 2013-03-05 2020-03-24 Gartner, Inc. Methods and systems for improving engagement with a recommendation engine that recommends items, peers, and services
US9104718B1 (en) 2013-03-07 2015-08-11 Vast.com, Inc. Systems, methods, and devices for measuring similarity of and generating recommendations for unique items
US10007946B1 (en) 2013-03-07 2018-06-26 Vast.com, Inc. Systems, methods, and devices for measuring similarity of and generating recommendations for unique items
US9465873B1 (en) 2013-03-07 2016-10-11 Vast.com, Inc. Systems, methods, and devices for identifying and presenting identifications of significant attributes of unique items
US9449106B2 (en) * 2013-03-08 2016-09-20 Opentable, Inc. Context-based queryless presentation of recommendations
US20140280133A1 (en) * 2013-03-13 2014-09-18 Google Inc. Structured Data to Aggregate Analytics
US9830635B1 (en) 2013-03-13 2017-11-28 Vast.com, Inc. Systems, methods, and devices for determining and displaying market relative position of unique items
US20140282036A1 (en) * 2013-03-15 2014-09-18 Turn Inc. Universal tag for page analytics and campaign creation
US20140278741A1 (en) * 2013-03-15 2014-09-18 International Business Machines Corporation Customer community analytics
US9311294B2 (en) 2013-03-15 2016-04-12 International Business Machines Corporation Enhanced answers in DeepQA system according to user preferences
US20140279724A1 (en) * 2013-03-15 2014-09-18 Turn Inc. Taxonomy configuration for page analytics and campaign creation
US11676192B1 (en) * 2013-03-15 2023-06-13 Overstock.Com, Inc. Localized sort of ranked product recommendations based on predicted user intent
US11023947B1 (en) * 2013-03-15 2021-06-01 Overstock.Com, Inc. Generating product recommendations using a blend of collaborative and content-based data
US10810654B1 (en) 2013-05-06 2020-10-20 Overstock.Com, Inc. System and method of mapping product attributes between different schemas
JP6191248B2 (en) 2013-06-04 2017-09-06 富士通株式会社 Information processing apparatus and information processing program
US10417685B2 (en) 2013-06-14 2019-09-17 Oracle International Corporation Context dependent data management and display
US9904949B1 (en) * 2013-06-21 2018-02-27 Amazon Technologies, Inc. Product recommendations
US20150007012A1 (en) * 2013-06-27 2015-01-01 International Business Machines Corporation System and method for using shared highlighting for various contexts to drive a recommendation engine
US9600571B2 (en) * 2013-07-11 2017-03-21 Neura, Inc. Interoperability mechanisms for internet of things integration platform
US10990894B2 (en) 2013-07-11 2021-04-27 Neura, Inc. Situation forecast mechanisms for internet of things integration platform
US9871865B2 (en) 2013-07-11 2018-01-16 Neura, Inc. Physical environment profiling through internet of things integration platform
US9576066B2 (en) * 2013-07-31 2017-02-21 Captora Inc. Webpage form optimization
US9342608B2 (en) 2013-08-01 2016-05-17 International Business Machines Corporation Clarification of submitted questions in a question and answer system
US10929890B2 (en) 2013-08-15 2021-02-23 Overstock.Com, Inc. System and method of personalizing online marketing campaigns
US9383965B1 (en) 2013-08-16 2016-07-05 Amazon Technologies, Inc. Media library analyzer
US9852448B2 (en) 2013-09-19 2017-12-26 Samsung Electronics Co., Ltd. Identifying gaps in search results
US20150095183A1 (en) * 2013-09-30 2015-04-02 International Business Machines Corporation Methods and Apparatus for Generating Recommendations
US11188543B2 (en) 2013-10-14 2021-11-30 International Business Machines Corporation Utilizing social information for recommending an application
US11238056B2 (en) 2013-10-28 2022-02-01 Microsoft Technology Licensing, Llc Enhancing search results with social labels
US9542440B2 (en) 2013-11-04 2017-01-10 Microsoft Technology Licensing, Llc Enterprise graph search based on object and actor relationships
US20150142607A1 (en) * 2013-11-20 2015-05-21 Cui Yang Shopping mind reader
US9298802B2 (en) 2013-12-03 2016-03-29 International Business Machines Corporation Recommendation engine using inferred deep similarities for works of literature
US10872350B1 (en) 2013-12-06 2020-12-22 Overstock.Com, Inc. System and method for optimizing online marketing based upon relative advertisement placement
US10013500B1 (en) * 2013-12-09 2018-07-03 Amazon Technologies, Inc. Behavior based optimization for content presentation
US10127596B1 (en) 2013-12-10 2018-11-13 Vast.com, Inc. Systems, methods, and devices for generating recommendations of unique items
US9015730B1 (en) 2013-12-17 2015-04-21 International Business Machines Corporation Natural language access to application programming interfaces
US10133589B2 (en) * 2013-12-31 2018-11-20 Microsoft Technology Licensing, Llc Identifying help information based on application context
US9984127B2 (en) 2014-01-09 2018-05-29 International Business Machines Corporation Using typestyles to prioritize and rank search results
US11645289B2 (en) 2014-02-04 2023-05-09 Microsoft Technology Licensing, Llc Ranking enterprise graph queries
US9870432B2 (en) 2014-02-24 2018-01-16 Microsoft Technology Licensing, Llc Persisted enterprise graph queries
US11755629B1 (en) * 2014-02-24 2023-09-12 Entefy Inc. System and method of context-based predictive content tagging for encrypted data
US11657060B2 (en) 2014-02-27 2023-05-23 Microsoft Technology Licensing, Llc Utilizing interactivity signals to generate relationships and promote content
US10757201B2 (en) * 2014-03-01 2020-08-25 Microsoft Technology Licensing, Llc Document and content feed
US10169457B2 (en) 2014-03-03 2019-01-01 Microsoft Technology Licensing, Llc Displaying and posting aggregated social activity on a piece of enterprise content
US10872110B2 (en) * 2014-03-03 2020-12-22 Spotify Ab Systems, apparatuses, methods and computer-readable medium for automatically generating playlists based on taste profiles
US10394827B2 (en) 2014-03-03 2019-08-27 Microsoft Technology Licensing, Llc Discovering enterprise content based on implicit and explicit signals
US10255563B2 (en) 2014-03-03 2019-04-09 Microsoft Technology Licensing, Llc Aggregating enterprise graph content around user-generated topics
US10380649B2 (en) 2014-03-03 2019-08-13 Spotify Ab System and method for logistic matrix factorization of implicit feedback data, and application to media environments
US10147220B2 (en) 2014-03-14 2018-12-04 Carnegie Mellon University Precomputing data for an interactive system having discrete control inputs
US20150262217A1 (en) * 2014-03-17 2015-09-17 Google Inc. User path abandonment analysis
US10551999B2 (en) 2014-03-26 2020-02-04 Unanimous A.I., Inc. Multi-phase multi-group selection methods for real-time collaborative intelligence systems
US11269502B2 (en) 2014-03-26 2022-03-08 Unanimous A. I., Inc. Interactive behavioral polling and machine learning for amplification of group intelligence
WO2015148738A1 (en) 2014-03-26 2015-10-01 Unanimous A.I. LLC Methods and systems for real-time closed-loop collaborative intelligence
US10110664B2 (en) 2014-03-26 2018-10-23 Unanimous A. I., Inc. Dynamic systems for optimization of real-time collaborative intelligence
US10222961B2 (en) 2014-03-26 2019-03-05 Unanimous A. I., Inc. Methods for analyzing decisions made by real-time collective intelligence systems
US11151460B2 (en) 2014-03-26 2021-10-19 Unanimous A. I., Inc. Adaptive population optimization for amplifying the intelligence of crowds and swarms
US10817158B2 (en) 2014-03-26 2020-10-27 Unanimous A. I., Inc. Method and system for a parallel distributed hyper-swarm for amplifying human intelligence
US10712929B2 (en) 2014-03-26 2020-07-14 Unanimous A. I., Inc. Adaptive confidence calibration for real-time swarm intelligence systems
US10353551B2 (en) 2014-03-26 2019-07-16 Unanimous A. I., Inc. Methods and systems for modifying user influence during a collaborative session of real-time collective intelligence system
US10122775B2 (en) 2014-03-26 2018-11-06 Unanimous A.I., Inc. Systems and methods for assessment and optimization of real-time collaborative intelligence systems
US10133460B2 (en) 2014-03-26 2018-11-20 Unanimous A.I., Inc. Systems and methods for collaborative synchronous image selection
US10439836B2 (en) 2014-03-26 2019-10-08 Unanimous A. I., Inc. Systems and methods for hybrid swarm intelligence
US10277645B2 (en) 2014-03-26 2019-04-30 Unanimous A. I., Inc. Suggestion and background modes for real-time collaborative intelligence systems
US10416666B2 (en) 2014-03-26 2019-09-17 Unanimous A. I., Inc. Methods and systems for collaborative control of a remote vehicle
US10310802B2 (en) 2014-03-26 2019-06-04 Unanimous A. I., Inc. System and method for moderating real-time closed-loop collaborative decisions on mobile devices
US10817159B2 (en) 2014-03-26 2020-10-27 Unanimous A. I., Inc. Non-linear probabilistic wagering for amplified collective intelligence
US9940006B2 (en) 2014-03-26 2018-04-10 Unanimous A. I., Inc. Intuitive interfaces for real-time collaborative intelligence
US9710546B2 (en) * 2014-03-28 2017-07-18 Microsoft Technology Licensing, Llc Explicit signals personalized search
US9836765B2 (en) 2014-05-19 2017-12-05 Kibo Software, Inc. System and method for context-aware recommendation through user activity change detection
US10402886B2 (en) * 2014-06-23 2019-09-03 Rakuten, Inc. Information processing device, information processing method, program, and storage medium
WO2016007992A1 (en) * 2014-07-17 2016-01-21 Bigtincan Holdings Pty Ltd Method and system for providing contextual electronic content
US10061826B2 (en) 2014-09-05 2018-08-28 Microsoft Technology Licensing, Llc. Distant content discovery
CN104253862B (en) * 2014-09-12 2018-04-17 北京诺亚星云科技有限责任公司 Immersion interaction guide to visitors support service system and equipment based on digital panoramic
US20160098416A1 (en) * 2014-10-03 2016-04-07 Yahoo! Inc. Auto complete search box based on the user's context to reduce user's input
US11100557B2 (en) 2014-11-04 2021-08-24 International Business Machines Corporation Travel itinerary recommendation engine using inferred interests and sentiments
US20160140186A1 (en) * 2014-11-14 2016-05-19 Manfred Langen Identifying Subject Matter Experts
IL236016A0 (en) * 2014-12-01 2015-02-26 Bg Negev Technologies & Applic Ltd A process and system for providing businesses with the ability to supply sets of coupons to potential customers
US10163118B2 (en) * 2015-02-18 2018-12-25 Adobe Systems Incorporated Method and apparatus for associating user engagement data received from a user with portions of a webpage visited by the user
US20160261544A1 (en) * 2015-03-02 2016-09-08 Linkedln Corporation Increasing interaction between clusters with low connectivity in a social network
RU2637882C2 (en) 2015-03-31 2017-12-07 Общество С Ограниченной Ответственностью "Яндекс" Method for managing web-resource displays in browser window, method of placing tabs in stack in browser window, electronic device and server
EP3093803A1 (en) * 2015-04-30 2016-11-16 Tata Consultancy Services Limited Systems and methods for contextual recommendation of learning content
US20160350658A1 (en) * 2015-06-01 2016-12-01 Microsoft Technology Licensing, Llc Viewport-based implicit feedback
US10142908B2 (en) 2015-06-02 2018-11-27 Liveperson, Inc. Dynamic communication routing based on consistency weighting and routing rules
EP3281122A4 (en) * 2015-07-24 2018-04-25 Samsung Electronics Co., Ltd. Method for automatically generating dynamic index for content displayed on electronic device
US10509806B2 (en) 2015-08-17 2019-12-17 Accenture Global Solutions Limited Recommendation engine for aggregated platform data
US10210453B2 (en) * 2015-08-17 2019-02-19 Adobe Inc. Behavioral prediction for targeted end users
US20170076318A1 (en) * 2015-09-16 2017-03-16 Microsoft Technology Licensing, Llc Enhanced content quality using content features
CN106570008B (en) 2015-10-09 2020-03-27 阿里巴巴集团控股有限公司 Recommendation method and device
WO2017062179A1 (en) * 2015-10-09 2017-04-13 Alibaba Group Holding Limited Recommendation method and device
TWI648223B (en) * 2015-10-29 2019-01-21 富鈞水資股份有限公司 Sewage treatment monitoring system
CN106713106A (en) * 2015-11-12 2017-05-24 北京国双科技有限公司 Data analysis method and device and mail sending method and device
CN106708888B (en) 2015-11-17 2020-06-26 阿里巴巴集团控股有限公司 Recommendation method and device
US10904360B1 (en) 2015-12-02 2021-01-26 Zeta Global Corp. Method and apparatus for real-time personalization
US20170169333A1 (en) * 2015-12-10 2017-06-15 International Business Machines Corporation Utilizing crowd source viewing behavior of users to recommend a viewing order to a user
AU2016366555A1 (en) * 2015-12-11 2018-05-24 Mastercard International Incorporated Systems and methods for generating recommendations using a corpus of data
CN105512326B (en) * 2015-12-23 2019-03-22 成都品果科技有限公司 A kind of method and system that picture is recommended
US10447828B2 (en) * 2016-03-01 2019-10-15 Microsoft Technology Licensing, Llc Cross-application service-driven contextual messages
US10534845B2 (en) 2016-05-11 2020-01-14 Overstock.Com, Inc. System and method for optimizing electronic document layouts
US11477302B2 (en) 2016-07-06 2022-10-18 Palo Alto Research Center Incorporated Computer-implemented system and method for distributed activity detection
CN116528220A (en) 2016-08-14 2023-08-01 利维帕尔森有限公司 Method and system for real-time remote control of mobile applications
CN107871259A (en) * 2016-09-26 2018-04-03 阿里巴巴集团控股有限公司 A kind of processing method of information recommendation, device and client
KR101852964B1 (en) * 2016-09-30 2018-04-27 에스케이플래닛 주식회사 System for recommend the customized information, method thereof, and recordable medium storing the method
US10762140B2 (en) 2016-11-02 2020-09-01 Microsoft Technology Licensing, Llc Identifying content in a content management system relevant to content of a published electronic document
US10762135B2 (en) * 2016-11-21 2020-09-01 Adobe Inc. Recommending software actions to create an image and recommending images to demonstrate the effects of software actions
US20180225605A1 (en) * 2017-02-06 2018-08-09 American Express Travel Related Services Company, Inc. Risk assessment and alert system
US11157836B2 (en) * 2017-02-28 2021-10-26 Verizon Media Inc. Changing machine learning classification of digital content
CN108738036B (en) * 2017-04-14 2021-06-18 广州杰赛科技股份有限公司 Method and system for extracting key users of mobile communication
CN107330023B (en) * 2017-06-21 2021-02-12 北京百度网讯科技有限公司 Text content recommendation method and device based on attention points
US10268704B1 (en) 2017-10-12 2019-04-23 Vast.com, Inc. Partitioned distributed database systems, devices, and methods
CN109697282B (en) * 2017-10-20 2023-06-06 阿里巴巴集团控股有限公司 Sentence user intention recognition method and device
JP6853159B2 (en) * 2017-10-31 2021-03-31 トヨタ自動車株式会社 State estimator
US11157503B2 (en) * 2017-11-15 2021-10-26 Stochastic Processes, LLC Systems and methods for using crowd sourcing to score online content as it relates to a belief state
US10853736B2 (en) * 2017-11-17 2020-12-01 Microsoft Technology Licensing, Llc Preventing notification blindness
US10635748B2 (en) * 2017-12-14 2020-04-28 International Business Machines Corporation Cognitive auto-fill content recommendation
WO2019125082A1 (en) * 2017-12-22 2019-06-27 Samsung Electronics Co., Ltd. Device and method for recommending contact information
CN110033338A (en) * 2018-01-11 2019-07-19 北京搜狗科技发展有限公司 Recommended method and device, the device for recommendation
CN108563681B (en) * 2018-03-07 2021-02-02 创新先进技术有限公司 Content recommendation method and device, electronic equipment and system
KR20190107990A (en) * 2018-03-13 2019-09-23 에스케이플래닛 주식회사 Service server and method for providing customized landing page
EP3553728A1 (en) * 2018-04-12 2019-10-16 Siemens Schweiz AG Recommendation engine
US20190378194A1 (en) * 2018-06-12 2019-12-12 Capital One Services, Llc Systems and methods for adjusting transaction affinity information
JP6679664B2 (en) * 2018-06-29 2020-04-15 ヤフー株式会社 Information processing apparatus, information processing method, and program
US11243957B2 (en) * 2018-07-10 2022-02-08 Verizon Patent And Licensing Inc. Self-organizing maps for adaptive individualized user preference determination for recommendation systems
US20200045037A1 (en) * 2018-07-31 2020-02-06 Salesforce.Com, Inc. Token store service for platform authentication
KR102159956B1 (en) * 2018-08-10 2020-09-25 엔에이치엔 주식회사 Server and method to provide web pages including product images and information associated with product images
US11526567B2 (en) * 2018-10-17 2022-12-13 International Business Machines Corporation Contextualizing searches in a collaborative session
US11163777B2 (en) 2018-10-18 2021-11-02 Oracle International Corporation Smart content recommendations for content authors
US11106822B2 (en) 2018-12-05 2021-08-31 At&T Intellectual Property I, L.P. Privacy-aware content recommendations
US10979528B2 (en) * 2018-12-17 2021-04-13 At&T Intellectual Property I, L.P. System for trend discovery and curation from content metadata and context
US11151217B2 (en) 2019-01-15 2021-10-19 Microstrategy Incorporated Dynamic contextual library
US10764386B1 (en) * 2019-02-15 2020-09-01 Citrix Systems, Inc. Activity detection in web applications
US11514493B1 (en) 2019-03-25 2022-11-29 Overstock.Com, Inc. System and method for conversational commerce online
US11205179B1 (en) 2019-04-26 2021-12-21 Overstock.Com, Inc. System, method, and program product for recognizing and rejecting fraudulent purchase attempts in e-commerce
CN110309864B (en) * 2019-06-18 2022-01-11 北京化工大学 Collaborative filtering recommendation method fusing local similarity and global similarity
CN110248249A (en) * 2019-06-26 2019-09-17 深圳市轱辘汽车维修技术有限公司 A kind of video recommendation method, video recommendations equipment and server
US11176218B2 (en) * 2019-07-30 2021-11-16 Ebay Inc. Presenting a customized landing page as a preview at a search engine
US11921881B2 (en) * 2019-08-01 2024-03-05 EMC IP Holding Company LLC Anonymous ranking service
US20210056636A1 (en) * 2019-08-20 2021-02-25 Deep Forecast Inc Systems and methods for measurement of data to provide decision support
RU2757406C1 (en) * 2019-09-09 2021-10-15 Общество С Ограниченной Ответственностью «Яндекс» Method and system for providing a level of service when advertising content element
US11734368B1 (en) 2019-09-26 2023-08-22 Overstock.Com, Inc. System and method for creating a consistent personalized web experience across multiple platforms and channels
US20210097133A1 (en) * 2019-09-27 2021-04-01 Microsoft Technology Licensing, Llc Personalized proactive pane pop-up
TWI712315B (en) * 2019-10-25 2020-12-01 中華電信股份有限公司 Method and system for recommending video
US11556822B2 (en) * 2020-05-27 2023-01-17 Yahoo Assets Llc Cross-domain action prediction
US11650998B2 (en) * 2020-06-30 2023-05-16 Microsoft Technology Licensing, Llc Determining authoritative documents based on implicit interlinking and communication signals
US11568469B1 (en) 2020-07-20 2023-01-31 Amazon Technologies, Inc. Systems and methods for generating recommendations based on multi-channel inputs
US11908003B2 (en) * 2020-08-24 2024-02-20 Capital One Services, Llc Methods and systems for updating a user interface based on level of user interest
US11698942B2 (en) * 2020-09-21 2023-07-11 International Business Machines Corporation Composite display of relevant views of application data
CN112132628A (en) * 2020-09-28 2020-12-25 科大讯飞股份有限公司 User intention prediction method, information recommendation method and related equipment
CN112417288A (en) * 2020-11-25 2021-02-26 南京大学 Task cross-domain recommendation method for crowdsourcing software testing
CN112507229A (en) * 2020-12-15 2021-03-16 北京明略软件系统有限公司 Document recommendation method and system and computer equipment
WO2023158676A1 (en) * 2022-02-16 2023-08-24 10G Consulting, Llc Systems and methods for automated data procurement
US11853335B1 (en) 2022-06-13 2023-12-26 International Business Machines Corporation Cooperative build and content annotation for conversational design of virtual assistants

Family Cites Families (110)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5619709A (en) * 1993-09-20 1997-04-08 Hnc, Inc. System and method of context vector generation and retrieval
US7251637B1 (en) * 1993-09-20 2007-07-31 Fair Isaac Corporation Context vector generation and retrieval
US6202058B1 (en) * 1994-04-25 2001-03-13 Apple Computer, Inc. System for ranking the relevance of information objects accessed by computer users
US5675819A (en) * 1994-06-16 1997-10-07 Xerox Corporation Document information retrieval using global word co-occurrence patterns
US6112186A (en) * 1995-06-30 2000-08-29 Microsoft Corporation Distributed system for facilitating exchange of user information and opinion using automated collaborative filtering
US6041311A (en) * 1995-06-30 2000-03-21 Microsoft Corporation Method and apparatus for item recommendation using automated collaborative filtering
US6049777A (en) * 1995-06-30 2000-04-11 Microsoft Corporation Computer-implemented collaborative filtering based method for recommending an item to a user
US5867799A (en) * 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US5890149A (en) * 1996-06-20 1999-03-30 Wisdomware, Inc. Organization training, coaching and indexing system
US6016475A (en) * 1996-10-08 2000-01-18 The Regents Of The University Of Minnesota System, method, and article of manufacture for generating implicit ratings based on receiver operating curves
US5924105A (en) * 1997-01-27 1999-07-13 Michigan State University Method and product for determining salient features for use in information searching
US6128613A (en) 1997-06-26 2000-10-03 The Chinese University Of Hong Kong Method and apparatus for establishing topic word classes based on an entropy cost function to retrieve documents represented by the topic words
US6070133A (en) * 1997-07-21 2000-05-30 Battelle Memorial Institute Information retrieval system utilizing wavelet transform
US6865715B2 (en) * 1997-09-08 2005-03-08 Fujitsu Limited Statistical method for extracting, and displaying keywords in forum/message board documents
WO1999018530A1 (en) * 1997-10-06 1999-04-15 Nexprise, Inc. Trackpoint-based computer-implemented systems and methods for facilitating collaborative project development and communication
US6240407B1 (en) * 1998-04-29 2001-05-29 International Business Machines Corp. Method and apparatus for creating an index in a database system
US6256633B1 (en) * 1998-06-25 2001-07-03 U.S. Philips Corporation Context-based and user-profile driven information retrieval
US6574632B2 (en) * 1998-11-18 2003-06-03 Harris Corporation Multiple engine information retrieval and visualization system
US6842877B2 (en) * 1998-12-18 2005-01-11 Tangis Corporation Contextual responses based on automated learning techniques
US6327590B1 (en) * 1999-05-05 2001-12-04 Xerox Corporation System and method for collaborative ranking of search results employing user and group profiles derived from document collection content analysis
US6493702B1 (en) * 1999-05-05 2002-12-10 Xerox Corporation System and method for searching and recommending documents in a collection using share bookmarks
US6493703B1 (en) * 1999-05-11 2002-12-10 Prophet Financial Systems System and method for implementing intelligent online community message board
AU5934900A (en) * 1999-07-16 2001-02-05 Agentarts, Inc. Methods and system for generating automated alternative content recommendations
US7219073B1 (en) * 1999-08-03 2007-05-15 Brandnamestores.Com Method for extracting information utilizing a user-context-based search engine
US6839680B1 (en) * 1999-09-30 2005-01-04 Fujitsu Limited Internet profiling
US6418448B1 (en) * 1999-12-06 2002-07-09 Shyam Sundar Sarkar Method and apparatus for processing markup language specifications for data and metadata used inside multiple related internet documents to navigate, query and manipulate information from a plurality of object relational databases over the web
US6415368B1 (en) * 1999-12-22 2002-07-02 Xerox Corporation System and method for caching
US6981040B1 (en) * 1999-12-28 2005-12-27 Utopy, Inc. Automatic, personalized online information and product services
US6502091B1 (en) * 2000-02-23 2002-12-31 Hewlett-Packard Company Apparatus and method for discovering context groups and document categories by mining usage logs
AU2001247789A1 (en) * 2000-03-22 2001-10-03 Sidestep, Inc. Method and apparatus for dynamic information connection engine
US8352331B2 (en) * 2000-05-03 2013-01-08 Yahoo! Inc. Relationship discovery engine
AU2001274818A1 (en) 2000-05-25 2001-12-11 Symbionautics Corporation Simulating human intelligence in computers using natural language dialog
JP5105456B2 (en) 2000-05-30 2012-12-26 株式会社ホットリンク Distributed monitoring system that provides knowledge services
US20040205643A1 (en) * 2000-06-22 2004-10-14 Harrington Steven J. Reproduction of documents using intent information
US20020078091A1 (en) * 2000-07-25 2002-06-20 Sonny Vu Automatic summarization of a document
US6687696B2 (en) * 2000-07-26 2004-02-03 Recommind Inc. System and method for personalized search, information filtering, and for generating recommendations utilizing statistical latent class models
AU2001290682A1 (en) * 2000-09-06 2002-03-22 Daryl Craig Josephson Data gathering and distribution apparatus and methods
NO313399B1 (en) * 2000-09-14 2002-09-23 Fast Search & Transfer Asa Procedure for searching and analyzing information in computer networks
US20020103698A1 (en) * 2000-10-31 2002-08-01 Christian Cantrell System and method for enabling user control of online advertising campaigns
US7065568B2 (en) * 2000-11-30 2006-06-20 Microsoft Corporation System and method for managing states and user context over stateless protocols
US20020075320A1 (en) * 2000-12-14 2002-06-20 Philips Electronics North America Corp. Method and apparatus for generating recommendations based on consistency of selection
US7440943B2 (en) * 2000-12-22 2008-10-21 Xerox Corporation Recommender system and method
US7089237B2 (en) * 2001-01-26 2006-08-08 Google, Inc. Interface and system for providing persistent contextual relevance for commerce activities in a networked environment
US6785676B2 (en) * 2001-02-07 2004-08-31 International Business Machines Corporation Customer self service subsystem for response set ordering and annotation
US6643639B2 (en) * 2001-02-07 2003-11-04 International Business Machines Corporation Customer self service subsystem for adaptive indexing of resource solutions and resource lookup
US6853998B2 (en) * 2001-02-07 2005-02-08 International Business Machines Corporation Customer self service subsystem for classifying user contexts
US6778193B2 (en) * 2001-02-07 2004-08-17 International Business Machines Corporation Customer self service iconic interface for portal entry and search specification
US20020116421A1 (en) * 2001-02-17 2002-08-22 Fox Harold L. Method and system for page-like display, formating and processing of computer generated information on networked computers
EP1388093A4 (en) * 2001-05-15 2007-01-24 Nokia Corp Method and business process to maintain privacy in distributed recommendation systems
US6968334B2 (en) * 2001-05-15 2005-11-22 Nokia Corporation Method and business process to maintain privacy in distributed recommendation systems
US7016968B2 (en) * 2001-06-22 2006-03-21 International Business Machines Corporation Method and apparatus for facilitating the providing of content
US20040230572A1 (en) * 2001-06-22 2004-11-18 Nosa Omoigui System and method for semantic knowledge retrieval, management, capture, sharing, discovery, delivery and presentation
US20030004996A1 (en) * 2001-06-29 2003-01-02 International Business Machines Corporation Method and system for spatial information retrieval for hyperlinked documents
US7188141B2 (en) * 2001-06-29 2007-03-06 International Business Machines Corporation Method and system for collaborative web research
US6744729B2 (en) * 2001-08-17 2004-06-01 Interactive Sapience Corp. Intelligent fabric
US7092936B1 (en) * 2001-08-22 2006-08-15 Oracle International Corporation System and method for search and recommendation based on usage mining
US6938035B2 (en) * 2001-10-03 2005-08-30 International Business Machines Corporation Reduce database monitor workload by employing predictive query threshold
US7295995B1 (en) * 2001-10-30 2007-11-13 A9.Com, Inc. Computer processes and systems for adaptively controlling the display of items
JP3997774B2 (en) * 2001-12-11 2007-10-24 ソニー株式会社 Data processing system, data processing method, information processing apparatus, and computer program
CN1299505C (en) * 2001-12-13 2007-02-07 皇家飞利浦电子股份有限公司 Recommending media content on a media system
US7343365B2 (en) * 2002-02-20 2008-03-11 Microsoft Corporation Computer system architecture for automatic context associations
US7743045B2 (en) * 2005-08-10 2010-06-22 Google Inc. Detecting spam related and biased contexts for programmable search engines
US7203909B1 (en) * 2002-04-04 2007-04-10 Microsoft Corporation System and methods for constructing personalized context-sensitive portal pages or views by analyzing patterns of users' information access activities
US20040039630A1 (en) * 2002-08-12 2004-02-26 Begole James M.A. Method and system for inferring and applying coordination patterns from individual work and communication activity
JP2005536816A (en) * 2002-08-19 2005-12-02 チョイスストリーム インコーポレイテッド Statistical specific personal recommendation system
US7349827B1 (en) * 2002-09-18 2008-03-25 Doubleclick Inc. System and method for reporting website activity based on inferred attribution methodology
US7103609B2 (en) * 2002-10-31 2006-09-05 International Business Machines Corporation System and method for analyzing usage patterns in information aggregates
US20040088322A1 (en) * 2002-10-31 2004-05-06 International Business Machines Corporation System and method for determining connections between information aggregates
US7065532B2 (en) * 2002-10-31 2006-06-20 International Business Machines Corporation System and method for evaluating information aggregates by visualizing associated categories
US20040088315A1 (en) * 2002-10-31 2004-05-06 International Business Machines Corporation System and method for determining membership of information aggregates
US7257569B2 (en) * 2002-10-31 2007-08-14 International Business Machines Corporation System and method for determining community overlap
US7249123B2 (en) * 2002-10-31 2007-07-24 International Business Machines Corporation System and method for building social networks based on activity around shared virtual objects
US20040117222A1 (en) * 2002-12-14 2004-06-17 International Business Machines Corporation System and method for evaluating information aggregates by generation of knowledge capital
BRPI0407451A (en) 2003-02-14 2006-02-07 Nervana Inc System and method for knowledge retrieval, semantics, management and presentation
JP2004348241A (en) * 2003-05-20 2004-12-09 Hitachi Ltd Information providing method, server, and program
US20040263639A1 (en) * 2003-06-26 2004-12-30 Vladimir Sadovsky System and method for intelligent image acquisition
US7162473B2 (en) * 2003-06-26 2007-01-09 Microsoft Corporation Method and system for usage analyzer that determines user accessed sources, indexes data subsets, and associated metadata, processing implicit queries based on potential interest to users
US6943877B2 (en) * 2003-06-30 2005-09-13 Emhart Glass S.A. Container inspection machine
US7836010B2 (en) * 2003-07-30 2010-11-16 Northwestern University Method and system for assessing relevant properties of work contexts for use by information services
US7577655B2 (en) * 2003-09-16 2009-08-18 Google Inc. Systems and methods for improving the ranking of news articles
US20050091184A1 (en) * 2003-10-24 2005-04-28 Praveen Seshadri Personalized folders
US20050108630A1 (en) * 2003-11-19 2005-05-19 Wasson Mark D. Extraction of facts from text
US7526458B2 (en) * 2003-11-28 2009-04-28 Manyworlds, Inc. Adaptive recommendations systems
US7363282B2 (en) * 2003-12-03 2008-04-22 Microsoft Corporation Search system using user behavior data
US20070074102A1 (en) * 2005-09-29 2007-03-29 Reiner Kraft Automatically determining topical regions in a document
US7958115B2 (en) * 2004-07-29 2011-06-07 Yahoo! Inc. Search systems and methods using in-line contextual queries
US7856441B1 (en) * 2005-01-10 2010-12-21 Yahoo! Inc. Search systems and methods using enhanced contextual queries
EP1672484A1 (en) * 2004-12-17 2006-06-21 Sap Ag System for identification of context related information in knowledge sources
WO2009021198A1 (en) * 2007-08-08 2009-02-12 Baynote, Inc. Method and apparatus for context-based content recommendation
US7698270B2 (en) * 2004-12-29 2010-04-13 Baynote, Inc. Method and apparatus for identifying, extracting, capturing, and leveraging expertise and knowledge
US20060161553A1 (en) * 2005-01-19 2006-07-20 Tiny Engine, Inc. Systems and methods for providing user interaction based profiles
US20060218115A1 (en) * 2005-03-24 2006-09-28 Microsoft Corporation Implicit queries for electronic documents
US20070043609A1 (en) * 2005-07-18 2007-02-22 Razi Imam Automated systems for defining, implementing and tracking the sales cycle
US9558498B2 (en) * 2005-07-29 2017-01-31 Excalibur Ip, Llc System and method for advertisement management
US20070050253A1 (en) * 2005-08-29 2007-03-01 Microsoft Corporation Automatically generating content for presenting in a preview pane for ADS
US7644101B2 (en) * 2005-09-07 2010-01-05 Ricoh Co., Ltd. System for generating and managing context information
US8117549B2 (en) * 2005-10-26 2012-02-14 Bruce Reiner System and method for capturing user actions within electronic workflow templates
US7895223B2 (en) * 2005-11-29 2011-02-22 Cisco Technology, Inc. Generating search results based on determined relationships between data objects and user connections to identified destinations
US7856446B2 (en) * 2005-12-27 2010-12-21 Baynote, Inc. Method and apparatus for determining usefulness of a digital asset
KR100673023B1 (en) * 2005-12-28 2007-01-24 삼성전자주식회사 Semiconductor memory device using pipelined-buffer programming scheme
US20080301541A1 (en) * 2006-04-27 2008-12-04 Ron Karidi Online internet navigation system and method
US7424488B2 (en) * 2006-06-27 2008-09-09 International Business Machines Corporation Context-aware, adaptive approach to information selection for interactive information analysis
US10110687B2 (en) * 2006-10-06 2018-10-23 International Business Machines Corporation Session based web usage reporter
KR101622742B1 (en) * 2006-10-18 2016-06-01 구글 인코포레이티드 Generic online ranking system and method suitable for syndication
US7590616B2 (en) * 2006-11-17 2009-09-15 Yahoo! Inc. Collaborative-filtering contextual model based on explicit and implicit ratings for recommending items
US8671021B2 (en) * 2006-12-13 2014-03-11 Quickplay Media Inc. Consumption profile for mobile media
WO2008097810A2 (en) * 2007-02-02 2008-08-14 Veoh Networks, Inc. Indicator-based recommendation system
US20080228537A1 (en) * 2007-02-26 2008-09-18 Andrew Monfried Systems and methods for targeting advertisements to users of social-networking and other web 2.0 websites and applications
US9405830B2 (en) * 2007-02-28 2016-08-02 Aol Inc. Personalization techniques using image clouds
US8271878B2 (en) * 2007-12-28 2012-09-18 Amazon Technologies, Inc. Behavior-based selection of items to present on affiliate sites

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10936824B2 (en) 2013-12-03 2021-03-02 International Business Machines Corporation Detecting literary elements in literature and their importance through semantic analysis and literary correlation

Also Published As

Publication number Publication date
WO2009021198A1 (en) 2009-02-12
CN101779180A (en) 2010-07-14
US20090037355A1 (en) 2009-02-05
JP2010536102A (en) 2010-11-25
CN101779180B (en) 2012-08-15
US8095523B2 (en) 2012-01-10
US20120311139A1 (en) 2012-12-06
EP2176730A4 (en) 2011-04-20
EP2176730A1 (en) 2010-04-21

Similar Documents

Publication Publication Date Title
US8095523B2 (en) Method and apparatus for context-based content recommendation
US20220020056A1 (en) Systems and methods for targeted advertising
Tang et al. Social recommendation: a review
Kumar et al. Approaches, issues and challenges in recommender systems: a systematic review
US8751430B2 (en) Methods and system of filtering irrelevant items from search and match operations using emotional codes
US10198503B2 (en) System and method for performing a semantic operation on a digital social network
US11188830B2 (en) Method and system for user profiling for content recommendation
US11449553B2 (en) Systems and methods for generating real-time recommendations
US20110010354A1 (en) Using scenario-related information to customize user experiences
Meyer Recommender systems in industrial contexts
Santesteban et al. How big data confers market power to big tech: Leveraging the perspective of data science
Pera et al. Exploiting the wisdom of social connections to make personalized recommendations on scholarly articles
Thomas et al. Machine learning techniques for recommender systems–a comparative case analysis
Markellou et al. Personalized e-commerce recommendations
Gursel et al. Improving search in social networks by agent based mining
Farid et al. User profiling approaches, modeling, and personalization
Sood et al. Survey on news recommendation
Srifi et al. A concise survey on content recommendations
Adeoye Hybrid Recommendation Systems
Esquerrà Schaefer Allyn, A Recommender Assistant for Online Bookstores
Hu Non-IID recommender systems: a machine learning approach
Aïmeur et al. WSRS: A web service recommender system
Tsatsou et al. deliverable 4.1 Specification of user profiling and contextualisation
Khalifeh User-Interface Web Recommender System
ÇUBUK Hybrid recommendation engine based on anonymous users

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200880102339.5

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08782688

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2008782688

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2010520330

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE