Like many other parts of SEO, TF-IDF is a concept that is highly debated. On the one hand, it’s known of as an amazing technique to rank your content on Google (and other search engines), and on the other, it’s apparently too old-school to be worth any effort!
As with most things, the truth is usually somewhere in between – that is, that TF-IDF is useful and works with SEO but might not be the be all and end all of SEO techniques.
This post will explore the definition of TF-IDF for SEO, and how the two are related.
What is TF-IDF?
TF-IDF is a method of a machine calculating what an article is about. It achieves this by using numeric representation. Put simply, a human brain can understand this article is about TF-IDF, yet a machine ‘brain’ needs to evaluate the relevancy of different articles, it looks at numeric representation to illustrate that Article A is about TF-IDF, whilst Article B is more about TF-IDF than Article B.
It doesn’t just do this by calculating the number of times the keyword, TF-IDF appears in an article. Instead, it looks at the size of documents and calculates how many times the keyword appears in relation to the document size.
This is called keyword density and is a widely used content optimization metric. Relying on this comes with a few problems – the words ‘and’ and ‘the’ may, for example, be more prominent than TF-IDF in this article.
TF-IDF is clever because it adjusts calculations to the fact that some words, like ‘and’ and ‘the’, appear more frequently in general. That’s how the algorithm works out what the article is really about – it calculates how a keyword is used in an article and compares its average frequency to other documents across the web.
So, the algorithm is able to pay less attention to all commonly used words, and to work out what word is the specific topic for any one piece of content.
The formula looks a bit like this: Wi, j = tf I,j x log (N/df i). TFij is the number of occurrences of I in j, df is the number of documents containing I, and N is the total number of documents.
Still with us?
Essentially, to analyze TF-IDF, it’s term frequency (TF) x inverse document frequency (IDF). When TF is multiplied for IDF, TF becomes lower for commonly used words and higher for unique words.
For example, the word ‘and’ and ‘the’ is used in every single article in English. But very few articles mention ‘TF-IDF’, ‘keywords’, ‘content’ and other subtopics in SEO optimization. So, the TF-IDF for these keywords becomes higher, and the machine brain knows what this article is about!
TF-IDF and SEO
In the past, TF-IDF was used when a machine needed to identify topics in a huge set of documents, for example in a library as it is digitized, and recommendation systems are put in place.
Recently, it’s gaining more popularity in SEO optimisation. Google has moved to semantic search, which is essentially that Google is now trying to match the meaning of a search query to topically relevant content. This is in opposition to Google ranking content based on query keywords to the same keywords on webpages.
This means that instead of just counting keywords, Google is counting co-occurrences, using the content around keywords to understand the meaning of the article.
For example: the word ‘cashew’ will be understood in relation to the content around it. Such as ‘high fat’, ‘protein’, ‘healthy’ etc. Based on this simple understanding of context, Google is able to better understand user queries and rank content based on relevance, not simply keyword density.
Although it’s probably not wise to try to engineer the entire system to work in your favor, by inserting more relevant content, you will rank higher on SERPs. This is because Google will see your content is more relevant – you’ll also be producing more relevant content, which is a plus for your readers!
In essence, TF-IDF can aid you with SEO by ranking your content higher on SERPs. This is because Google algorithms are now looking for relevance of content and seeking to direct users to truly useful content, not just to sites containing the specific keywords.
