Google SMITH Algorithm
Do you already know everything about Google's SMITH algorithm? Google SMITH is an algorithm designed to complement the BERT algorithm and help with understanding longer texts. In this blog post, we answer the most important questions about the SMITH algorithm, including what its functions are, whether it is actually active, and how much it supports the BERT algorithm in Google Search.
To answer these questions, let's first take a look at what Google SMITH actually means and clarify its area of application.
Definition: Siamese Multi-depth Transformer-based Hierarchical (SMITH)
Google recently published a new research paper on an algorithm called "SMITH", which outperforms the BERT algorithm when understanding longer queries and longer documents and therefore supports the BERT algorithm. What makes this new model particularly effective is its ability to understand passages within documents in a similar way to how the BERT algorithm understands words and sentences. This enables the algorithm to understand longer and more complex documents. Through this extension, SMITH increases the maximum input text length that can be processed from 512 to 2,048 tokens. This allows SMITH to outperform other approaches such as HAN, SMASH, and BERT when matching long documents.
Is the SMITH algorithm active?
Google has stated that it intends to make greater use of the SMITH algorithm in the future. This represents one of the major developments in Google's algorithms. Estimates have indicated that Google processes more than 3 billion search queries worldwide every day. Overall, SMITH could therefore help provide different and more accurate results for more than 300 million searches per day.
General information about the SMITH algorithm
SMITH is a new algorithm designed to help Google better analyze longer texts and entire documents. Google currently uses the BERT algorithm, which is designed to understand individual words within the context of a sentence.
Simply put, the SMITH algorithm attempts to analyze and understand text passages within the context of the entire document.
However, the SMITH algorithm is not trained on datasets in the same way as BERT. Instead, it is trained to predict what the next sentence block should be. The BERT algorithm, by comparison, predicts hidden words based on the context within a sentence. The researchers believe that this type of training enables SMITH to understand larger documents significantly better than the BERT algorithm.
The BERT algorithm has limitations:
In recent years, self-attention-based models such as BERT and Transformers have performed very well in text matching. However, because of the quadratic computational complexity of self-attention in relation to the length of the text they can process, these models are highly limited and can sometimes analyze only sentences or smaller passages.
Researchers clearly state that the BERT algorithm is not suitable for understanding longer texts for a variety of reasons, but rather for short sentence-level inferences, sentences, or individual words.
Why are long texts particularly difficult to analyze?
1) For longer texts, so-called matching requires a better understanding than it does for individual sentences. It requires a semantic relationship, including matching patterns between text fragments that may be far apart.
2) Long documents always have an internal structure, such as passages, sections, sentences, etc. For readers, structure plays an important role and is essential for understanding a topic. Therefore, the model must take this structure into account to enable better document matching.
3) When processing complex texts, inadequate model design often leads to problems.
BERT is limited to a certain text length (512 tokens). The longer the text, the worse BERT performs, whereas the SMITH algorithm tends to perform better as the text becomes longer.
Nevertheless, the SMITH model is not intended to replace the BERT algorithm, but rather to serve as an extension.
SMITH is one of the developments that Google planned to implement in 2021.
Do you have any questions about this topic? Then feel free to contact us!
