{"id":489,"date":"2026-04-11T20:43:27","date_gmt":"2026-04-11T19:43:27","guid":{"rendered":"https:\/\/who-offers.site\/index.php\/2026\/04\/11\/understanding-transformers-part-5-queries-keys-and-similarity\/"},"modified":"2026-04-11T20:43:33","modified_gmt":"2026-04-11T19:43:33","slug":"understanding-transformers-part-5-queries-keys-and-similarity","status":"publish","type":"post","link":"https:\/\/who-offers.site\/index.php\/2026\/04\/11\/understanding-transformers-part-5-queries-keys-and-similarity\/","title":{"rendered":"Understanding Transformers Part 5: Queries, Keys, and Similarity"},"content":{"rendered":"
\n

The topic Understanding Transformers Part 5: Queries, Keys, and Similarity<\/strong> is currently the subject of lively discussion \u2014 readers and analysts are keeping a close eye on developments.<\/p>\n

This is taking place in a dynamic environment: companies\u2019 decisions and competitors\u2019 reactions can quickly change the picture.<\/p>\n

In the previous article, we explored the self-attention concept for transformers, in this article we will go deeper into how the comparisons are performed.<\/p>\n

We have already added positional encoding to the words \u201cLet\u2019s\u201d and \u201cgo\u201d.<\/p>\n

The first step is to multiply the position-encoded values for the word \u201cLet\u2019s\u201d by a set of weights.<\/p>\n

Next, we repeat the same process using a different set of weights, which gives us another value (for example, 3.7).<\/p>\n

We do this twice because we started with two position-encoded values representing the word \u201cLet\u2019s\u201d.<\/p>\n

\"\"<\/figure>\n

These resulting values together represent \u201cLet\u2019s\u201d in a new form.<\/p>\n

Now, we use these query values to measure similarity with other words, such as \u201cgo\u201d.<\/p>\n

To do this, we first create a new set of values for each word, similar to how we created the query values.<\/p>\n

We will use these key values along with the query values to calculate how similar \u201cLet\u2019s\u201d is to \u201cgo\u201d.<\/p>\n

We will explore how this similarity is calculated in the next article.<\/p>\n

Looking for an easier way to install tools, libraries, or entire repositories?
\nTry Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.<\/p>\n

\"\"<\/figure>\n

Templates let you quickly answer FAQs or store snippets for re-use.<\/p>\n

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment’s permalink.<\/p>\n

For further actions, you may consider blocking this person and\/or reporting abuse<\/p>\n

Thank you to our Diamond Sponsors for supporting the DEV Community<\/p>\n

Google AI is the official AI Model and Platform Partner of DEV<\/p>\n

DEV Community \u2014 A space to discuss and keep up software development and manage your software career<\/p>\n

Built on Forem \u2014 the open source software that powers DEV and other inclusive communities.<\/p>\n

We’re a place where coders share, stay up-to-date and grow their careers.<\/p>\n