In order to gain a better understanding how AI uses resources to give you better answers (when you set up a context database), you need to get familiar with the concept of a vector database and terms like chunking, embedding or similarity. You will then see how AI, going beyond a simple keyword match, uses vectors to identify useful context based on meaning.
Phrases turned into vectors
To give an oversimplified view, let me say that words, phrases, sentences or even whole paragraphs in such database exist in a multidimensional space and their location is explained by their vectors (arrays of coordinates). In such space, words like “dog” or “cat” will stay relatively close, while a “helicopter” will be located further away. If you then ask about a “goldfish”, AI will go and look somewhere closer to dogs and cats (relative to helicopter) where also words like “pet” and “animal”, are located, to search for additional context on caring for a fish and give you an answer you are looking for.
The same thing will happen to entire paragraphs – a piece of context will be split into chunks, these chunks will be embedded into vectors and represented in this space next to other vectors that have the highest similarities between them. AI will later turn your question also into a vector, and will dig up pieces of context from the database with the highest similarity scores to your question.
While it is easier to understand it by looking into a 3-dimensional space, you have to remember vectors consist of hundreds and often thousands of dimensions, in order to more precisely assign the meaning of text.
See this visualized
All these concepts are difficult to understand for a non-technical person (“what do you mean there are more than 3 dimensions?”). This is why as a test for newly released Astra model I requested a page where it will explain all that using practical examples, visualizations and a simplified simulation of what AI does in the backend, to give you the answers you are looking for.
I think it did a pretty decent job. Have a look:
Explore the interactive vector database simulation →
Make sure to use the first section to simulate chunking or embedding (incl. chunking size), and see a visualization of their placement in a 2D, 3D and 12D space.