I’m curious how this approach differs from the Qwen Embeddings 3 model. As far as I can understand they generate query and answer values for a variety of roles from a document corpus with a Qwen LLM. Then they filter these for “high quality” by removing any pairs with a cosine similarity of less than 0.7.
So they use LLMs to come up with the queries and answers to learn the embedding, while you seem to use real queries but ask an LLM to remove any “wrong” answers? I’m not an expert on this stuff so apologies if I’m misstating your approach.
What would help audience to engage more with this article is to start with why focusing on embedding model is important?
May I ask what you studied for your bachelors?
I’m curious how this approach differs from the Qwen Embeddings 3 model. As far as I can understand they generate query and answer values for a variety of roles from a document corpus with a Qwen LLM. Then they filter these for “high quality” by removing any pairs with a cosine similarity of less than 0.7.
So they use LLMs to come up with the queries and answers to learn the embedding, while you seem to use real queries but ask an LLM to remove any “wrong” answers? I’m not an expert on this stuff so apologies if I’m misstating your approach.
Could it be scale?
how much does it cost to run this?