Interesting idea! What are you suggesting to do with relevance? I feel like one MEANS request returns just one best matching result while a user may want to have more. We decided to express it in SQL logic like what you see below but I'm really curious how the MEANS operator could be applied.
-----
SELECT id
FROM index_name
ORDER BY emb <-> $query_vector
LIMIT k;
-----
$query_vector may come from anywhere including being transformed on the fly from 'some text'.
I built it using Apache Datafusion. It adds semantic operators to SQL such as MEANS which can be used to filter data by its meaning. The core idea is that a user should be able to process unstructured data while semcast takes care of optimizing the query, including LLM calls.
Interesting idea! What are you suggesting to do with relevance? I feel like one MEANS request returns just one best matching result while a user may want to have more. We decided to express it in SQL logic like what you see below but I'm really curious how the MEANS operator could be applied.
-----
SELECT id
FROM index_name
ORDER BY emb <-> $query_vector
LIMIT k;
-----
$query_vector may come from anywhere including being transformed on the fly from 'some text'.
I built it using Apache Datafusion. It adds semantic operators to SQL such as MEANS which can be used to filter data by its meaning. The core idea is that a user should be able to process unstructured data while semcast takes care of optimizing the query, including LLM calls.
Would love some feedback