Thank you! Prela’s `.and` operator is exactly the fork in fork algebra. I also suspect you will get `.select` and `.and` if you lift >>> and &&& to the category of Rel.
I wonder where the limitations of this approach are seen?
Compile time. Rustc takes forever to compile, much longer than the time it takes to run the query. There’s plan to build a JIT for Prela, which would also improve the interop.
Most of the language design is orthogonal to the embedded implementation though, and Prela could very well be implemented in a vectorized engine.
Got it! One more question if I may: in https://arxiv.org/pdf/2607.26356, all the JOB queries are around a second or less, if I read correctly.
My understanding is the query:
movie
.with(keyword.eq("my-kw"))
.select(title)
is inlined to something by the compiler approximately like
for movie in 0..movie_count {
let lo = keyword_offsets[movie];
let hi = keyword_offsets[movie + 1];
for pos in lo..hi {
let kw = keyword_ids[pos];
if keyword_text[kw] == "my-kw" {
emit(movie, movie_title[movie]);
break;
}
}
}
So there's no index lookup on keyword, right? E.g., to use a hash index on keyword to find the resulting movie rows. If I wanted to do so, would I re-normalize the data in some way? This is what surprised me: that even without the secondary indexes, it is still the same (or more) performant than DuckDB.
Perhaps the index-lookup version would use something like
let movies_by_keyword: HashIdx<_, _> =
keyword.text().inv().collect();
but it does not seem like it does (even though I assume DuckDB may).
Yes basically. But DuckDB actually does not build an index on the keyword text. It only builds primary key indices automatically on data load. The standard benchmark schema specifies fk indices but not one for the keyword text.
Comments
Thank you! Prela’s `.and` operator is exactly the fork in fork algebra. I also suspect you will get `.select` and `.and` if you lift >>> and &&& to the category of Rel.
Compile time. Rustc takes forever to compile, much longer than the time it takes to run the query. There’s plan to build a JIT for Prela, which would also improve the interop.
Most of the language design is orthogonal to the embedded implementation though, and Prela could very well be implemented in a vectorized engine.
Got it! One more question if I may: in https://arxiv.org/pdf/2607.26356, all the JOB queries are around a second or less, if I read correctly.
My understanding is the query:
movie
is inlined to something by the compiler approximately likefor movie in 0..movie_count {
}So there's no index lookup on keyword, right? E.g., to use a hash index on keyword to find the resulting movie rows. If I wanted to do so, would I re-normalize the data in some way? This is what surprised me: that even without the secondary indexes, it is still the same (or more) performant than DuckDB.
Perhaps the index-lookup version would use something like
let movies_by_keyword: HashIdx<_, _> =
but it does not seem like it does (even though I assume DuckDB may).Yes basically. But DuckDB actually does not build an index on the keyword text. It only builds primary key indices automatically on data load. The standard benchmark schema specifies fk indices but not one for the keyword text.