feat: partition table query optimize

[zealchen]

Rationale

#1441

Detailed Changes

TLDR

The performance issue with inlist queries is due to the extra overhead from bloom-filter-like directory lookups when scanning each SST file for rows. The solution is to create a separate predicate for each partition, containing only the keys relevant to that partition. Since the current partition filter only supports BinaryExpr(Column, operator, Literal) and non-negated InList expressions, this solution will address only those specific cases.

Changes

1. During the scan building process, when identifying the partitions for a query, we create a PartitionedFilterKeyIndex variable to store the predicate key indices for each expression.
2. In the compute_partition_for_keys_group function, we use a HashMap<partition_id, HashMap<filter_index, BTreeSet<key_index>>> to record the indices of keys involved in partition computation for each group. Since DatumView only implements PartialEq or PartialOrd, it cannot be directly stored in a HashSet or BTreeSet. To handle this, I use a key_index in a set to ensure deduplication of all inlist keys when each key group is built in multi_cartesian_product.
3. In the partitioned_predicates function, we construct the final predicates for each partition.
4. In resolve_partitioned_scan_internal, we generate separate requests for each partition.

e.g.
conditions:

1. table schema: col_ts, col1, col2, in which col1 and col2 are both keys,
   and with two partitions
2. sql: select * from table where col1 = '33' and col2 in ("aa", "bb",
   "cc", "dd")

partition expectations:
yield two predicates
p0: col1 = '33' and col2 in ("aa", "bb", "cc");
p1: col1 = '33' and col2 in ("dd")

Other issues discovered

When the inlist key args length is less than three, Expr will be refactored to nested BinaryExpr which bypasses the FilterExtractor.

e.g.
SQL: select * from table where col1 in ("aa", "bb") and col2 in (1,2,3,4,5...1000)
Since ("aa", "bb") has fewer than three elements, the col1 key filter is not included in partition computation, which interrupts the partitioning process in the get_candidate_partition_keys_groups function, as contains_empty_filter is set to true.

Test Plan

1. UT: test_partitioned_predicate
2. Manual test.

[jiacai2050]

Add comments why we need to overwrite old predicate.

[jiacai2050]

Our codebase should prefer absolute import over super import.

[jiacai2050]

Suggested change

	.map_err(|e| DataFusionError::Internal(format!("err:{e}")))?,
	.map_err(|e| DataFusionError::Internal(format!("partition predicates failed, err:{e}")))?,

[jiacai2050]

DataFusion provider some helper functions to define expr:

• https://docs.rs/datafusion/latest/datafusion/logical_expr/index.html#functions

[jiacai2050]

Add comments explaining the type meaning of those usize.

[jiacai2050]

horaedb/src/common_types/src/datum.rs

Line 1313 in e065ba7

impl<'a> std::hash::Hash for DatumView<'a> {

DatumView already impl Hash