Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VTJ-JP][feature] arrow_fdw仮想列での検索でOR/IN等への対応 #854

Open
sakaik opened this issue Nov 14, 2024 · 0 comments
Open
Labels
feature developer considers it is the feature to be supported on the future version

Comments

@sakaik
Copy link

sakaik commented Nov 14, 2024

リクエスト内容

 2021年11月に対応してくださった arrow_fdw の仮想列を v_col としたとき、
WHERE v_col='00123' のようなズバリ指定では使用arrowファイルの絞り込みをしてくれるが、
ここに OR や INを使用すると arrowファイルの絞り込みをしてくれなくなります。
OR/IN 等にも対応してほしいです。

arrowファイルをテーブルにする手順

  • arrowをテーブルに
IMPORT FOREIGN SCHEMA mydata
  FROM SERVER arrow_fdw INTO public 
 OPTIONS(dir '/opt/nvme/arrow_files/', pattern 'data_${v_fileid}_@{v_gnx}_@{v_gny}.arrow');
  • 定義確認
db=# \d mydata
                         Foreign table "public.mydata"
  Column  |       Type       | Collation | Nullable | Default |     FDW options      
----------+------------------+-----------+----------+---------+----------------------
 id       | character(5)     |           |          |         | 
(略)
 v_fileid | text             |           |          |         | (virtual 'v_fileid')
 v_gnx    | bigint           |           |          |         | (virtual 'v_gnx')
 v_gny    | bigint           |           |          |         | (virtual 'v_gny')
Server: arrow_fdw
FDW options: (dir '/opt/nvme/arrow_files/', pattern 'data_${v_fileid}_@{v_gnx}_@{v_gny}.arrow')

クエリ例

基本クエリ(arrowが絞り込まれる)

SELECT MAX(eps) FROM myata WHERE v_fileid='00135';

実行計画中に以下の行が出力されている。

stats-Hint: (v_fileid = '00135'::text)  [loaded: 6, skipped: 12282]

LIKEやINやORでは絞られない(stats-Hintが出力されていない)

SELECT v_fileid, MAX(eps) FROM mydata 
 WHERE v_fileid like '0013%' GROUP BY v_fileid;
SELECT v_fileid, MAX(eps) FROM mydata 
 WHERE v_fileid IN('00130','00131','00132','00133','00134','00135','00136','00137','00138','00139') GROUP BY v_fileid;
SELECT v_fileid, MAX(eps) FROM mydata 
 WHERE v_fileid='00130' OR v_fileid='00131' GROUP BY v_fileid;

ズバリ指定をUNIONで(絞り込まれる)

絞り込まれることがわかっているクエリをUNIONでまとめた状態であれば、ひとつひとつは絞り込まれた状態で実施されるので(=stats-Hintが出力される)、高速。

SELECT v_fileid, MAX(eps) FROM mydata WHERE v_fileid='00140' GROUP BY v_fileid
UNION ALL
SELECT v_fileid, MAX(eps) FROM mydata WHERE v_fileid='00141' GROUP BY v_fileid;
@kaigai kaigai added the feature developer considers it is the feature to be supported on the future version label Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature developer considers it is the feature to be supported on the future version
Projects
None yet
Development

No branches or pull requests

2 participants