Add unit test to investigate torch issues (scaled_dot_product_attention, index_put) #1864
Conversation
Signed-off-by: xadupre <[email protected]>
Signed-off-by: xadupre <[email protected]>
xadupre
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1864 +/- ##
==========================================
+ Coverage 75.26% 75.32% +0.06%
==========================================
Files 251 251
Lines 27446 27446
Branches 5032 5032
==========================================
+ Hits 20656 20673 +17
+ Misses 5822 5808 -14
+ Partials 968 965 -3 ☔ View full report in Codecov by Sentry. |
Signed-off-by: xadupre <[email protected]>
| key_states = torch.randn(batch_size, seq_length_kv, embedding_dim) | ||
| value_states = torch.randn(batch_size, seq_length_kv, embedding_dim) | ||
|
|
||
| output = model(query_states, key_states, value_states) |
Check warning
Code scanning / CodeQL
Variable defined multiple times Warning test
| else: | ||
| raise AssertionError(f"Unknown exporter {exporter!r}") | ||
|
|
||
| import onnxruntime |
Check notice
Code scanning / lintrunner
PYLINT/C0415 Note test
| onnx_file_path = f"scaled_dot_product_attention_{exporter}.onnx" | ||
|
|
||
| if exporter == "script": | ||
| torch.onnx.export( |
Check failure
Code scanning / lintrunner
PYLINT/E1123 Error test
| else: | ||
| raise AssertionError(f"Unknown exporter {exporter!r}") | ||
|
|
||
| import onnxruntime |
Check notice
Code scanning / lintrunner
PYLINT/C0415 Note test
|
Do you have plans to merge or is this for investigation only? Marking as draft for now |
|
We do need to rewrite our index put implementation. #1749 |
|
I don't have time to implement a fix this week but anybody doing it should check the with unit tests I made and decide whether or not they should be kept. |
See pytorch/pytorch#135615, pytorch/pytorch#135233.