Skip to content

Commit

Permalink
[MINOR][PYTHON][TESTS] Retry test_map_in_pandas_with_column_vector
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Retry `test_map_in_pandas_with_column_vector` and its parity test

### Why are the changes needed?
I am seeing this test and its parity test failing from time to time, and then fails `pyspark-sql` and `pyspark-connect`.

It seems due to some log4j issue, e.g

https://github.com/zhengruifeng/spark/actions/runs/7459243602/job/20294868487
```
test_map_in_pandas_with_column_vector (pyspark.sql.tests.pandas.test_pandas_map.MapInPandasTests) ... ERROR StatusConsoleListener An exception occurred processing Appender File
 java.lang.IllegalArgumentException: found 1 argument placeholders, but provided 0 for pattern `0, VisitedIndex{visitedIndexes={}}: [] r:0`
	at org.apache.logging.log4j.message.ParameterFormatter.formatMessage(ParameterFormatter.java:233)
```

https://github.com/apache/spark/actions/runs/7460093200/job/20297508703
```
  test_map_in_pandas_with_column_vector (pyspark.sql.tests.connect.test_parity_pandas_map.MapInPandasParityTests) ... ERROR StatusConsoleListener An exception occurred processing Appender File
 java.lang.IllegalArgumentException: found 1 argument placeholders, but provided 0 for pattern `0, VisitedIndex{visitedIndexes={}}: [] r:0`
	at org.apache.logging.log4j.message.ParameterFormatter.formatMessage(ParameterFormatter.java:233)
	at org.apache.logging.log4j.message.ParameterizedMessage.formatTo(ParameterizedMessage.java:266)
	at
```

this PR simply attempt to retry it after failures

### Does this PR introduce _any_ user-facing change?
no, test-only

### How was this patch tested?
ci

### Was this patch authored or co-authored using generative AI tooling?
no

Closes #44641 from zhengruifeng/py_test_retry_mip.

Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
zhengruifeng authored and HyukjinKwon committed Jan 10, 2024
1 parent 2cf07f9 commit d203328
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion python/pyspark/sql/tests/pandas/test_pandas_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
pandas_requirement_message,
pyarrow_requirement_message,
)
from pyspark.testing.utils import QuietTest
from pyspark.testing.utils import QuietTest, eventually

if have_pandas:
import pandas as pd
Expand Down Expand Up @@ -381,6 +381,7 @@ def test_self_join(self):
self.assertEqual(sorted(actual), sorted(expected))

# SPARK-33277
@eventually(timeout=180, catch_assertions=True)
def test_map_in_pandas_with_column_vector(self):
path = tempfile.mkdtemp()
shutil.rmtree(path)
Expand Down

0 comments on commit d203328

Please sign in to comment.