Merged

BUG: concat coercing arrow to object with null type #53702

Show file tree
Hide file tree
Changes from all commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6a6c0c0
BUG: concat coercing arrow to object with null type
phoflJun 16, 2023
76640f6
BUG: concat coercing arrow to object with null type
phoflJun 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Failed to load files.
Original file line numberDiff line numberDiff line change
Expand Up@@ -471,6 +471,7 @@ Groupby/resample/rolling

Reshaping
^^^^^^^^^
- Bug in :func:`concat` coercing to ``object`` dtype when one column has ``pa.null()`` dtype (:issue:`53702`)
- Bug in :func:`crosstab` when ``dropna=False`` would not keep ``np.nan`` in the result (:issue:`10772`)
- Bug in :func:`merge_asof` raising ``KeyError`` for extension dtypes (:issue:`52904`)
- Bug in :func:`merge_asof` raising ``ValueError`` for data backed by read-only ndarrays (:issue:`53513`)
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -2872,6 +2872,15 @@ def test_conversion_large_dtypes_from_numpy_array(data, arrow_dtype):
tm.assert_extension_array_equal(result, expected)


def test_concat_null_array():
df = pd.DataFrame({"a": [None, None]}, dtype=ArrowDtype(pa.null()))
df2 = pd.DataFrame({"a": [0, 1]}, dtype="int64[pyarrow]")

result = pd.concat([df, df2], ignore_index=True)
expected = pd.DataFrame({"a": [None, None, 0, 1]}, dtype="int64[pyarrow]")
tm.assert_frame_equal(result, expected)


@pytest.mark.parametrize("pa_type", tm.ALL_INT_PYARROW_DTYPES + tm.FLOAT_PYARROW_DTYPES)
def test_describe_numeric_data(pa_type):
# GH 52470
Expand Down