Skip to content

Fix pandas_to_eland append to an aliased index#829

Open
vismaytiwari wants to merge 2 commits into
elastic:mainfrom
vismaytiwari:fix-pandas-to-eland-alias-747
Open

Fix pandas_to_eland append to an aliased index#829
vismaytiwari wants to merge 2 commits into
elastic:mainfrom
vismaytiwari:fix-pandas-to-eland-alias-747

Conversation

@vismaytiwari

Copy link
Copy Markdown

What

pandas_to_eland(..., es_if_exists="append") raised KeyError when es_dest_index is an alias rather than a concrete index name.

The append path verifies mapping compatibility via:

dest_mapping = es_client.indices.get_mapping(index=es_dest_index)[es_dest_index]

indices.get_mapping keys its response by the concrete index name, which differs from the alias passed in. So for an alias (e.g. an ILM rollover write alias) the subscript [es_dest_index] misses and raises KeyError: '<alias>', exactly as reported in #747.

Fix

Iterate the returned mappings instead of assuming the response is keyed by es_dest_index:

for dest_mapping in es_client.indices.get_mapping(index=es_dest_index).values():
    verify_mapping_compatibility(...)

This keeps the existing behavior for a concrete index (a single entry whose key equals the index), fixes the alias case, and also handles an alias that resolves to more than one index (each resolved index is verified).

Tests

Added test_es_if_exists_append_to_alias in tests/etl/test_pandas_to_eland.py: it creates an index, points an alias at it, and appends through the alias. The test fails on main with KeyError and passes with this change.

Closes #747

When es_dest_index is an alias, indices.get_mapping keys its response by
the concrete index name, so the previous response[es_dest_index] lookup
raised KeyError and made appends to an aliased index fail. Verify mapping
compatibility against every index the name resolves to instead, which
also covers an alias spanning multiple indices.

Closes elastic#747
@cla-checker-service

cla-checker-service Bot commented Jun 29, 2026

Copy link
Copy Markdown

💚 CLA has been signed

@prodsecmachine

prodsecmachine commented Jun 29, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@vismaytiwari

Copy link
Copy Markdown
Author

Gentle nudge - CLA is signed and CI/Snyk are green. Happy to change anything you'd like; otherwise it's ready for a review whenever someone has time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pandas_to_eland - cannot write into index by alias name

2 participants