When trying to write a dataframe with categorical columns to a HDF store, I get the following error:
Cannot store a category dtype in a HDF5 dataset that uses format="fixed". Use format="table"
This is resolved when writing out using table as the output format instead of fixed. So my question is: should the output format be another option on the run method, or should categoricals be converted back to strings and then output using the fixed format?
As an aside, the output will also fail, even with table as the output format, if the categorical was built from a column with unicode strings. Calling column = column.astype(str) before creating the categorical seems to take care of this.
Or should we just avoid categoricals all together?
When trying to write a dataframe with categorical columns to a HDF store, I get the following error:
Cannot store a category dtype in a HDF5 dataset that uses format="fixed". Use format="table"
This is resolved when writing out using table as the output format instead of fixed. So my question is: should the output format be another option on the run method, or should categoricals be converted back to strings and then output using the fixed format?
As an aside, the output will also fail, even with table as the output format, if the categorical was built from a column with unicode strings. Calling
column = column.astype(str)before creating the categorical seems to take care of this.Or should we just avoid categoricals all together?