Table

class featherstore.table.Table(table_name, store_name)

Bases: object

A class for saving and loading DataFrames as partitioned Feather files.

Tables supports several operations that can be done without loading in the full data:

• Partial reading of data

• Append data

• Insert rows and columns

• Update data

• Drop data

• Read metadata (column names, index, table shape, etc)

• Changing column types

Parameters:

• table_name (str) – The name of the table.

• store_name (str) – The name of the store.

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• StoreNotFoundError – If the store does not exist.

• ForbiddenTableNameError – If table_name is reserved.

• TypeError – If table_name is not a str.

read_arrow(*, cols=None, rows=None, mmap=None)

Reads the data as a PyArrow Table

Parameters:

• cols (Collection, optional) – List of column names or, filter-predicates in the form of {‘like’: pattern}. If not provided, all columns are read.

• rows (Collection, optional) – List of index values or filter-predicates in the form of {keyword: value}, where keyword can be either before, after, or between. If not provided, all rows are read.

• mmap (bool, optional) – Use memory mapping when opening table on disk, by default False on Windows and True on other systems.

Return type:

pyarrow.Table

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• ColumnNotFoundError – If any requested column is not in the table.

• RowNotFoundError – If any requested row is not in the table.

• IndexTypeMismatchError – If row values do not match the table index dtype.

• TypeError – If cols or rows has an invalid type.

• ValueError – If mmap is not a bool or None.

read_pandas(*, cols=None, rows=None, mmap=None)

Reads the data as a Pandas DataFrame or Series

Parameters:

• cols (Collection, optional) – List of column names or, filter-predicates in the form of {‘like’: pattern}. If not provided, all columns are read.

• rows (Collection, optional) – List of index values or filter-predicates in the form of {keyword: value}, where keyword can be either before, after, or between. If not provided, all rows are read.

• mmap (bool, optional) – Use memory mapping when opening table on disk, by default False on Windows and True on other systems.

Return type:

pandas.DataFrame or pandas.Series

:raises Same exceptions as read_arrow().:

read_polars(*, cols=None, rows=None, mmap=None)

Reads the data as a Polars DataFrame or Series

Parameters:

• cols (Collection, optional) – List of column names or, filter-predicates in the form of {‘like’: pattern}. If not provided, all columns are read.

• rows (Collection, optional) – List of index values or filter-predicates in the form of {keyword: value}, where keyword can be either before, after, or between. If not provided, all rows are read.

• mmap (bool, optional) – Use memory mapping when opening table on disk, by default False on Windows and True on other systems.

Return type:

polars.DataFrame or polars.Series

:raises Same exceptions as read_arrow().:

write(df, /, index=None, *, partition_size=134217728, errors='raise', warnings='warn')

Writes a DataFrame to the current table.

The DataFrame index column, if provided, must be either of type int, str, or datetime. FeatherStore sorts the DataFrame by the index before storage.

Parameters:

• df (pandas DataFrame or Series, polars DataFrame or Series, or pyarrow Table) – The DataFrame to be stored

• index (str, optional) – The name of the column to be used as index. Uses current index for Pandas or a standard integer index for Arrow and Polars if index not provided, by default None

• partition_size (int, optional) – The size of each partition in bytes. A partition_size value of -1 disables partitioning, by default 128 MB

• errors (str, optional) – Whether or not to raise an error if the table already exist. Can be either raise or ignore, ignore overwrites existing table, by default raise

• warnings (str, optional) – Whether or not to warn if a unsorted index is about to get sorted. Can be either warn or ignore, by default warn

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableAlreadyExistsError – If errors='raise' and the table already exists.

• DuplicateColumnNamesError – If column names are not unique.

• DuplicateIndexValuesError – If index values are not unique.

• IndexNotInColumnsError – If index is not among the table columns.

• UnsupportedIndexTypeError – If the index type is not supported.

• MultiTypeColumnError – If a column contains multiple dtypes.

• TypeError – If arguments have invalid types.

• ValueError – If errors or warnings is invalid.

append(df, *, warnings='warn')

Appends data to the current table

Parameters:

• df (pandas DataFrame or Series, polars DataFrame or Series, or pyarrow Table) – The data to be appended

• warnings (str, optional) – Whether or not to warn if a unsorted index is about to get sorted. Can be either warn or ignore, by default warn

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• AppendIndexError – If append index is not strictly after stored data.

• ColumnDtypeMismatchError – If column dtypes are incompatible.

• ColumnMismatchError – If column names do not match the stored table.

• DuplicateColumnNamesError – If column names are not unique.

• DuplicateIndexValuesError – If index values are not unique.

• IndexNameMismatchError – If the index name does not match the stored table.

• IndexTypeMismatchError – If the index type does not match the stored table.

• MissingIndexError – If an index is required but not provided.

• TypeError – If df has an invalid type.

• ValueError – If warnings is invalid.

update(df)

Updates data in the current table.

Note: You can’t use this method to update index values. Updating index values can be accomplished by deleting the old records and inserting new ones with the updated index values.

Parameters:

df (pandas DataFrame or Series, polars DataFrame, or pyarrow Table) – The updated data. The index of df is the rows to be updated, while the columns of df are the new values.

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• ColumnDtypeMismatchError – If column dtypes are incompatible.

• ColumnNotFoundError – If any column is not in the stored table.

• DuplicateColumnNamesError – If column names are not unique.

• DuplicateIndexValuesError – If index values are not unique.

• IndexNameMismatchError – If the index name does not match the stored table.

• IndexTypeMismatchError – If the index type does not match the stored table.

• RowNotFoundError – If any row is not in the stored table.

• TypeError – If df is not a supported table type.

insert(df, *, idx=-1, warnings='warn')

Insert one or more rows or columns into the current table.

If df column names match the stored table, rows are inserted. Otherwise, columns are inserted at position idx.

Parameters:

• df (pandas DataFrame or Series, polars DataFrame, or pyarrow Table) – The data to be inserted.

• idx (int or Sequence[int], optional) – The position(s) to insert new column(s). Only valid when inserting columns. If a sequence is provided, it must have one position per new column. Default is to add columns to the end.

• warnings (str, optional) – Whether or not to warn if a unsorted index is about to get sorted. Can be either warn or ignore, by default warn

Raises:

TypeError – If idx is passed when inserting rows.

:raises Same exceptions as insert_rows() and insert_columns().:

insert_rows(df, *, warnings='warn')

Insert one or more rows into the current table.

Parameters:

• df (pandas DataFrame or Series, polars DataFrame, or pyarrow Table) – The data to be inserted. df must have the same index and column types as the stored data.

• warnings (str, optional) – Whether or not to warn if a unsorted index is about to get sorted. Can be either warn or ignore, by default warn

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• ColumnDtypeMismatchError – If column dtypes are incompatible.

• ColumnMismatchError – If column names do not match the stored table.

• DuplicateColumnNamesError – If column names are not unique.

• DuplicateIndexValuesError – If index values are not unique.

• IndexNameMismatchError – If the index name does not match the stored table.

• IndexTypeMismatchError – If the index type does not match the stored table.

• RowAlreadyExistsError – If any row already exists in the stored table.

• TypeError – If df is not a supported table type.

• ValueError – If warnings is invalid.

insert_columns(df, *, idx=-1, warnings='warn')

Insert one or more columns into the current table.

Parameters:

• df (pandas DataFrame or Series, polars DataFrame, or pyarrow Table) – The data to be inserted. df must have the same index as the stored data.

• idx (int or Sequence[int], optional) – The position(s) to insert the new column(s). If a sequence is provided, it must have one position per new column. Default is to add columns to the end.

• warnings (str, optional) – Whether or not to warn if a unsorted index is about to get sorted. Can be either warn or ignore, by default warn

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• ColumnAlreadyExistsError – If a new column name already exists.

• ColumnLengthMismatchError – If new column length does not match stored row count.

• DuplicateColumnNamesError – If column names are not unique.

• IndexMismatchError – If indices do not match the stored table.

• IndexNameInColumnsError – If a new column uses the index name.

• IndexTypeMismatchError – If the index type does not match the stored table.

• TypeError – If df or idx has an invalid type.

• ValueError – If idx length does not match the number of new columns, or warnings is invalid.

drop(*, cols=None, rows=None)

Drop specified labels from rows or columns.

Parameters:

• cols (Collection, optional) – list of column names or filter-predicates in the form of {‘like’: pattern}, by default None

• rows (Collection, optional) – list of index values or, filter-predicates in the form of {keyword: value}, where keyword can be either before, after, or between, by default None

Raises:

• AttributeError – If neither rows nor cols is provided.

• CannotDropAllColumnsError – If dropping all columns.

• CannotDropAllRowsError – If dropping all rows.

• ColumnNotFoundError – If any column to drop is not in the table.

• IndexNameInColumnsError – If attempting to drop the index column.

• IndexTypeMismatchError – If row values do not match the table index dtype.

• NotConnectedError – If FeatherStore is not connected to a database.

• RowNotFoundError – If any row to drop is not in the table.

• TableNotFoundError – If the table does not exist.

• TypeError – If cols or rows has an invalid type.

drop_rows(rows)

Drops specified rows from table

Same as Table.drop(rows=value)

Parameters:

rows (Collection, optional) – list of index values or, filter-predicates in the form of {keyword: value}, where keyword can be either before, after, or between, by default None

:raises Same exceptions as drop() when dropping rows.:

drop_columns(cols)

Drops specified rows from table

Same as Table.drop(cols=value)

Parameters:

cols (Collection, optional) – list of column names or filter-predicates in the form of {‘like’: pattern}, by default None

:raises Same exceptions as drop() when dropping columns.:

rename_columns(cols, *, to=None)

Rename one or more columns.

rename_columns supports two different call-syntaxes:

• rename_columns({‘c1’: ‘new_c1’, ‘c2’: ‘new_c2’})

• rename_columns([‘c1’, ‘c2’], to=[‘new_c1’, ‘new_c2’])

Parameters:

• cols (Collection) – Either a list of columns to be renamed, or a dict mapping columns to be renamed to new column names

• to (Collection[str], optional) – New column names, by default None

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• AttributeError – If to is provided twice or not provided when required.

• ColumnNotFoundError – If any column to rename is not in the table.

• DuplicateColumnNamesError – If renamed columns would not be unique.

• IndexNameInColumnsError – If a column is renamed to the index name.

• TypeError – If cols or to has an invalid type.

• ValueError – If the number of columns and new names do not match.

property columns

Fetches the names of the table columns

Returns:

The table columns

Return type:

list

reorder_columns(cols)

Reorder the current columns

Parameters:

cols (Sequence[str]) – The new column ordering. The column names provided must be the same as the column names used in the table.

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• ColumnMismatchError – If column names do not match the stored table.

• DuplicateColumnNamesError – If column names are not unique.

• IndexNameInColumnsError – If the index name is included in cols.

• TypeError – If cols has an invalid type.

property index

Fetches the table index

Return type:

pandas.Index

astype(cols, *, to=None)

Change data type of one or more columns.

astype supports two different call-syntaxes:

• astype({‘c1’: pa.int64(), ‘c2’: pa.int16()})

• astype([‘c1’, ‘c2’], to=[pa.int64(), pa.int16()])

Parameters:

• cols (Sequence[str] or dict) – Either a sequence of columns to have its data types changed, or a dict mapping columns to new column data types.

• to (Sequence[Pyarrow DataType], optional) – New column data types, by default None

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• TableNotFoundError – If the table does not exist.

• AttributeError – If to is provided twice or not provided when required.

• DuplicateColumnNamesError – If column names are not unique.

• UnsupportedIndexTypeError – If a forbidden index dtype is requested.

• TypeError – If cols or to has an invalid type.

• ValueError – If the number of columns and dtypes do not match.

rename_table(*, to)

Renames the current table

Parameters:

to (str) – The new name of the table.

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• ForbiddenTableNameError – If to is reserved.

• TableAlreadyExistsError – If a table with the new name already exists.

• TypeError – If to is not a str.

drop_table(*, warnings='warn')

Deletes the current table

Parameters:

warnings (str, optional) – Whether or not to warn if the table doesn’t exist. Can be either warn or ignore, by default warn

Raises:

ValueError – If warnings is not 'warn' or 'ignore'.

create_snapshot(path)

Creates a compressed backup of the table.

The table can later be restored by using snapshot.restore_table().

Parameters:

path (str) – The path to the snapshot archive.

Raises:

• NotConnectedError – If FeatherStore is not connected to a database.

• SnapshotTargetNotFoundError – If the table path does not exist.

• TypeError – If path is not a str.

repartition(new_partition_size)

Repartitions a table so that each partition is new_partition_size big.

Parameters:

new_partition_size (int) – The size of each partition in bytes. A new_partition_size value of -1 disables partitioning

property shape

Fetches the shape of the stored table as (rows, columns).

Returns:

The shape of the table

Return type:

tuple(int, int)

property partition_size

Fetches the table partition size in bytes.

Returns:

The partition size in bytes.

Return type:

int

exists()

property name