feast.loaders package¶
Submodules¶
feast.loaders.file module¶
-
feast.loaders.file.
export_dataframe_to_local
(df: pandas.core.frame.DataFrame, dir_path: Optional[str] = None)[source]¶ Exports a pandas dataframe to the local filesystem
- Parameters
df – Pandas dataframe to save
dir_path – (optional) Absolute directory path ‘/data/project/subfolder/’
-
feast.loaders.file.
export_dataframe_to_staging_location
(df: pandas.core.frame.DataFrame, staging_location_uri: str) → str[source]¶ Uploads a dataframe to a remote staging location
- Parameters
df – Pandas dataframe
staging_location_uri –
Remote staging location where dataframe should be written .. rubric:: Examples
gs://bucket/path/ file:///data/subfolder/
- Returns
Returns the full path to the file in the remote staging location
-
feast.loaders.file.
upload_file_to_gcs
(local_path: str, bucket: str, remote_path: str)[source]¶ Upload a file from the local file system to Google Cloud Storage (GCS)
- Parameters
local_path – Local filesystem path of file to upload
bucket – GCS bucket to upload to
remote_path – Path within GCS bucket to upload file to, includes file name
feast.loaders.ingest module¶
-
feast.loaders.ingest.
ingest_table_to_kafka
(feature_set: feast.feature_set.FeatureSet, table: pyarrow.lib.Table, max_workers: int, chunk_size: int = 5000, disable_pbar: bool = False, timeout: int = None) → None[source]¶ Ingest a PyArrow Table to a Kafka topic based for a Feature Set
- Parameters
feature_set – FeatureSet describing PyArrow table.
table – PyArrow table to be processed.
max_workers – Maximum number of workers.
chunk_size – Maximum size of each chunk when PyArrow table is batched.
disable_pbar – Flag to indicate if tqdm progress bar should be disabled.
timeout – Maximum time before method times out
feast.loaders.yaml module¶
-
feast.loaders.yaml.
yaml_loader
(yml, load_single=False)[source]¶ Loads one or more Feast resources from a YAML path or string. Multiple resources can be divided by three hyphens ‘—’
- Parameters
yml – A path ending in .yaml or .yml, or a YAML string
load_single – Expect only a single YAML resource, fail otherwise
- Returns
Either a single YAML dictionary or a list of YAML dictionaries