feast.loaders package

Submodules

feast.loaders.file module

feast.loaders.file.export_dataframe_to_local(df: pandas.core.frame.DataFrame, dir_path: Optional[str] = None)[source]

Exports a pandas dataframe to the local filesystem

Parameters
  • df – Pandas dataframe to save

  • dir_path – (optional) Absolute directory path ‘/data/project/subfolder/’

feast.loaders.file.export_dataframe_to_staging_location(df: pandas.core.frame.DataFrame, staging_location_uri: str) → str[source]

Uploads a dataframe to a remote staging location

Parameters
  • df – Pandas dataframe

  • staging_location_uri

    Remote staging location where dataframe should be written .. rubric:: Examples

    gs://bucket/path/ file:///data/subfolder/

Returns

Returns the full path to the file in the remote staging location

feast.loaders.file.upload_file_to_gcs(local_path: str, bucket: str, remote_path: str)[source]

Upload a file from the local file system to Google Cloud Storage (GCS)

Parameters
  • local_path – Local filesystem path of file to upload

  • bucket – GCS bucket to upload to

  • remote_path – Path within GCS bucket to upload file to, includes file name

feast.loaders.ingest module

feast.loaders.ingest.ingest_table_to_kafka(feature_set: feast.feature_set.FeatureSet, table: pyarrow.lib.Table, max_workers: int, chunk_size: int = 5000, disable_pbar: bool = False, timeout: int = None) → None[source]

Ingest a PyArrow Table to a Kafka topic based for a Feature Set

Parameters
  • feature_set – FeatureSet describing PyArrow table.

  • table – PyArrow table to be processed.

  • max_workers – Maximum number of workers.

  • chunk_size – Maximum size of each chunk when PyArrow table is batched.

  • disable_pbar – Flag to indicate if tqdm progress bar should be disabled.

  • timeout – Maximum time before method times out

feast.loaders.yaml module

feast.loaders.yaml.yaml_loader(yml, load_single=False)[source]

Loads one or more Feast resources from a YAML path or string. Multiple resources can be divided by three hyphens ‘—’

Parameters
  • yml – A path ending in .yaml or .yml, or a YAML string

  • load_single – Expect only a single YAML resource, fail otherwise

Returns

Either a single YAML dictionary or a list of YAML dictionaries

Module contents