feast package¶
Subpackages¶
Submodules¶
feast.client module¶
-
class
feast.client.
Client
(core_url: str = None, serving_url: str = None, verbose: bool = False)[source]¶ Bases:
object
Feast Client: Used for creating, managing, and retrieving features.
-
apply
(feature_sets: Union[List[feast.feature_set.FeatureSet], feast.feature_set.FeatureSet])[source]¶ Idempotently registers feature set(s) with Feast Core. Either a single feature set or a list can be provided.
- Parameters
feature_sets – List of feature sets that will be registered
-
property
core_url
¶ Retrieve Feast Core URL
-
get_batch_features
(feature_ids: List[str], entity_rows: pandas.core.frame.DataFrame) → feast.job.Job[source]¶ Retrieves historical features from a Feast Serving deployment.
- Parameters
feature_ids – List of feature ids that will be returned for each entity. Each feature id should have the following format “feature_set_name:version:feature_name”.
entity_rows – Pandas dataframe containing entities and a ‘datetime’ column. Each entity in a feature set must be present as a column in this dataframe. The datetime column must
- Returns
Returns a job object that can be used to monitor retrieval progress asynchronously, and can be used to materialize the results
Examples
>>> from feast import Client >>> from datetime import datetime >>> >>> feast_client = Client(core_url="localhost:6565", serving_url="localhost:6566") >>> feature_ids = ["customer:1:bookings_7d"] >>> entity_rows = pd.DataFrame( >>> { >>> "datetime": [pd.datetime.now() for _ in range(3)], >>> "customer": [1001, 1002, 1003], >>> } >>> ) >>> feature_retrieval_job = feast_client.get_batch_features(feature_ids, entity_rows) >>> df = feature_retrieval_job.to_dataframe() >>> print(df)
-
get_feature_set
(name: str, version: int = None, fail_if_missing: bool = False) → Optional[feast.feature_set.FeatureSet][source]¶ Retrieves a feature set. If no version is specified then the latest version will be returned.
- Parameters
name – Name of feature set
version – Version of feature set
fail_if_missing – Raise an error if feature set is not found
- Returns
Returns either the specified feature set, or None if not found
-
get_online_features
(feature_ids: List[str], entity_rows: List[feast.serving.ServingService_pb2.EntityRow]) → feast.serving.ServingService_pb2.GetOnlineFeaturesResponse[source]¶ Retrieves the latest online feature data from Feast Serving
- Parameters
feature_ids –
List of feature Ids in the following format [feature_set_name]:[version]:[feature_name] .. rubric:: Example
[“feature_set_1:6:my_feature_1”, “feature_set_1:6:my_feature_2”,]
entity_rows – List of GetFeaturesRequest.EntityRow where each row contains entities. Timestamp should not be set for online retrieval. All entity types within a feature
- Returns
Returns a list of maps where each item in the list contains the latest feature values for the provided entities
-
ingest
(feature_set: Union[str, feast.feature_set.FeatureSet], source: Union[pandas.core.frame.DataFrame, str], version: int = None, force_update: bool = False, max_workers: int = 4, disable_progress_bar: bool = False, chunk_size: int = 5000, timeout: int = None)[source]¶ Loads feature data into Feast for a specific feature set.
- Parameters
feature_set – Name of feature set or a feature set object
source – Either a file path or Pandas Dataframe to ingest into Feast Files that are currently supported: * parquet * csv * json
version – Feature set version
force_update – Automatically update feature set based on source data prior to ingesting. This will also register changes to Feast
max_workers – Number of worker processes to use to encode values
disable_progress_bar – Disable printing of progress statistics
chunk_size – Maximum amount of rows to load into memory and ingest at a time
timeout – Seconds to wait before ingestion times out
-
list_entities
() → Dict[str, feast.entity.Entity][source]¶ Returns a dictionary of entities across all feature sets
- Returns
Dictionary of entities, indexed by name
-
list_feature_sets
() → List[feast.feature_set.FeatureSet][source]¶ Retrieve a list of feature sets from Feast Core
- Returns
List of feature sets
-
property
serving_url
¶ Retrieve Serving Core URL
-
feast.config module¶
-
feast.config.
get_config_property_or_fail
(prop: str, force_config: Dict[str, str] = None) → str[source]¶ Gets a single property in the users configuration
- Parameters
prop – Property to retrieve
force_config – Configuration dictionary containing properties that should be overridden. This will take precedence over file based properties.
- Returns
Returns a string property
feast.constants module¶
feast.entity module¶
-
class
feast.entity.
Entity
(name: str, dtype: feast.value_type.ValueType)[source]¶ Bases:
feast.field.Field
Entity field type
feast.exceptions module¶
feast.feature module¶
-
class
feast.feature.
Feature
(name: str, dtype: feast.value_type.ValueType)[source]¶ Bases:
feast.field.Field
Feature field type
feast.feature_set module¶
-
class
feast.feature_set.
FeatureSet
(name: str, features: List[feast.feature.Feature] = None, entities: List[feast.entity.Entity] = None, source: feast.source.Source = None, max_age: Optional[google.protobuf.duration_pb2.Duration] = None)[source]¶ Bases:
object
Represents a collection of features and associated metadata.
-
add
(resource)[source]¶ Adds a resource (Feature, Entity) to this Feature Set. Does not register the updated Feature Set with Feast Core
- Parameters
resource – A resource can be either a Feature or an Entity object
-
drop
(name: str)[source]¶ Removes a Feature or Entity from a Feature Set. This does not apply any changes to Feast Core until the apply() method is called.
- Parameters
name – Name of Feature or Entity to be removed
-
property
entities
¶ Returns list of entities from this feature set
-
property
features
¶ Returns a list of features from this feature set
-
property
fields
¶ Returns a dict of fields from this feature set
-
classmethod
from_dict
(fs_dict)[source]¶ Creates a feature set from a dict
- Parameters
fs_dict – A dict representation of a feature set
- Returns
Returns a FeatureSet object based on the feature set dict
-
classmethod
from_proto
(feature_set_proto: feast.core.FeatureSet_pb2.FeatureSetSpec)[source]¶ Creates a feature set from a protobuf representation of a feature set
- Parameters
from_proto – A protobuf representation of a feature set
- Returns
Returns a FeatureSet object based on the feature set protobuf
-
classmethod
from_yaml
(yml: str)[source]¶ Creates a feature set from a YAML string body or a file path
- Parameters
yml – Either a file path containing a yaml file or a YAML string
- Returns
Returns a FeatureSet object based on the YAML file
-
get_kafka_source_topic
() → str[source]¶ Get the topic that this feature set has been configured to use as source
-
infer_fields_from_df
(df: pandas.core.frame.DataFrame, entities: Optional[List[feast.entity.Entity]] = None, features: Optional[List[feast.feature.Feature]] = None, replace_existing_features: bool = False, replace_existing_entities: bool = False, discard_unused_fields: bool = False, rows_to_sample: int = 100)[source]¶ Adds fields (Features or Entities) to a feature set based on the schema of a Datatframe. Only Pandas dataframes are supported. All columns are detected as features, so setting at least one entity manually is advised.
- Parameters
df – Pandas dataframe to read schema from
entities – List of entities that will be set manually and not inferred. These will take precedence over any existing entities or entities found in the dataframe.
features – List of features that will be set manually and not inferred. These will take precedence over any existing feature or features found in the dataframe.
replace_existing_features – If true, will replace existing features in this feature set with features found in dataframe. If false, will skip conflicting features.
replace_existing_entities – If true, will replace existing entities in this feature set with features found in dataframe. If false, will skip conflicting entities.
discard_unused_fields – Boolean flag. Setting this to True will discard any existing fields that are not found in the dataset or provided by the user
rows_to_sample – Number of rows to sample to infer types. All rows must have consistent types, even values within list types must be homogeneous
-
is_valid
()[source]¶ Validates the state of a feature set locally :return: (bool, str) True if valid, false if invalid. Contains a message string with a reason
-
property
max_age
¶ Returns the maximum age of this feature set. This is the total maximum amount of staleness that will be allowed during feature retrieval for each specific feature row that is looked up.
-
property
name
¶ Returns the name of this feature set
-
property
source
¶ Returns the source of this feature set
-
to_proto
() → feast.core.FeatureSet_pb2.FeatureSetSpec[source]¶ Converts a feature set object to its protobuf representation
- Returns
FeatureSetSpec protobuf
-
property
version
¶ Returns the version of this feature set
-
feast.field module¶
feast.job module¶
-
class
feast.job.
Job
(job_proto: feast.serving.ServingService_pb2.Job, serving_stub: feast.serving.ServingService_pb2_grpc.ServingServiceStub)[source]¶ Bases:
object
A class representing a job for feature retrieval in Feast.
-
property
id
¶ Getter for the Job Id
-
result
(timeout_sec: int = 21600)[source]¶ Wait until job is done to get an iterable rows of result. The row can only represent an Avro row in Feast 0.3.
- Parameters
timeout_sec – max no of seconds to wait until job is done. If “timeout_sec” is exceeded, an exception will be raised.
Returns: Iterable of Avro rows
-
property
status
¶ Getter for the Job status from Feast Core
-
property
feast.resource module¶
feast.source module¶
-
class
feast.source.
KafkaSource
(brokers: str = '', topic: str = '')[source]¶ Bases:
feast.source.Source
Kafka feature set source type.
-
property
brokers
¶ Returns the list of broker addresses for this Kafka source
-
property
source_type
¶ Returns the type of source. For a Kafka source this will always return “kafka”
-
to_proto
() → feast.core.Source_pb2.Source[source]¶ Converts this Source into its protobuf representation
-
property
topic
¶ Returns the topic for this feature set
-
property
-
class
feast.source.
Source
[source]¶ Bases:
object
Source is the top level class that represents a data source for finding feature data. Source must be extended with specific implementations to be useful
-
classmethod
from_proto
(source_proto: feast.core.Source_pb2.Source)[source]¶ Creates a source from a protobuf representation. This will instantiate and return a specific source type, depending on the protobuf that is passed in.
- Parameters
source_proto – SourceProto python object
- Returns
Source object
-
property
source_type
¶ The type of source. If not implemented, this will return “None”
-
classmethod
feast.type_map module¶
-
feast.type_map.
convert_df_to_feature_rows
(dataframe: pandas.core.frame.DataFrame, feature_set)[source]¶ Returns a function that converts a Pandas Series to a Feast FeatureRow for a given Feature Set and Pandas Dataframe
- Parameters
dataframe – Dataframe that will be converted
feature_set – Feature set used as schema for conversion
- Returns
Function that will do conversion
-
feast.type_map.
convert_dict_to_proto_values
(row: dict, df_datetime_dtype: <property object at 0x7f1d51582d18>, feature_set) → feast.types.FeatureRow_pb2.FeatureRow[source]¶ Encode a dictionary describing a feature row into a FeatureRows object.
- Parameters
row – Dictionary describing a feature row.
df_datetime_dtype – Pandas dtype of datetime column.
feature_set – Feature set describing feature row.
- Returns
FeatureRow
-
feast.type_map.
python_type_to_feast_value_type
(name: str, value, recurse: bool = True) → feast.value_type.ValueType[source]¶ Finds the equivalent Feast Value Type for a Python value. Both native and Pandas types are supported. This function will recursively look for nested types when arrays are detected. All types must be homogenous.
- Parameters
name – Name of the value or field
value – Value that will be inspected
recurse – Whether to recursively look for nested types in arrays
- Returns
Feast Value Type
feast.value_type module¶
-
class
feast.value_type.
ValueType
[source]¶ Bases:
enum.Enum
Feature value type. Used to define data types in Feature Sets.
-
BOOL
= 7¶
-
BOOL_LIST
= 17¶
-
BYTES
= 1¶
-
BYTES_LIST
= 11¶
-
DOUBLE
= 5¶
-
DOUBLE_LIST
= 15¶
-
FLOAT
= 6¶
-
FLOAT_LIST
= 16¶
-
INT32
= 3¶
-
INT32_LIST
= 13¶
-
INT64
= 4¶
-
INT64_LIST
= 14¶
-
STRING
= 2¶
-
STRING_LIST
= 12¶
-
UNKNOWN
= 0¶
-