components package
Submodules
components.config module
- class components.config.Config(*, domain: str, reports_csv_path: str, reports_list: list[dict[str, Any]] = [], summary_filepath: str | None = None, log_path: str | None = None, report: str = '', path: str = '', threads: int = 0, stdout_loglevel: str = 'WARNNING', file_loglevel: str = 'INFO', verbose=False)[source]
Bases:
object
Concrete class representing Config object. Contains entire configuration required for a program.
- class components.config.ConfigProtocol(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for config object.
- Parameters:
reports_list_path (str) – CLI argument for input report list path.
report (str) – CLI argument for single report params.
path (str) – CLI argument for save location path override.
threads (int) – CLI argument for number of threads to use.
- domain: str
- file_loglevel: str
- log_filepath: str
- path: str
- report: str
- reports_csv_path: str
- reports_list: list[dict[str, Any]]
- stdout_loglevel: str
- summary_filepath: str
- threads: int
- verbose: bool
components.connectors module
- class components.connectors.Connector(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for connector object.
- Parameters:
queue (Queue) – Shared queue object.
timeout (int) – Request’s timeout value in seconds.
headers (dict[str, str]) – Headers required to establish the connection.
- check_connection() bool [source]
Checks connection with given domain.
- Returns:
Flag, True if connection is established, False otherwise.
- Return type:
bool
- headers: dict[str, str]
- queue: Queue
- async report_gathering(reports: list[components.containers.ReportProtocol], session: ClientSession) None [source]
Collects asynchronous responses from the servers.
- Parameters:
reports (list[ReportProt]) – Collection of ReportProtocol objects.
session (aiohttp.ClientSession) – HTTP client session object to handle request in transaction.
- timeout: int
- class components.connectors.SfdcConnector(queue: Queue, *, domain: str, verbose: bool = False, timeout: int = 900, headers: dict[str, str] = {'Content-Type': 'application/csv', 'X-PrettyPrint': '1'})[source]
Bases:
object
Concrete class representing Connector object for SFDC
- Parameters:
queue (Queue) – Shared queue object.
verbose – CLI parameter used as switch between progress bar and logging to stdout on INFO level. Defaults to False.
timeout (int) – Request’s timeout value in seconds. Defaults to 900.
headers (dict[str, str]) – Headers required to establish the connection. Defaults to {‘Content-Type’: ‘application/csv’, ‘X-PrettyPrint’: ‘1’}.
export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.
- check_connection() bool [source]
Checks the connection with given domain.
- Returns:
Flag, True if connection was successful, False wasn’t.
- Return type:
bool
- async handle_requests(reports: list[components.containers.ReportProtocol]) None [source]
Creates session and process asynchronous tasks.
- Parameters:
reports (list[ReportProtocol]) – Collection of ReportProtocol instances.
components.containers module
- class components.containers.ReportProtocol(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for report object.
- Parameters:
name (str) – Report name, propagated to report file name
id (str) – Report id, identification number of the report in SFDC
path (PathLike) – Report path, save location for the report in form of Path object
type (str) – Report type, allowed options [‘SFDC’], type drives connector and report objects selection
export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.
downloaded (bool) – Flag indicating whether the reports has been succesfully downloaded or not
valid (bool) – Flag indicating whether the response has been succesfully retrieved or not
created_date (datetime) – Report save completition date
pull_date (timedelta) – Report response completition date
processing_time – The time it took to process the report in seconds
attempt_count (int) – Number of attempts to process the report
size (float) – Size of saved report file in Mb
response (str) – Container for request response
content (DataFrame) – Pandas DataFrame based on response
- attempt_count: int
- content: DataFrame
- created_date: datetime
- downloaded: bool
- export_params: str
- id: str
- name: str
- path: PathLike
- processing_time: timedelta
- pull_date: datetime
- response: str
- size: float
- type: str
- valid: bool
- class components.containers.ReportsContainer(reports_params_list: list[dict[str, Any]], summary_path: PathLike | None)[source]
Bases:
object
Concrete class representing ReportContainer object.
- create_reports() list[components.containers.ReportProtocol] [source]
Orchestrating method to handle report objects factory
- Returns:
Collection of Reports
- Return type:
list[ReportProtocol]
- class components.containers.ReportsContainerProtocol(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for report container object.
- Parameters:
report_params_list (list[dict[str, Any]]) – Collection of dicts with parameters for object crafting.
summary_report_path (PathLike) – Path to save location of summary report.
- create_reports() list[components.containers.ReportProtocol] [source]
Orchestrating method to handle report objects factory
- Returns:
Collection of Reports
- Return type:
list[ReportProtocol]
- class components.containers.SfdcReport(name: str, id: str, path: ~os.PathLike, type: str = 'SFDC', export_params: str = '?export=csv&enc=UTF-8&isdtp=p1', downloaded: bool = False, valid: bool = False, created_date: ~datetime.datetime = datetime.datetime(2023, 4, 4, 0, 1, 37, 61391), pull_date: ~datetime.datetime = datetime.datetime(2023, 4, 4, 0, 1, 37, 61391), processing_time: ~datetime.timedelta = datetime.timedelta(0), attempt_count: int = 0, size: float = 0.0, response: str = '', content: ~pandas.core.frame.DataFrame = <factory>)[source]
Bases:
object
Concrete class representing Report object from SFDC.
- Parameters:
name (str) – Report name, propagated to report file name.
id (str) – Report id, identification number of the report in SFDC.
path (PathLike) – Report path, save location for the report in form of Path object.
type (str) – Report type, allowed options [‘SFDC’], type drives connector and report objects selection. Defaults to ‘SFDC’.
export_params (str) – Default parameters required by SFDC. Defaults to ‘?export=csv&enc=UTF-8&isdtp=p1’.
downloaded (bool) – Flag indicating whether the reports has been succesfully downloaded or not. Defaults to False.
valid (bool) – Flag indicating whether the response has been succesfully retrieved or not. Defaults to False.
created_date (datetime) – Report save completition date. Defaults to current datetime.
pull_date (timedelta) – Report response completition date. Defaults to current datetime.
processing_time – The time it took to process the report in seconds. Defaults to 0 microseconds.
attempt_count (int) – Number of attempts to process the report. Defaults to 0 .
size (float) – Size of saved report file in Mb. Defaults to 0.0 .
response (str) – Container for request response. Defaults to empty string.
content (DataFrame) – Pandas DataFrame based on response. Defaults to empty Pandas DataFrame.
- attempt_count: int
- content: DataFrame
- created_date: datetime
- downloaded: bool
- export_params: str
- id: str
- name: str
- path: PathLike
- processing_time: timedelta
- pull_date: datetime
- response: str
- size: float
- type: str
- valid: bool
components.exceptions module
components.handlers module
- class components.handlers.Worker(queue: Queue)[source]
Bases:
Thread
Concrete class representing Worker object.
- process_report(report: ReportProtocol) None [source]
Orchiestrates entire process of downloading the report.
- Parameters:
report (ReportProtocol) – Instance of the ReportProtocol object.
- class components.handlers.WorkerFactory(queue: Queue, *, threads: int = 1)[source]
Bases:
object
Concrete class representing WorkerFactory object.
- class components.handlers.WorkerFactoryProtocol(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for worker factory objects.
- Parameters:
queue (Queue) – Shared, thread-safe queue.
threads (int) – Number of threads, equal to number of Workers to be deployed.
- static active_workers() int [source]
Counts active works in current time.
- Returns:
Number of active workers.
- Return type:
int
- queue: Queue
- threads: int
- class components.handlers.WorkerProtocol(*args, **kwargs)[source]
Bases:
Protocol
Protocol class for worker factory objects.
- Parameters:
queue (Queue) – Shared, thread-safe queue.
- report_processing(report: ReportProtocol) None [source]
Orchiestrates the report processing.
- Parameters:
report (ReportProtocol) – Instance of the ReportProtocol object.
components.loggers module
requests.adapters
This module contains the transport adapters that Requests uses to define and maintain connections.
- components.loggers.logger_configurer(*, stdout_loglevel: str, file_loglevel: str, log_path: PathLike | None, verbose: bool) None [source]
Configures logger settings for file and stdout handlers.
- Parameters:
cli_stdout_loglevel (str) – LogLevel for stdout logger handler based on CLI option. Defaults to ERROR.
cli_file_loglevel – LogLevel for file logger handler based on CLI option. Defaults to INFO.
verbose (str) – Flag toggling LogLevel for stdout logger handler, if True sets to ERROR, else INFO.