know.base

Base objects for know

The main object of this module is SlabsIter, and object to source and manipulate multiple streams.

A slab is a collection of items of a same interval of time. We represent a slab using a dict or mapping. Typically, a slab will be the aggregation of multiple information streams that happened around the same time.

SlabsIter is a tool that allows you to source multiple streams into a stream of slabs that can contain the original data, or other datas computed from it, or both.

Note to developers looking into the code: The overview of the SlabsIter class:

>>> class SlabsIter:
...     def _call_on_scope(self, scope):
...         '''Calls the components 1 by 1, sourcing inputs and writing outputs in scope'''
...
...     def __next__(self):
...         '''Get the next slab by calling _call_on_scope on an new empty scope.
...         At least one of the components will have to be argument-less and provide
...         some data for other components to get their inputs from, if any are needed.
...         '''
...         return self._call_on_scope(scope={})
...
...     def __iter__(self):
...         '''Iterates over slabs until a handle exception is raised.'''
...         # Simplified code:
...         with self:  # enter all the contexts that need to be entered
...             while True:  # loop until you encounter a handled exception
...                 try:
...                     yield next(self)
...                 except self.handle_exceptions as exc_val:
...                     # use specific exceptions to signal that iteration should stop
...                     break
exception know.base.ExceptionalException[source]

Raised when an exception was supposed to be handled, but no matching handler was found.

See the _handle_exception function, where it is raised.

exception know.base.IteratorExit[source]

Raised when an iterator should quit being iterated on, signaling this event any process that cares to catch the signal. We chose to inherit directly from BaseException instead of Exception for the same reason that GeneratorExit does: Because it’s not technically an error.

See: https://docs.python.org/3/library/exceptions.html#GeneratorExit

class know.base.SlabsIter(handle_exceptions=(<class 'StopIteration'>, <class 'know.base.IteratorExit'>, <class 'KeyboardInterrupt'>), **components)[source]

Object to source and manipulate multiple streams.

A slab is a collection of items of a same interval of time. We represent a slab using a dict or mapping. Typically, a slab will be the aggregation of multiple information streams that happened around the same time.

For example, say and edge device had a microphone, light, and movement sensor. An aggregate reading of these sensors could give you something like:

>>> slab = {'audio': [1, 2, 4], 'light': 126, 'movement': None}

movement is None because the sensor is off. If it were on, we’d have True or False as values.

From this information, you’d like to compute a turn_mov_on value based on the formula

>>> from statistics import stdev
>>> vol = stdev
>>> should_turn_movement_sensor_on = lambda audio, light: vol(audio) * light > 50000

The produce of the volume and the lumens gives you 192, so you now have…

>>> slab = {'audio': [1, 2, 4], 'light': 126, 'turn_mov_on': False, 'movement': None}

The next slab that comes in is

>>> slab = {'audio': [-96, 89, -92], 'light': 501, 'movement': None}

which puts us over the threshold so

>>> slab = {
...     'audio': [-96, 89, -92], 'light': 501, 'turn_mov_on': True, 'movement': None
... }

and the movement sensor is turned on, the movement is detected, a human_presence signal is computed, and a notification sent if that metric is above a given theshold.

The point here is that we incrementally compute various fields, enhancing our slab of information, and we do so iteratively over over slab that is streaming to us from our smart home device.

SlabsIter is there to help you create such slabs, from source to enhanced.

The situation above would look something along like this:

>>> from know.base import SlabsIter
>>> from statistics import stdev
>>>
>>> vol = stdev
>>>
>>> # Making a slabs iter object
>>> def make_a_slabs_iter():
...
...     # Mocking the sensor readers
...     audio_sensor_read = iter([[1, 2, 3], [-96, 87, -92], [320, -96, 99]]).__next__
...     light_sensor_read = iter([126, 501, 523]).__next__
...     movement_sensor_read = iter([None, None, True]).__next__
...
...     return SlabsIter(
...         # The first three components get data from the sensors.
...         # The *_read objects are all callable, returning the next
...         # chunk of data for that sensor, if any.
...         audio=audio_sensor_read,
...         light=light_sensor_read,
...         movement=movement_sensor_read,
...         # The next
...         should_turn_movement_sensor_on = lambda audio, light: vol(audio) * light > 50000,
...         human_presence_score = lambda audio, light, movement: movement and sum([vol(audio), light]),
...         should_notify = lambda human_presence_score: human_presence_score and human_presence_score > 700,
...         notify = lambda should_notify: print('someone is there') if should_notify else None
...     )
...
>>>
>>> si = make_a_slabs_iter()
>>> next(si)  
{'audio': [1, 2, 3],
 'light': 126,
 'movement': None,
 'should_turn_movement_sensor_on': False,
 'human_presence_score': None,
 'should_notify': None,
 'notify': None}
>>> next(si)  
{'audio': [-96, 87, -92],
 'light': 501,
 'movement': None,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': None,
 'should_notify': None,
 'notify': None}
>>> next(si)  
someone is there
{'audio': [320, -96, 99],
 'light': 523,
 'movement': True,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': 731.1353726143957,
 'should_notify': True,
 'notify': None}

If you ask for the next slab, you’ll get a StopIteration (raised by the mocked sources since they reached the end of their iterators).

>>> next(si)  
Traceback (most recent call last):
  ...
StopIteration

That said, if you iterate through a SlabsIter that handles the StopIteration exception (it does by default), you’ll reach the end of you iteration gracefully.

>>> si = make_a_slabs_iter()
>>> for slab in si:
...     pass
someone is there
>>> si = make_a_slabs_iter()
>>> slabs = list(si)  # gather all the slabs
someone is there
>>> len(slabs)
3
>>> slabs[-1]  
{'audio': [320, -96, 99],
 'light': 523,
 'movement': True,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': 731.1353726143957,
 'should_notify': True,
 'notify': None}