Common Utils
Generic utility functions that are used in multiple places in the library that for now don’t have a clear other place to be
Functions:
|
List all classes within a module/package without importing by parsing the syntax tree directly with |
|
Given a full package.module.ClassName string, return the relevant class |
|
Given some class, find its subclasses recursively |
|
Given a dictionary of a pilot_db, return the subjects that are in it. |
|
Try to load the file_db |
|
Coerce a discrete/string column of a pandas dataframe into numeric values |
|
Find all instances of a key in a dictionary, recursively. |
|
Find an entry in a list of dictionaries where dict[key] == value. |
|
Recursively yield key/value pairs, returning keys as tuples corresponding to the recursive keys in the dict |
|
Flatten a nested dictionary to a dictionary with tuples of the nested keys |
Classes:
|
Thread whose .join() method returns the value from the function thx to https://stackoverflow.com/a/6894023 |
|
Allow json serialization of objects containing numpy arrays. |
|
Allow json deserialization of objects containing numpy arrays. |
- list_classes(module) List[Tuple[str, str]] [source]
List all classes within a module/package without importing by parsing the syntax tree directly with
ast
.- Parameters
module (module, str) – either the imported module to be queried, or its name as a string. if passed a string, attempt to import with
importlib.import_module()
- Returns
list of tuples [(‘ClassName’, ‘module1.module2.ClassName’)] a la
inspect.getmembers()
- find_class(cls_str: str)[source]
Given a full package.module.ClassName string, return the relevant class
- Parameters
cls_str (str) – a full package.module.ClassName string, like
'autopilot.hardware.Hardware'
- Returns
the class indicated by
cls_str
- recurse_subclasses(cls, leaves_only=False) list [source]
Given some class, find its subclasses recursively
See: https://stackoverflow.com/a/17246726/13113166
- Parameters
leaves_only (bool) – If True, only include classes that have no further subclasses, if False (default), return all subclasses.
- Returns
list of subclasses
- class ReturnThread(group=None, target=None, name=None, args=(), kwargs={}, Verbose=None)[source]
Bases:
threading.Thread
Thread whose .join() method returns the value from the function thx to https://stackoverflow.com/a/6894023
This constructor should always be called with keyword arguments. Arguments are:
group should be None; reserved for future extension when a ThreadGroup class is implemented.
target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.
name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.
args is the argument tuple for the target invocation. Defaults to ().
kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.
If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
Methods:
run
()Method representing the thread's activity.
join
([timeout])Wait until the thread terminates.
- run()[source]
Method representing the thread’s activity.
You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.
- join(timeout=None)[source]
Wait until the thread terminates.
This blocks the calling thread until the thread whose join() method is called terminates – either normally or through an unhandled exception or until the optional timeout occurs.
When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call is_alive() after join() to decide whether a timeout happened – if the thread is still alive, the join() call timed out.
When the timeout argument is not present or None, the operation will block until the thread terminates.
A thread can be join()ed many times.
join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.
- list_subjects(pilot_db=None)[source]
Given a dictionary of a pilot_db, return the subjects that are in it.
- Parameters
pilot_db (dict) – a pilot_db. if None tried to load pilot_db with :method:`.load_pilotdb`
- Returns
a list of currently active subjects
- Return type
subjects (list)
- load_pilotdb(file_name=None, reverse=False)[source]
Try to load the file_db
- Parameters
reverse
file_name
Returns:
- coerce_discrete(df, col, mapping={'L': 0, 'R': 1})[source]
Coerce a discrete/string column of a pandas dataframe into numeric values
Default is to map ‘L’ to 0 and ‘R’ to 1 as in the case of Left/Right 2AFC tasks
- Parameters
df (
pandas.DataFrame
) – dataframe with the column to transformcol (str) – name of column
mapping (dict) – mapping of strings to numbers
- Returns
transformed dataframe
- Return type
df (
pandas.DataFrame
)
- find_key_recursive(key, dictionary)[source]
Find all instances of a key in a dictionary, recursively.
- Parameters
key
dictionary
- Returns
list
- find_key_value(dicts: List[dict], key: str, value: str, single=True)[source]
Find an entry in a list of dictionaries where dict[key] == value.
- Parameters
dicts ()
key ()
value ()
single (bool) – if
True
(default), raise an exception if multiple results are matched
- walk_dicts(adict, keys: Optional[List] = None) tuple [source]
Recursively yield key/value pairs, returning keys as tuples corresponding to the recursive keys in the dict
- Parameters
adict (dict) – dict to walk over
- Yields
tuple of key value pairs
- flatten_dict(nested: dict, keys=(), skip=()) dict [source]
Flatten a nested dictionary to a dictionary with tuples of the nested keys
Similar to
walk_dicts()
, excepts not a generator, and returns a flattened dictionary rather than a series of tuples.Examples
nested_dict = { 'a': 1, 'b': { 'c': 2, 'd': { 'e': 3 }, 'f': 4 } } flatten_dict(nested_dict) { ('a',): 1, ('b', 'c'): 2, ('b', 'd', 'e'): 3, ('b', 'f'): 4 }
- Parameters
nested (dict) – A nested dictionary
keys (tuple) – A tuple of keys used in the recursive function to create the returned key
skip (tuple[str]) – Tuple of keys to skip flattening
- Returns
A flattened dictionary
- Return type
- class NumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]
Bases:
json.encoder.JSONEncoder
Allow json serialization of objects containing numpy arrays.
Use like
json.dump(obj, fp, cls=NumpyEncoder)
Deserialize with
NumpyDecoder
References
https://github.com/mpld3/mpld3/issues/434#issuecomment-340255689
https://gist.github.com/massgh/297a73f2dba017ffd28dbc34b9a40e90
Constructor for JSONEncoder, with sensible defaults.
If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.
If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.
If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.
If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.
If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.
If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.
If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is
None
and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a
TypeError
.Methods:
default
(obj)Implement this method in a subclass such that it returns a serializable object for
o
, or calls the base implementation (to raise aTypeError
).- default(obj)[source]
Implement this method in a subclass such that it returns a serializable object for
o
, or calls the base implementation (to raise aTypeError
).For example, to support arbitrary iterators, you could implement default like this:
def default(self, o): try: iterable = iter(o) except TypeError: pass else: return list(iterable) # Let the base class default method raise the TypeError return JSONEncoder.default(self, o)
- class NumpyDecoder(*args, **kwargs)[source]
Bases:
json.decoder.JSONDecoder
Allow json deserialization of objects containing numpy arrays.
Use like
json.load(fp, cls=NumpyDecoder)
Serialize with
NumpyEncoder
References
https://github.com/mpld3/mpld3/issues/434#issuecomment-340255689
https://gist.github.com/massgh/297a73f2dba017ffd28dbc34b9a40e90
object_hook
, if specified, will be called with the result of every JSON object decoded and its return value will be used in place of the givendict
. This can be used to provide custom deserializations (e.g. to support JSON-RPC class hinting).object_pairs_hook
, if specified will be called with the result of every JSON object decoded with an ordered list of pairs. The return value ofobject_pairs_hook
will be used instead of thedict
. This feature can be used to implement custom decoders. Ifobject_hook
is also defined, theobject_pairs_hook
takes priority.parse_float
, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).parse_int
, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).parse_constant
, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN. This can be used to raise an exception if invalid JSON numbers are encountered.If
strict
is false (true is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including'\t'
(tab),'\n'
,'\r'
and'\0'
.Methods:
object_hook
(obj)