v0.3.5 (February 22, 2021)
Bugfixes
Very minor one, fixes to the way
Terminal
accesses thepilot_db.json
file to useTerminal.pilots
property that makes a new pilot_db.json file if one doesn’t exist, but otherwise loads the one that is found inprefs.get('PILOT_DB')
Reorganized
Terminal
source to group properties together & minor additions of type hintingFixed some bad fallback behavior looking for files in old hardcoded default directories, eg. in the ye olde
utils.get_pilotdb()
v0.3.4 (December 13, 2020)
Improvements
Unify the creation of loggers!!!! See the docs ;)
autopilot.core.loggers
: https://github.com/wehr-lab/autopilot/pull/52/commits/d55638f985ab38044fc95ffeff5945021c2e198e https://github.com/wehr-lab/autopilot/issues/38Unify prefs, including sensible defaults, refactoring of scripts into a reasonable format, multiprocess-safety, and just generally a big weight off my mind. Note that this is a breaking change to the way prefs are accessed. Previously one would do prefs.PREF_NAME, but that made it very difficult to provide default values or handle missing prefs. the new syntax is prefs.get(‘PREF_NAME’) which returns defaults with a warning and None if the pref is not set: https://github.com/wehr-lab/autopilot/pull/52/commits/c40a212bcaf5f184f2a6a606027fe15b1b4df59c https://github.com/wehr-lab/autopilot/issues/38
completely clean up scripts, and together that opened the path to clean up setup as well. so all things configuration got a major promotion
We’re on the board with CI and automated testing with a positively massive 3% code coverage!!! https://github.com/wehr-lab/autopilot/pull/52/commits/743bb8fe67a69fcc556fa76e81f72f97f510dff7
new scripts to eg. create autopilot alias: https://github.com/wehr-lab/autopilot/pull/52/commits/211919b05922e18a85d8ef6216973f4000fd32c5
Bugfixes
cleanup scripts on object deletion: https://github.com/wehr-lab/autopilot/pull/52/commits/e8218304bd7ef2e13d2adfc236f3e781abea5f78 https://github.com/wehr-lab/autopilot/issues/41
don’t drop ‘floats’ from gui when we say we can use them…: https://github.com/wehr-lab/autopilot/pull/52/commits/743bb8fe67a69fcc556fa76e81f72f97f510dff7
pigpio scripts dont like floats: https://github.com/wehr-lab/autopilot/pull/52/commits/9f939cd78a5296db3bf318115bee0213bcd1afc0
Docs
Clarification of supported systems: https://github.com/wehr-lab/autopilot/pull/52/commits/ce0ddf78b7f59f5487fec2ca7e8fb3c0ad162051
Solved an ancient sphinx riddle of how to get data objects/constants to pretty-print: https://github.com/wehr-lab/autopilot/pull/52/commits/ec6d5a75dada05688b6bd3c1a53b3d9e5923870f
Clarify hardware prefs https://github.com/wehr-lab/autopilot/pull/52/commits/f3a7609995c84848004891a0f41c7847cb754aae
what numbering system do we use: https://github.com/wehr-lab/autopilot/pull/52/commits/64267249d7b1ec1040b522308cd60f928f2b2ee6
Logging
catch pigpio script init exception: https://github.com/wehr-lab/autopilot/pull/52/commits/3743f8abde7bbd3ed7766bdd75aee52afedf47e2
more of it idk https://github.com/wehr-lab/autopilot/pull/52/commits/b682d088dbad0f206c3630543e96a5a00ceabe25
v0.3.3 (October 25, 2020)
Bugfixes
Fix layout in batch reassign gui widget from python 3 float division
Cleaner close by catching KeyboardInterrupt in networking modules
Fixing audioserver boot options – if ‘AUDIOSERVER’ is set even if ‘AUDIO’ isn’t set in prefs, should still start server. Not full fixed, need to make single plugin handler, single point of enabling/disabling optional services like audio server
Fix conflict between polarity and pull in initializing pulls in pilot
Catch
tables.HDF5ExtError
if local .h5 file corrupt in pilotFor some reason ‘fs’ wasn’t being replaced in the jackd string, reinstated.
Fix comparison in LED_RGB that caused ‘0’ to turn on full becuse ‘value’ was being checked for its truth value (0 is false) rather than checking if value is None.
obj.next()
tonext(obj)`
in jackdserver
Improvements
Better internal handling of pigpiod – you’re now able to import and use hardware modules without needing to explicitly start pigpiod!!
Hopefully better killing of processes on exit, though still should work into unified process manager so don’t need to reimplement everything (eg. as is done with launching pigpiod and jackd)
Environment scripts have been split out into
setup/scripts.py
and you can now run them withpython -m autopilot.setup.run_script
(use--help
to see how!)Informative error when setup is run with too narrow terminal: https://github.com/wehr-lab/autopilot/issues/23
More loggers, but increased need to unify logger creation!!!
Cleanup
remove unused imports in main
__init__.py
that made cyclical imports happen more frequently than necessarysingle-sourcing version number from
__init__.py
more cleanup of unnecessary meta and header stuff left from early days
more debugging flags
filter
NaturalNameWarning
from pytablesquieter cleanups for hardware objects
v0.3.2 (September 28, 2020)
Bugfixes
https://github.com/wehr-lab/autopilot/issues/19 - previously, I attempted to package binaries for the lightly modified pigpio and for jackd (the apt binary used to not work), but after realizing that was the worst possible way of going about it I changed install strategies, but didn’t entirely remove the vestiges of the prior attempt. The installation expected certain directories to exist (in autopilot/external) that didn’t, which crashed and choked install. Still need to formalize a configuration and plugin system, but getting there.
https://github.com/wehr-lab/autopilot/issues/20 - the jackd binary in the apt repos for the raspi used to not work, so i was in the habit of compiling jackd audio from source. I had build that into the install routine, but something about that now causes the JACK-Client python interface to throw segfaults. Somewhere along the line someone fixed the apt repo version of jackd so we use that now.
previously I had only tested in a virtual environment, but now the installation routine properly handles not being in a venv.
Cleanup
remove bulky static files like fonts and css from /docs/ where they were never needed and god knows how they got there
use a forked sphinx-sass when building docs that doesn’t specify a required sphinx version (which breaks sphinx)
removed skbuild requirements from install
fixed pigpio install requirement in requirements_pilot.txt
included various previously missed files in MANIFEST.in
added installation of system libraries to the pilot configuration menu
v0.3.1 (August 4, 2020)
Practice version!!! still figuring out pypi
v0.3.0 (August 4, 2020)
Major Updates
Python 3 - We’ve finally made it to Python 3! Specifically we have brought Autopilot up to compatibility with Python 3.8 – though the Spinnaker SDK is currently only available through Python 3.7, so we have formally required 3.7 for now while we work on moving acquisition to Aravis. I will not attempt to keep Autopilot compatible with Python 2, but no decision has been made about compatibility with other versions of Python 3. Until then, expect that Autopilot will attempt to keep up with major version changes. The switch also let up update PySide (Qt library used for the GUI) to PySide2, which uses Qt5 and has a whole raft of other improvements.
Continuous Data Handling - The
Subject
class andnetworking
modules have been improved to handle continuous data (eg. streaming data, generally non-trialwise or non-event-sampled data). Continuous data can be set in a Task description either with atables
column descriptor as trial data is, but also can be set as'infer'
, for which theSubject
class will wait until it receives the first data and automatically create atables
column depending on its type and shape. While previously we intended to nudge users to be explicit about declaring their data, this was necessary to allow for data that might be variable in type and shape to be included in a Task – eg. it should be possible to record video data without needing to specify the resolution or bit depth as a hardcoded parameter in a task class. I have come to like type inference, and may make it a general practice for all types of data. That would potentially allow tasks to be written without explicitly declaring the data that they produce at all, but I haven’t decided if that’s a good thing or not yet.The GPIO engine has been rebuilt, relying more on
pigpio
’s function interface. This means that GPIO timing is now ~microsecond precise, important for reward delivery, LED flashing, and a number of other basic infrastructural needs. The reorganization of hardware modules resulted in generalGPIO
,Digital_In
andDigital_Out
metaclasses, making common operations like setting polarity, triggers, and pullup/down resistors much easier.Setup has been greatly improved. This includes proper packaging and installation with setuptools & sk-build, allowing us to finally join PyPI :) https://pypi.org/project/auto-pi-lot/ . Setup has been unified into a single npyscreen-based set of prompts that allow the user to run scripts to install libraries or configure their environment (also see
run_script()
andlist_scripts()
), setprefs
, configure hardware objects (based on some very fun signature introspection), setup autopilot as a systemd service, etc. Getting started with Autopilot is now three commands!:pip install auto-pi-lot autopilot.setup.setup_autopilot ~/autopilot/launch_autopilot.sh
Minor Updates
Logging level is now set from
prefs
, so where before, eg. every message through the networking modules would be logged to stdout, now only warnings and exceptions are. This gives a surprisingly large performance boost.Logging has also been much improved in
networking
modules, where rather than an awkwarddo_logging
flag that was used to avoid logging performance-critical events like streaming data, logging is controlled by log level throughout the system. By default, logging of most messages is set atdebug
level so they don’t drown out important messages in the logs as they used to.
Networking modules now only deserialize messages if they are the final recipient, saving lots of processing time – particularly with streamed arrays.
Message
objects also only re-serialize messages if they have been changed. Message structure has been changed such that serialized messages are now of the general format:[sender, (optional) intermediate_node_1, intermediate_node_2, ... final_recipient, message_contents]
Configuration will continue to be a point of improvement, but a few minor updates were made:
prefs.CONFIG
will be used to signal multiple, potentially overlapping agent configurations, each of which may have their own system dependencies, external daemons, etc. Eg. a Pilot could be configured to play audio (which requires a jackd daemon to be started before Autopilot) and video (which requires Autopilot to be started in a X session). Checks ofprefs.CONFIG
are nowin
rather than==
to reflect that.prefs.PINS
was renamedprefs.HARDWARE
, and now allows hardware to be configured with dictionaries rather than integers only. InitiallyPINS
was meant to just contain pin numbering for GPIO objects, but having a single point of hardware configuration is preferable.Task.init_hardware()
now respects all parameters set inprefs
.
Throughout the code, minimal
get_this
type methods have begun to be replaced with@property
attributes. This is because a) I love them and think they are magical, but b) will also be building Autopilot’s closed-loop infrastructure around a Qt-style signal/slot architecture that wraps@property
attributes so they can be.connected
to one another easily.Previously it was possible to control presentation by groups of stimuli, but now it is possible to control the presentation frequency of individual stimuli.
PySide2
has proper support for CSS Stylesheets, so the design of Autopilot’s GUI has been marginally improved, a process that will continue in the ceaseless quest for aesthetic perfection.Several setup routines have been added to make installation of opencv, pyspin, etc. easier. I also wrote a routine to
download_box()
files from a URL, which is mysteriously hard to do.The To-Do page now reflects the full ambition of Autopilot, where before this vision was contained only in the whitepaper and a disorganized plaintext file in the repo.
The
Subject
class can now export trial datato_csv()
. A very minor update, but one that is the first in a number of planned improvements to data export.I have also opened up a message board in google groups to make feature requests and discuss use and development, hope to see you there :)
New Features
TRANSFORMS have been introduced!!!
Transform
objects have aprocess()
method that, well, transforms data in some way. Multiple transforms can be added together to make a transformation chain. This module is still very young and doesn’t have a developed API, but will be built to to automatic type compatibility checking, coersion, parallelization, and rhythm (FIFO/FILO) control. Transforms are implemented with different modalities (image, selection, logical) that imply different types of input and output data structures, but the hierarchical structure of the modules is still quite flat.Autopilot is now integrated with DeepLabCut-live!!!! You can now use realtime pose tracking in your experiments. See the dlclive_example
HARDWARE has been substantially refactored to give objects an appropriate inheritance structure. This substantially reduces effort duplication across hardware objects and makes a bunch of obvious capabilities available to all of them, for example all hardware objects are now network (
init_networking()
) and logging (init_logging()
) capable.Cameras: The
cameras.Camera_CV
class allows webcams/other simple cameras to be accessed through OpenCV, and thecameras.Camera_Spinnaker
class allows FLIR and other cameras to be accessed through the Spinnaker SDK. Cameras are capable of encoding videos locally (with x264), streaming frames over the network, and making acquired frames available to other objects on the same computer. TheCamera_Spinnaker
class provides simple@property
setter/getter methods for common parameters, but also makes allPySpin
attributes available to the user with itsget()
andset()
methods. Thecameras.Camera
metaclass is written so that new camera types can be added by overriding a few methods. A newVideo_Child
can be used to run a camera on a Child agent.9DOF Motion Sensor: The
i2c.I2C_9DOF
class can use the LSM9DS1 sensor to collect accelerometer, magnetometer, and gyroscopic data to compute unambiguous position and orientation information. We will be including calibration and computation routines that make it easier to extract properties of interest – eg. computing vertical motion by combining readings from the three sensors.Temperature Sensor: The
i2c.MLX90640
class can use the MLX90640 sensor to measure temperature. The sensor is 32x24px, which the class caninterpolate()
. The class also allows frames to be integrated and averaged over time, substantially reducing noise. I modified the driver library to enable capture at the full 64fps on the Raspberry Pi.
NETWORKING modules can stream continuous data better in a few ways:
Net_Node
modules were given aget_stream()
method that lets objects, well, stream data. Specifically, they are given aqueue.Queue
to shovel data into, which is then picked up by a dedicatedzmq.Socket
in its own thread, which handles batching, serialization, and load balancing. Streamed messages are batched (ie. contain multiple messages), but behave like normal message when received – they are split and contain aninner_key
that is used to call thelisten
with each message (seel_stream()
).networking
objects also now compress arrays-in-transit with the superfast blosc compression library. This increases their throughput dramatically, as many data streams in neuroscience are relatively low-entropy (eg. the pixels in a video of a mostly-white arena are mostly unchanged frame-to-frame and are thus highly compressible). See theMessage._serialize_numpy()
andMessage._deserialize_numpy()
methods.
STIMULI - The
JackClient
can now play continuous sounds rather than discrete sounds. An example can be found in theNafc_Gap
task, which plays continuous white noise. All sounds now have aplay_continuous()
method, which continually dumps samples in a cycle into a queue for theJackClient
. The continuous sound will be interrupted if another sound has itsJack_Sound.play()
method called, but the continuous sound will resume seamlessly even if number of samples in the played sound aren’t a multiple of the jack buffer size. We use this for gaps in noise (using the newGap
class), which we have confirmed are sample-accurate.UI & VIZ
A
Video
window has been created to display streaming video. TheTerminal_Networking.l_continuous()
method meters frames such that even if high-speed video is being acquired, frames are only sent at a rate ofprefs.DRAWFPS
. TheVideo
class uses theImageItem_TimedUpdate
object, a slight modification ofpyqtgraph.ImageItem
, that calls itsupdate
method according to aPySide2.QtCore.QTimer
.A
plots_menu
menu has been added to the Terminal, and a GUI dialog (gui.Psychometric
) has been added to create simple psychometric curves with theviz.psychometric
module, which uses altair. Plans for developing visualization are described in To-Do.A general
gui.pop_dialog()
function simplifies displaying messages to the user using the Terminal UI. This was an initial step towards improving status/error reporting from other agents, further detailed in To-Do.
Bugfixes
Some objects, particularly several
gui
objects, had the old mouse/mice terminology updated to subject/subjects.Net_Node
objects were only implicitly destroyed by theirrelease
method which ends the threaded loop by setting theclosing
event.Embarassingly,
Pilot
objects were not prevented from running multiple tasks at a time. This led to some very confusing and hard-to-debug problems, as well as frequent conflicts over hardware access and resources. Typically what would happen is the Terminal would send aSTART
message to begin a task, and if it wouldn’t received a message receipt quickly enough would resend it, resulting in two tasks being started – but this would happen whenever twoSTART
messages were sent to a pilot. This was fixed with a simple check ofPilot.state
before a task is initialized. Similar bugs were fixed inPlot
objects.The
Subject
class would sometimes fail to get and increment the trial session. This has been fixed by saving the session number as an attribute in theinfo
node.The
Subject
class would reset the session counter even when the same task was being reassigned (eg. if updated), now it preserves session number if the protocol name is unchanged.The
update_protocols()
method didn’t report which subjects had their protocols updated, and so if there was some exception when setting new protocols it happened silently, making it so a user would never know their task was never updated. This was fixed with a noisier protocol update method for the Subject class and by displaying a list of subjects that were updated after the method is called.Correction trials were being calculated incorrectly by the
Stim_Manager
, such that rather than only repeating a stimulus if the subject got the previous trial incorrect, the stimulus was always repeated at least once.
Code Structure
Modified versions of external libraries have been added as git submodules in autopilot/external.
Requirements files have been split out to better differentiate between different agents and use-cases. eg. requirements for Terminal agents are in
requirements/requirements_terminal.txt
, requirements for build the docs are inrequirements/requirements_docs.txt
, etc. This is a temporary arrangement, as a future design goal is restructuring setup routines so that they can flexibly install components as-needed (see To-Do)autopilot.core.hardware
has been refactored into its own module,autopilot.hardware
, and split by device type, currently…autopilot.cameras
autopilot.gpio
- devices that use the GPIO pins for standard digital I/O logicautopilot.i2c
- devices that use the GPIO pins for I2Cautopilot.usb
The docs are hosted on readthedocs again, so the docs structure has been collapsed to a single folder without built documentation
The autopilot user directory is now
~/autopilot
rather than/usr/autopilot
, which was always a mistake anyway. Autopilot creates a wayfinder~/.autopilot
file that is used to find the user directory if it’s set elsewhere
External Libraries
External libraries can now be built and packaged along with autopilot using cmake, see CMakeLists.txt. Still uh having a little bit of trouble getting this to work, so code is in place to build and package the custom pigpio repo and jack audio but this will likely need some more work.
pigpio https://github.com/sneakers-the-rat/pigpio/
Added the ability to return absolute timestamps rather than system ticks. pigpio typically returns 1 32-bit integer of ticks since the daemon started, absolute timestamps are 64-bit, so the pigpio daemon and python interface (pi) were given two new methods:
synchronize gets several (default 5) sets of paired timestamps and ticks using get_sync_time. It then computes an offset for translating ticks to timestamps
ticks_to_timestamp converts ticks to timestamps based on the offset found with synchronize
get_current_time sends two requests to the daemon to get the seconds and microseconds of the complete timestamp and returns an isoformatted string
mlx90640-library https://github.com/pimoroni/mlx90640-library
Removed building examples by default which require additional dependencies
When using the raspi I2C driver, the baudrate would never be set to 1MHz, which is necessary to achieve full 64fps. This was fixed to use 1MHz by default.
Regressions
Message confirmation (holding a message to resend if confirmation isn’t received) was causing a huge amount of problems and needed to be rethought. There are in general very low rates (near-zero) of messages being dropped without some larger bug causing them, so confirmation has been disabled for now.
The same is true of
heartbeat()
- which polled for status of connected pilots. this will be repaired and restored, as the terminal currently has a pretty bad idea of the status of what’s connected to it. this will be part of a broader networking overhaul