This is a live document under consistent revision, please check back for updates regularly, or sign up to our mailing list to be notified.

Project

Goals

The primary goal of this project is to develop an easy to use, personalized and continually improving digital assistant which analyzes software usage for repetitive patterns and learns to automate these procedures. The assistant should;

 

  1. Record all data necessary to be able to extract procedural patterns in software operation
  2. Provide contextually aware autocompletion predictions across any software program
  3. Execute predicted autocompletion procedures in any software context
  4. Provide interfaces to analyze performance and submit hints to assist in the discovery of patterns
  5. Operate seamlessly across operating system
  6. Be able to automate and assist without voice commands (silent operation)

Roadmap

The plan is to develop the assistant in stages, incrementally building towards achieving all the goals outlined above. The currently planned releases target the following feature sets;

 

  1. Basic data collection and analysis
  2. Expanded data collection
  3. Real time predictions and inference of operational context using pre-trained deep neural nets
  4. Integrated deep net training pipeline
  5. Procedure triggers and conditional execution
  6. Third party assistant/RPA integrations
  7. HID control autocompletion
  8. Scene based reasoning in 3D and robotic environments

Status

Version 0.1 – Basic data collection and analysis is currently in development. Currently targetting Q2 2019 for limited alpha release.

APIs

Plugin API

The plugin system is used to hook into triggers and state changes, for building RPA style workflows.

Plugins are dynamic link libraries, and the interface definition and source code will be made publically available as a repository once stablized, around release 3-4.

Guides

Adding a new Deep Model

Raw data streams are processed through plugins, or computation graphs. You can replace or install tensorflow models to transform a data stream to extract signals for use in autocompletion procedures.