Creating Reports From Trello Boards

Where I work, we use Trello to help track tasks between globally distributed team members but it has no reporting features. I needed a quick way to create reports and wanted to use tools I am familiar with. I chose to use Python, the official Trello Python package and Jinja2. The code for this quick report hack is on GitHub.

board_report.py --board_name my-trello-board --config ~/.mytrello.conf --template ~/test-report.txt -o /tmp/my-board-report.txt

Alternatively, if you don’t store your secrets in a conf file you can also pass them as arguments to the script.

board_report.py --board_name my-trello-board --api_key 23432423hj32g43 --access_token 89238432842394 --template ~/test-report.txt -o /tmp/my-board-report.txt

As a quick example, say you want to create a quick text format report with the cards found in the open lists on your board. You might use a simple Jinja2 template that looks like this:
The resulting report will look like this: 

Because you are querying for the entire board in this example, you can access each of the properties on the cards and create a much more detailed report if needed. If you wanted to change the query to multiple boards or a subset of one board, you could change the following lines of code found in the board_report.py file[1]

[1] https://github.com/rbrady/trello-extras/blob/master/trello_extras/board_report.py#L91-L99

Improving Support For Sensitive Data in Mistral

At the recent OpenStack Project Teams Gathering in Atlanta, GA, one of the topics discussed by the Mistral developer community attendees is the security issue where data that might be sensitive could be exposed in Mistral log files, stored in the database or when transferred over the network.

This was one of my favorite sessions as every attendee in the room participated in the discussion and provided a valuable insight or opinion and the proposed idea for a solution was truly a collaborative effort. The idea agreed upon included adding a section to workflows called “secret” where a developer can identify data of a sensitive nature. All of the items in the secret section will be protected whenever they are persisted either to log files or the database. Actions can be executed independently from workflows, so a decorator will also be added to mistral-lib to provide a means for custom action developers to identify constructor arguments to be protected.

Proposed Workflows Syntax

Proposed Custom Actions Syntax

It is great to see so many developers in the Mistral community concerned about security.

Mistral Custom Actions API

The Mistral community has been discussing improving the developer experience for creating custom actions for nearly a year.  As my first experience with Mistral has been as a consumer via TripleO, the custom actions experience has been important to me and I’ve been fortunate enough to be a part of the planning and implementation.  Mistral allows developers to implement new actions and plug them into Mistral via a mechanism based on Stevedore.  The ability to create custom actions is important as some things are easier to implementor more easily readable in Python instead of YAML/YAQL.  Developers wanting to integrate with Mistral may already have code they want to reuse or integrate with other products and systems that Mistral doesn’t provide support for directly.

According to the spec, one of the primary motivations for the new custom actions effort is to make it clear what parts of Mistral are publicly callable.  This was originally a source of confusion when creating custom actions for TripleO.  We decided to use a utility class for getting information from Keystone and it wasn’t initially clear in the review process on whether or not we were accessing private internals or not.  unclear on what parts of the Mistral code base you can or should be importing into your own actions and what can be used without fear of change.  This was a good indicator that Mistral needs to give developers an explicit public interface.

As the discussion has continued, the efforts to improve custom action development have also spawned additional ideas that will likely have a positive impact on the overall Mistral project ecosystem.  At the recent OpenStack Project Teams Gathering in Atlanta, GA, the attendees from the Mistral developer community decided to move forward on the following changes:

All of the common code between the Mistral ecosystem of projects and general 3rd party integration code to include custom actions API and custom yaql functions API will be contained in the mistral-lib project.

All of the OpenStack specific integration code and examples will reside in the mistral-extra project.

There is already a patch up for review, but it will likely see some additional changes to land and I’ll write a more detailed example of creating custom actions once it lands.

 

Debugging a Custom Mistral Action

When I first started creating custom Mistral actions for TripleO, I quickly found the need to debug them interactively.  I wasn’t satisfied with dumping out information to logs and searching through them.  I started with creating scripts that used the same code so I could run them directly, but I soon found that approach inadequate.  I needed to be able to debug an action that used something internal to Mistral (e.g. mistral.context) or when an action is passing a unit test but fails within a workflow execution.   I found myself wanting a solution where you could create a Mistral execution and on any error have it open a pdb session in the shell.

To solve this problem, a coworker suggested remote debugging with pdb.  A quick search led to the Remote PDB package.  It turned out to be quick and simple to use and it worked immediately without a ton of effort.  To use remote-pdb on the undercloud, you need to perform the following steps:

# https://pypi.python.org/pypi/remote-pdb
pip install remote-pdb

# open port for remote-pdb access on undercloud
sudo iptables -A INPUT -p tcp -m multiport --dports 4445 -m comment --comment "remote pdb" -m state --state NEW -j ACCEPT

# add two lines to any action you want to debug
from remote_pdb import RemotePdb
RemotePdb('127.0.0.1', 4445).set_trace()

# run action or create exececution
mistral run-action tripleo.get_capabilities '{"container": "overcloud"}'

# open netcat in another terminal on undercloud
nc -C 127.0.0.1 4445

Reboot, Refresh, Renew

“Think for yourself and let others enjoy the privilege of doing so too.” – Voltaire

 

The last couple of years have been part of a journey that took me away from public writing. My choice of employment at the time came with many constraints that makes sharing your ideas and experiences difficult. My blog had become stale and virtually abandoned. Most of my previous writing was either quickly written or vague. I tried to force myself to write when I really didn’t have the time or wasn’t allowed to write to any level of detail about my projects.

 

Life has changed for me. I recently started working for Red Hat and their company culture based on collaboration, sharing, and open communication is a sharp contrast to the last four years of my career. I love working with a bunch of talented people who are very passionate about what they do.  My work is now in the community; it’s open and available for anyone to see.  It will be exciting to see what unfolds for me in this new phase of my career.

 

I have hopes to write more and to share a voice.  Hopefully the choices of my past haven’t formed bad habits in procrastination too much (I am a developer though).