This is a retelling of a presentation I gave at work. In it, I describe a mechanism I've started using to raise the quality of artifacts I check into version control.
I've written before about how I use MediaWiki for taking notes and as one of my study tools. This has worked well for many years. But a problem started to develop: while I wrote my technical notes in MediaWiki, I wrote my day-to-day notes (books I want to read, notes from podcasts I listen to, and even my weekly planner) in Notion. This meant I had to use different apps for reading/writing in each tool, remember two different markup languages, and couldn't (cleanly) link pieces of content between the two. The final straw was realizing how much more effort I had to expend to maintain my MediaWiki instance; I just didn't have the time or will to keep up with new releases not to mention maintain the server itself.
For these reasons, I decided to move all of my MediaWiki content to Notion and unify all of my notes. But this revealed a new problem: there was no tooling to automate this. So I created my own. Here's how it works.
The AWS Cloud Development Kit (CDK) is an "open source software development framework to define your cloud application resources using familiar programming languages". When CDK launched in 2019, I remember reading the announcement and thinking, "Ok, AWS wants their own Terraform-esque tool. No surprise given how popular Terraform is." Months later, my friend and colleague Matt M. was telling me how he was using CDK in a project he was working on and how crazy cool it was.
I finally decided to give CDK a go for one of my projects. Here is what I discovered.
I spent a long time creating my first Spark bot, Zpark. The first commit was in August and the first release was posted in January. So, six months elapsed time. It's also over-engineered. I mean, all it does is post messages back and forth between a back-end system and some Spark spaces and I ended up with something so complex that I had to draw a damn block diagram in the user guide to give people a fighting chance at comprehending how it works.
Its internals could've been much simpler. But that was part of the point of creating the bot: examining the proper architecture for a scalable application, learning about new technologies for building my own API, learning about message brokers, pulling my hair out over git's eccentricities and ultimately, having enough material to write this blog post.
In this post I'm going to break down the different functional components of Zpark, discuss what each does, and why-or not-that component is necessary. If I can achieve one goal, it will be to retire to a tropical island ASAP. If I can achieve a second goal, it will be to give aspiring bot creaters (like yourself, presumably) a strong mental model of a Spark bot to aid their development.
For a long while now I've been brainstorming how I could leverage the API that's present in the Cisco Spark collaboration platform to create a bot. There are lots of goofy and fun examples of bots (ie, Gifbot) that I might be able to draw inspiration from, but I wanted to create something that would provide high value to myself and anyone else that choose to download and use it. The idea finally hit me after I started using Zabbix for system monitoring. Since Zabbix also has a feature-rich API, all the pieces were in place to create a bot that would act as a bit of middle-ware between Zabbix and Spark. I call the bot: Zpark.
For the benefit of readers who haven't worked with Flask or don't know what Flask is, it's a so-called microframework for writing web-based applications in Python. Basically, the framework takes care of all the obvious tasks that are needed to run a web app. Things like talking HTTP to clients, routing incoming requests to the appropriate handler in the app, and formatting output to send back to the client in response to their request. When you use a framework like this, you as the developer can concentrate a lot more on the application logic and worry a lot less about hooking the app into the web.
As you may have guessed from the title of this post, one of the other tasks that Flask manages is logging within the application. For example, if you want to emit a log message when a user logs in or when they upload a new photo, Flask provides a simple interface to generate those log messages.
Flask has a large community of active users built around it and as a result, there's tons of best practice information out there on scaling, talking to a database, and even whole tutorials on how to build full applications. But it's been my experience that there is very little information devoted to the topic of logging. Granted logging isn't very fun and does nothing to get your app to market quicker or increase user counts, but it is really important if you want to get any sort of feedback from the app about what it's doing during normal operation or what exactly happened if something goes wrong.
In this post, I'll look very briefly at how logging works in Flask and then examine three common methods used to record log files within Flask and how each of them can shoot you in the foot. I'll close by offering some recommendations that should keep you on safe ground.
At Cisco's GSX conference at the start of FY17, the DevNet team made a programming scavenger hunt by posting daily challenges that required using things like containers, Cisco Shipped, Python, and RESTful APIs in Cisco software in order to solve puzzles. In order to submit an answer, the team created an API that contestants had to use (in effect creating another challenge that contestants had to solve).
This post contains the artifacts I created while solving some of the challenges.