Notify me of new content

Using GitHub as a Flat Data Store and AWS Lambda to Update it

I spend most of my day, every day, knee deep in code. Optimizing, building, fixing and thinking through workflows can be taxing. This means that the last thing I want to do when I come home is deal with more programming. But I also like learning new things and communicating my experiences so they can help others. I do that through the posts in this website.

Maintaining a web presence without dealing with code means you get to use as many off-the-shelf components as possible. You consider things like WordPress or static site generators that let you concentrate on content, while handling the user interface for you. Write in markdown, build the website, rinse repeat with updates. It’s all very easy, until you need a little more interactivity, like a comments section, or a newsletter signup.

Continue reading

Controlling Python Async Creep

Python added formal asynchronicity in the base language a while ago. It’s fun to play with asyncio tasks and coroutines, the basic constructs that execute almost in parallel. But as you start to integrate more with a regular codebase, you may find that things can get tricky. Especially if you’re forced to interact with synchronous code.

The complication arises when invoking awaitable functions. Doing so requires an async defined code block or coroutine. A non-issue except that if your caller has to be async, then you can’t call it either unless its caller is async. Which then forces its caller into an async block as well, and so on. This is “async creep”.

Continue reading

Making 3D Interfaces for Python with Unity3D

A while back I wrote a post on using with game engines as frontend interfaces. I want to enable rich interfaces in the Python ecosystem that are usable in virtual and augmented reality. Since then, I was able to build some basic concepts on top of Sofi and I’m here to share them.

Using Sofi and WebSockets to command game engines

Sofi works as a WebSocket server that can issue commands and handle events from a client. It’s written to simplify the use of frontend web technologies as interfaces to a Python backend. You can even package it as a desktop app with a desktop look and feel.

It functions by sending the user to a webpage with the basics needed to open a WebSocket client. This client then registers handlers that process server commands telling it how to alter the DOM or respond to events. All the logic resides with Python, the webpage is the user interface.

Continue reading

How to Turn a Web App Into a Desktop App, Using Chromium and PyInstaller

Packaging and distributing your app sounds simple in principle. It’s just software. But in practice, it’s quite challenging.

I’ve been working on a Python module called Sofi that generates user interfaces. It can deliver a desktop feel while using standard single-page web technologies. For flexibility, I designed it to work through two methods of distribution: in-browser and executable.

Running in the browser, it functions much like a normal webpage. You can load it by opening a file, or launch it from your shell. I also built an executable that runs as a packaged app, independent and without external requirements.

Over time, as I hacked at code in Atom — my editor of choice these days — I remembered that Atom is actually a browser. It uses Node.js as a back end, and the Electron framework for its user interface.This inspired me to start poking at Electron’s internals, hoping to find examples and best practices on how they solved desktop packaging.

Continue reading

Threaded Asynchronous Magic and How to Wield It

Ok let’s face it. Clock speeds no longer govern the pace at which computer processors improve. Instead we see increased transistor density and higher core counts. Translating to software terms, this means that code won’t run faster, but more of it can run in parallel.

Although making good use of our new-found silicon real estate requires improvements in software, a lot of programming languages have already started down this path by adding features that help with parallel execution. In fact, they’ve been there for years waiting for us to take advantage.

So why don’t we? A good engineer always has an ear to the ground, listening for the latest trends in his industry, so let’s take a look at what Python is building for us.

Continue reading

The Trusted Packaging Index

A proposal for funding PyPI infrastructure and development

A few days ago, I was listening to the latest episode of Talk Python To Me: Are we failing to fund Python’s core infrastructure?, which had a panel of guests from the Python Software Foundation, PyPI and Read The Docs. As someone that writes open source code, the topic of sustainability is always floating around in my mind. Being able to mostly work on the things that tickle my brain would definitely be awesome, but even if you had a fantastically successful project — which I don’t — it still is extraordinarily difficult to achieve.

I always wondered how organizations like the PSF made it all work, especially with infrastructure and systems that have the level of traffic we see in PyPI. The closest parallel I can draw is to research projects, where a considerable amount of time is dedicated towards finding the right kind of funding.

Continue reading

How To Write Your Own Python Documentation Generator

In my early days with Python, one of the things that I really liked was using the built-in help function to examine classes and methods while sitting at the interpreter, trying to determine what to type next. This function imports an object and walks through its members, pulling out docstrings and generating manpage-like output to help give you an idea of how to use the object it was examining.

The beauty about it being built into the standard library is that with output being generated straight from code, it indirectly emphasizes a coding style for lazy people like me, who want to do as little extra work as possible to maintain documentation. Especially if you already choose straight forward names for your variables and functions. This style involves things like adding docstrings to your functions and classes, as well as properly identifying private and protected members by prefixing them with underscores.

Continue reading

On Democratizing the Next Generation of User Interfaces

Musings of a dev trying to plan for the future

I remember making a program come alive back in the days of the IBM System 23. It was my first endeavor into programming. I must have been 7 years old and already using terminals for data entry to help my dad out in his business. I wrote code that highlighted items on the screen in different shades. For the younger me, it was an achievement to be proud of, even though color was not yet available in these monitors.

Fast forward a few decades — yes, only a few — and we’re in a completely different universe. One where 3D gaming with thousands of people at the same time is possible. A space that collides with realistic physics, represented by lifelike graphics and beautiful scenery.

While a gigantic leap from the olden days of monochrome monitors and ASCII graphics. From a human’s perspective, there’s still a layer of separation between reality and the digital world. But we’re about ready to break through that barrier.

Continue reading

Hacking Together a Simple Graphical Python Debugger

Zero-to-Debugging in 15 mins

You don’t realize the value of a debugger until you’re stuck working on a hard-to-visualize problem. But once you fire up a development environment with decent debugging capabilities, you’ll never look back.

Want to know where you’re at in code execution? What’s taking so long? Just pause it and check.

Wonder what value is assigned to that variable? Mouse over it.

Want to skip a bunch of code and continue running from a different section? Go for it.

Sometimes print(variable_name) is just not enough to give you an idea of what’s going on with your project. This is when a good debugger can help you figuring things out.

Continue reading

A Python Ate My GUI - Part 3: Implementation

Time to Bootstrap your D3, pick up that Python and hop on the Autobahn

If you aren’t aware of my earlier posts, check out Part 1 and Part 2 of this series so that you get some context for this ongoing exercise.

Since I started thinking about and working on these posts, I’ve also been developing the ideas on GitHub as a side project I called sofi.

Sofi is a Python 3 package that serves as the starter implementation of the design discussed in Part 2: a system that will generate the necessary HTML and JavaScript code typically needed to produce a single-page application and serve it up through WebSockets (not an HTTP server).

Continue reading

Engineering the Tests that Matter

As I sit in my standard issued cubicle, going through a typical test results report from the typical external test shop, I begin to shake my head in disgust.

I’m looking at a spreadsheet with endless rows and columns that associate a thumbs up / thumbs down with each one-liner that’s supposed to describe the test that executed.

What does all of this mean? Does that mean the test completed? Aborted? What does a thumbs up mean if a high priority issue was logged against it? Wait! If I read through this issue, the comments trail indicates that the test itself — not the item being tested — was modified to get a passing result! What is all this? Does any of it actually imply any type of quality level in the product that I’m testing?

Continue reading

A Python Ate My GUI - Part 2: Design

In the first part of this series, we discussed a few existing modules that give us tools for building interfaces. I promised to come back with some ideas on how I would attempt to solve the situation, and this post is intended to cover aspects of my initial design.

However, first I need to discuss one more point on existing modules. In case you’re not aware, PyCon 2016 happened a few wks ago in Portland, and amongst the many wonderful talks, keynotes and open spaces, there was one in particular that is relevant to the topic of our discussion here: Russell Keith Magee’s talk on BeeWare and the work he’s done to solve the same problem. I highly recommend you take a look and help out with his projects if you can. It’s definitely a bold and brave solution that’s highly complex, but he already has it at a usable state and deserves massive props for going down that path. The Podcast.__init__ guys also have a great interview with him going over the details.

Continue reading

Using Python to Interact with Docker Containers

“You should really be looking at Vagrant” — he said while I struggled with the keyboard, as if pressing the keys harder was going to magically make it work.

I had recently completed a utility — you know, one of those things that slurp in data from some black hole in a distant corner of the universe, marries it with the structure of a different time-space continuum, and magically spawns a pretty visual representation for mere mortals to easily consume — and I was having the hardest time getting all the pieces together for a demo on a laptop running that OS (yes, that one).

My buddy was trying to point me to a system that helps provision virtual machines from a simple configuration file. A solution which was becoming common in application distribution and deployment, as well as a method to standardize development environments.

Continue reading

A Python Ate My GUI

Thoughts on the future of Python and graphical interfaces

Staring at my coworkers, already knowing the inevitability of the situation, my eyes roll as the argument starts anew:

“I told you I can write that code twice as fast, in half as many lines and they’ll be cleaner and more readable than yours will ever be! Python is awesome!” — said the one guy.

“Whatever you say, you’ll never be able to make a UI that’s half as good as this. It won’t look pretty and no one will want to use it!” — replied the other — “Probably can’t even make it run on Windows” — he mumbled to himself while walking away.

Some years ago this was a regular exchange between coworkers, and while they were mostly messing around, there was still an element of truth to it. Regardless of its capabilities, Python was mostly known as a “scripting” language — not really for graphical interfaces — and the world was still looking for a native OS feel in their GUI applications, which was not really accessible from Python without a lot of work.

Continue reading

Open Source Contributions Are Part of My Professional Development

“Boy did I spend a bunch of time organizing all my issues today” — he said while trying not to sound like a whiner — “You know… I found this neat tool yesterday that easily integrates with our repository management software. It adds project management capabilities that would save us time. Might be interesting to take a look?” — He ended his sentence with a question, something that he absolutely hates doing, a sticking point he’s had since spending a lot of time with a coworker that incessantly did that. He definitely did not want to sound insecure while talking to his manager.

The manager looks up, away from the computer screen in which he was diligently reassigning issues to their newly requested releases, and instead answers with the dreaded: “Is that tool freeware? Did you download it?”. The programmer, a lonely, isolated peon that was simply trying to make his own life easier, was definitely not expecting that response, and so he answered with the unfiltered truth (those of you that often communicate with management know what I’m talking about).

Continue reading

Cristian Medina


Founder of tryexceptpass.org and maintainer of sofi, sofi-unity3d and korv. Cris likes to write about building real world systems, their tradeoffs and the practicality of common software development practices. He also enjoys fiddling with game engines and plugging them into other technologies and programming languages.

Newer posts