tryexceptpass

Blog | Podcast | Categories | Tags | Authors
Join Our Newsletter
Episode 3 - Decoupling Database Migrations at Application Startup - Header

Episode 3 - Decoupling Database Migrations at Application Startup

13 mins
2020-02-09 engineering databases docker

Based On

Decoupling database migrations from server startup: why and how
by Itamar Turner-Trauring

  • Data models change and evolve with your application.
  • There’s plenty of tools that keep track of database schemas and automatically generate scripts to upgrade or downgrade them.
  • It’s common for developers to run a migration at the start of their app before running app code.
  • Our author explains two common problems with this approach.
    1. Modern day production deployments and horizontal scaling can get you into a race condition.
    2. You start assuming that new code will only ever run with the new schema.
  • You can decouple migrations from code changes by disabling parallelism during this time.
  • Make it a separate command or lock the database during the upgrade.
  • We can easily implement locking ourselves in any language.
    • Use Redis locks if you’re ok with something external to the DB.
    • Use the DB itself by writing to an extra table to say that you’re upgrading it.
  • Plan your deployment appropriately so you can run old code with new by making migrations additive in the short term.
  • Using a script at startup that optionally performs the migration based on an environment variable integrates wel with Docker and cloud services.
  • Upgrades of both code and data should be part of your testing BEFORE releasing to production.
Continue reading
practicality orm alembic sql sqlalchemy docker
12 Trending Alternatives for Distributing Python Applications in 2020 - Header

12 Trending Alternatives for Distributing Python Applications in 2020

2019-12-10 python distribution docker Cristian Medina

One of the more prevalent topics in the Python ecosystem of 2019 was that of packaging and distribution. As the year comes to an end, I wanted to put together a summary of the many paths we currently have available to distribute apps built with Python. Though some of these also apply to any language.

Whether delivering an executable, a virtual environment, your packaged code, or a full application, the following list includes both standard systems and some up-and-comers to keep in mind as we enter 2020.

Continue reading
python pyinstaller pyoxidizer docker vagrant pex shiv flatpak snapcraft
Easy Clustering with Docker Swarm - Designing CI/CD Systems - Header

Easy Clustering with Docker Swarm - Designing CI/CD Systems

2019-08-23 python engineering docker Cristian Medina
Building code is sometimes as simple as executing a script. But a full-featured build system requires a lot more supporting infrastructure to handle multiple build requests at the same time, manage compute resources, distribute artifacts, etc. After our last chapter discussing build events, this next iteration in the CI/CD design series covers how to spin-up a container inside Docker Swarm to run a build and test it. What is Docker Swarm When running the Docker engine in Swarm mode your effectively creating a cluster. Continue reading
python docker ci cd builds swarm clustering
Effortless Parsing of Build Specifications - Designing CI/CD Systems - Header

Effortless Parsing of Build Specifications - Designing CI/CD Systems

2019-07-15 python engineering api docker Cristian Medina

Every code repository is different. The execution environment, the framework, the deliverables, or even the linters, all need some sort of customization. Creating a flexible build system requires a mechanism that specifies the steps to follow at different stages of a pipeline.

As the next chapter in the Comprehensive CI/CD Pipeline and System Design series, this article examines which instructions you’ll want to convey to your custom system and how to parse them. The focus is around a common solution, adding a file into the repository’s root directory that’s read by your execution engine when receiving new webhooks.

Continue reading
python docker ci cd builds parsing yaml toml github REST
Comprehensive CI/CD System Design - Header

Comprehensive CI/CD System Design

2019-04-24 python engineering docker Cristian Medina

Continuous integration and delivery is finally becoming a common goal for teams of all sizes. After building a couple of these systems at small and medium scales, I wanted to write down ideas, design choices and lessons learned. This post is the first in a series that explores the design of a custom build system created around common development workflows, using off-the-shelf components where possible. You’ll get an understanding of the basic components, how they interact, and maybe an open source project with example code from which to start your own.

Continue reading
python docker ci cd builds design git
Using Python to Interact with Docker Containers - Header

Using Python to Interact with Docker Containers

2016-05-31 python docker Cristian Medina

“You should really be looking at Vagrant” — he said while I struggled with the keyboard, as if pressing the keys harder was going to magically make it work.

I had recently completed a utility — you know, one of those things that slurp in data from some black hole in a distant corner of the universe, marries it with the structure of a different time-space continuum, and magically spawns a pretty visual representation for mere mortals to easily consume — and I was having the hardest time getting all the pieces together for a demo on a laptop running that OS (yes, that one).

My buddy was trying to point me to a system that helps provision virtual machines from a simple configuration file. A solution which was becoming common in application distribution and deployment, as well as a method to standardize development environments.

Continue reading
docker python
© Copyright 2020 - tryexceptpass, llc