Welcome to my personal blog. Here I share tutorials and updates on my research.
Construct a robust, containerized ETL pipeline using Kestra to orchestrate the ingestion of Greek road traffic data into PostgreSQL. The guide details extracting API data, normalizing JSON with Python/Pandas, and ensuring data integrity through staging tables and SQL merge strategies
Lets explore three strategies for running Python in Kestra (runtime installation, custom Docker images, and custom Worker images) which help us balance speed, isolation, and maintenance.
This guide introduces Kestra, a declarative orchestration platform, explaining how to define Flows using YAML, manage tasks like logging and downloading, and automate execution with Triggers and Cron.
This article breaks down the containerization setup for a Python application which connects to a PostgreSQL database, specifically focusing on the build process (Dockerfile) and the application service configuration (docker-compose.yml).
Bridge the gap between your local Python environment and a Dockerized database using SQLAlchemy, while securing your credentials with environment variables.
Learn how to use Docker Compose by orchestrating a complete architecture by connecting a PostgreSQL database and pgAdmin interface using declarative configuration, internal networking, and Service Discovery.
uvDiscover uv, a high-speed Python package manager, and see how to initialize projects, lock dependencies, and share reproducible environments instantly.
This guide builds on Docker fundamentals by demonstrating how to use Volumes and Bind Mounts to extract and persist data generated by ephemeral containers.
This guide is a practical, Python-centric introduction to Docker that takes you from basic concepts like images and containers to building and optimizing efficient data pipelines.