Joong Hoon Shin
EN KO

CLOUD / INFRA ENGINEER

I design and run services
that keep running.

I built and operated a hybrid AWS–on-prem infrastructure joined by VPN, Kafka-based real-time pipelines, and Kubernetes run with GitOps.

See projects ↓ GitHub
BARO · vehicle position flow
Site-to-Site VPNVehicles ×1500asyncio simulatorMosquittoMQTT · EC2controlshared subKafka4 partitionsdispatchGEO dispatchTimescaleDBon-prem storePrice feedsyfinance · KIS WSKafkastock.raw.*Detection% · Z · ETFNewsNewsAPI · RSSLLM analysisGroq · DLQDashboardWebSocket · Slack

The core flows of both projects play in turn

1,500

simulated vehicles handled
over MQTT → Kafka

BARO

2.9M

Kafka consumer lag
traced and cleared

BARO

26

Kubernetes workloads
run with ArgoCD GitOps

StockPulse

108

Korean and US tickers
watched minute by minute

StockPulse

Projects

Site-to-Site VPNVehicles ×1500asyncio simulatorMosquittoMQTT · EC2controlshared subKafka4 partitionsdispatchGEO dispatchTimescaleDBon-prem store

BARO — Autonomous Taxi Dispatch & Control Platform

Apr – Jun 2026 · Team of 5 · Backend / DevOps

Takes positions from 1,500 vehicles over MQTT → Kafka to dispatch and relocate cars, with AWS services and on-prem analytics and observability joined by VPN in a hybrid cloud.

  • AWS ECS
  • Terraform
  • MQTT
  • Kafka
  • Valkey
  • OpenStack · K3s
  • Airflow
  • Prometheus
Case study →
Price feedsyfinance · KIS WSKafkastock.raw.*Detection% · Z · ETFNewsNewsAPI · RSSLLM analysisGroq · DLQDashboardWebSocket · Slack

StockPulse — Stock Anomaly Detection & AI Analysis Pipeline

Mar 2026 · Solo project · Design to operations

Streams 1-minute bars for 108 Korean and US tickers through a Kafka pipeline to catch sharp moves, has an LLM explain the cause, and delivers it to a dashboard and Slack. Deployed and run on on-prem Kubernetes with ArgoCD GitOps.

  • Kafka
  • Kubernetes
  • ArgoCD
  • Sealed Secrets
  • TimescaleDB
  • Prometheus
  • FastAPI
  • Next.js
Case study →

Operations log

Full logs are in each case study →

BARO

When Kafka stalled, the control console stalled too

Symptom
The control console showed only one vehicle, and Mosquitto logs filled with broken pipes.
Cause
The Kafka producer’s default wait (max.block.ms, 60 s) held the MQTT receive thread, and the broker’s queue overflowed.
Fix
Cut max.block.ms to 500 ms and retries to 0 so a Kafka outage can no longer spill into MQTT receiving.

BARO

Consumer lag hit 2.9M — the culprit was a DB query on the hot path

Symptom
dispatch fell behind on positions and Kafka consumer lag grew to 2.9M.
Cause
For every position message (about 333 per second), it looked up the vehicle’s active dispatch in the database.
Fix
Cached the vehicle → dispatch mapping in memory so message handling no longer touches the database.

StockPulse

ArgoCD stuck in OutOfSync forever

Symptom
Some resources stayed OutOfSync after every deploy, and large manifests failed to apply at all.
Cause
Fields the cluster fills in — StatefulSet volumeClaimTemplates, CronJob status — never matched Git, and large resources exceeded the 262 KB limit of the last-applied annotation that client-side apply leaves behind.
Fix
Excluded those fields with ignoreDifferences and switched to ServerSideApply=true so resources apply without the annotation.

Tech stack

Cloud · AWS

  • ECS Fargate
  • EC2
  • RDS
  • ElastiCache
  • ALB
  • Route53
  • Site-to-Site VPN
  • Secrets Manager
  • CloudWatch

IaC · CI/CD

  • Terraform
  • GitHub Actions
  • self-hosted runner
  • CodeDeploy Blue/Green
  • ArgoCD
  • Sealed Secrets

Container · Orchestration

  • Kubernetes
  • K3s
  • OpenStack
  • Docker
  • Harbor
  • MetalLB
  • NFS

Messaging · Data

  • Kafka
  • MQTT · Mosquitto
  • TimescaleDB
  • PostgreSQL · PostGIS
  • Valkey · Redis
  • Airflow

Observability

  • Prometheus
  • Grafana
  • Alertmanager
  • CloudWatch exporter
  • blackbox exporter
  • kafka-exporter

Backend

  • Kotlin
  • Spring Boot
  • Python
  • FastAPI
  • TypeScript
  • Next.js

About

An introduction is on its way.