🏗️ Seasoned Data Platform Solutions Architect and Principal Data Engineer with 20+ years of experience designing and delivering enterprise data platforms across automotive, banking, and manufacturing sectors in Europe.
☁️ I architect end-to-end solutions on Azure and Snowflake — from data ingestion and data modeling to governance and self-service analytics — while staying hands-on with 🐍 Python, SQL, CI/CD (Azure DevOps & GitHub Actions), and Infrastructure as Code.
🤖 I actively pioneer the use of LLMs and AI Agents (RAG, PydanticAI, Semantic Kernel, MCP) to enhance data engineering workflows and platform intelligence.
🎓 Certified on Azure, Snowflake, Google Cloud, with an MSc in Software & Systems Engineering and executive courses from Stanford and Harvard Business School.
Some of my projects are also available through agent plugin marketplaces and Microsoft's Awesome AZD catalog:
| Project | Where to find it | Quick start |
|---|---|---|
| iParq | Codex and GitHub Copilot plugin marketplace, VS Code source install, PyPI, and Homebrew | codex plugin marketplace add MiguelElGallo/iparqcodex plugin add iparq@iparqVS Code: Chat: Install Plugin From Source |
| md-to-pdf | Codex and GitHub Copilot plugin marketplace, VS Code source install, and GitHub Releases | codex plugin marketplace add MiguelElGallo/md-to-pdfcodex plugin add md-to-pdf@md-to-pdfVS Code: Chat: Install Plugin From Source |
| FastAPI + Snowflake on Azure Functions | Awesome AZD catalog | azd init --template MiguelElGallo/simple-fastapi-snow-azd |
| Simple Streamlit on Azure App Service | Awesome AZD catalog | azd init --template MiguelElGallo/simple-streamlit-azd |
| FastMCP on Azure App Service | Awesome AZD catalog | azd init --template MiguelElGallo/myfirstmcp-openai |
Here are the repositories I maintain or contribute to:
| Repository | Short Description | Python libraries | Azure services | Data? | AI? |
|---|---|---|---|---|---|
| mpzsql ⭐28 | Arrow Flight SQL DuckDB lakehouse | azure-identity, duckdb, grpcio, pyarrow, pyjwt | Blob Storage, Container Apps, PostgreSQL | ✅ | - |
| iparq ⭐25 | Parquet metadata inspection CLI tool | pyarrow, pydantic, rich, typer | - | ✅ | - |
| modelando ⭐11 | Bilingual data modeling educational materials | - | - | ✅ | - |
| api-elt ⭐9 | API to DuckDB ELT pipeline | dlt, python-dotenv | - | ✅ | - |
| azquack ⭐6 | DuckDB Quack protocol on Azure | duckdb | Blob Storage, Container Apps | ✅ | - |
| snowflake-semantic-view-skill ⭐5 | Snowflake semantic view AI skill | - | - | ✅ | ✅ |
| FastAPI-in-Snowflake ⭐4 | FastAPI in Snowflake container services | fastapi, passlib, python-jose, snowflake-connector-python, uvicorn | - | ✅ | - |
| SynapseApacheIceBergExperiment ⭐4 | Apache Iceberg on Synapse Spark | - | Synapse Analytics | ✅ | - |
| evsnow ⭐3 | Event Hubs to Snowflake streaming | azure-eventhub, azure-identity, pydantic-ai, snowflake-connector-python | Event Hubs | ✅ | ✅ |
| simple-streamlit-azd ⭐3 | Streamlit on Azure App Service | numpy, pandas, streamlit | App Service | ✅ | - |
| sensormatrix ⭐1 | ESP32 sensor testing and validation | pydantic, pyserial, pyyaml, typer | - | ✅ | - |
| snapshottest ⭐1 | Inline snapshot testing demonstration | httpx, inline-snapshot, rich, typer | - | - | - |
| ragsql ⭐1 | RAG-powered SQL generation from databases | langchain, openai, snowflake-connector-python | - | ✅ | ✅ |
| skill-metric-view-dbr | Databricks metric view creation skill | - | - | ✅ | ✅ |
| Modelando2025 | Data modeling concepts 2025 edition | mkdocs-git-committers-plugin, mkdocs-i18n, mkdocs-material | - | ✅ | - |
| agent-plugin-forge | Agent Skills to validated Plugins | jsonschema, license-expression, pydantic, pyyaml, semantic-version | - | - | ✅ |
| api-test-pilot | Contract-driven API test generation | httpx, jsonschema, pydantic, pyyaml, typer | - | - | ✅ |
| azure-ducklake-quack | Azure-native DuckLake with Entra routing | - | Container Apps, PostgreSQL | ✅ | - |
| bricksgdpr-publi | Databricks GDPR pseudonymization with dbt | dbt-core, dbt-databricks | - | ✅ | - |
| bricksgdpr | Databricks dbt GDPR demonstration | dbt-core, dbt-databricks | - | ✅ | - |
| md-to-pdf | Markdown to PDF converter | zensical | - | - | - |
| homebrew-iparq | Homebrew tap for iparq CLI | - | - | - | - |
| dbtobsb | dbt Core observability for Databricks | - | - | ✅ | - |
| evbricks | Event Hubs to Databricks streaming | azure-eventhub, azure-identity, databricks-zerobus-ingest-sdk | Event Hubs | ✅ | - |
| bricks-cli | dbt on Databricks with CLI | - | - | ✅ | - |
| codemode | Code Mode vs MCP comparison | anthropic, mcp, openai, pydantic-ai-harness | - | - | ✅ |
| neoquack | DuckDB Quack on free tiers | duckdb, fastapi, httpx, psycopg | - | ✅ | - |
| SnowflakeCortexCLI | Single-role Snowflake Cortex CLI setup | - | - | ✅ | ✅ |
| snowdag | Airflow dbt in Snowflake containers | - | - | ✅ | - |
| fastapi-free | FastAPI free-threading benchmark | fastapi, prometheus-client, uvicorn | - | - | - |
| RioArriba | River shooter arcade game | - | - | - | - |
| ir-support-site | iOS app support site | - | - | - | - |
| snow_iceberg_snowstorage | Snowflake Iceberg with DuckDB demo | - | - | ✅ | - |
| snowmcpaz | Snowflake MCP with Azure auth | - | - | ✅ | ✅ |
| jwtaztoken | Azure JWT token inspector | cryptography, httpx, pyjwt, rich, typer | - | - | - |
| dlthubarrow | dlt Arrow mode on Azure | azure-monitor-opentelemetry, dlt, psutil, pyarrow, snowflake-connector-python | Container Apps, Key Vault | ✅ | - |
| snowdcm | Snowflake DCM objects demonstration | - | - | ✅ | - |
| dlthubsnow | dlt in Snowflake containers | dlt, requests | - | ✅ | - |
| Arrow_as_source | Arrow as universal dataset demonstration | arrow, duckdb, polars, pyarrow | - | ✅ | - |
| ghs | Sync env files with GitHub | pydantic, python-dotenv, typer | - | - | - |
| myfirstmcp-openai | FastMCP on Azure App Service | gunicorn, mcp, uvicorn | App Service | - | ✅ |
| CallAPIfromLLM | LLM API calling demonstration | - | - | - | ✅ |
| snowtofu | Snowflake with OpenTofu IaC | - | Storage Account | ✅ | - |
| simple-fastapi-snow-azd | FastAPI Snowflake on Azure Functions | fastapi, snowflake-sqlalchemy | Functions | ✅ | - |
| embeddindataengineering | OpenAI embeddings in Fabric | - | - | ✅ | ✅ |
| mycv | Personal CV repository | - | - | - | - |
| dltHub-teesting | dltHub pytest testing | dlt, duckdb, pytest | - | ✅ | - |
| ChartToMD | Chart to markdown for RAG | aiohttp, azure-identity, pydantic, typer | - | - | ✅ |
| SnowCLI | Snowflake CLI with GitHub Actions | - | - | ✅ | - |
| rag-graph | Graph approaches for RAG testing | - | - | - | ✅ |
| opeanai-rag-test | OpenAI RAG Spanish document testing | openai, python-dotenv | - | - | ✅ |
| ragvectordb | Embeddings in multiple vector databases | - | AI Search | ✅ | ✅ |
| azdcodespaces | AZD in Codespaces configuration | - | - | - | - |
| azureaisearch | Azure AI Search embeddings demo | - | AI Search | - | ✅ |
| azopenai_adx | Azure Data Explorer embeddings storage | - | Data Explorer | ✅ | ✅ |
| semker | Semantic Kernel Python examples | semantic-kernel | - | - | ✅ |
| dvsat | Data Vault extended satellite demo | - | - | ✅ | - |
| JoinOrAssociation | BI tool fan trap handling | - | - | ✅ | - |
Last updated: 2026-08-30