Voltar ao MarketplaceScore
DataAvançadoVerificado
ETL Pipeline Builder
86
Builds ETL pipelines with extraction, transformation and loading. Supports batch and streaming with Spark, Airflow or Prefect.
etlpipelinesparkairflowdata-engineering
Linguagens
PythonSQLShell
Documento do Skill
SKILL.mdetl-pipeline-builder/workflow
1
SENSE — Map sources and destinations
Identify data sources (APIs, DBs, files, streams)
Destination schema and SLAs
2
CONTEXTUALIZE — Choose architecture
Batch vs streaming vs hybrid
Orchestrator (Airflow, Prefect, Dagster)
3
HYPOTHESIZE — Design pipeline
Transformation DAG
Partitioning strategy
Error handling and retry logic
4
ACT — Implement
Extract: connectors per source
Transform: cleaning, join, aggregation
Load: upsert to destination with idempotency
5
REFLECT — Validate
Data quality checks (Great Expectations)
Freshness and completeness monitoring
Alerts for data drift
Telemetria de Agentes
Execuções
14.5K
total
Taxa de Sucesso
84%
últimos 30d
Latência Média
5.6s
p50
Alucinação
3.0%
detecção
Uso por Plataforma
cursor7.2K
claude-code5.1K
codex2.2K
Árvore do Skill
ETL Pipeline Builder
etl-pipeline-builder
Fases Cognitivas5
1.SENSE
2.CONTEXTUALIZE
3.HYPOTHESIZE
4.ACT
5.REFLECT
Triggers4
criar pipeline etlpipeline de dadosetldata pipeline
MCP Servers3
🔌
Spark MCP
Distributed data processing with Apache Spark
ConfigConfigure SPARK_MASTER_URL
🗄️
PostgreSQL MCP
Query execution, migrations and schema introspection
npx @modelcontextprotocol/server-postgresConfigConfigure DATABASE_URL in .env
🔌
AWS S3 MCP
Upload, download and object management in S3 buckets
ConfigConfigure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Avaliar este Skill
Score Breakdown
⭐Avaliação Humana88%
🤖Sucesso de Agentes84%
🕐Atualidade89%
🔗Saúde de Dependências95%
🕸️Centralidade no Grafo70%
🛡️Segurança48%
CompositeScore = α·Humano + β·Agente + γ·Recência + δ·Deps + ε·Centralidade + ζ·Segurança
Instalação
$ skillschain install etl-pipeline-builder
$ skillschain use etl-pipeline-builder