Synaptic SkillsSynapticSkills
MarketplaceSkill GraphCriar SkillMCP ServerPlataformaEnterprise
v0.1.0-beta
Voltar ao Marketplace
DataAvançado

Web scraping methodology

porjamditis·jamditis· v1.0.0 · atualizado em 2026-04-10
81
Score

Web scraping with anti-bot bypass, content extraction, and poison pill detection. Use when extracting content from websites, handling paywalls, implementing scraping cascades, or processing social media. Covers trafilatura, Playwright with stealth mode, yt-dlp, and instaloader patterns.

web-scrapingdata-extractionanti-bot-bypasscontent-extractionpoison-pill-detectiondata-pipelinedata-acquisition
0Stars
0Forks
0Usos
Fork

Documento do Skill

SKILL.mdweb-scraping/workflow
1
Identify Target: — Determine the website and data to scrape.
2
Choose Scraper: — Select the appropriate scraping method (Trafilatura, Requests, Playwright).
3
Implement Anti-Bot Bypass: — Use techniques like rotating user agents and Playwright stealth mode.
4
Extract Content: — Extract the desired data from the website.
5
Detect Poison Pills: — Check for paywalls, CAPTCHAs, and other anti-scraping measures.
6
Handle Social Media: — Use yt-dlp or instaloader for YouTube, TikTok, or Instagram.
7
Process Data: — Clean and format the extracted data.
8
Store Data: — Save the extracted data to a database or file.

Telemetria de Agentes

Execuções
0
total
Taxa de Sucesso
0%
últimos 30d
Latência Média
0.0s
p50
Alucinação
0.0%
detecção
Tokens Entrada
0
avg 0/exec
Tokens Saída
0
avg 0/exec

Uso por Plataforma

Skills Relacionados

Compõe com ←浏览器自动化 Skill
70%
Hebbian Synapse
Composite0.700
w = 0.3·α + 0.5·β + 0.2·γ
79
Compõe com ←Browser Automation
70%
Hebbian Synapse
Composite0.700
w = 0.3·α + 0.5·β + 0.2·γ
81
Compõe com ←Browser Automation
70%
Hebbian Synapse
Composite0.700
w = 0.3·α + 0.5·β + 0.2·γ
78
Compõe com ←Chrome CDP
70%
Hebbian Synapse
Composite0.700
w = 0.3·α + 0.5·β + 0.2·γ
78
Compõe com ←opencli-rs
70%
Hebbian Synapse
Composite0.700
w = 0.3·α + 0.5·β + 0.2·γ
77
Similar a ←WeChat Article Extractor
60%
Hebbian Synapse
Composite0.600
w = 0.3·α + 0.5·β + 0.2·γ
77
Similar a ←Weather
60%
Hebbian Synapse
Composite0.600
w = 0.3·α + 0.5·β + 0.2·γ
82
Similar a ←DAG Diagnosis
60%
Hebbian Synapse
Composite0.600
w = 0.3·α + 0.5·β + 0.2·γ
83
Similar a ←Data Freshness Check
60%
Hebbian Synapse
Composite0.600
w = 0.3·α + 0.5·β + 0.2·γ
83

Árvore do Skill

Web scraping methodology
web-scraping
Fases Cognitivas5
1.SENSE
2.CONTEXTUALIZE
3.HYPOTHESIZE
4.ACT
5.REFLECT
Triggers9
scrape a websiteextract data from a webpagebypass anti-bot measuresdetect paywallsscrape YouTube videosscrape Instagram postsextract content from social mediascrape with playwrightscrape with trafilatura

Avaliar este Skill

Score Breakdown

⭐Avaliação Humana0%
🤖Sucesso de Agentes0%
🕐Atualidade100%
🔗Saúde de Dependências100%
🕸️Centralidade no Grafo0%
🛡️Segurança50%
CompositeScore = α·Humano + β·Agente + γ·Recência + δ·Deps + ε·Centralidade + ζ·Segurança

Instalação

$ synaptic mcp download web-scraping
$ synaptic skills detail web-scraping
$ synaptic skills live web-scraping

Dependências

trafilaturarequestsbeautifulsoup4playwrightplaywright-stealthyt-dlpinstaloader

Links

GitHub Repository