AI Newsletter Digest improvements: fixed QP soft line break decoding, URL extraction, and content cleaning
This commit is contained in:
Krilly
@@ -0,0 +1,179 @@
|
||||
# YouTube Summarizer
|
||||
|
||||
Automatically fetch YouTube video transcripts, generate structured summaries, and deliver full transcripts to messaging platforms.
|
||||
|
||||
## Features
|
||||
|
||||
✅ **Automatic detection** - Recognizes YouTube URLs in messages
|
||||
✅ **Cloud-friendly** - Works from VPS/cloud IPs where yt-dlp fails
|
||||
✅ **Structured summaries** - Main thesis, key insights, and takeaways
|
||||
✅ **Full transcripts** - Downloadable text file with complete video content
|
||||
✅ **Platform-aware** - Auto-sends files to Telegram, text-only elsewhere
|
||||
✅ **Multi-language** - Supports multiple languages with English fallback
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. **Node.js 18+** installed
|
||||
2. **Clawdbot** running
|
||||
|
||||
### Install via ClawdHub
|
||||
|
||||
```bash
|
||||
clawdhub install youtube-summarizer
|
||||
```
|
||||
|
||||
### Manual Installation
|
||||
|
||||
```bash
|
||||
# 1. Clone the skill
|
||||
cd /root/clawd/skills
|
||||
git clone <this-repo-url> youtube-summarizer
|
||||
|
||||
# 2. Install MCP YouTube Transcript dependency
|
||||
cd /root/clawd
|
||||
git clone https://github.com/kimtaeyoon83/mcp-server-youtube-transcript.git
|
||||
cd mcp-server-youtube-transcript
|
||||
npm install && npm run build
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Simply share a YouTube URL in chat:
|
||||
|
||||
```
|
||||
You: https://youtu.be/dQw4w9WgXcQ
|
||||
|
||||
Agent: 📹 **Video:** Never Gonna Give You Up
|
||||
👤 **Channel:** Rick Astley | 👁️ **Views:** 1.4B | 📅 **Published:** 2009-10-25
|
||||
|
||||
**🎯 Main Thesis:**
|
||||
A declaration of unwavering commitment and loyalty in a relationship...
|
||||
|
||||
[structured summary follows]
|
||||
|
||||
📄 Full transcript attached (Telegram) or saved to transcripts/
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Detects** YouTube URLs automatically
|
||||
2. **Fetches** transcript using MCP server (bypasses cloud IP blocks)
|
||||
3. **Generates** structured summary with metadata
|
||||
4. **Saves** full transcript to `transcripts/YYYY-MM-DD_VIDEO_ID.txt`
|
||||
5. **Sends** file to Telegram (if in Telegram context)
|
||||
6. **Replies** with formatted summary
|
||||
|
||||
## Supported URL Formats
|
||||
|
||||
- `https://www.youtube.com/watch?v=VIDEO_ID`
|
||||
- `https://youtu.be/VIDEO_ID`
|
||||
- `https://www.youtube.com/shorts/VIDEO_ID`
|
||||
- Direct video ID: `VIDEO_ID`
|
||||
|
||||
## Output Format
|
||||
|
||||
### Summary Structure
|
||||
|
||||
```markdown
|
||||
📹 **Video:** [Title]
|
||||
👤 **Channel:** [Author] | 👁️ **Views:** [Count] | 📅 **Published:** [Date]
|
||||
|
||||
**🎯 Main Thesis:**
|
||||
Core message in 1-2 sentences
|
||||
|
||||
**💡 Key Insights:**
|
||||
- Insight 1
|
||||
- Insight 2
|
||||
- Insight 3
|
||||
|
||||
**📝 Notable Points:**
|
||||
- Supporting detail 1
|
||||
- Supporting detail 2
|
||||
|
||||
**🔑 Takeaway:**
|
||||
Practical conclusion
|
||||
```
|
||||
|
||||
### Transcript File
|
||||
|
||||
Saved to `/root/clawd/transcripts/YYYY-MM-DD_VIDEO_ID.txt` with:
|
||||
- Video metadata header
|
||||
- Full transcript text
|
||||
- URL reference
|
||||
|
||||
## Configuration
|
||||
|
||||
No configuration needed! The skill automatically:
|
||||
- Detects your messaging platform
|
||||
- Chooses appropriate delivery method
|
||||
- Handles language fallback
|
||||
- Creates transcript directory if needed
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Transcript not available"
|
||||
- Video may not have captions enabled
|
||||
- Try a different video
|
||||
- Use YouTube's manual transcript feature
|
||||
|
||||
### "MCP server not found"
|
||||
Install the dependency:
|
||||
```bash
|
||||
cd /root/clawd
|
||||
git clone https://github.com/kimtaeyoon83/mcp-server-youtube-transcript.git
|
||||
cd mcp-server-youtube-transcript
|
||||
npm install && npm run build
|
||||
```
|
||||
|
||||
### "Language not available"
|
||||
The skill automatically falls back to English if requested language isn't available.
|
||||
|
||||
## Why This Skill?
|
||||
|
||||
### Problem
|
||||
- yt-dlp blocked on cloud/VPS IPs
|
||||
- YouTube aggressive with bot detection
|
||||
- Manual transcript extraction tedious
|
||||
- Need structured summaries, not raw text
|
||||
|
||||
### Solution
|
||||
- Uses MCP server with Android client emulation
|
||||
- Bypasses cloud IP restrictions
|
||||
- Auto-generates structured summaries
|
||||
- Platform-aware file delivery
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [MCP YouTube Transcript](https://github.com/kimtaeyoon83/mcp-server-youtube-transcript) - Fetches transcripts via Android client emulation
|
||||
- Node.js 18+ - Runtime environment
|
||||
- Clawdbot - AI agent framework
|
||||
|
||||
## Credits
|
||||
|
||||
- Built by **abe238**
|
||||
- Uses [mcp-server-youtube-transcript](https://github.com/kimtaeyoon83/mcp-server-youtube-transcript) by kimtaeyoon83
|
||||
- Inspired by the need for reliable YouTube transcription on cloud servers
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
## Contributing
|
||||
|
||||
Improvements welcome! Consider:
|
||||
- Additional summary templates
|
||||
- Multi-language summary generation
|
||||
- Timestamp-based chapter extraction
|
||||
- Video metadata enrichment
|
||||
|
||||
## Changelog
|
||||
|
||||
### v1.0.0 (2026-01-26)
|
||||
- Initial release
|
||||
- Auto-detect YouTube URLs
|
||||
- Generate structured summaries
|
||||
- Save full transcripts
|
||||
- Telegram file delivery
|
||||
- Cloud IP bypass via MCP server
|
||||
@@ -0,0 +1,161 @@
|
||||
---
|
||||
name: youtube-summarizer
|
||||
description: Automatically fetch YouTube video transcripts, generate structured summaries, and send full transcripts to messaging platforms. Detects YouTube URLs and provides metadata, key insights, and downloadable transcripts.
|
||||
version: 1.0.0
|
||||
author: abe238
|
||||
tags: [youtube, transcription, summarization, video, telegram]
|
||||
---
|
||||
|
||||
# YouTube Summarizer Skill
|
||||
|
||||
Automatically fetch transcripts from YouTube videos, generate structured summaries, and deliver full transcripts to messaging platforms.
|
||||
|
||||
## When to Use
|
||||
|
||||
Activate this skill when:
|
||||
- User shares a YouTube URL (youtube.com/watch, youtu.be, youtube.com/shorts)
|
||||
- User asks to summarize or transcribe a YouTube video
|
||||
- User requests information about a YouTube video's content
|
||||
|
||||
## Dependencies
|
||||
|
||||
**Required:** MCP YouTube Transcript server must be installed at:
|
||||
`/root/clawd/mcp-server-youtube-transcript`
|
||||
|
||||
If not present, install it:
|
||||
```bash
|
||||
cd /root/clawd
|
||||
git clone https://github.com/kimtaeyoon83/mcp-server-youtube-transcript.git
|
||||
cd mcp-server-youtube-transcript
|
||||
npm install && npm run build
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Detect YouTube URL
|
||||
Extract video ID from these patterns:
|
||||
- `https://www.youtube.com/watch?v=VIDEO_ID`
|
||||
- `https://youtu.be/VIDEO_ID`
|
||||
- `https://www.youtube.com/shorts/VIDEO_ID`
|
||||
- Direct video ID: `VIDEO_ID` (11 characters)
|
||||
|
||||
### 2. Fetch Transcript
|
||||
Run this command to get the transcript:
|
||||
```bash
|
||||
cd /root/clawd/mcp-server-youtube-transcript && node --input-type=module -e "
|
||||
import { getSubtitles } from './dist/youtube-fetcher.js';
|
||||
const result = await getSubtitles({ videoID: 'VIDEO_ID', lang: 'en' });
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
" > /tmp/yt-transcript.json
|
||||
```
|
||||
|
||||
Replace `VIDEO_ID` with the extracted ID. Read the output from `/tmp/yt-transcript.json`.
|
||||
|
||||
### 3. Process the Data
|
||||
|
||||
Parse the JSON to extract:
|
||||
- `result.metadata.title` - Video title
|
||||
- `result.metadata.author` - Channel name
|
||||
- `result.metadata.viewCount` - Formatted view count
|
||||
- `result.metadata.publishDate` - Publication date
|
||||
- `result.actualLang` - Language used
|
||||
- `result.lines` - Array of transcript segments
|
||||
|
||||
Full text: `result.lines.map(l => l.text).join(' ')`
|
||||
|
||||
### 4. Generate Summary
|
||||
|
||||
Create a structured summary using this template:
|
||||
|
||||
```markdown
|
||||
📹 **Video:** [title]
|
||||
👤 **Channel:** [author] | 👁️ **Views:** [views] | 📅 **Published:** [date]
|
||||
|
||||
**🎯 Main Thesis:**
|
||||
[1-2 sentence core argument/message]
|
||||
|
||||
**💡 Key Insights:**
|
||||
- [insight 1]
|
||||
- [insight 2]
|
||||
- [insight 3]
|
||||
- [insight 4]
|
||||
- [insight 5]
|
||||
|
||||
**📝 Notable Points:**
|
||||
- [additional point 1]
|
||||
- [additional point 2]
|
||||
|
||||
**🔑 Takeaway:**
|
||||
[Practical application or conclusion]
|
||||
```
|
||||
|
||||
Aim for:
|
||||
- Main thesis: 1-2 sentences maximum
|
||||
- Key insights: 3-5 bullets, each 1-2 sentences
|
||||
- Notable points: 2-4 supporting details
|
||||
- Takeaway: Actionable conclusion
|
||||
|
||||
### 5. Save Full Transcript
|
||||
|
||||
Save the complete transcript to a timestamped file:
|
||||
```
|
||||
/root/clawd/transcripts/YYYY-MM-DD_VIDEO_ID.txt
|
||||
```
|
||||
|
||||
Include in the file:
|
||||
- Video metadata header
|
||||
- Full transcript text
|
||||
- URL reference
|
||||
|
||||
### 6. Platform-Specific Delivery
|
||||
|
||||
**If channel is Telegram:**
|
||||
```bash
|
||||
message --action send --channel telegram --target CHAT_ID \
|
||||
--filePath /root/clawd/transcripts/YYYY-MM-DD_VIDEO_ID.txt \
|
||||
--caption "📄 YouTube Transcript: [title]"
|
||||
```
|
||||
|
||||
**If channel is other/webchat:**
|
||||
Just reply with the summary (no file attachment).
|
||||
|
||||
### 7. Reply with Summary
|
||||
|
||||
Send the structured summary as your response to the user.
|
||||
|
||||
## Error Handling
|
||||
|
||||
**If transcript fetch fails:**
|
||||
- Check if video has captions enabled
|
||||
- Try with `lang: 'en'` fallback if requested language unavailable
|
||||
- Inform user that transcript is not available and suggest alternatives:
|
||||
- Manual YouTube transcript feature
|
||||
- Video may not have captions
|
||||
- Try a different video
|
||||
|
||||
**If MCP server not installed:**
|
||||
- Provide installation instructions
|
||||
- Offer to install it automatically if in appropriate context
|
||||
|
||||
**If video ID extraction fails:**
|
||||
- Ask user to provide the full YouTube URL or video ID
|
||||
|
||||
## Examples
|
||||
|
||||
See `examples/` directory for sample outputs.
|
||||
|
||||
## Quality Guidelines
|
||||
|
||||
- **Be concise:** Summary should be scannable in 30 seconds
|
||||
- **Be accurate:** Don't add information not in the transcript
|
||||
- **Be structured:** Use consistent formatting for easy reading
|
||||
- **Be contextual:** Adjust detail level based on video length
|
||||
- Short videos (<5 min): Brief summary
|
||||
- Long videos (>30 min): More detailed breakdown
|
||||
|
||||
## Notes
|
||||
|
||||
- MCP server uses Android client emulation to bypass YouTube's cloud IP blocking
|
||||
- Works reliably from VPS/cloud environments where yt-dlp often fails
|
||||
- Supports multiple languages with automatic fallback to English
|
||||
- Transcript quality depends on YouTube's auto-generated captions or manual captions
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"ownerId": "kn76wvzbk1fwrzfg04x9n7hc7x7zy3rf",
|
||||
"slug": "youtube-summarizer",
|
||||
"version": "1.0.0",
|
||||
"publishedAt": 1769398410293
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
# Sample Output
|
||||
|
||||
## Example 1: AI Productivity Video
|
||||
|
||||
**Input:** `https://youtu.be/EZ4EjJ0iDDQ`
|
||||
|
||||
**Output:**
|
||||
|
||||
```
|
||||
📹 **Video:** THE MISTAKE COSTING YOU 40% IN PRODUCTIVITY - Stop Teaching AI Basics
|
||||
👤 **Channel:** AI News & Strategy Daily | 👁️ **Views:** 22.2k | 📅 **Published:** 2025-12-03
|
||||
|
||||
**🎯 Main Thesis:**
|
||||
Most AI training focuses on the wrong layer—either basic "101" prompting or advanced "401" technical implementation. The missing middle ("201 level") is where actual productivity gains happen, and it requires management skills, not tool skills.
|
||||
|
||||
**💡 Key Insights:**
|
||||
- Microsoft study of 300K employees showed 80% stopped using AI after initial excitement crater; survivors treated AI as a management skill, not a tool skill
|
||||
- The "201 level" gap includes six critical skills: context assembly, quality judgment, task decomposition, iterative refinement, workflow integration, and frontier recognition
|
||||
- BCG/Harvard study found consultants were 19% less accurate on tasks outside AI's capability frontier, showing the danger of applying one mental model without understanding boundaries
|
||||
- Two effective work patterns emerged: "Centaur mode" (clearly divided tasks) for high-stakes work, and "Cyborg mode" (fluid integration) for creative work
|
||||
- The apprentice model is collapsing—juniors miss routine work that builds judgment, creating a long-term expertise deficit
|
||||
|
||||
**📝 Notable Points:**
|
||||
- Token consumption leaderboards at major orgs are dominated by senior execs and distinguished engineers—not because of technical skill, but management capability and domain knowledge
|
||||
- Organizations need experts mapping the frontier and creating guardrails, while enabling non-experts to work safely within those boundaries
|
||||
- Fear of doing it wrong is the biggest adoption blocker—without clear organizational guidance, talented people see AI as risk and avoid it
|
||||
- Employees who receive more than 5 hours of formal AI training are double-digit percentage points more likely to become regular users
|
||||
|
||||
**🔑 Takeaway:**
|
||||
Getting 80% of your organization to "201 level" AI fluency—treating AI like a capable but inexperienced collaborator who needs management—distinguishes companies that realize real gains from those stuck with a few power users and mass disengagement. Invest in the judgment layer, not just tool deployment.
|
||||
```
|
||||
|
||||
**File saved:** `/root/clawd/transcripts/2026-01-26_EZ4EjJ0iDDQ.txt`
|
||||
|
||||
---
|
||||
|
||||
## Example 2: Short Tutorial Video
|
||||
|
||||
**Input:** `https://youtu.be/dQw4w9WgXcQ`
|
||||
|
||||
**Output:**
|
||||
|
||||
```
|
||||
📹 **Video:** How to Build a REST API in 10 Minutes
|
||||
👤 **Channel:** DevTips | 👁️ **Views:** 450k | 📅 **Published:** 2024-08-15
|
||||
|
||||
**🎯 Main Thesis:**
|
||||
You can build a production-ready REST API quickly using modern frameworks and best practices without overengineering.
|
||||
|
||||
**💡 Key Insights:**
|
||||
- Express.js with TypeScript provides type safety and rapid development
|
||||
- Middleware pattern handles authentication, validation, and error handling cleanly
|
||||
- Environment variables manage configuration across dev, staging, and production
|
||||
- Simple folder structure (routes, controllers, services) scales better than complex architectures
|
||||
|
||||
**📝 Notable Points:**
|
||||
- Use Zod for runtime validation instead of manual checks
|
||||
- Prisma ORM eliminates SQL boilerplate while maintaining type safety
|
||||
- Docker containerization ensures consistency across environments
|
||||
|
||||
**🔑 Takeaway:**
|
||||
Start simple with proven patterns, add complexity only when needed, and prioritize developer experience to ship faster.
|
||||
```
|
||||
|
||||
**File saved:** `/root/clawd/transcripts/2024-08-15_dQw4w9WgXcQ.txt`
|
||||
|
||||
---
|
||||
|
||||
## Platform-Specific Behavior
|
||||
|
||||
### Telegram
|
||||
- Summary sent as text message
|
||||
- Full transcript sent as `.txt` file attachment
|
||||
- Caption on file: "📄 YouTube Transcript: [Video Title]"
|
||||
|
||||
### Webchat / Discord / Other
|
||||
- Summary sent as text message
|
||||
- Transcript saved locally but not sent as file
|
||||
- User informed of save location
|
||||
|
||||
---
|
||||
|
||||
## Transcript File Format
|
||||
|
||||
```
|
||||
VIDEO: THE MISTAKE COSTING YOU 40% IN PRODUCTIVITY - Stop Teaching AI Basics
|
||||
URL: https://youtu.be/EZ4EjJ0iDDQ
|
||||
Channel: AI News & Strategy Daily
|
||||
Views: 22.2k
|
||||
Published: 2025-12-03
|
||||
|
||||
=== TRANSCRIPT ===
|
||||
|
||||
Toward the very end of 2025, a study happened that we didn't pay attention to. Microsoft tracked 300,000 employees using AI C-Pilot. Excitement peaked for the first three weeks. Then there was a crater of disappointment...
|
||||
|
||||
[full transcript continues...]
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"name": "youtube-summarizer",
|
||||
"version": "1.0.0",
|
||||
"description": "Automatically fetch YouTube video transcripts, generate structured summaries, and deliver full transcripts to messaging platforms",
|
||||
"author": "abe238",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"youtube",
|
||||
"transcript",
|
||||
"summarization",
|
||||
"video",
|
||||
"telegram",
|
||||
"clawdbot",
|
||||
"ai-agent"
|
||||
],
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/abe238/youtube-summarizer"
|
||||
},
|
||||
"clawdbot": {
|
||||
"skill": true,
|
||||
"requires": {
|
||||
"node": ">=18.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"external": [
|
||||
{
|
||||
"name": "mcp-server-youtube-transcript",
|
||||
"url": "https://github.com/kimtaeyoon83/mcp-server-youtube-transcript",
|
||||
"path": "/root/clawd/mcp-server-youtube-transcript",
|
||||
"install": "git clone https://github.com/kimtaeyoon83/mcp-server-youtube-transcript.git /root/clawd/mcp-server-youtube-transcript && cd /root/clawd/mcp-server-youtube-transcript && npm install && npm run build"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user