bug fixing

TROUBLESHOOTING.md

@@ -0,0 +1,307 @@
+# Troubleshooting Guide
+
+## Common Issues and Solutions
+
+### Issue: Only 1-2 Articles in Digest (Expected 10-15)
+
+**Cause:** Filtering score is too strict or AI model is rating articles too harshly.
+
+**Solutions:**
+
+1. **Lower the filtering threshold** in `config.yaml`:
+   ```yaml
+   ai:
+     filtering:
+       min_score: 5.0  # Lower = more articles (default was 6.5)
+   ```
+
+2. **Check what scores articles are getting:**
+   ```bash
+   # View recent article scores in database
+   sqlite3 data/articles.db "SELECT title, relevance_score FROM articles WHERE processed=1 ORDER BY fetched_at DESC LIMIT 20;"
+   ```
+   
+   If most scores are between 4-6, lower your `min_score` to 4.5 or 5.0.
+
+3. **Adjust your interests** in `config.yaml` to be broader:
+   ```yaml
+   ai:
+     interests:
+       - "Technology news"  # Broader
+       - "Software development"  # Broader
+       # Instead of very specific interests
+   ```
+
+4. **Temporarily disable filtering** to see all articles:
+   ```yaml
+   ai:
+     filtering:
+       enabled: false  # Get ALL articles (not recommended long-term)
+   ```
+
+### Issue: Hit Rate Limit on Free Model
+
+**Cause:** Free models have strict rate limits (e.g., 10 requests/minute).
+
+**Solution:** Switch to a paid model in `config.yaml`:
+
+```yaml
+ai:
+  model: "openai/gpt-4o-mini"  # ~$0.05-0.10/day, no rate limits
+  # OR
+  model: "anthropic/claude-3.5-haiku"  # ~$0.10/day, excellent quality
+```
+
+Add credits to your OpenRouter account: https://openrouter.ai/credits
+
+### Issue: No Email Received
+
+**Check these in order:**
+
+1. **Look for error in logs:**
+   ```bash
+   tail -n 50 data/logs/news-agent.log | grep -i email
+   ```
+
+2. **Verify SMTP settings:**
+   ```bash
+   # Check config
+   grep -A 6 "smtp:" config.yaml
+   
+   # Check credentials
+   grep SMTP .env
+   ```
+
+3. **Test email independently:**
+   ```bash
+   python -m src.test_email_simple
+   ```
+
+4. **Common fixes:**
+   - Wrong port (use 587 for TLS, 465 for SSL)
+   - Missing credentials in `.env`
+   - Firewall blocking SMTP port
+   - Wrong hostname
+   - See `SMTP_CONFIG.md` for detailed troubleshooting
+
+### Issue: Articles Are All Duplicates
+
+**Cause:** Database hasn't been cleared and articles were already fetched.
+
+**Solution:**
+
+```bash
+# Option 1: Clear the database completely
+rm data/articles.db
+python -m src.main
+
+# Option 2: Just clear processed status to re-process
+sqlite3 data/articles.db "UPDATE articles SET processed=0, included_in_digest=0;"
+python -m src.main
+```
+
+### Issue: Email Not Nicely Formatted
+
+**Check:**
+1. Are you viewing in a web-based email client? (Gmail, Outlook web, etc.)
+   - HTML emails look best in web clients
+   
+2. Plain text client?
+   - The email includes both HTML and plain text versions
+   - Plain text is readable but less styled
+
+3. Want to customize?
+   - Edit `src/email/templates/daily_digest.html`
+   - CSS is inline for maximum email client compatibility
+
+### Issue: Model Name Error (404)
+
+```
+ERROR - No endpoints found for [model-name]
+```
+
+**Cause:** Model name is wrong or model is no longer available.
+
+**Solution:**
+
+1. Check current models at: https://openrouter.ai/models
+
+2. Update `config.yaml` with correct name:
+   ```yaml
+   ai:
+     model: "openai/gpt-4o-mini"  # Copy exact name from OpenRouter
+   ```
+
+3. See `MODELS.md` for recommended models
+
+### Issue: Too Expensive
+
+**Current cost too high?**
+
+Check usage: https://openrouter.ai/activity
+
+**Solutions:**
+
+1. **Use cheaper model:**
+   ```yaml
+   ai:
+     model: "openai/gpt-4o-mini"  # Cheapest reliable option (~$0.05/day)
+   ```
+
+2. **Process fewer articles:**
+   ```yaml
+   ai:
+     filtering:
+       max_articles: 10  # Reduce from 15
+       min_score: 6.0  # Higher score = fewer articles
+   ```
+
+3. **Remove some RSS sources** from `config.yaml`
+
+4. **Use free model** (with rate limits):
+   ```yaml
+   ai:
+     model: "google/gemini-2.0-flash-exp:free"
+   ```
+   Note: May fail during runs due to rate limits
+
+### Issue: Summaries Are Poor Quality
+
+**Solutions:**
+
+1. **Use better model:**
+   ```yaml
+   ai:
+     model: "anthropic/claude-3.5-haiku"  # Best summarization
+   ```
+
+2. **Adjust prompts** in `src/ai/prompts.py`:
+   - Edit `SUMMARIZATION_SYSTEM_PROMPT`
+   - Make it more specific to your needs
+
+3. **Check article content:**
+   - Some RSS feeds have limited content
+   - AI can only summarize what's in the feed
+
+### Issue: Articles Not Relevant
+
+**Solutions:**
+
+1. **Update your interests** in `config.yaml`:
+   ```yaml
+   ai:
+     interests:
+       - "Specific topic you care about"
+       - "Another specific interest"
+   ```
+
+2. **Check article scores:**
+   ```bash
+   sqlite3 data/articles.db "SELECT title, relevance_score, category FROM articles WHERE included_in_digest=1 ORDER BY relevance_score DESC LIMIT 10;"
+   ```
+
+3. **Increase filtering threshold:**
+   ```yaml
+   ai:
+     filtering:
+       min_score: 7.0  # Only very relevant articles
+   ```
+
+### Issue: SystemD Timer Not Running
+
+**Check timer status:**
+```bash
+systemctl --user status news-agent.timer
+systemctl --user list-timers
+```
+
+**Common fixes:**
+
+1. **Enable lingering:**
+   ```bash
+   sudo loginctl enable-linger $USER
+   ```
+
+2. **Reload systemd:**
+   ```bash
+   systemctl --user daemon-reload
+   systemctl --user restart news-agent.timer
+   ```
+
+3. **Check service logs:**
+   ```bash
+   journalctl --user -u news-agent.service -f
+   ```
+
+4. **Verify paths in service file:**
+   ```bash
+   nano ~/.config/systemd/user/news-agent.service
+   # Make sure WorkingDirectory and ExecStart paths are correct
+   ```
+
+### Issue: Permission Denied
+
+**Running as root?**
+- Don't run as root - use your normal user account
+- systemd user services run as your user
+
+**File permissions:**
+```bash
+chmod 600 .env
+chmod 644 config.yaml
+chmod +x setup.sh
+```
+
+### Issue: Import Errors
+
+```
+ModuleNotFoundError: No module named 'feedparser'
+```
+
+**Solution:**
+
+1. **Activate virtual environment:**
+   ```bash
+   source .venv/bin/activate
+   ```
+
+2. **Install dependencies:**
+   ```bash
+   pip install feedparser httpx openai pydantic pydantic-settings jinja2 premailer python-dotenv pyyaml aiosqlite
+   ```
+
+## Getting More Help
+
+1. **Check logs:**
+   ```bash
+   cat data/logs/news-agent.log
+   ```
+
+2. **Run in verbose mode** (shows all output):
+   ```bash
+   python -m src.main
+   ```
+
+3. **Test components individually:**
+   ```bash
+   # Test email
+   python -m src.test_email_simple
+   
+   # Test database
+   sqlite3 data/articles.db ".tables"
+   
+   # Test config
+   python -c "from src.config import get_config; c = get_config(); print(f'Model: {c.ai.model}')"
+   ```
+
+4. **Review documentation:**
+   - `README.md` - Overview
+   - `SETUP.md` - Installation
+   - `SMTP_CONFIG.md` - Email configuration
+   - `MODELS.md` - AI model selection
+   - `CHANGES.md` - Recent changes
+
+5. **OpenRouter support:**
+   - Dashboard: https://openrouter.ai/activity
+   - Docs: https://openrouter.ai/docs
+   - Models: https://openrouter.ai/models