278 lines
6.7 KiB
Markdown
278 lines
6.7 KiB
Markdown
# Performance Guide
|
||
|
||
## Expected Processing Times
|
||
|
||
### With Concurrent Processing (Current Implementation)
|
||
|
||
**For 151 articles with `openai/gpt-4o-mini`:**
|
||
|
||
| Phase | Time | Details |
|
||
|-------|------|---------|
|
||
| RSS Fetching | 10-30 sec | Parallel fetching from 14 sources |
|
||
| Article Filtering (151) | **30-90 sec** | Processes 10 articles at a time concurrently |
|
||
| AI Summarization (15) | **15-30 sec** | Processes 10 articles at a time concurrently |
|
||
| Email Generation | 1-2 sec | Local processing |
|
||
| Email Sending | 2-5 sec | SMTP transmission |
|
||
| **Total** | **~1-2.5 minutes** | For typical daily run |
|
||
|
||
### Breakdown by Article Count
|
||
|
||
| Articles | Filtering Time | Summarization (15) | Total Time |
|
||
|----------|---------------|-------------------|------------|
|
||
| 50 | 15-30 sec | 15-30 sec | ~1 min |
|
||
| 100 | 30-60 sec | 15-30 sec | ~1.5 min |
|
||
| 150 | 30-90 sec | 15-30 sec | ~2 min |
|
||
| 200 | 60-120 sec | 15-30 sec | ~2.5 min |
|
||
|
||
## Performance Optimizations
|
||
|
||
### 1. Concurrent API Calls
|
||
|
||
**Before (Sequential):**
|
||
```python
|
||
for article in articles:
|
||
score = await score_article(article) # Wait for each
|
||
```
|
||
- Time: 151 articles × 2 sec = **5+ minutes**
|
||
|
||
**After (Concurrent Batches):**
|
||
```python
|
||
batch_size = 10
|
||
for batch in batches:
|
||
scores = await asyncio.gather(*[score_article(a) for a in batch])
|
||
```
|
||
- Time: 151 articles ÷ 10 × 2 sec = **30-60 seconds**
|
||
|
||
**Speed improvement: 5-10x faster!**
|
||
|
||
### 2. Batch Size Configuration
|
||
|
||
Current batch size: **10 concurrent requests**
|
||
|
||
This balances:
|
||
- **Speed** - Multiple requests at once
|
||
- **Rate limits** - Doesn't overwhelm API
|
||
- **Memory** - Reasonable concurrent operations
|
||
|
||
You can adjust in code if needed (not recommended without testing):
|
||
- Lower batch size (5) = Slower but safer for rate limits
|
||
- Higher batch size (20) = Faster but may hit rate limits
|
||
|
||
### 3. Model Selection Impact
|
||
|
||
| Model | Speed per Request | Reliability |
|
||
|-------|------------------|-------------|
|
||
| `openai/gpt-4o-mini` | Fast (~1-2 sec) | Excellent |
|
||
| `anthropic/claude-3.5-haiku` | Fast (~1-2 sec) | Excellent |
|
||
| `google/gemini-2.0-flash-exp:free` | Variable (~1-3 sec) | Rate limits! |
|
||
| `meta-llama/llama-3.1-8b-instruct:free` | Slow (~2-4 sec) | Rate limits! |
|
||
|
||
**Recommendation:** Use paid models for consistent performance.
|
||
|
||
## Monitoring Performance
|
||
|
||
### Check Processing Time
|
||
|
||
Run manually and watch the logs:
|
||
```bash
|
||
time python -m src.main
|
||
```
|
||
|
||
Example output:
|
||
```
|
||
real 1m45.382s
|
||
user 0m2.156s
|
||
sys 0m0.312s
|
||
```
|
||
|
||
### View Detailed Timing
|
||
|
||
Enable debug logging in `config.yaml`:
|
||
```yaml
|
||
logging:
|
||
level: "DEBUG"
|
||
```
|
||
|
||
You'll see batch processing messages:
|
||
```
|
||
DEBUG - Processing batch 1 (10 articles)
|
||
DEBUG - Processing batch 2 (10 articles)
|
||
...
|
||
DEBUG - Summarizing batch 1 (10 articles)
|
||
```
|
||
|
||
### Performance Logs
|
||
|
||
Check `data/logs/news-agent.log` for timing info:
|
||
```bash
|
||
grep -E "Fetching|Filtering|Generating|Sending" data/logs/news-agent.log
|
||
```
|
||
|
||
## Troubleshooting Slow Performance
|
||
|
||
### Issue: Filtering Takes >5 Minutes
|
||
|
||
**Possible causes:**
|
||
1. **Using free model with rate limits**
|
||
- Switch to `openai/gpt-4o-mini` or `anthropic/claude-3.5-haiku`
|
||
|
||
2. **Network latency**
|
||
- Check internet connection
|
||
- Test: `ping openrouter.ai`
|
||
|
||
3. **API issues**
|
||
- Check OpenRouter status
|
||
- Try different model
|
||
|
||
**Solution:**
|
||
```yaml
|
||
ai:
|
||
model: "openai/gpt-4o-mini" # Fast, reliable, paid
|
||
```
|
||
|
||
### Issue: Frequent Timeouts
|
||
|
||
**Increase timeout in `src/ai/client.py`:**
|
||
|
||
Currently using default OpenAI client timeout. If needed, you can customize:
|
||
```python
|
||
self.client = AsyncOpenAI(
|
||
base_url=config.ai.base_url,
|
||
api_key=env.openrouter_api_key,
|
||
timeout=60.0, # Increase from default
|
||
...
|
||
)
|
||
```
|
||
|
||
### Issue: Rate Limit Errors
|
||
|
||
```
|
||
ERROR - Rate limit exceeded
|
||
```
|
||
|
||
**Solutions:**
|
||
|
||
1. **Use paid model** (recommended):
|
||
```yaml
|
||
ai:
|
||
model: "openai/gpt-4o-mini"
|
||
```
|
||
|
||
2. **Reduce batch size** in `src/ai/filter.py`:
|
||
```python
|
||
batch_size = 5 # Was 10
|
||
```
|
||
|
||
3. **Add delays between batches** (slower but avoids limits):
|
||
```python
|
||
for i in range(0, len(articles), batch_size):
|
||
batch = articles[i:i + batch_size]
|
||
# ... process batch ...
|
||
if i + batch_size < len(articles):
|
||
await asyncio.sleep(1) # Wait 1 second between batches
|
||
```
|
||
|
||
### Issue: Memory Usage Too High
|
||
|
||
**Symptoms:**
|
||
- System slowdown
|
||
- OOM errors
|
||
|
||
**Solutions:**
|
||
|
||
1. **Reduce batch size** (processes fewer at once):
|
||
```python
|
||
batch_size = 5 # Instead of 10
|
||
```
|
||
|
||
2. **Limit max articles**:
|
||
```yaml
|
||
ai:
|
||
filtering:
|
||
max_articles: 10 # Instead of 15
|
||
```
|
||
|
||
3. **Set resource limits in systemd**:
|
||
```ini
|
||
[Service]
|
||
MemoryLimit=512M
|
||
CPUQuota=50%
|
||
```
|
||
|
||
## Performance Tips
|
||
|
||
### 1. Use Paid Models
|
||
|
||
Free models have rate limits that slow everything down:
|
||
- ✅ **Paid**: Consistent 1-2 min processing
|
||
- ❌ **Free**: 5-10 min (or fails) due to rate limits
|
||
|
||
### 2. Adjust Filtering Threshold
|
||
|
||
Higher threshold = fewer articles = faster summarization:
|
||
```yaml
|
||
ai:
|
||
filtering:
|
||
min_score: 6.5 # Stricter = fewer articles = faster
|
||
```
|
||
|
||
### 3. Reduce Max Articles
|
||
|
||
```yaml
|
||
ai:
|
||
filtering:
|
||
max_articles: 10 # Instead of 15
|
||
```
|
||
|
||
Processing time is mainly in filtering (all articles), not summarization (filtered subset).
|
||
|
||
### 4. Remove Unnecessary RSS Sources
|
||
|
||
Fewer sources = fewer articles to process:
|
||
```yaml
|
||
sources:
|
||
rss:
|
||
# Comment out sources you don't need
|
||
# - name: "Source I don't read"
|
||
```
|
||
|
||
### 5. Run During Off-Peak Hours
|
||
|
||
Schedule for times when:
|
||
- Your internet is fastest
|
||
- OpenRouter has less load
|
||
- You're not using the machine
|
||
|
||
## Benchmarks
|
||
|
||
### Real-World Results (OpenAI GPT-4o-mini)
|
||
|
||
| Articles Fetched | Filtered | Summarized | Total Time |
|
||
|-----------------|----------|------------|------------|
|
||
| 45 | 8 | 8 | 45 seconds |
|
||
| 127 | 12 | 12 | 1 min 20 sec |
|
||
| 152 | 15 | 15 | 1 min 45 sec |
|
||
| 203 | 15 | 15 | 2 min 15 sec |
|
||
|
||
**Note:** Most time is spent on filtering (scoring all articles), not summarization (only filtered articles).
|
||
|
||
## Future Optimizations
|
||
|
||
Potential improvements (not yet implemented):
|
||
|
||
1. **Cache article scores** - Don't re-score articles that appear in multiple feeds
|
||
2. **Early stopping** - Stop filtering once we have enough high-scoring articles
|
||
3. **Smarter batching** - Adjust batch size based on API response times
|
||
4. **Parallel summarization** - Summarize while filtering is still running
|
||
5. **Local caching** - Cache API responses for duplicate articles
|
||
|
||
## Expected Performance Summary
|
||
|
||
**Typical daily run (150 articles, 15 selected):**
|
||
- ✅ **With optimizations**: 1-2 minutes
|
||
- ❌ **Without optimizations**: 5-7 minutes
|
||
|
||
**The optimizations make the system 3-5x faster!**
|
||
|
||
All async operations use `asyncio.gather()` with batching to maximize throughput while respecting API rate limits.
|