diff --git a/.gitignore b/.gitignore index 107b564..3c2ad4a 100644 --- a/.gitignore +++ b/.gitignore @@ -22,8 +22,9 @@ Pipfile.lock # Consider if you want to include or exclude ._* *~.nib *~.xib -README.md.old -oai.zip + +# Added by author +*.zip .note diagnose.py *.log @@ -33,4 +34,8 @@ build* compiled/ images/oai-iOS-Default-1024x1024@1x.png images/oai.icon/ -b0.sh \ No newline at end of file +b0.sh +*.bak +*.old +*.sh +*.back diff --git a/README.md b/README.md index 0c91bd6..a3c2e5d 100644 --- a/README.md +++ b/README.md @@ -1,53 +1,77 @@ # oAI - OpenRouter AI Chat -A terminal-based chat interface for OpenRouter API with conversation management, cost tracking, and rich formatting. +A powerful terminal-based chat interface for OpenRouter API with **MCP (Model Context Protocol)** support, enabling AI agents to access local files and query SQLite databases directly. ## Description -oAI is a command-line chat application that provides an interactive interface to OpenRouter's AI models. It features conversation persistence, file attachments, export capabilities, and detailed session metrics. +oAI is a feature-rich command-line chat application that provides an interactive interface to OpenRouter's AI models. It now includes **MCP integration** for local file system access and read-only database querying, allowing AI to help with code analysis, data exploration, and more. ## Features -- Interactive chat with multiple AI models via OpenRouter -- Model selection with search functionality -- Conversation save/load/export (Markdown, JSON, HTML) -- File attachment support (code files and images) -- Session cost tracking and credit monitoring -- Rich terminal formatting with syntax highlighting -- Persistent command history -- Configurable system prompts and token limits -- SQLite-based configuration and conversation storage +### Core Features +- 🤖 Interactive chat with 300+ AI models via OpenRouter +- 🔍 Model selection with search and capability filtering +- 💾 Conversation save/load/export (Markdown, JSON, HTML) +- 📎 File attachment support (images, PDFs, code files) +- 💰 Session cost tracking and credit monitoring +- 🎨 Rich terminal formatting with syntax highlighting +- 📝 Persistent command history with search (Ctrl+R) +- ⚙️ Configurable system prompts and token limits +- 🗄️ SQLite-based configuration and conversation storage +- 🌐 Online mode (web search capabilities) +- 🧠 Conversation memory toggle (save costs with stateless mode) + +### NEW: MCP (Model Context Protocol) v2.1.0-beta +- 🔧 **File Mode**: AI can read, search, and list your local files + - Automatic .gitignore filtering + - Virtual environment exclusion (venv, node_modules, etc.) + - Supports code files, text, JSON, YAML, and more + - Large file handling (auto-truncates >50KB) + +- 🗄️ **Database Mode**: AI can query your SQLite databases + - Read-only access (no data modification possible) + - Schema inspection (tables, columns, indexes) + - Full-text search across all tables + - SQL query execution (SELECT, JOINs, CTEs, subqueries) + - Query validation and timeout protection + - Result limiting (max 1000 rows) + +- 🔒 **Security Features**: + - Explicit folder/database approval required + - System directory blocking + - Read-only database access + - SQL injection protection + - Query timeout (5 seconds) ## Requirements -- Python 3.7 or higher +- Python 3.10-3.13 (3.14 not supported yet) - OpenRouter API key (get one at https://openrouter.ai) +- Function-calling model required for MCP features (GPT-4, Claude, etc.) -## Screenshot (from version 1.0) +## Screenshot [](https://gitlab.pm/rune/oai/src/branch/main/README.md) -Screenshot of `/help` screen. +*Screenshot from version 1.0 - MCP interface shows mode indicators like `[🔧 MCP: Files]` or `[🗄️ MCP: DB #1]`* ## Installation -### 1. Install Dependencies +### Option 1: From Source (Recommended for Development) -Use the included `requirements.txt` file to install the dependencies: +#### 1. Install Dependencies ```bash pip install -r requirements.txt ``` -### 2. Make the Script Executable +#### 2. Make Executable ```bash chmod +x oai.py ``` -### 3. Copy to PATH - -Copy the script to a directory in your `$PATH` environment variable. Common locations include: +#### 3. Copy to PATH ```bash # Option 1: System-wide (requires sudo) @@ -57,145 +81,408 @@ sudo cp oai.py /usr/local/bin/oai mkdir -p ~/.local/bin cp oai.py ~/.local/bin/oai -# Add to PATH if not already (add to ~/.bashrc or ~/.zshrc) +# Add to PATH if needed (add to ~/.bashrc or ~/.zshrc) export PATH="$HOME/.local/bin:$PATH" ``` -### 4. Verify Installation +#### 4. Verify Installation + +```bash +oai --version +``` + +### Option 2: Pre-built Binaries + +Download platform-specific binaries: +- **macOS (Apple Silicon)**: `oai_vx.x.x_mac_arm64.zip` +- **Linux (x86_64)**: `oai_vx.x.x-linux-x86_64.zip` + +```bash +# Extract and install +unzip oai_vx.x.x_mac_arm64.zip # or `oai_vx.x.x-linux-x86_64.zip` +chmod +x oai +mkdir -p ~/.local/bin +mv oai ~/.local/bin/ +``` + +### Option 3: Build Your Own Binary + +```bash +# Install build dependencies +pip install -r requirements.txt +pip install nuitka ordered-set zstandard + +# Run build script +chmod +x build.sh +./build.sh + +# Binary will be in dist/oai +cp dist/oai ~/.local/bin/ +``` + +### Alternative: Shell Alias + +```bash +# Add to ~/.bashrc or ~/.zshrc +alias oai='python3 /path/to/oai.py' +``` + +## Quick Start + +### First Run Setup ```bash oai ``` -### 5. Alternative Installation (for *nix systems) +On first run, you'll be prompted to enter your OpenRouter API key. -If you have issues with the above method you can add an alias in your `.bashrc`, `.zshrc` etc. - -```bash -alias oai='python3 ' -``` - -On first run, you will be prompted to enter your OpenRouter API key. - -### 6. Use Binaries - -You can also just download the supplied binary for either Mac wit Mx (M1, M2 etc) `oai_mac_arm64.zip` and follow [#3](https://gitlab.pm/rune/oai#3-copy-to-path). Or download for Linux (64bit) `oai_linux_x86_64.zip` and also follow [#3](https://gitlab.pm/rune/oai#3-copy-to-path). - -## Usage - -### Starting the Application +### Basic Usage ```bash +# Start chatting oai + +# Select a model +You> /model + +# Enable MCP for file access +You> /mcp enable +You> /mcp add ~/Documents + +# Ask AI to help with files +[🔧 MCP: Files] You> List all Python files in Documents +[🔧 MCP: Files] You> Read and explain main.py + +# Switch to database mode +You> /mcp add db ~/myapp/data.db +You> /mcp db 1 +[🗄️ MCP: DB #1] You> Show me all tables +[🗄️ MCP: DB #1] You> Find all users created this month ``` -### Basic Commands +## MCP Guide -``` -/help Show all available commands -/model Select an AI model -/config api Set OpenRouter API key -exit Quit the application +### File Mode (Default) + +**Setup:** +```bash +/mcp enable # Start MCP server +/mcp add ~/Projects # Grant access to folder +/mcp add ~/Documents # Add another folder +/mcp list # View all allowed folders ``` -### Configuration - -All configuration is stored in `~/.config/oai/`: -- `oai_config.db` - SQLite database for settings and conversations -- `oai.log` - Application log file -- `history.txt` - Command history - -### Common Workflows - -**Select a Model:** +**Natural Language Usage:** ``` -/model +"List all Python files in Projects" +"Read and explain config.yaml" +"Search for files containing 'TODO'" +"What's in my Documents folder?" ``` -**Paste from clipboard:** -Paste and send content to model -``` -/paste +**Available Tools:** +- `read_file` - Read complete file contents +- `list_directory` - List files/folders (recursive optional) +- `search_files` - Search by name or content + +**Features:** +- ✅ Automatic .gitignore filtering +- ✅ Skips virtual environments (venv, node_modules) +- ✅ Handles large files (auto-truncates >50KB) +- ✅ Cross-platform (macOS, Linux, Windows via WSL) + +### Database Mode + +**Setup:** +```bash +/mcp add db ~/app/database.db # Add SQLite database +/mcp db list # View all databases +/mcp db 1 # Switch to database #1 ``` -Paste with prompt and send content to model +**Natural Language Usage:** ``` -/paste Analyze this text +"Show me all tables in this database" +"Find records mentioning 'error'" +"How many users registered last week?" +"Get the schema for the orders table" +"Show me the 10 most recent transactions" ``` -**Start Chatting:** -``` -You> Hello, how are you? -``` +**Available Tools:** +- `inspect_database` - View schema, tables, columns, indexes +- `search_database` - Full-text search across tables +- `query_database` - Execute read-only SQL queries -**Attach Files:** -``` -You> Debug this code @/path/to/script.py -You> Analyze this image @/path/to/image.png -``` +**Supported Queries:** +- ✅ SELECT statements +- ✅ JOINs (INNER, LEFT, RIGHT, FULL) +- ✅ Subqueries +- ✅ CTEs (Common Table Expressions) +- ✅ Aggregations (COUNT, SUM, AVG, etc.) +- ✅ WHERE, GROUP BY, HAVING, ORDER BY, LIMIT +- ❌ INSERT/UPDATE/DELETE (blocked for safety) -**Save Conversation:** -``` -/save my_conversation -``` +### Mode Management -**Export to File:** +```bash +/mcp status # Show current mode, stats, folders/databases +/mcp files # Switch to file mode +/mcp db # Switch to database mode +/mcp gitignore on # Enable .gitignore filtering (default) +/mcp remove 2 # Remove folder/database by number ``` -/export md notes.md -/export json backup.json -/export html report.html -``` - -**View Session Stats:** - -``` -/stats -/credits -``` - -**Prevous commands input:** - -Use the up/down arrows to see earlier `/`commands and earlier input to model and `` to execute the same command or resend the same input. ## Command Reference -Use `/help` within the application for a complete command reference organized by category: -- Session Commands -- Model Commands -- Configuration -- Token & System -- Conversation Management -- Monitoring & Stats -- File Attachments +### Session Commands +``` +/help [command] Show help menu or detailed command help +/help mcp Comprehensive MCP guide +/clear or /cl Clear terminal screen (or Ctrl+L) +/memory on|off Toggle conversation memory (save costs) +/online on|off Enable/disable web search +/paste [prompt] Paste clipboard content +/retry Resend last prompt +/reset Clear history and system prompt +/prev View previous response +/next View next response +``` + +### MCP Commands +``` +/mcp enable Start MCP server +/mcp disable Stop MCP server +/mcp status Show comprehensive status +/mcp add Add folder for file access +/mcp add db Add SQLite database +/mcp list List all folders +/mcp db list List all databases +/mcp db Switch to database mode +/mcp files Switch to file mode +/mcp remove Remove folder/database +/mcp gitignore on Enable .gitignore filtering +``` + +### Model Commands +``` +/model [search] Select/change AI model +/info [model_id] Show model details (pricing, capabilities) +``` + +### Configuration +``` +/config View all settings +/config api Set API key +/config model Set default model +/config online Set default online mode (on|off) +/config stream Enable/disable streaming (on|off) +/config maxtoken Set max token limit +/config costwarning Set cost warning threshold ($) +/config loglevel Set log level (debug/info/warning/error) +/config log Set log file size (MB) +``` + +### Conversation Management +``` +/save Save conversation +/load Load saved conversation +/delete Delete conversation +/list List saved conversations +/export md|json|html Export conversation +``` + +### Token & System +``` +/maxtoken [value] Set session token limit +/system [prompt] Set system prompt (use 'clear' to reset) +/middleout on|off Enable prompt compression +``` + +### Monitoring +``` +/stats View session statistics +/credits Check OpenRouter credits +``` + +### File Attachments +``` +@/path/to/file Attach file (images, PDFs, code) + +Examples: + Debug @script.py + Analyze @data.json + Review @screenshot.png +``` ## Configuration Options -- API Key: `/config api` -- Base URL: `/config url` -- Streaming: `/config stream on|off` -- Default Model: `/config model` -- Cost Warning: `/config costwarning ` -- Max Token Limit: `/config maxtoken ` +All configuration stored in `~/.config/oai/`: -## File Support +### Files +- `oai_config.db` - SQLite database (settings, conversations, MCP config) +- `oai.log` - Application logs (rotating, configurable size) +- `history.txt` - Command history (searchable with Ctrl+R) -**Supported Code Extensions:** -.py, .js, .ts, .cs, .java, .c, .cpp, .h, .hpp, .rb, .ruby, .php, .swift, .kt, .kts, .go, .sh, .bat, .ps1, .R, .scala, .pl, .lua, .dart, .elm, .xml, .json, .yaml, .yml, .md, .txt +### Key Settings +- **API Key**: OpenRouter authentication +- **Default Model**: Auto-select on startup +- **Streaming**: Real-time response display +- **Max Tokens**: Global and session limits +- **Cost Warning**: Alert threshold for expensive requests +- **Online Mode**: Default web search setting +- **Log Level**: debug/info/warning/error/critical +- **Log Size**: Rotating file size in MB -**Image Support:** -Any image format with proper MIME type (PNG, JPEG, GIF, etc.) +## Supported File Types -## Data Storage +### Code Files +`.py, .js, .ts, .cs, .java, .c, .cpp, .h, .hpp, .rb, .ruby, .php, .swift, .kt, .kts, .go, .sh, .bat, .ps1, .R, .scala, .pl, .lua, .dart, .elm` -- Configuration: `~/.config/oai/oai_config.db` -- Logs: `~/.config/oai/oai.log` -- History: `~/.config/oai/history.txt` +### Data Files +`.json, .yaml, .yml, .xml, .csv, .txt, .md` + +### Images +All standard formats: PNG, JPEG, JPG, GIF, WEBP, BMP + +### Documents +PDF (models with document support) + +### Size Limits +- Images: 10 MB max +- Code/Text: Auto-truncates files >50KB +- Binary data: Displayed as `` + +## MCP Security + +### Access Control +- ✅ Explicit folder/database approval required +- ✅ System directories blocked automatically +- ✅ User confirmation for each addition +- ✅ .gitignore patterns respected (file mode) + +### Database Safety +- ✅ Read-only mode (cannot modify data) +- ✅ SQL query validation (blocks INSERT/UPDATE/DELETE) +- ✅ Query timeout (5 seconds max) +- ✅ Result limits (1000 rows max) +- ✅ Database opened in `mode=ro` + +### File System Safety +- ✅ Read-only access (no write/delete) +- ✅ Virtual environment exclusion +- ✅ Build artifact filtering +- ✅ Maximum file size (10 MB) + +## Tips & Tricks + +### Command History +- **↑/↓ arrows**: Navigate previous commands +- **Ctrl+R**: Search command history +- **Auto-complete**: Start typing `/` for command suggestions + +### Cost Optimization +```bash +/memory off # Disable context (stateless mode) +/maxtoken 1000 # Limit response length +/config costwarning 0.01 # Set alert threshold +``` + +### MCP Best Practices +```bash +# Check status frequently +/mcp status + +# Use specific paths to reduce search time +"List Python files in Projects/app/" # Better than +"List all Python files" # Slower + +# Database queries - be specific +"SELECT * FROM users LIMIT 10" # Good +"SELECT * FROM users" # May hit row limit +``` + +### Debugging +```bash +# Enable debug logging +/config loglevel debug + +# Check log file +tail -f ~/.config/oai/oai.log + +# View MCP statistics +/mcp status # Shows tool call counts +``` + +## Troubleshooting + +### MCP Not Working +```bash +# 1. Check if MCP is installed +python3 -c "import mcp; print('MCP OK')" + +# 2. Verify model supports function calling +/info # Look for "tools" in supported parameters + +# 3. Check MCP status +/mcp status + +# 4. Review logs +tail ~/.config/oai/oai.log +``` + +### Import Errors +```bash +# Reinstall dependencies +pip install --force-reinstall -r requirements.txt +``` + +### Binary Issues (macOS) +```bash +# Remove quarantine +xattr -cr ~/.local/bin/oai + +# Check security settings +# System Settings > Privacy & Security > "Allow Anyway" +``` + +### Database Errors +```bash +# Verify it's a valid SQLite database +sqlite3 database.db ".tables" + +# Check file permissions +ls -la database.db +``` + +## Version History + +### v2.1.0-beta (Current) +- ✨ **NEW**: MCP (Model Context Protocol) integration +- ✨ **NEW**: File system access (read, search, list) +- ✨ **NEW**: SQLite database querying (read-only) +- ✨ **NEW**: Dual mode support (Files & Database) +- ✨ **NEW**: .gitignore filtering +- ✨ **NEW**: Binary data handling in databases +- ✨ **NEW**: Mode indicators in prompt +- ✨ **NEW**: Comprehensive `/help mcp` guide +- 🔧 Improved error handling for tool calls +- 🔧 Enhanced logging for MCP operations +- 🔧 Statistics tracking for tool usage + +### v1.9.6 +- Base version with core chat functionality +- Conversation management +- File attachments +- Cost tracking +- Export capabilities ## License MIT License -Copyright (c) 2024 Rune Olsen +Copyright (c) 2024-2025 Rune Olsen Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -221,12 +508,22 @@ Full license: https://opensource.org/licenses/MIT **Rune Olsen** -Blog: https://blog.rune.pm +- Blog: https://blog.rune.pm +- Project: https://iurl.no/oai -## Version +## Contributing -1.0 +Contributions welcome! Please: +1. Fork the repository +2. Create a feature branch +3. Submit a pull request with detailed description -## Support +## Acknowledgments -For issues, questions, or contributions, visit https://iurl.no/oai and create an issue. \ No newline at end of file +- OpenRouter team for the unified AI API +- Rich library for beautiful terminal output +- MCP community for the protocol specification + +--- + +**Star ⭐ this project if you find it useful!** diff --git a/oai.py b/oai.py index 26b92ef..e679ce5 100644 --- a/oai.py +++ b/oai.py @@ -2,7 +2,8 @@ import sys import os import requests -import time # For response time tracking +import time +import asyncio from pathlib import Path from typing import Optional, List, Dict, Any import typer @@ -21,20 +22,34 @@ import sqlite3 import json import datetime import logging -from logging.handlers import RotatingFileHandler # Added for log rotation +from logging.handlers import RotatingFileHandler from prompt_toolkit import PromptSession from prompt_toolkit.history import FileHistory from rich.logging import RichHandler from prompt_toolkit.auto_suggest import AutoSuggestFromHistory from packaging import version as pkg_version -import io # Added for custom handler +import io +import platform +import shutil +import subprocess +import fnmatch +import signal -# App version. Changes by author with new releases. -version = '1.9.6' +# MCP imports +try: + from mcp import ClientSession, StdioServerParameters + from mcp.client.stdio import stdio_client + MCP_AVAILABLE = True +except ImportError: + MCP_AVAILABLE = False + print("Warning: MCP library not found. Install with: pip install mcp") + +# App version +version = '2.1.0-beta' app = typer.Typer() -# Application identification for OpenRouter +# Application identification APP_NAME = "oAI" APP_URL = "https://iurl.no/oai" @@ -42,25 +57,26 @@ APP_URL = "https://iurl.no/oai" home = Path.home() config_dir = home / '.config' / 'oai' cache_dir = home / '.cache' / 'oai' -history_file = config_dir / 'history.txt' # Persistent input history file +history_file = config_dir / 'history.txt' database = config_dir / 'oai_config.db' log_file = config_dir / 'oai.log' -# Create dirs if needed +# Create dirs config_dir.mkdir(parents=True, exist_ok=True) cache_dir.mkdir(parents=True, exist_ok=True) -# Rich console for chat UI (separate from logging) +# Rich console console = Console() -# Valid commands list for validation +# Valid commands VALID_COMMANDS = { '/retry', '/online', '/memory', '/paste', '/export', '/save', '/load', '/delete', '/list', '/prev', '/next', '/stats', '/middleout', '/reset', - '/info', '/model', '/maxtoken', '/system', '/config', '/credits', '/clear', '/cl', '/help' + '/info', '/model', '/maxtoken', '/system', '/config', '/credits', '/clear', + '/cl', '/help', '/mcp' } -# Detailed command help database +# Command help database (COMPLETE - includes MCP comprehensive guide) COMMAND_HELP = { '/clear': { 'aliases': ['/cl'], @@ -74,13 +90,155 @@ COMMAND_HELP = { }, '/help': { 'description': 'Display help information for commands.', - 'usage': '/help [command]', + 'usage': '/help [command|topic]', 'examples': [ ('Show all commands', '/help'), ('Get help for a specific command', '/help /model'), - ('Get help for config', '/help /config'), + ('Get detailed MCP help', '/help mcp'), ], - 'notes': 'Use /help without arguments to see the full command list, or /help for detailed information about a specific command.' + 'notes': 'Use /help without arguments to see the full command list, /help for detailed command info, or /help mcp for comprehensive MCP documentation.' + }, + 'mcp': { + 'description': 'Complete guide to MCP (Model Context Protocol) - file access and database querying for AI agents.', + 'usage': 'See detailed examples below', + 'examples': [], + 'notes': ''' +MCP (Model Context Protocol) gives your AI assistant direct access to: + • Local files and folders (read, search, list) + • SQLite databases (inspect, search, query) + +╭─────────────────────────────────────────────────────────╮ +│ 🗂️ FILE MODE │ +╰─────────────────────────────────────────────────────────╯ + +SETUP: + /mcp enable Start MCP server + /mcp add ~/Documents Grant access to folder + /mcp add ~/Code/project Add another folder + /mcp list View all allowed folders + +USAGE (just ask naturally): + "List all Python files in my Code folder" + "Read the contents of Documents/notes.txt" + "Search for files containing 'budget'" + "What's in my Documents folder?" + +FEATURES: + ✓ Automatically respects .gitignore patterns + ✓ Skips virtual environments (venv, node_modules, etc.) + ✓ Handles large files (auto-truncates >50KB) + ✓ Cross-platform (macOS, Linux, Windows) + +MANAGEMENT: + /mcp remove ~/Desktop Remove folder access + /mcp remove 2 Remove by number + /mcp gitignore on|off Toggle .gitignore filtering + /mcp status Show comprehensive status + +╭─────────────────────────────────────────────────────────╮ +│ 🗄️ DATABASE MODE │ +╰─────────────────────────────────────────────────────────╯ + +SETUP: + /mcp add db ~/app/data.db Add specific database + /mcp add db ~/.config/oai/oai_config.db + /mcp db list View all databases + +SWITCH TO DATABASE: + /mcp db 1 Work with database #1 + /mcp db 2 Switch to database #2 + /mcp files Switch back to file mode + +USAGE (after selecting a database): + "Show me all tables in this database" + "Find any records mentioning 'Santa Clause'" + "How many users are in the database?" + "Show me the schema for the orders table" + "Get the 10 most recent transactions" + +FEATURES: + ✓ Read-only mode (no data modification possible) + ✓ Smart schema inspection + ✓ Full-text search across all tables + ✓ SQL query execution (SELECT only) + ✓ JOINs, subqueries, CTEs supported + ✓ Automatic query timeout (5 seconds) + ✓ Result limits (max 1000 rows) + +SAFETY: + • All queries are read-only (INSERT/UPDATE/DELETE blocked) + • Database opened in read-only mode + • Query validation prevents dangerous operations + • Timeout protection prevents infinite loops + +╭─────────────────────────────────────────────────────────╮ +│ 💡 TIPS & TRICKS │ +╰─────────────────────────────────────────────────────────╯ + +MODE INDICATORS: + [🔧 MCP: Files] You're in file mode + [🗄️ MCP: DB #1] You're querying database #1 + +QUICK REFERENCE: + /mcp status See current mode, stats, folders/databases + /mcp files Switch to file mode (default) + /mcp db Switch to database mode + /mcp db list List all databases with details + /mcp list List all folders + +TROUBLESHOOTING: + • No results? Check /mcp status to see what's accessible + • Wrong mode? Use /mcp files or /mcp db to switch + • Database errors? Ensure file exists and is valid SQLite + • .gitignore not working? Check /mcp status shows patterns + +SECURITY NOTES: + • MCP only accesses explicitly added folders/databases + • File mode: read-only access (no write/delete) + • Database mode: SELECT queries only (no modifications) + • System directories are blocked automatically + • Each addition requires your explicit confirmation + +For command-specific help: /help /mcp + ''' + }, + '/mcp': { + 'description': 'Manage MCP (Model Context Protocol) for local file access and SQLite database querying. When enabled with a function-calling model, AI can automatically search, read, list files, and query databases. Supports two modes: Files (default) and Database.', + 'usage': '/mcp [args]', + 'examples': [ + ('Enable MCP server', '/mcp enable'), + ('Disable MCP server', '/mcp disable'), + ('Show MCP status and current mode', '/mcp status'), + ('', ''), + ('━━━ FILE MODE ━━━', ''), + ('Add folder for file access', '/mcp add ~/Documents'), + ('Remove folder by path', '/mcp remove ~/Desktop'), + ('Remove folder by number', '/mcp remove 2'), + ('List allowed folders', '/mcp list'), + ('Toggle .gitignore filtering', '/mcp gitignore on'), + ('', ''), + ('━━━ DATABASE MODE ━━━', ''), + ('Add SQLite database', '/mcp add db ~/app/data.db'), + ('List all databases', '/mcp db list'), + ('Switch to database #1', '/mcp db 1'), + ('Remove database', '/mcp remove db 2'), + ('Switch back to file mode', '/mcp files'), + ], + 'notes': '''MCP allows AI to read local files and query SQLite databases. Works automatically with function-calling models (GPT-4, Claude, etc.). + +FILE MODE (default): +- Automatically loads and respects .gitignore patterns +- Skips virtual environments and build artifacts +- Supports search, read, and list operations + +DATABASE MODE: +- Read-only access (no data modification) +- Execute SELECT queries with JOINs, subqueries, CTEs +- Full-text search across all tables +- Schema inspection and data exploration +- Automatic query validation and timeout protection + +Use /help mcp for comprehensive guide with examples.''' }, '/memory': { 'description': 'Toggle conversation memory. When ON, the AI remembers conversation history. When OFF, each request is independent (saves tokens and cost).', @@ -272,7 +430,7 @@ COMMAND_HELP = { }, } -# Supported code file extensions +# Supported code extensions SUPPORTED_CODE_EXTENSIONS = { '.py', '.js', '.ts', '.cs', '.java', '.c', '.cpp', '.h', '.hpp', '.rb', '.ruby', '.php', '.swift', '.kt', '.kts', '.go', @@ -280,16 +438,16 @@ SUPPORTED_CODE_EXTENSIONS = { '.elm', '.xml', '.json', '.yaml', '.yml', '.md', '.txt' } -# Session metrics constants (per 1M tokens, in USD; adjustable) +# Pricing MODEL_PRICING = { - 'input': 3.0, # $3/M input tokens (adjustable) - 'output': 15.0 # $15/M output tokens (adjustable) + 'input': 3.0, + 'output': 15.0 } -LOW_CREDIT_RATIO = 0.1 # Warn if credits left < 10% of total -LOW_CREDIT_AMOUNT = 1.0 # Warn if credits left < $1 in absolute terms -HIGH_COST_WARNING = "cost_warning_threshold" # Configurable key for cost threshold, default $0.01 +LOW_CREDIT_RATIO = 0.1 +LOW_CREDIT_AMOUNT = 1.0 +HIGH_COST_WARNING = "cost_warning_threshold" -# Valid log levels mapping +# Valid log levels VALID_LOG_LEVELS = { 'debug': logging.DEBUG, 'info': logging.INFO, @@ -298,14 +456,18 @@ VALID_LOG_LEVELS = { 'critical': logging.CRITICAL } -# DB configuration -database = config_dir / 'oai_config.db' -DB_FILE = str(database) +# System directories to block (cross-platform) +SYSTEM_DIRS_BLACKLIST = { + '/System', '/Library', '/private', '/usr', '/bin', '/sbin', # macOS + '/boot', '/dev', '/proc', '/sys', '/root', # Linux + 'C:\\Windows', 'C:\\Program Files', 'C:\\Program Files (x86)' # Windows +} +# Database functions def create_table_if_not_exists(): - """Ensure the config and conversation_sessions tables exist.""" + """Ensure tables exist.""" os.makedirs(config_dir, exist_ok=True) - with sqlite3.connect(DB_FILE) as conn: + with sqlite3.connect(str(database)) as conn: conn.execute('''CREATE TABLE IF NOT EXISTS config ( key TEXT PRIMARY KEY, value TEXT NOT NULL @@ -314,32 +476,106 @@ def create_table_if_not_exists(): id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, timestamp TEXT NOT NULL, - data TEXT NOT NULL -- JSON of session_history + data TEXT NOT NULL + )''') + # MCP configuration table + conn.execute('''CREATE TABLE IF NOT EXISTS mcp_config ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL + )''') + # MCP statistics table + conn.execute('''CREATE TABLE IF NOT EXISTS mcp_stats ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + timestamp TEXT NOT NULL, + tool_name TEXT NOT NULL, + folder TEXT, + success INTEGER NOT NULL, + error_message TEXT + )''') + # MCP databases table + conn.execute('''CREATE TABLE IF NOT EXISTS mcp_databases ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + path TEXT NOT NULL UNIQUE, + name TEXT NOT NULL, + size INTEGER, + tables TEXT, + added_timestamp TEXT NOT NULL )''') conn.commit() def get_config(key: str) -> Optional[str]: create_table_if_not_exists() - with sqlite3.connect(DB_FILE) as conn: + with sqlite3.connect(str(database)) as conn: cursor = conn.execute('SELECT value FROM config WHERE key = ?', (key,)) result = cursor.fetchone() return result[0] if result else None def set_config(key: str, value: str): create_table_if_not_exists() - with sqlite3.connect(DB_FILE) as conn: + with sqlite3.connect(str(database)) as conn: conn.execute('INSERT OR REPLACE INTO config (key, value) VALUES (?, ?)', (key, value)) conn.commit() -# ============================================================================ -# ROTATING RICH HANDLER - Combines RotatingFileHandler with Rich formatting -# ============================================================================ +def get_mcp_config(key: str) -> Optional[str]: + """Get MCP configuration value.""" + create_table_if_not_exists() + with sqlite3.connect(str(database)) as conn: + cursor = conn.execute('SELECT value FROM mcp_config WHERE key = ?', (key,)) + result = cursor.fetchone() + return result[0] if result else None + +def set_mcp_config(key: str, value: str): + """Set MCP configuration value.""" + create_table_if_not_exists() + with sqlite3.connect(str(database)) as conn: + conn.execute('INSERT OR REPLACE INTO mcp_config (key, value) VALUES (?, ?)', (key, value)) + conn.commit() + +def log_mcp_stat(tool_name: str, folder: Optional[str], success: bool, error_message: Optional[str] = None): + """Log MCP tool usage statistics.""" + create_table_if_not_exists() + timestamp = datetime.datetime.now().isoformat() + with sqlite3.connect(str(database)) as conn: + conn.execute( + 'INSERT INTO mcp_stats (timestamp, tool_name, folder, success, error_message) VALUES (?, ?, ?, ?, ?)', + (timestamp, tool_name, folder, 1 if success else 0, error_message) + ) + conn.commit() + +def get_mcp_stats() -> Dict[str, Any]: + """Get MCP usage statistics.""" + create_table_if_not_exists() + with sqlite3.connect(str(database)) as conn: + cursor = conn.execute(''' + SELECT + COUNT(*) as total_calls, + SUM(CASE WHEN tool_name = 'read_file' THEN 1 ELSE 0 END) as reads, + SUM(CASE WHEN tool_name = 'list_directory' THEN 1 ELSE 0 END) as lists, + SUM(CASE WHEN tool_name = 'search_files' THEN 1 ELSE 0 END) as searches, + SUM(CASE WHEN tool_name = 'inspect_database' THEN 1 ELSE 0 END) as db_inspects, + SUM(CASE WHEN tool_name = 'search_database' THEN 1 ELSE 0 END) as db_searches, + SUM(CASE WHEN tool_name = 'query_database' THEN 1 ELSE 0 END) as db_queries, + MAX(timestamp) as last_used + FROM mcp_stats + ''') + row = cursor.fetchone() + return { + 'total_calls': row[0] or 0, + 'reads': row[1] or 0, + 'lists': row[2] or 0, + 'searches': row[3] or 0, + 'db_inspects': row[4] or 0, + 'db_searches': row[5] or 0, + 'db_queries': row[6] or 0, + 'last_used': row[7] + } + +# Rotating Rich Handler class RotatingRichHandler(RotatingFileHandler): - """Custom handler that combines RotatingFileHandler with Rich formatting.""" + """Custom handler combining RotatingFileHandler with Rich formatting.""" def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) - # Create a Rich console that writes to a string buffer self.rich_console = Console(file=io.StringIO(), width=120, force_terminal=False) self.rich_handler = RichHandler( console=self.rich_console, @@ -351,46 +587,31 @@ class RotatingRichHandler(RotatingFileHandler): def emit(self, record): try: - # Let RichHandler format the record self.rich_handler.emit(record) - - # Get the formatted output from the string buffer output = self.rich_console.file.getvalue() - - # Clear the buffer for next use self.rich_console.file.seek(0) self.rich_console.file.truncate(0) - - # Write the Rich-formatted output to our rotating file if output: self.stream.write(output) self.flush() - except Exception: self.handleError(record) -# ============================================================================ -# LOGGING SETUP - MUST BE DONE AFTER CONFIG IS LOADED -# ============================================================================ - -# Load log configuration from DB FIRST (before creating handler) +# Logging setup LOG_MAX_SIZE_MB = int(get_config('log_max_size_mb') or "10") LOG_BACKUP_COUNT = int(get_config('log_backup_count') or "2") LOG_LEVEL_STR = get_config('log_level') or "info" LOG_LEVEL = VALID_LOG_LEVELS.get(LOG_LEVEL_STR.lower(), logging.INFO) -# Global reference to the handler for dynamic reloading app_handler = None app_logger = None def setup_logging(): - """Setup or reset logging configuration with current settings.""" + """Setup or reset logging configuration.""" global app_handler, LOG_MAX_SIZE_MB, LOG_BACKUP_COUNT, LOG_LEVEL, app_logger - # Get the root logger root_logger = logging.getLogger() - # Remove existing handler if present if app_handler is not None: root_logger.removeHandler(app_handler) try: @@ -398,13 +619,12 @@ def setup_logging(): except: pass - # Check if log file needs immediate rotation + # Check if log needs rotation if os.path.exists(log_file): current_size = os.path.getsize(log_file) max_bytes = LOG_MAX_SIZE_MB * 1024 * 1024 if current_size >= max_bytes: - # Perform immediate rotation import shutil timestamp = datetime.datetime.now().strftime('%Y%m%d_%H%M%S') backup_file = f"{log_file}.{timestamp}" @@ -413,7 +633,7 @@ def setup_logging(): except Exception as e: print(f"Warning: Could not rotate log file: {e}") - # Clean up old backups if exceeding limit + # Clean old backups log_dir = os.path.dirname(log_file) log_basename = os.path.basename(log_file) backup_pattern = f"{log_basename}.*" @@ -421,7 +641,6 @@ def setup_logging(): import glob backups = sorted(glob.glob(os.path.join(log_dir, backup_pattern))) - # Keep only the most recent backups while len(backups) > LOG_BACKUP_COUNT: oldest = backups.pop(0) try: @@ -429,7 +648,6 @@ def setup_logging(): except: pass - # Create new handler with current settings app_handler = RotatingRichHandler( filename=str(log_file), maxBytes=LOG_MAX_SIZE_MB * 1024 * 1024, @@ -437,15 +655,11 @@ def setup_logging(): encoding='utf-8' ) - # Set handler level to NOTSET so it processes all records app_handler.setLevel(logging.NOTSET) - - # Configure root logger - set to WARNING to suppress third-party library noise root_logger.setLevel(logging.WARNING) root_logger.addHandler(app_handler) - # Suppress noisy third-party loggers - # These libraries create DEBUG logs that pollute our log file + # Suppress noisy loggers logging.getLogger('asyncio').setLevel(logging.WARNING) logging.getLogger('urllib3').setLevel(logging.WARNING) logging.getLogger('requests').setLevel(logging.WARNING) @@ -454,19 +668,14 @@ def setup_logging(): logging.getLogger('openai').setLevel(logging.WARNING) logging.getLogger('openrouter').setLevel(logging.WARNING) - # Get or create app logger and set its level (this filters what gets logged) app_logger = logging.getLogger("oai_app") app_logger.setLevel(LOG_LEVEL) - # Don't propagate to avoid root logger filtering app_logger.propagate = True return app_logger -# Initial logging setup -app_logger = setup_logging() - def set_log_level(level_str: str) -> bool: - """Set the application log level. Returns True if successful.""" + """Set application log level.""" global LOG_LEVEL, LOG_LEVEL_STR, app_logger level_str_lower = level_str.lower() if level_str_lower not in VALID_LOG_LEVELS: @@ -474,40 +683,1911 @@ def set_log_level(level_str: str) -> bool: LOG_LEVEL = VALID_LOG_LEVELS[level_str_lower] LOG_LEVEL_STR = level_str_lower - # Update the logger level immediately if app_logger: app_logger.setLevel(LOG_LEVEL) return True def reload_logging_config(): - """Reload logging configuration from database and reinitialize handler.""" + """Reload logging configuration.""" global LOG_MAX_SIZE_MB, LOG_BACKUP_COUNT, LOG_LEVEL, LOG_LEVEL_STR, app_logger - # Reload from database LOG_MAX_SIZE_MB = int(get_config('log_max_size_mb') or "10") LOG_BACKUP_COUNT = int(get_config('log_backup_count') or "2") LOG_LEVEL_STR = get_config('log_level') or "info" LOG_LEVEL = VALID_LOG_LEVELS.get(LOG_LEVEL_STR.lower(), logging.INFO) - # Reinitialize logging app_logger = setup_logging() - return app_logger -# ============================================================================ -# END OF LOGGING SETUP -# ============================================================================ - +app_logger = setup_logging() logger = logging.getLogger(__name__) -def check_for_updates(current_version: str) -> str: - """ - Check if a new version is available using semantic versioning. +# Load configs +API_KEY = get_config('api_key') +OPENROUTER_BASE_URL = get_config('base_url') or "https://openrouter.ai/api/v1" +STREAM_ENABLED = get_config('stream_enabled') or "on" +DEFAULT_MODEL_ID = get_config('default_model') +MAX_TOKEN = int(get_config('max_token') or "100000") +COST_WARNING_THRESHOLD = float(get_config(HIGH_COST_WARNING) or "0.01") +DEFAULT_ONLINE_MODE = get_config('default_online_mode') or "off" + +# Fetch models +models_data = [] +text_models = [] +try: + headers = { + "Authorization": f"Bearer {API_KEY}", + "HTTP-Referer": APP_URL, + "X-Title": APP_NAME + } if API_KEY else { + "HTTP-Referer": APP_URL, + "X-Title": APP_NAME + } + response = requests.get(f"{OPENROUTER_BASE_URL}/models", headers=headers) + response.raise_for_status() + models_data = response.json()["data"] + text_models = [m for m in models_data if "modalities" not in m or "video" not in (m.get("modalities") or [])] + selected_model_default = None + if DEFAULT_MODEL_ID: + selected_model_default = next((m for m in text_models if m["id"] == DEFAULT_MODEL_ID), None) + if not selected_model_default: + console.print(f"[bold yellow]Warning: Default model '{DEFAULT_MODEL_ID}' unavailable. Use '/config model'.[/]") +except Exception as e: + models_data = [] + text_models = [] + app_logger.error(f"Failed to fetch models: {e}") + +# ============================================================================ +# MCP INTEGRATION CLASSES +# ============================================================================ + +class CrossPlatformMCPConfig: + """Handle OS-specific MCP configuration.""" - Returns: - Formatted status string for display - """ + def __init__(self): + self.system = platform.system() + self.is_macos = self.system == "Darwin" + self.is_linux = self.system == "Linux" + self.is_windows = self.system == "Windows" + + app_logger.info(f"Detected OS: {self.system}") + + def get_default_allowed_dirs(self) -> List[Path]: + """Get safe default directories.""" + home = Path.home() + + if self.is_macos: + return [ + home / "Documents", + home / "Desktop", + home / "Downloads" + ] + elif self.is_linux: + dirs = [home / "Documents"] + + try: + for xdg_dir in ["DOCUMENTS", "DESKTOP", "DOWNLOAD"]: + result = subprocess.run( + ["xdg-user-dir", xdg_dir], + capture_output=True, + text=True, + timeout=1 + ) + if result.returncode == 0: + dir_path = Path(result.stdout.strip()) + if dir_path.exists(): + dirs.append(dir_path) + except: + dirs.extend([ + home / "Desktop", + home / "Downloads" + ]) + + return list(set(dirs)) + + elif self.is_windows: + return [ + home / "Documents", + home / "Desktop", + home / "Downloads" + ] + + return [home] + + def get_python_command(self) -> str: + """Get Python command.""" + import sys + return sys.executable + + def get_filesystem_warning(self) -> str: + """Get OS-specific security warning.""" + if self.is_macos: + return """ +⚠️ macOS Security Notice: +The Filesystem MCP server needs access to your selected folder. +You may see a security prompt - click 'Allow' to proceed. +(System Settings > Privacy & Security > Files and Folders) +""" + elif self.is_linux: + return """ +⚠️ Linux Security Notice: +The Filesystem MCP server will access your selected folder. +Ensure oAI has appropriate file permissions. +""" + elif self.is_windows: + return """ +⚠️ Windows Security Notice: +The Filesystem MCP server will access your selected folder. +You may need to grant file access permissions. +""" + return "" + + def normalize_path(self, path: str) -> Path: + """Normalize path for current OS.""" + return Path(os.path.expanduser(path)).resolve() + + def is_system_directory(self, path: Path) -> bool: + """Check if path is a system directory.""" + path_str = str(path) + for blocked in SYSTEM_DIRS_BLACKLIST: + if path_str.startswith(blocked): + return True + return False + + def is_safe_path(self, requested_path: Path, allowed_dirs: List[Path]) -> bool: + """Check if path is within allowed directories.""" + try: + requested = requested_path.resolve() + + for allowed in allowed_dirs: + try: + allowed_resolved = allowed.resolve() + requested.relative_to(allowed_resolved) + return True + except ValueError: + continue + + return False + except: + return False + + def get_folder_stats(self, folder: Path) -> Dict[str, Any]: + """Get folder statistics.""" + try: + if not folder.exists() or not folder.is_dir(): + return {'exists': False} + + file_count = 0 + total_size = 0 + + for item in folder.rglob('*'): + if item.is_file(): + file_count += 1 + try: + total_size += item.stat().st_size + except: + pass + + return { + 'exists': True, + 'file_count': file_count, + 'total_size': total_size, + 'size_mb': total_size / (1024 * 1024) + } + except Exception as e: + app_logger.error(f"Error getting folder stats for {folder}: {e}") + return {'exists': False, 'error': str(e)} + + +class GitignoreParser: + """Parse .gitignore files and check if paths should be ignored.""" + + def __init__(self): + self.patterns = [] # List of (pattern, is_negation, source_dir) + + def add_gitignore(self, gitignore_path: Path): + """Parse and add patterns from a .gitignore file.""" + if not gitignore_path.exists(): + return + + try: + source_dir = gitignore_path.parent + with open(gitignore_path, 'r', encoding='utf-8') as f: + for line_num, line in enumerate(f, 1): + line = line.rstrip('\n\r') + + # Skip empty lines and comments + if not line or line.startswith('#'): + continue + + # Check for negation pattern + is_negation = line.startswith('!') + if is_negation: + line = line[1:] + + # Remove leading slash (make relative to gitignore location) + if line.startswith('/'): + line = line[1:] + + self.patterns.append((line, is_negation, source_dir)) + + app_logger.debug(f"Loaded {len(self.patterns)} patterns from {gitignore_path}") + except Exception as e: + app_logger.warning(f"Error reading {gitignore_path}: {e}") + + def should_ignore(self, path: Path) -> bool: + """Check if a path should be ignored based on gitignore patterns.""" + if not self.patterns: + return False + + ignored = False + + for pattern, is_negation, source_dir in self.patterns: + # Only apply pattern if path is under the source directory + try: + rel_path = path.relative_to(source_dir) + except ValueError: + # Path is not relative to this gitignore's directory + continue + + rel_path_str = str(rel_path) + + # Check if pattern matches + if self._match_pattern(pattern, rel_path_str, path.is_dir()): + if is_negation: + ignored = False # Negation patterns un-ignore + else: + ignored = True + + return ignored + + def _match_pattern(self, pattern: str, path: str, is_dir: bool) -> bool: + """Match a gitignore pattern against a path.""" + # Directory-only pattern (ends with /) + if pattern.endswith('/'): + if not is_dir: + return False + pattern = pattern[:-1] + + # ** matches any number of directories + if '**' in pattern: + # Convert ** to regex-like pattern + pattern_parts = pattern.split('**') + if len(pattern_parts) == 2: + prefix, suffix = pattern_parts + # Match if path starts with prefix and ends with suffix + if prefix: + if not path.startswith(prefix.rstrip('/')): + return False + if suffix: + suffix = suffix.lstrip('/') + if not (path.endswith(suffix) or f'/{suffix}' in path): + return False + return True + + # Direct match + if fnmatch.fnmatch(path, pattern): + return True + + # Match as subdirectory pattern + if '/' not in pattern: + # Pattern without / matches in any directory + parts = path.split('/') + if any(fnmatch.fnmatch(part, pattern) for part in parts): + return True + + return False + + +class SQLiteQueryValidator: + """Validate SQLite queries for read-only safety.""" + + @staticmethod + def is_safe_query(query: str) -> tuple[bool, str]: + """ + Validate that query is a safe read-only SELECT. + Returns (is_safe, error_message) + """ + query_upper = query.strip().upper() + + # Must start with SELECT or WITH (for CTEs) + if not (query_upper.startswith('SELECT') or query_upper.startswith('WITH')): + return False, "Only SELECT queries are allowed (including WITH/CTE)" + + # Block dangerous keywords even in SELECT + dangerous_keywords = [ + 'INSERT', 'UPDATE', 'DELETE', 'DROP', 'CREATE', + 'ALTER', 'TRUNCATE', 'REPLACE', 'ATTACH', 'DETACH', + 'PRAGMA', 'VACUUM', 'REINDEX' + ] + + # Check for dangerous keywords (but allow them in string literals) + # Simple check: look for keywords outside of quotes + query_no_strings = re.sub(r"'[^']*'", '', query_upper) + query_no_strings = re.sub(r'"[^"]*"', '', query_no_strings) + + for keyword in dangerous_keywords: + if re.search(r'\b' + keyword + r'\b', query_no_strings): + return False, f"Keyword '{keyword}' not allowed in read-only mode" + + return True, "" + # P2 +class MCPFilesystemServer: + """MCP Filesystem Server with file access and SQLite database querying.""" + + def __init__(self, allowed_folders: List[Path]): + self.allowed_folders = allowed_folders + self.config = CrossPlatformMCPConfig() + self.max_file_size = 10 * 1024 * 1024 # 10MB limit + self.max_list_items = 1000 # Max items to return in list_directory + self.respect_gitignore = True # Default enabled + + # Initialize gitignore parser + self.gitignore_parser = GitignoreParser() + self._load_gitignores() + + # SQLite configuration + self.max_query_timeout = 5 # seconds + self.max_query_results = 1000 # max rows + self.default_query_limit = 100 # default rows + + app_logger.info(f"MCP Filesystem Server initialized with {len(allowed_folders)} folders") + + def _load_gitignores(self): + """Load all .gitignore files from allowed folders.""" + gitignore_count = 0 + for folder in self.allowed_folders: + if not folder.exists(): + continue + + # Load root .gitignore + root_gitignore = folder / '.gitignore' + if root_gitignore.exists(): + self.gitignore_parser.add_gitignore(root_gitignore) + gitignore_count += 1 + app_logger.info(f"Loaded .gitignore from {folder}") + + # Load nested .gitignore files + try: + for gitignore_path in folder.rglob('.gitignore'): + if gitignore_path != root_gitignore: + self.gitignore_parser.add_gitignore(gitignore_path) + gitignore_count += 1 + app_logger.debug(f"Loaded nested .gitignore: {gitignore_path}") + except Exception as e: + app_logger.warning(f"Error loading nested .gitignores from {folder}: {e}") + + if gitignore_count > 0: + app_logger.info(f"Loaded {gitignore_count} .gitignore file(s) with {len(self.gitignore_parser.patterns)} total patterns") + + def reload_gitignores(self): + """Reload all .gitignore files (call when allowed_folders changes).""" + self.gitignore_parser = GitignoreParser() + self._load_gitignores() + app_logger.info("Reloaded .gitignore patterns") + + def is_allowed_path(self, path: Path) -> bool: + """Check if path is allowed.""" + return self.config.is_safe_path(path, self.allowed_folders) + + async def read_file(self, file_path: str) -> Dict[str, Any]: + """Read file contents.""" + try: + path = self.config.normalize_path(file_path) + + if not self.is_allowed_path(path): + error_msg = f"Access denied: {path} not in allowed folders" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + if not path.exists(): + error_msg = f"File not found: {path}" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + if not path.is_file(): + error_msg = f"Not a file: {path}" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Check if file should be ignored + if self.respect_gitignore and self.gitignore_parser.should_ignore(path): + error_msg = f"File ignored by .gitignore: {path}" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + file_size = path.stat().st_size + if file_size > self.max_file_size: + error_msg = f"File too large: {file_size / (1024*1024):.1f}MB (max: {self.max_file_size / (1024*1024):.0f}MB)" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Try to read as text + try: + content = path.read_text(encoding='utf-8') + + # If file is large (>50KB), provide summary instead of full content + max_content_size = 50 * 1024 # 50KB + if file_size > max_content_size: + lines = content.split('\n') + total_lines = len(lines) + + # Return first 500 lines + last 100 lines with notice + head_lines = 500 + tail_lines = 100 + + if total_lines > (head_lines + tail_lines): + truncated_content = ( + '\n'.join(lines[:head_lines]) + + f"\n\n... [TRUNCATED: {total_lines - head_lines - tail_lines} lines omitted to stay within size limits] ...\n\n" + + '\n'.join(lines[-tail_lines:]) + ) + + app_logger.info(f"Read file (truncated): {path} ({file_size} bytes, {total_lines} lines)") + log_mcp_stat('read_file', str(path.parent), True) + + return { + 'content': truncated_content, + 'path': str(path), + 'size': file_size, + 'truncated': True, + 'total_lines': total_lines, + 'lines_shown': head_lines + tail_lines, + 'note': f'File truncated: showing first {head_lines} and last {tail_lines} lines of {total_lines} total' + } + + app_logger.info(f"Read file: {path} ({file_size} bytes)") + log_mcp_stat('read_file', str(path.parent), True) + return { + 'content': content, + 'path': str(path), + 'size': file_size + } + except UnicodeDecodeError: + error_msg = f"Cannot decode file as UTF-8: {path}" + app_logger.warning(error_msg) + log_mcp_stat('read_file', str(path.parent), False, error_msg) + return {'error': error_msg} + + except Exception as e: + error_msg = f"Error reading file: {e}" + app_logger.error(error_msg) + log_mcp_stat('read_file', file_path, False, str(e)) + return {'error': error_msg} + + async def list_directory(self, dir_path: str, recursive: bool = False) -> Dict[str, Any]: + """List directory contents.""" + try: + path = self.config.normalize_path(dir_path) + + if not self.is_allowed_path(path): + error_msg = f"Access denied: {path} not in allowed folders" + app_logger.warning(error_msg) + log_mcp_stat('list_directory', str(path), False, error_msg) + return {'error': error_msg} + + if not path.exists(): + error_msg = f"Directory not found: {path}" + app_logger.warning(error_msg) + log_mcp_stat('list_directory', str(path), False, error_msg) + return {'error': error_msg} + + if not path.is_dir(): + error_msg = f"Not a directory: {path}" + app_logger.warning(error_msg) + log_mcp_stat('list_directory', str(path), False, error_msg) + return {'error': error_msg} + + items = [] + pattern = '**/*' if recursive else '*' + + # Directories to skip (hardcoded common patterns) + skip_dirs = { + '.venv', 'venv', 'env', 'virtualenv', + 'site-packages', 'dist-packages', + '__pycache__', '.pytest_cache', '.mypy_cache', + 'node_modules', '.git', '.svn', + '.idea', '.vscode', + 'build', 'dist', 'eggs', '.eggs' + } + + def should_skip_path(item_path: Path) -> bool: + """Check if path should be skipped.""" + # Check hardcoded skip directories + path_parts = item_path.parts + if any(part in skip_dirs for part in path_parts): + return True + + # Check gitignore patterns (if enabled) + if self.respect_gitignore and self.gitignore_parser.should_ignore(item_path): + return True + + return False + + for item in path.glob(pattern): + # Stop if we hit the limit + if len(items) >= self.max_list_items: + break + + # Skip excluded directories + if should_skip_path(item): + continue + + try: + stat = item.stat() + items.append({ + 'name': item.name, + 'path': str(item), + 'type': 'directory' if item.is_dir() else 'file', + 'size': stat.st_size if item.is_file() else 0, + 'modified': datetime.datetime.fromtimestamp(stat.st_mtime).isoformat() + }) + except: + continue + + truncated = len(items) >= self.max_list_items + + app_logger.info(f"Listed directory: {path} ({len(items)} items, recursive={recursive}, truncated={truncated})") + log_mcp_stat('list_directory', str(path), True) + + result = { + 'path': str(path), + 'items': items, + 'count': len(items), + 'truncated': truncated + } + + if truncated: + result['note'] = f'Results limited to {self.max_list_items} items. Use more specific search path or disable recursive mode.' + + return result + + except Exception as e: + error_msg = f"Error listing directory: {e}" + app_logger.error(error_msg) + log_mcp_stat('list_directory', dir_path, False, str(e)) + return {'error': error_msg} + + async def search_files(self, pattern: str, search_path: Optional[str] = None, content_search: bool = False) -> Dict[str, Any]: + """Search for files.""" + try: + # Determine search roots + if search_path: + path = self.config.normalize_path(search_path) + if not self.is_allowed_path(path): + error_msg = f"Access denied: {path} not in allowed folders" + app_logger.warning(error_msg) + log_mcp_stat('search_files', str(path), False, error_msg) + return {'error': error_msg} + search_roots = [path] + else: + search_roots = self.allowed_folders + + matches = [] + + # Directories to skip (virtual environments, caches, etc.) + skip_dirs = { + '.venv', 'venv', 'env', 'virtualenv', + 'site-packages', 'dist-packages', + '__pycache__', '.pytest_cache', '.mypy_cache', + 'node_modules', '.git', '.svn', + '.idea', '.vscode', + 'build', 'dist', 'eggs', '.eggs' + } + + def should_skip_path(item_path: Path) -> bool: + """Check if path should be skipped.""" + # Check hardcoded skip directories + path_parts = item_path.parts + if any(part in skip_dirs for part in path_parts): + return True + + # Check gitignore patterns (if enabled) + if self.respect_gitignore and self.gitignore_parser.should_ignore(item_path): + return True + + return False + + for root in search_roots: + if not root.exists(): + continue + + # Filename search + if not content_search: + for item in root.rglob(pattern): + if item.is_file() and not should_skip_path(item): + try: + stat = item.stat() + matches.append({ + 'path': str(item), + 'name': item.name, + 'size': stat.st_size, + 'modified': datetime.datetime.fromtimestamp(stat.st_mtime).isoformat() + }) + except: + continue + else: + # Content search (slower) + for item in root.rglob('*'): + if item.is_file() and not should_skip_path(item): + try: + # Skip large files + if item.stat().st_size > self.max_file_size: + continue + + # Try to read and search content + try: + content = item.read_text(encoding='utf-8') + if pattern.lower() in content.lower(): + stat = item.stat() + matches.append({ + 'path': str(item), + 'name': item.name, + 'size': stat.st_size, + 'modified': datetime.datetime.fromtimestamp(stat.st_mtime).isoformat() + }) + except: + continue + except: + continue + + search_type = "content" if content_search else "filename" + app_logger.info(f"Searched files: pattern='{pattern}', type={search_type}, found={len(matches)}") + log_mcp_stat('search_files', str(search_roots[0]) if search_roots else None, True) + + return { + 'pattern': pattern, + 'search_type': search_type, + 'matches': matches, + 'count': len(matches) + } + + except Exception as e: + error_msg = f"Error searching files: {e}" + app_logger.error(error_msg) + log_mcp_stat('search_files', search_path or "all", False, str(e)) + return {'error': error_msg} + + # ======================================================================== + # SQLite DATABASE METHODS + # ======================================================================== + + async def inspect_database(self, db_path: str, table_name: Optional[str] = None) -> Dict[str, Any]: + """Inspect SQLite database schema.""" + try: + path = Path(db_path).resolve() + + if not path.exists(): + error_msg = f"Database not found: {path}" + app_logger.warning(error_msg) + log_mcp_stat('inspect_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + if not path.is_file(): + error_msg = f"Not a file: {path}" + app_logger.warning(error_msg) + log_mcp_stat('inspect_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Open database in read-only mode + try: + conn = sqlite3.connect(f'file:{path}?mode=ro', uri=True) + cursor = conn.cursor() + except sqlite3.DatabaseError as e: + error_msg = f"Not a valid SQLite database: {path} ({e})" + app_logger.warning(error_msg) + log_mcp_stat('inspect_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + try: + if table_name: + # Get info for specific table + cursor.execute("SELECT sql FROM sqlite_master WHERE type='table' AND name=?", (table_name,)) + result = cursor.fetchone() + + if not result: + return {'error': f"Table '{table_name}' not found"} + + create_sql = result[0] + + # Get column info + cursor.execute(f"PRAGMA table_info({table_name})") + columns = [] + for row in cursor.fetchall(): + columns.append({ + 'id': row[0], + 'name': row[1], + 'type': row[2], + 'not_null': bool(row[3]), + 'default_value': row[4], + 'primary_key': bool(row[5]) + }) + + # Get indexes + cursor.execute(f"PRAGMA index_list({table_name})") + indexes = [] + for row in cursor.fetchall(): + indexes.append({ + 'name': row[1], + 'unique': bool(row[2]) + }) + + # Get row count + cursor.execute(f"SELECT COUNT(*) FROM {table_name}") + row_count = cursor.fetchone()[0] + + app_logger.info(f"Inspected table: {table_name} in {path}") + log_mcp_stat('inspect_database', str(path.parent), True) + + return { + 'database': str(path), + 'table': table_name, + 'create_sql': create_sql, + 'columns': columns, + 'indexes': indexes, + 'row_count': row_count + } + else: + # Get all tables + cursor.execute("SELECT name, sql FROM sqlite_master WHERE type='table' ORDER BY name") + tables = [] + for row in cursor.fetchall(): + table = row[0] + # Get row count + cursor.execute(f"SELECT COUNT(*) FROM {table}") + count = cursor.fetchone()[0] + tables.append({ + 'name': table, + 'row_count': count + }) + + # Get indexes + cursor.execute("SELECT name, tbl_name FROM sqlite_master WHERE type='index' ORDER BY name") + indexes = [{'name': row[0], 'table': row[1]} for row in cursor.fetchall()] + + # Get triggers + cursor.execute("SELECT name, tbl_name FROM sqlite_master WHERE type='trigger' ORDER BY name") + triggers = [{'name': row[0], 'table': row[1]} for row in cursor.fetchall()] + + # Get database size + db_size = path.stat().st_size + + app_logger.info(f"Inspected database: {path} ({len(tables)} tables)") + log_mcp_stat('inspect_database', str(path.parent), True) + + return { + 'database': str(path), + 'size': db_size, + 'size_mb': db_size / (1024 * 1024), + 'tables': tables, + 'table_count': len(tables), + 'indexes': indexes, + 'triggers': triggers + } + finally: + conn.close() + + except Exception as e: + error_msg = f"Error inspecting database: {e}" + app_logger.error(error_msg) + log_mcp_stat('inspect_database', db_path, False, str(e)) + return {'error': error_msg} + + async def search_database(self, db_path: str, search_term: str, table_name: Optional[str] = None, column_name: Optional[str] = None) -> Dict[str, Any]: + """Search for a value across database tables.""" + try: + path = Path(db_path).resolve() + + if not path.exists(): + error_msg = f"Database not found: {path}" + app_logger.warning(error_msg) + log_mcp_stat('search_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Open database in read-only mode + try: + conn = sqlite3.connect(f'file:{path}?mode=ro', uri=True) + cursor = conn.cursor() + except sqlite3.DatabaseError as e: + error_msg = f"Not a valid SQLite database: {path} ({e})" + app_logger.warning(error_msg) + log_mcp_stat('search_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + try: + matches = [] + + # Get tables to search + if table_name: + cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name=?", (table_name,)) + tables = [row[0] for row in cursor.fetchall()] + if not tables: + return {'error': f"Table '{table_name}' not found"} + else: + cursor.execute("SELECT name FROM sqlite_master WHERE type='table'") + tables = [row[0] for row in cursor.fetchall()] + + for table in tables: + # Get columns for this table + cursor.execute(f"PRAGMA table_info({table})") + columns = [row[1] for row in cursor.fetchall()] + + # Filter columns if specified + if column_name: + if column_name not in columns: + continue + columns = [column_name] + + # Search in each column + for column in columns: + try: + # Use LIKE for partial matching + query = f"SELECT * FROM {table} WHERE {column} LIKE ? LIMIT {self.default_query_limit}" + cursor.execute(query, (f'%{search_term}%',)) + results = cursor.fetchall() + + if results: + # Get column names + col_names = [desc[0] for desc in cursor.description] + + for row in results: + # Convert row to dict, handling binary data + row_dict = {} + for col_name, value in zip(col_names, row): + if isinstance(value, bytes): + # Convert bytes to hex string or base64 + try: + # Try to decode as UTF-8 first + row_dict[col_name] = value.decode('utf-8') + except UnicodeDecodeError: + # If binary, show as hex string + row_dict[col_name] = f"" + else: + row_dict[col_name] = value + + matches.append({ + 'table': table, + 'column': column, + 'row': row_dict + }) + except sqlite3.Error: + # Skip columns that can't be searched (e.g., BLOBs) + continue + + app_logger.info(f"Searched database: {path} for '{search_term}' - found {len(matches)} matches") + log_mcp_stat('search_database', str(path.parent), True) + + result = { + 'database': str(path), + 'search_term': search_term, + 'matches': matches, + 'count': len(matches) + } + + if table_name: + result['table_filter'] = table_name + if column_name: + result['column_filter'] = column_name + + if len(matches) >= self.default_query_limit: + result['note'] = f'Results limited to {self.default_query_limit} matches. Use more specific search or query_database for full results.' + + return result + + finally: + conn.close() + + except Exception as e: + error_msg = f"Error searching database: {e}" + app_logger.error(error_msg) + log_mcp_stat('search_database', db_path, False, str(e)) + return {'error': error_msg} + + async def query_database(self, db_path: str, query: str, limit: Optional[int] = None) -> Dict[str, Any]: + """Execute a read-only SQL query on database.""" + try: + path = Path(db_path).resolve() + + if not path.exists(): + error_msg = f"Database not found: {path}" + app_logger.warning(error_msg) + log_mcp_stat('query_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Validate query + is_safe, error_msg = SQLiteQueryValidator.is_safe_query(query) + if not is_safe: + app_logger.warning(f"Unsafe query rejected: {error_msg}") + log_mcp_stat('query_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + # Open database in read-only mode + try: + conn = sqlite3.connect(f'file:{path}?mode=ro', uri=True) + cursor = conn.cursor() + except sqlite3.DatabaseError as e: + error_msg = f"Not a valid SQLite database: {path} ({e})" + app_logger.warning(error_msg) + log_mcp_stat('query_database', str(path.parent), False, error_msg) + return {'error': error_msg} + + try: + # Set query timeout + def timeout_handler(signum, frame): + raise TimeoutError(f"Query exceeded {self.max_query_timeout} second timeout") + + # Note: signal.alarm only works on Unix-like systems + if hasattr(signal, 'SIGALRM'): + signal.signal(signal.SIGALRM, timeout_handler) + signal.alarm(self.max_query_timeout) + + try: + # Execute query + cursor.execute(query) + + # Get results + result_limit = min(limit or self.default_query_limit, self.max_query_results) + results = cursor.fetchmany(result_limit) + + # Get column names + columns = [desc[0] for desc in cursor.description] if cursor.description else [] + + # Convert to list of dicts, handling binary data + rows = [] + for row in results: + row_dict = {} + for col_name, value in zip(columns, row): + if isinstance(value, bytes): + # Convert bytes to readable format + try: + # Try to decode as UTF-8 first + row_dict[col_name] = value.decode('utf-8') + except UnicodeDecodeError: + # If binary, show as hex string or summary + row_dict[col_name] = f"" + else: + row_dict[col_name] = value + rows.append(row_dict) + + # Check if more results available + has_more = len(results) == result_limit + if has_more: + # Try to fetch one more to check + one_more = cursor.fetchone() + has_more = one_more is not None + + finally: + if hasattr(signal, 'SIGALRM'): + signal.alarm(0) # Cancel timeout + + app_logger.info(f"Executed query on {path}: returned {len(rows)} rows") + log_mcp_stat('query_database', str(path.parent), True) + + result = { + 'database': str(path), + 'query': query, + 'columns': columns, + 'rows': rows, + 'count': len(rows) + } + + if has_more: + result['truncated'] = True + result['note'] = f'Results limited to {result_limit} rows. Use LIMIT clause in query for more control.' + + return result + + except TimeoutError as e: + error_msg = str(e) + app_logger.warning(error_msg) + log_mcp_stat('query_database', str(path.parent), False, error_msg) + return {'error': error_msg} + except sqlite3.Error as e: + error_msg = f"SQL error: {e}" + app_logger.warning(error_msg) + log_mcp_stat('query_database', str(path.parent), False, error_msg) + return {'error': error_msg} + finally: + if hasattr(signal, 'SIGALRM'): + signal.alarm(0) # Ensure timeout is cancelled + conn.close() + + except Exception as e: + error_msg = f"Error executing query: {e}" + app_logger.error(error_msg) + log_mcp_stat('query_database', db_path, False, str(e)) + return {'error': error_msg} + + +class MCPManager: + """Manage MCP server lifecycle, tool calls, and mode switching.""" + + def __init__(self): + self.enabled = False + self.mode = "files" # "files" or "database" + self.selected_db_index = None + + self.server: Optional[MCPFilesystemServer] = None + + # File/folder mode + self.allowed_folders: List[Path] = [] + + # Database mode + self.databases: List[Dict[str, Any]] = [] + + self.config = CrossPlatformMCPConfig() + self.session_start_time: Optional[datetime.datetime] = None + + # Load persisted data + self._load_folders() + self._load_databases() + + app_logger.info("MCP Manager initialized") + + def _load_folders(self): + """Load allowed folders from database.""" + folders_json = get_mcp_config('allowed_folders') + if folders_json: + try: + folder_paths = json.loads(folders_json) + self.allowed_folders = [self.config.normalize_path(p) for p in folder_paths] + app_logger.info(f"Loaded {len(self.allowed_folders)} folders from config") + except Exception as e: + app_logger.error(f"Error loading MCP folders: {e}") + self.allowed_folders = [] + + def _save_folders(self): + """Save allowed folders to database.""" + folder_paths = [str(p) for p in self.allowed_folders] + set_mcp_config('allowed_folders', json.dumps(folder_paths)) + app_logger.info(f"Saved {len(self.allowed_folders)} folders to config") + + def _load_databases(self): + """Load databases from database.""" + create_table_if_not_exists() + try: + with sqlite3.connect(str(database)) as conn: + cursor = conn.execute('SELECT id, path, name, size, tables, added_timestamp FROM mcp_databases ORDER BY id') + self.databases = [] + for row in cursor.fetchall(): + tables_list = json.loads(row[4]) if row[4] else [] + self.databases.append({ + 'id': row[0], + 'path': row[1], + 'name': row[2], + 'size': row[3], + 'tables': tables_list, + 'added': row[5] + }) + app_logger.info(f"Loaded {len(self.databases)} databases from config") + except Exception as e: + app_logger.error(f"Error loading databases: {e}") + self.databases = [] + + def _save_database(self, db_info: Dict[str, Any]): + """Save a database to config.""" + create_table_if_not_exists() + try: + with sqlite3.connect(str(database)) as conn: + conn.execute( + 'INSERT INTO mcp_databases (path, name, size, tables, added_timestamp) VALUES (?, ?, ?, ?, ?)', + (db_info['path'], db_info['name'], db_info['size'], json.dumps(db_info['tables']), db_info['added']) + ) + conn.commit() + # Get the ID + cursor = conn.execute('SELECT id FROM mcp_databases WHERE path = ?', (db_info['path'],)) + db_info['id'] = cursor.fetchone()[0] + app_logger.info(f"Saved database {db_info['name']} to config") + except Exception as e: + app_logger.error(f"Error saving database: {e}") + + def _remove_database_from_config(self, db_path: str): + """Remove database from config.""" + create_table_if_not_exists() + try: + with sqlite3.connect(str(database)) as conn: + conn.execute('DELETE FROM mcp_databases WHERE path = ?', (db_path,)) + conn.commit() + app_logger.info(f"Removed database from config: {db_path}") + except Exception as e: + app_logger.error(f"Error removing database from config: {e}") + + def enable(self) -> Dict[str, Any]: + """Enable MCP server.""" + if not MCP_AVAILABLE: + return { + 'success': False, + 'error': 'MCP library not installed. Run: pip install mcp' + } + + if self.enabled: + return { + 'success': False, + 'error': 'MCP is already enabled' + } + + try: + self.server = MCPFilesystemServer(self.allowed_folders) + self.enabled = True + self.session_start_time = datetime.datetime.now() + + set_mcp_config('mcp_enabled', 'true') + + app_logger.info("MCP Filesystem Server enabled") + + return { + 'success': True, + 'folder_count': len(self.allowed_folders), + 'database_count': len(self.databases), + 'message': 'MCP Filesystem Server started successfully' + } + except Exception as e: + app_logger.error(f"Error enabling MCP: {e}") + return { + 'success': False, + 'error': str(e) + } + + def disable(self) -> Dict[str, Any]: + """Disable MCP server.""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled' + } + + try: + self.server = None + self.enabled = False + self.session_start_time = None + self.mode = "files" + self.selected_db_index = None + + set_mcp_config('mcp_enabled', 'false') + + app_logger.info("MCP Filesystem Server disabled") + + return { + 'success': True, + 'message': 'MCP Filesystem Server stopped' + } + except Exception as e: + app_logger.error(f"Error disabling MCP: {e}") + return { + 'success': False, + 'error': str(e) + } + + def switch_mode(self, new_mode: str, db_index: Optional[int] = None) -> Dict[str, Any]: + """Switch between files and database mode.""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled. Use /mcp enable first' + } + + if new_mode == "files": + self.mode = "files" + self.selected_db_index = None + app_logger.info("Switched to file mode") + return { + 'success': True, + 'mode': 'files', + 'message': 'Switched to file mode', + 'tools': ['read_file', 'list_directory', 'search_files'] + } + elif new_mode == "database": + if db_index is None: + return { + 'success': False, + 'error': 'Database index required. Use /mcp db ' + } + + if db_index < 1 or db_index > len(self.databases): + return { + 'success': False, + 'error': f'Invalid database number. Use 1-{len(self.databases)}' + } + + self.mode = "database" + self.selected_db_index = db_index - 1 # Convert to 0-based + db = self.databases[self.selected_db_index] + + app_logger.info(f"Switched to database mode: {db['name']}") + return { + 'success': True, + 'mode': 'database', + 'database': db, + 'message': f"Switched to database #{db_index}: {db['name']}", + 'tools': ['inspect_database', 'search_database', 'query_database'] + } + else: + return { + 'success': False, + 'error': f'Invalid mode: {new_mode}' + } + + def add_folder(self, folder_path: str) -> Dict[str, Any]: + """Add folder to allowed list.""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled. Use /mcp enable first' + } + + try: + path = self.config.normalize_path(folder_path) + + # Validation checks + if not path.exists(): + return { + 'success': False, + 'error': f'Directory does not exist: {path}' + } + + if not path.is_dir(): + return { + 'success': False, + 'error': f'Not a directory: {path}' + } + + if self.config.is_system_directory(path): + return { + 'success': False, + 'error': f'Cannot add system directory: {path}' + } + + # Check if already added + if path in self.allowed_folders: + return { + 'success': False, + 'error': f'Folder already in allowed list: {path}' + } + + # Check for nested paths + parent_folder = None + for existing in self.allowed_folders: + try: + path.relative_to(existing) + parent_folder = existing + break + except ValueError: + continue + + # Get folder stats + stats = self.config.get_folder_stats(path) + + # Add folder + self.allowed_folders.append(path) + self._save_folders() + + # Update server and reload gitignores + if self.server: + self.server.allowed_folders = self.allowed_folders + self.server.reload_gitignores() + + app_logger.info(f"Added folder to MCP: {path}") + + result = { + 'success': True, + 'path': str(path), + 'stats': stats, + 'total_folders': len(self.allowed_folders) + } + + if parent_folder: + result['warning'] = f'Note: {parent_folder} is already allowed (parent folder)' + + return result + + except Exception as e: + app_logger.error(f"Error adding folder: {e}") + return { + 'success': False, + 'error': str(e) + } + + def remove_folder(self, folder_ref: str) -> Dict[str, Any]: + """Remove folder from allowed list (by path or number).""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled. Use /mcp enable first' + } + + try: + # Check if it's a number + if folder_ref.isdigit(): + index = int(folder_ref) - 1 + if 0 <= index < len(self.allowed_folders): + path = self.allowed_folders[index] + else: + return { + 'success': False, + 'error': f'Invalid folder number: {folder_ref}' + } + else: + # Treat as path + path = self.config.normalize_path(folder_ref) + if path not in self.allowed_folders: + return { + 'success': False, + 'error': f'Folder not in allowed list: {path}' + } + + # Remove folder + self.allowed_folders.remove(path) + self._save_folders() + + # Update server and reload gitignores + if self.server: + self.server.allowed_folders = self.allowed_folders + self.server.reload_gitignores() + + app_logger.info(f"Removed folder from MCP: {path}") + + result = { + 'success': True, + 'path': str(path), + 'total_folders': len(self.allowed_folders) + } + + if len(self.allowed_folders) == 0: + result['warning'] = 'No allowed folders remaining. Add one with /mcp add ' + + return result + + except Exception as e: + app_logger.error(f"Error removing folder: {e}") + return { + 'success': False, + 'error': str(e) + } + + def add_database(self, db_path: str) -> Dict[str, Any]: + """Add SQLite database.""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled. Use /mcp enable first' + } + + try: + path = Path(db_path).resolve() + + # Validation + if not path.exists(): + return { + 'success': False, + 'error': f'Database file not found: {path}' + } + + if not path.is_file(): + return { + 'success': False, + 'error': f'Not a file: {path}' + } + + # Check if already added + if any(db['path'] == str(path) for db in self.databases): + return { + 'success': False, + 'error': f'Database already added: {path.name}' + } + + # Validate it's a SQLite database and get schema + try: + conn = sqlite3.connect(f'file:{path}?mode=ro', uri=True) + cursor = conn.cursor() + + # Get tables + cursor.execute("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name") + tables = [row[0] for row in cursor.fetchall()] + + conn.close() + except sqlite3.DatabaseError as e: + return { + 'success': False, + 'error': f'Not a valid SQLite database: {e}' + } + + # Get file size + db_size = path.stat().st_size + + # Create database entry + db_info = { + 'path': str(path), + 'name': path.name, + 'size': db_size, + 'tables': tables, + 'added': datetime.datetime.now().isoformat() + } + + # Save to config + self._save_database(db_info) + + # Add to list + self.databases.append(db_info) + + app_logger.info(f"Added database: {path.name}") + + return { + 'success': True, + 'database': db_info, + 'number': len(self.databases), + 'message': f'Added database #{len(self.databases)}: {path.name}' + } + + except Exception as e: + app_logger.error(f"Error adding database: {e}") + return { + 'success': False, + 'error': str(e) + } + + def remove_database(self, db_ref: str) -> Dict[str, Any]: + """Remove database (by number or path).""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled' + } + + try: + # Check if it's a number + if db_ref.isdigit(): + index = int(db_ref) - 1 + if 0 <= index < len(self.databases): + db = self.databases[index] + else: + return { + 'success': False, + 'error': f'Invalid database number: {db_ref}' + } + else: + # Treat as path + path = str(Path(db_ref).resolve()) + db = next((d for d in self.databases if d['path'] == path), None) + if not db: + return { + 'success': False, + 'error': f'Database not found: {db_ref}' + } + index = self.databases.index(db) + + # If currently selected, deselect + if self.mode == "database" and self.selected_db_index == index: + self.mode = "files" + self.selected_db_index = None + + # Remove from config + self._remove_database_from_config(db['path']) + + # Remove from list + self.databases.pop(index) + + app_logger.info(f"Removed database: {db['name']}") + + result = { + 'success': True, + 'database': db, + 'message': f"Removed database: {db['name']}" + } + + if len(self.databases) == 0: + result['warning'] = 'No databases remaining. Add one with /mcp add db ' + + return result + + except Exception as e: + app_logger.error(f"Error removing database: {e}") + return { + 'success': False, + 'error': str(e) + } + + def list_databases(self) -> Dict[str, Any]: + """List all databases.""" + try: + db_list = [] + for idx, db in enumerate(self.databases, 1): + db_info = { + 'number': idx, + 'name': db['name'], + 'path': db['path'], + 'size_mb': db['size'] / (1024 * 1024), + 'tables': db['tables'], + 'table_count': len(db['tables']), + 'added': db['added'] + } + + # Check if file still exists + if not Path(db['path']).exists(): + db_info['warning'] = 'File not found' + + db_list.append(db_info) + + return { + 'success': True, + 'databases': db_list, + 'count': len(db_list) + } + + except Exception as e: + app_logger.error(f"Error listing databases: {e}") + return { + 'success': False, + 'error': str(e) + } + + def toggle_gitignore(self, enabled: bool) -> Dict[str, Any]: + """Toggle .gitignore filtering.""" + if not self.enabled: + return { + 'success': False, + 'error': 'MCP is not enabled' + } + + if not self.server: + return { + 'success': False, + 'error': 'MCP server not running' + } + + self.server.respect_gitignore = enabled + status = "enabled" if enabled else "disabled" + + app_logger.info(f".gitignore filtering {status}") + + return { + 'success': True, + 'message': f'.gitignore filtering {status}', + 'pattern_count': len(self.server.gitignore_parser.patterns) if enabled else 0 + } + + def list_folders(self) -> Dict[str, Any]: + """List all allowed folders with stats.""" + try: + folders_info = [] + total_files = 0 + total_size = 0 + + for idx, folder in enumerate(self.allowed_folders, 1): + stats = self.config.get_folder_stats(folder) + + folder_info = { + 'number': idx, + 'path': str(folder), + 'exists': stats.get('exists', False) + } + + if stats.get('exists'): + folder_info['file_count'] = stats['file_count'] + folder_info['size_mb'] = stats['size_mb'] + total_files += stats['file_count'] + total_size += stats['total_size'] + + folders_info.append(folder_info) + + return { + 'success': True, + 'folders': folders_info, + 'total_folders': len(self.allowed_folders), + 'total_files': total_files, + 'total_size_mb': total_size / (1024 * 1024) + } + + except Exception as e: + app_logger.error(f"Error listing folders: {e}") + return { + 'success': False, + 'error': str(e) + } + + def get_status(self) -> Dict[str, Any]: + """Get comprehensive MCP status.""" + try: + stats = get_mcp_stats() + + uptime = None + if self.session_start_time: + delta = datetime.datetime.now() - self.session_start_time + hours = delta.seconds // 3600 + minutes = (delta.seconds % 3600) // 60 + uptime = f"{hours}h {minutes}m" if hours > 0 else f"{minutes}m" + + folder_info = self.list_folders() + + gitignore_status = "enabled" if (self.server and self.server.respect_gitignore) else "disabled" + gitignore_patterns = len(self.server.gitignore_parser.patterns) if self.server else 0 + + # Current mode info + mode_info = { + 'mode': self.mode, + 'mode_display': '🔧 Files' if self.mode == 'files' else f'🗄️ DB #{self.selected_db_index + 1}' + } + + if self.mode == 'database' and self.selected_db_index is not None: + db = self.databases[self.selected_db_index] + mode_info['database'] = db + + return { + 'success': True, + 'enabled': self.enabled, + 'uptime': uptime, + 'mode_info': mode_info, + 'folder_count': len(self.allowed_folders), + 'database_count': len(self.databases), + 'total_files': folder_info.get('total_files', 0), + 'total_size_mb': folder_info.get('total_size_mb', 0), + 'stats': stats, + 'tools_available': self._get_current_tools(), + 'gitignore_status': gitignore_status, + 'gitignore_patterns': gitignore_patterns + } + + except Exception as e: + app_logger.error(f"Error getting MCP status: {e}") + return { + 'success': False, + 'error': str(e) + } + + def _get_current_tools(self) -> List[str]: + """Get tools available in current mode.""" + if not self.enabled: + return [] + + if self.mode == "files": + return ['read_file', 'list_directory', 'search_files'] + elif self.mode == "database": + return ['inspect_database', 'search_database', 'query_database'] + + return [] + + async def call_tool(self, tool_name: str, **kwargs) -> Dict[str, Any]: + """Call an MCP tool.""" + if not self.enabled or not self.server: + return { + 'error': 'MCP is not enabled' + } + + try: + # File mode tools + if tool_name == 'read_file': + return await self.server.read_file(kwargs.get('file_path', '')) + elif tool_name == 'list_directory': + return await self.server.list_directory( + kwargs.get('dir_path', ''), + kwargs.get('recursive', False) + ) + elif tool_name == 'search_files': + return await self.server.search_files( + kwargs.get('pattern', ''), + kwargs.get('search_path'), + kwargs.get('content_search', False) + ) + + # Database mode tools + elif tool_name == 'inspect_database': + if self.mode != 'database' or self.selected_db_index is None: + return {'error': 'Not in database mode. Use /mcp db first'} + db = self.databases[self.selected_db_index] + return await self.server.inspect_database( + db['path'], + kwargs.get('table_name') + ) + elif tool_name == 'search_database': + if self.mode != 'database' or self.selected_db_index is None: + return {'error': 'Not in database mode. Use /mcp db first'} + db = self.databases[self.selected_db_index] + return await self.server.search_database( + db['path'], + kwargs.get('search_term', ''), + kwargs.get('table_name'), + kwargs.get('column_name') + ) + elif tool_name == 'query_database': + if self.mode != 'database' or self.selected_db_index is None: + return {'error': 'Not in database mode. Use /mcp db first'} + db = self.databases[self.selected_db_index] + return await self.server.query_database( + db['path'], + kwargs.get('query', ''), + kwargs.get('limit') + ) + else: + return { + 'error': f'Unknown tool: {tool_name}' + } + except Exception as e: + app_logger.error(f"Error calling MCP tool {tool_name}: {e}") + return { + 'error': str(e) + } + + def get_tools_schema(self) -> List[Dict[str, Any]]: + """Get MCP tools as OpenAI function calling schema (for current mode).""" + if not self.enabled: + return [] + + if self.mode == "files": + return self._get_file_tools_schema() + elif self.mode == "database": + return self._get_database_tools_schema() + + return [] + + def _get_file_tools_schema(self) -> List[Dict[str, Any]]: + """Get file mode tools schema.""" + if len(self.allowed_folders) == 0: + return [] + + allowed_dirs_str = ", ".join(str(f) for f in self.allowed_folders) + + return [ + { + "type": "function", + "function": { + "name": "search_files", + "description": f"Search for files in the user's local filesystem. Allowed directories: {allowed_dirs_str}. Can search by filename pattern (e.g., '*.py' for Python files) or search within file contents. Automatically respects .gitignore patterns and excludes virtual environments.", + "parameters": { + "type": "object", + "properties": { + "pattern": { + "type": "string", + "description": "Search pattern. For filename search, use glob patterns like '*.py', '*.txt', 'report*'. For content search, use plain text to search for." + }, + "content_search": { + "type": "boolean", + "description": "If true, searches inside file contents for the pattern (SLOW - use only when specifically asked to search file contents). If false, searches only filenames (FAST - use for finding files by name). Default is false. Virtual environments and package directories are automatically excluded.", + "default": False + }, + "search_path": { + "type": "string", + "description": "Optional: specific directory to search in. If not provided, searches all allowed directories.", + } + }, + "required": ["pattern"] + } + } + }, + { + "type": "function", + "function": { + "name": "read_file", + "description": "Read the complete contents of a text file from the user's local filesystem. Only works for files within allowed directories. Maximum file size: 10MB. Files larger than 50KB are automatically truncated. Respects .gitignore patterns.", + "parameters": { + "type": "object", + "properties": { + "file_path": { + "type": "string", + "description": "Full path to the file to read (e.g., /Users/username/Documents/report.txt or ~/Documents/notes.md)" + } + }, + "required": ["file_path"] + } + } + }, + { + "type": "function", + "function": { + "name": "list_directory", + "description": "List all files and subdirectories in a directory from the user's local filesystem. Only works for directories within allowed paths. Automatically filters out virtual environments, build artifacts, and .gitignore patterns. Limited to 1000 items.", + "parameters": { + "type": "object", + "properties": { + "dir_path": { + "type": "string", + "description": "Directory path to list (e.g., ~/Documents or /Users/username/Projects)" + }, + "recursive": { + "type": "boolean", + "description": "If true, lists all files in subdirectories recursively. If false, lists only immediate children. Default is true. WARNING: Recursive listings can be very large - use specific paths when possible.", + "default": True + } + }, + "required": ["dir_path"] + } + } + } + ] + + def _get_database_tools_schema(self) -> List[Dict[str, Any]]: + """Get database mode tools schema.""" + if self.selected_db_index is None or self.selected_db_index >= len(self.databases): + return [] + + db = self.databases[self.selected_db_index] + db_name = db['name'] + tables_str = ", ".join(db['tables']) + + return [ + { + "type": "function", + "function": { + "name": "inspect_database", + "description": f"Inspect the schema of the currently selected SQLite database ({db_name}). Can get all tables or details for a specific table including columns, types, indexes, and row counts. Available tables: {tables_str}.", + "parameters": { + "type": "object", + "properties": { + "table_name": { + "type": "string", + "description": f"Optional: specific table to inspect. If not provided, returns info for all tables. Available: {tables_str}" + } + }, + "required": [] + } + } + }, + { + "type": "function", + "function": { + "name": "search_database", + "description": f"Search for a value across all tables in the database ({db_name}). Performs a LIKE search (partial matching) across all columns or specific table/column. Returns matching rows with all column data. Limited to {self.server.default_query_limit} results.", + "parameters": { + "type": "object", + "properties": { + "search_term": { + "type": "string", + "description": "Value to search for (partial match supported)" + }, + "table_name": { + "type": "string", + "description": f"Optional: limit search to specific table. Available: {tables_str}" + }, + "column_name": { + "type": "string", + "description": "Optional: limit search to specific column within the table" + } + }, + "required": ["search_term"] + } + } + }, + { + "type": "function", + "function": { + "name": "query_database", + "description": f"Execute a read-only SQL query on the database ({db_name}). Supports SELECT queries including JOINs, subqueries, CTEs, aggregations, etc. Maximum {self.server.max_query_results} rows returned. Query timeout: {self.server.max_query_timeout} seconds. INSERT/UPDATE/DELETE/DROP are blocked for safety.", + "parameters": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": f"SQL SELECT query to execute. Available tables: {tables_str}. Example: SELECT * FROM table WHERE column = 'value' LIMIT 10" + }, + "limit": { + "type": "integer", + "description": f"Optional: maximum rows to return (default {self.server.default_query_limit}, max {self.server.max_query_results}). You can also use LIMIT in your query." + } + }, + "required": ["query"] + } + } + } + ] + +# Global MCP manager +mcp_manager = MCPManager() + +# ============================================================================ +# HELPER FUNCTIONS +# ============================================================================ + +def supports_function_calling(model: Dict[str, Any]) -> bool: + """Check if model supports function calling.""" + supported_params = model.get("supported_parameters", []) + return "tools" in supported_params or "functions" in supported_params + +def check_for_updates(current_version: str) -> str: + """Check for updates.""" try: response = requests.get( 'https://gitlab.pm/api/v1/repos/rune/oai/releases/latest', @@ -534,36 +2614,20 @@ def check_for_updates(current_version: str) -> str: logger.debug(f"Already up to date: {current_version}") return f"[bold green]oAI version {current_version} (up to date)[/]" - except requests.exceptions.HTTPError as e: - logger.warning(f"HTTP error checking for updates: {e.response.status_code}") - return f"[bold green]oAI version {current_version}[/]" - except requests.exceptions.ConnectionError: - logger.warning("Network error checking for updates (offline?)") - return f"[bold green]oAI version {current_version}[/]" - except requests.exceptions.Timeout: - logger.warning("Timeout checking for updates") - return f"[bold green]oAI version {current_version}[/]" - except requests.exceptions.RequestException as e: - logger.warning(f"Request error checking for updates: {type(e).__name__}") - return f"[bold green]oAI version {current_version}[/]" - except (KeyError, ValueError) as e: - logger.warning(f"Invalid API response checking for updates: {e}") - return f"[bold green]oAI version {current_version}[/]" - except Exception as e: - logger.error(f"Unexpected error checking for updates: {e}") + except: return f"[bold green]oAI version {current_version}[/]" def save_conversation(name: str, data: List[Dict[str, str]]): - """Save conversation history to DB.""" + """Save conversation.""" timestamp = datetime.datetime.now().isoformat() data_json = json.dumps(data) - with sqlite3.connect(DB_FILE) as conn: + with sqlite3.connect(str(database)) as conn: conn.execute('INSERT INTO conversation_sessions (name, timestamp, data) VALUES (?, ?, ?)', (name, timestamp, data_json)) conn.commit() def load_conversation(name: str) -> Optional[List[Dict[str, str]]]: - """Load conversation history from DB (latest by timestamp).""" - with sqlite3.connect(DB_FILE) as conn: + """Load conversation.""" + with sqlite3.connect(str(database)) as conn: cursor = conn.execute('SELECT data FROM conversation_sessions WHERE name = ? ORDER BY timestamp DESC LIMIT 1', (name,)) result = cursor.fetchone() if result: @@ -571,15 +2635,15 @@ def load_conversation(name: str) -> Optional[List[Dict[str, str]]]: return None def delete_conversation(name: str) -> int: - """Delete all conversation sessions with the given name. Returns number of deleted rows.""" - with sqlite3.connect(DB_FILE) as conn: + """Delete conversation.""" + with sqlite3.connect(str(database)) as conn: cursor = conn.execute('DELETE FROM conversation_sessions WHERE name = ?', (name,)) conn.commit() return cursor.rowcount def list_conversations() -> List[Dict[str, Any]]: - """List all saved conversations from DB with metadata.""" - with sqlite3.connect(DB_FILE) as conn: + """List conversations.""" + with sqlite3.connect(str(database)) as conn: cursor = conn.execute(''' SELECT name, MAX(timestamp) as last_saved, data FROM conversation_sessions @@ -598,34 +2662,32 @@ def list_conversations() -> List[Dict[str, Any]]: return conversations def estimate_cost(input_tokens: int, output_tokens: int) -> float: - """Estimate cost in USD based on token counts.""" + """Estimate cost.""" return (input_tokens * MODEL_PRICING['input'] / 1_000_000) + (output_tokens * MODEL_PRICING['output'] / 1_000_000) def has_web_search_capability(model: Dict[str, Any]) -> bool: - """Check if model supports web search based on supported_parameters.""" + """Check web search capability.""" supported_params = model.get("supported_parameters", []) - # Web search is typically indicated by 'tools' parameter support return "tools" in supported_params def has_image_capability(model: Dict[str, Any]) -> bool: - """Check if model supports image input based on input modalities.""" + """Check image capability.""" architecture = model.get("architecture", {}) input_modalities = architecture.get("input_modalities", []) return "image" in input_modalities def supports_online_mode(model: Dict[str, Any]) -> bool: - """Check if model supports :online suffix for web search.""" - # Models that support tools parameter can use :online + """Check online mode support.""" return has_web_search_capability(model) def get_effective_model_id(base_model_id: str, online_enabled: bool) -> str: - """Get the effective model ID with :online suffix if enabled.""" + """Get effective model ID.""" if online_enabled and not base_model_id.endswith(':online'): return f"{base_model_id}:online" return base_model_id def export_as_markdown(session_history: List[Dict[str, str]], session_system_prompt: str = "") -> str: - """Export conversation history as Markdown.""" + """Export as Markdown.""" lines = ["# Conversation Export", ""] if session_system_prompt: lines.extend([f"**System Prompt:** {session_system_prompt}", ""]) @@ -651,7 +2713,7 @@ def export_as_markdown(session_history: List[Dict[str, str]], session_system_pro return "\n".join(lines) def export_as_json(session_history: List[Dict[str, str]], session_system_prompt: str = "") -> str: - """Export conversation history as JSON.""" + """Export as JSON.""" export_data = { "export_date": datetime.datetime.now().isoformat(), "system_prompt": session_system_prompt, @@ -661,8 +2723,7 @@ def export_as_json(session_history: List[Dict[str, str]], session_system_prompt: return json.dumps(export_data, indent=2, ensure_ascii=False) def export_as_html(session_history: List[Dict[str, str]], session_system_prompt: str = "") -> str: - """Export conversation history as HTML.""" - # Escape HTML special characters + """Export as HTML.""" def escape_html(text): return text.replace('&', '&').replace('<', '<').replace('>', '>').replace('"', '"').replace("'", ''') @@ -733,99 +2794,84 @@ def export_as_html(session_history: List[Dict[str, str]], session_system_prompt: return "\n".join(html_parts) -def show_command_help(command: str): - """Display detailed help for a specific command.""" - # Normalize command to ensure it starts with / - if not command.startswith('/'): - command = '/' + command +def show_command_help(command_or_topic: str): + """Display detailed help for command or topic.""" + # Handle topics (like 'mcp') + if not command_or_topic.startswith('/'): + if command_or_topic.lower() == 'mcp': + help_data = COMMAND_HELP['mcp'] + + help_content = [] + help_content.append(f"[bold cyan]Description:[/]") + help_content.append(help_data['description']) + help_content.append("") + help_content.append(help_data['notes']) + + console.print(Panel( + "\n".join(help_content), + title="[bold green]MCP - Model Context Protocol Guide[/]", + title_align="left", + border_style="green", + width=console.width - 4 + )) + + app_logger.info("Displayed MCP comprehensive guide") + return + else: + command_or_topic = '/' + command_or_topic - # Check if command exists - if command not in COMMAND_HELP: - console.print(f"[bold red]Unknown command: {command}[/]") + if command_or_topic not in COMMAND_HELP: + console.print(f"[bold red]Unknown command: {command_or_topic}[/]") console.print("[bold yellow]Type /help to see all available commands.[/]") - app_logger.warning(f"Help requested for unknown command: {command}") + console.print("[bold yellow]Type /help mcp for comprehensive MCP guide.[/]") + app_logger.warning(f"Help requested for unknown command: {command_or_topic}") return - help_data = COMMAND_HELP[command] + help_data = COMMAND_HELP[command_or_topic] - # Create detailed help panel help_content = [] - # Aliases if available if 'aliases' in help_data: aliases_str = ", ".join(help_data['aliases']) help_content.append(f"[bold cyan]Aliases:[/] {aliases_str}") help_content.append("") - # Description help_content.append(f"[bold cyan]Description:[/]") help_content.append(help_data['description']) help_content.append("") - # Usage help_content.append(f"[bold cyan]Usage:[/]") help_content.append(f"[yellow]{help_data['usage']}[/]") help_content.append("") - # Examples if 'examples' in help_data and help_data['examples']: help_content.append(f"[bold cyan]Examples:[/]") for desc, example in help_data['examples']: - help_content.append(f" [dim]{desc}:[/]") - help_content.append(f" [green]{example}[/]") - help_content.append("") + if not desc and not example: + help_content.append("") + elif desc.startswith('━━━'): + help_content.append(f"[bold yellow]{desc}[/]") + else: + help_content.append(f" [dim]{desc}:[/]" if desc else "") + help_content.append(f" [green]{example}[/]" if example else "") + help_content.append("") - # Notes if 'notes' in help_data: help_content.append(f"[bold cyan]Notes:[/]") help_content.append(f"[dim]{help_data['notes']}[/]") console.print(Panel( "\n".join(help_content), - title=f"[bold green]Help: {command}[/]", + title=f"[bold green]Help: {command_or_topic}[/]", title_align="left", border_style="green", width=console.width - 4 )) - app_logger.info(f"Displayed detailed help for command: {command}") - -# Load configs (AFTER logging is set up) -API_KEY = get_config('api_key') -OPENROUTER_BASE_URL = get_config('base_url') or "https://openrouter.ai/api/v1" -STREAM_ENABLED = get_config('stream_enabled') or "on" -DEFAULT_MODEL_ID = get_config('default_model') -MAX_TOKEN = int(get_config('max_token') or "100000") -COST_WARNING_THRESHOLD = float(get_config(HIGH_COST_WARNING) or "0.01") # Configurable cost threshold for alerts -DEFAULT_ONLINE_MODE = get_config('default_online_mode') or "off" # New: Default online mode setting - -# Fetch models with app identification headers -models_data = [] -text_models = [] -try: - headers = { - "Authorization": f"Bearer {API_KEY}", - "HTTP-Referer": APP_URL, - "X-Title": APP_NAME - } if API_KEY else { - "HTTP-Referer": APP_URL, - "X-Title": APP_NAME - } - response = requests.get(f"{OPENROUTER_BASE_URL}/models", headers=headers) - response.raise_for_status() - models_data = response.json()["data"] - text_models = [m for m in models_data if "modalities" not in m or "video" not in (m.get("modalities") or [])] - selected_model_default = None - if DEFAULT_MODEL_ID: - selected_model_default = next((m for m in text_models if m["id"] == DEFAULT_MODEL_ID), None) - if not selected_model_default: - console.print(f"[bold yellow]Warning: Default model '{DEFAULT_MODEL_ID}' unavailable. Use '/config model'.[/]") -except Exception as e: - models_data = [] - text_models = [] - app_logger.error(f"Failed to fetch models: {e}") + app_logger.info(f"Displayed detailed help for command: {command_or_topic}") def get_credits(api_key: str, base_url: str = OPENROUTER_BASE_URL) -> Optional[Dict[str, str]]: + """Get credits.""" if not api_key: return None url = f"{base_url}/credits" @@ -851,7 +2897,7 @@ def get_credits(api_key: str, base_url: str = OPENROUTER_BASE_URL) -> Optional[D return None def check_credit_alerts(credits_data: Optional[Dict[str, str]]) -> List[str]: - """Check and return list of credit-related alerts.""" + """Check credit alerts.""" alerts = [] if credits_data: credits_left_value = float(credits_data['credits_left'].strip('$')) @@ -863,26 +2909,23 @@ def check_credit_alerts(credits_data: Optional[Dict[str, str]]) -> List[str]: return alerts def clear_screen(): + """Clear screen.""" try: print("\033[H\033[J", end="", flush=True) except: print("\n" * 100) def display_paginated_table(table: Table, title: str): - """Display a table with pagination support using Rich console for colored output, repeating header on each page.""" - # Get terminal height (subtract some lines for prompt and margins) + """Display paginated table.""" try: terminal_height = os.get_terminal_size().lines - 8 except: - terminal_height = 20 # Fallback if terminal size can't be determined + terminal_height = 20 - # Create a segment-based approach to capture Rich-rendered output from rich.segment import Segment - # Render the table to segments segments = list(console.render(table)) - # Convert segments to lines while preserving style current_line_segments = [] all_lines = [] @@ -893,27 +2936,22 @@ def display_paginated_table(table: Table, title: str): else: current_line_segments.append(segment) - # Add last line if not empty if current_line_segments: all_lines.append(current_line_segments) total_lines = len(all_lines) - # If fits on one screen after segment analysis if total_lines <= terminal_height: console.print(Panel(table, title=title, title_align="left")) return - # Separate header from data rows header_lines = [] data_lines = [] - # Find where the header ends header_end_index = 0 found_header_text = False for i, line_segments in enumerate(all_lines): - # Check if this line contains header-style text has_header_style = any( seg.style and ('bold' in str(seg.style) or 'magenta' in str(seg.style)) for seg in line_segments @@ -922,56 +2960,44 @@ def display_paginated_table(table: Table, title: str): if has_header_style: found_header_text = True - # After finding header text, the next line with box-drawing chars is the separator if found_header_text and i > 0: line_text = ''.join(seg.text for seg in line_segments) if any(char in line_text for char in ['─', '━', '┼', '╪', '┤', '├']): header_end_index = i break - # If we found a header separator, split there if header_end_index > 0: header_lines = all_lines[:header_end_index + 1] data_lines = all_lines[header_end_index + 1:] else: - # Fallback: assume first 3 lines are header header_lines = all_lines[:min(3, len(all_lines))] data_lines = all_lines[min(3, len(all_lines)):] - # Calculate how many data lines fit per page lines_per_page = terminal_height - len(header_lines) - # Display with pagination current_line = 0 page_number = 1 while current_line < len(data_lines): - # Clear screen for each page clear_screen() - # Print title console.print(f"[bold cyan]{title} (Page {page_number})[/]") - # Print header on every page for line_segments in header_lines: for segment in line_segments: console.print(segment.text, style=segment.style, end="") console.print() - # Calculate how many data lines to show on this page end_line = min(current_line + lines_per_page, len(data_lines)) - # Print data lines for this page for line_segments in data_lines[current_line:end_line]: for segment in line_segments: console.print(segment.text, style=segment.style, end="") console.print() - # Update position current_line = end_line page_number += 1 - # If there's more content, wait for user if current_line < len(data_lines): console.print(f"\n[dim yellow]--- Press SPACE for next page, or any other key to finish (Page {page_number - 1}, showing {end_line}/{len(data_lines)} data rows) ---[/dim yellow]") try: @@ -991,16 +3017,20 @@ def display_paginated_table(table: Table, title: str): finally: termios.tcsetattr(fd, termios.TCSADRAIN, old_settings) except: - # Fallback for Windows or if termios not available input_char = input().strip() if input_char != '': break else: break + # P3 + # ============================================================================ +# MAIN CHAT FUNCTION WITH FULL MCP SUPPORT (FILES + DATABASE) +# ============================================================================ @app.command() def chat(): global API_KEY, OPENROUTER_BASE_URL, STREAM_ENABLED, MAX_TOKEN, COST_WARNING_THRESHOLD, DEFAULT_ONLINE_MODE, LOG_MAX_SIZE_MB, LOG_BACKUP_COUNT, LOG_LEVEL, LOG_LEVEL_STR, app_logger + session_max_token = 0 session_system_prompt = "" session_history = [] @@ -1009,11 +3039,11 @@ def chat(): total_output_tokens = 0 total_cost = 0.0 message_count = 0 - middle_out_enabled = False # Session-level middle-out transform flag - conversation_memory_enabled = True # Memory ON by default - memory_start_index = 0 # Track when memory was last enabled - saved_conversations_cache = [] # Cache for /list results to use with /load by number - online_mode_enabled = DEFAULT_ONLINE_MODE == "on" # Initialize from config + middle_out_enabled = False + conversation_memory_enabled = True + memory_start_index = 0 + saved_conversations_cache = [] + online_mode_enabled = DEFAULT_ONLINE_MODE == "on" app_logger.info("Starting new chat session with memory enabled") @@ -1035,7 +3065,6 @@ def chat(): console.print("[bold red]No models available. Check API key/URL.[/]") raise typer.Exit() - # Check for credit alerts at startup credits_data = get_credits(API_KEY, OPENROUTER_BASE_URL) startup_credit_alerts = check_credit_alerts(credits_data) if startup_credit_alerts: @@ -1045,7 +3074,6 @@ def chat(): selected_model = selected_model_default - # Initialize OpenRouter client client = OpenRouter(api_key=API_KEY) if selected_model: @@ -1057,18 +3085,35 @@ def chat(): if not selected_model: console.print("[bold yellow]No model selected. Use '/model'.[/]") - # Persistent input history session = PromptSession(history=FileHistory(str(history_file))) while True: try: - user_input = session.prompt("You> ", auto_suggest=AutoSuggestFromHistory()).strip() + # ============================================================ + # BUILD PROMPT PREFIX WITH MODE INDICATOR + # ============================================================ + prompt_prefix = "You> " + if mcp_manager.enabled: + if mcp_manager.mode == "files": + prompt_prefix = "[🔧 MCP: Files] You> " + elif mcp_manager.mode == "database" and mcp_manager.selected_db_index is not None: + db = mcp_manager.databases[mcp_manager.selected_db_index] + prompt_prefix = f"[🗄️ MCP: DB #{mcp_manager.selected_db_index + 1}] You> " - # Handle // escape sequence - convert to single / and treat as regular text + # ============================================================ + # INITIALIZE LOOP VARIABLES + # ============================================================ + text_part = "" + file_attachments = [] + content_blocks = [] + + user_input = session.prompt(prompt_prefix, auto_suggest=AutoSuggestFromHistory()).strip() + + # Handle escape sequence if user_input.startswith("//"): - user_input = user_input[1:] # Remove first slash, keep the rest + user_input = user_input[1:] - # Check for unknown commands + # Check unknown commands elif user_input.startswith("/") and user_input.lower() not in ["exit", "quit", "bye"]: command_word = user_input.split()[0].lower() if user_input.split() else user_input.lower() @@ -1084,7 +3129,479 @@ def chat(): console.print("[bold yellow]Goodbye![/]") return - # Commands with logging + # ============================================================ + # MCP COMMANDS + # ============================================================ + if user_input.lower().startswith("/mcp"): + parts = user_input[5:].strip().split(maxsplit=1) + mcp_command = parts[0].lower() if parts else "" + mcp_args = parts[1] if len(parts) > 1 else "" + + if mcp_command == "enable": + result = mcp_manager.enable() + if result['success']: + console.print(f"[bold green]✓ {result['message']}[/]") + if result['folder_count'] == 0 and result['database_count'] == 0: + console.print("\n[bold yellow]No folders or databases configured yet.[/]") + console.print("\n[bold cyan]To get started:[/]") + console.print(" [bold yellow]Files:[/] /mcp add ~/Documents") + console.print(" [bold yellow]Databases:[/] /mcp add db ~/app/data.db") + else: + folders_msg = f"{result['folder_count']} folder(s)" if result['folder_count'] else "no folders" + dbs_msg = f"{result['database_count']} database(s)" if result['database_count'] else "no databases" + console.print(f"\n[bold cyan]MCP active with {folders_msg} and {dbs_msg}.[/]") + + warning = mcp_manager.config.get_filesystem_warning() + if warning: + console.print(warning) + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command == "disable": + result = mcp_manager.disable() + if result['success']: + console.print(f"[bold green]✓ {result['message']}[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command == "add": + # Check if it's a database or folder + if mcp_args.startswith("db "): + # Database + db_path = mcp_args[3:].strip() + if not db_path: + console.print("[bold red]Usage: /mcp add db [/]") + continue + + result = mcp_manager.add_database(db_path) + + if result['success']: + db = result['database'] + + console.print("\n[bold yellow]⚠️ Database Check:[/]") + console.print(f"Adding: [bold]{db['name']}[/]") + console.print(f"Size: {db['size'] / (1024 * 1024):.2f} MB") + console.print(f"Tables: {', '.join(db['tables'])} ({len(db['tables'])} total)") + + if db['size'] > 100 * 1024 * 1024: # > 100MB + console.print("\n[bold yellow]⚠️ Large database detected! Queries may be slow.[/]") + + console.print("\nMCP will be able to:") + console.print(" [green]✓[/green] Inspect database schema") + console.print(" [green]✓[/green] Search for data across tables") + console.print(" [green]✓[/green] Execute read-only SQL queries") + console.print(" [red]✗[/red] Modify data (read-only mode)") + + try: + confirm = typer.confirm("\nProceed?", default=True) + if not confirm: + # Remove it + mcp_manager.remove_database(str(result['number'])) + console.print("[bold yellow]Cancelled. Database not added.[/]") + continue + except (EOFError, KeyboardInterrupt): + mcp_manager.remove_database(str(result['number'])) + console.print("\n[bold yellow]Cancelled. Database not added.[/]") + continue + + console.print(f"\n[bold green]✓ {result['message']}[/]") + console.print(f"\n[bold cyan]Use '/mcp db {result['number']}' to start querying this database.[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + else: + # Folder + folder_path = mcp_args + if not folder_path: + console.print("[bold red]Usage: /mcp add or /mcp add db [/]") + continue + + result = mcp_manager.add_folder(folder_path) + + if result['success']: + stats = result['stats'] + + console.print("\n[bold yellow]⚠️ Security Check:[/]") + console.print(f"You are granting MCP access to: [bold]{result['path']}[/]") + + if stats.get('exists'): + file_count = stats['file_count'] + size_mb = stats['size_mb'] + console.print(f"This folder contains: [bold]{file_count} files ({size_mb:.1f} MB)[/]") + + console.print("\nMCP will be able to:") + console.print(" [green]✓[/green] Read files in this folder") + console.print(" [green]✓[/green] List and search files") + console.print(" [green]✓[/green] Access subfolders recursively") + console.print(" [green]✓[/green] Automatically respect .gitignore patterns") + console.print(" [red]✗[/red] Delete or modify files (read-only)") + + try: + confirm = typer.confirm("\nProceed?", default=True) + if not confirm: + mcp_manager.allowed_folders.remove(mcp_manager.config.normalize_path(folder_path)) + mcp_manager._save_folders() + console.print("[bold yellow]Cancelled. Folder not added.[/]") + continue + except (EOFError, KeyboardInterrupt): + mcp_manager.allowed_folders.remove(mcp_manager.config.normalize_path(folder_path)) + mcp_manager._save_folders() + console.print("\n[bold yellow]Cancelled. Folder not added.[/]") + continue + + console.print(f"\n[bold green]✓ Added {result['path']} to MCP allowed folders[/]") + console.print(f"[dim cyan]MCP now has access to {result['total_folders']} folder(s) total.[/]") + + if result.get('warning'): + console.print(f"\n[bold yellow]⚠️ {result['warning']}[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command in ["remove", "rem"]: + # Check if it's a database or folder + if mcp_args.startswith("db "): + # Database + db_ref = mcp_args[3:].strip() + if not db_ref: + console.print("[bold red]Usage: /mcp remove db [/]") + continue + + result = mcp_manager.remove_database(db_ref) + + if result['success']: + console.print(f"\n[bold yellow]Removing: {result['database']['name']}[/]") + + try: + confirm = typer.confirm("Confirm removal?", default=False) + if not confirm: + console.print("[bold yellow]Cancelled.[/]") + # Re-add it + mcp_manager.databases.append(result['database']) + mcp_manager._save_database(result['database']) + continue + except (EOFError, KeyboardInterrupt): + console.print("\n[bold yellow]Cancelled.[/]") + mcp_manager.databases.append(result['database']) + mcp_manager._save_database(result['database']) + continue + + console.print(f"\n[bold green]✓ {result['message']}[/]") + + if result.get('warning'): + console.print(f"\n[bold yellow]⚠️ {result['warning']}[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + else: + # Folder + if not mcp_args: + console.print("[bold red]Usage: /mcp remove or /mcp remove db [/]") + continue + + result = mcp_manager.remove_folder(mcp_args) + + if result['success']: + console.print(f"\n[bold yellow]Removing: {result['path']}[/]") + + try: + confirm = typer.confirm("Confirm removal?", default=False) + if not confirm: + mcp_manager.allowed_folders.append(mcp_manager.config.normalize_path(result['path'])) + mcp_manager._save_folders() + console.print("[bold yellow]Cancelled. Folder not removed.[/]") + continue + except (EOFError, KeyboardInterrupt): + mcp_manager.allowed_folders.append(mcp_manager.config.normalize_path(result['path'])) + mcp_manager._save_folders() + console.print("\n[bold yellow]Cancelled. Folder not removed.[/]") + continue + + console.print(f"\n[bold green]✓ Removed {result['path']} from MCP allowed folders[/]") + console.print(f"[dim cyan]MCP now has access to {result['total_folders']} folder(s) total.[/]") + + if result.get('warning'): + console.print(f"\n[bold yellow]⚠️ {result['warning']}[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command == "list": + result = mcp_manager.list_folders() + + if result['success']: + if result['total_folders'] == 0: + console.print("[bold yellow]No folders configured.[/]") + console.print("\n[bold cyan]Add a folder with: /mcp add ~/Documents[/]") + continue + + status_indicator = "[green]✓[/green]" if mcp_manager.enabled else "[red]✗[/red]" + + table = Table( + "No.", "Path", "Files", "Size", + show_header=True, + header_style="bold magenta" + ) + + for folder_info in result['folders']: + number = str(folder_info['number']) + path = folder_info['path'] + + if folder_info['exists']: + files = f"📁 {folder_info['file_count']}" + size = f"{folder_info['size_mb']:.1f} MB" + else: + files = "[red]Not found[/red]" + size = "-" + + table.add_row(number, path, files, size) + + gitignore_info = "" + if mcp_manager.server: + gitignore_status = "on" if mcp_manager.server.respect_gitignore else "off" + pattern_count = len(mcp_manager.server.gitignore_parser.patterns) + gitignore_info = f" | .gitignore: {gitignore_status} ({pattern_count} patterns)" + + console.print(Panel( + table, + title=f"[bold green]MCP Folders: {'Active' if mcp_manager.enabled else 'Inactive'} {status_indicator}[/]", + title_align="left", + subtitle=f"[dim]Total: {result['total_folders']} folders, {result['total_files']} files ({result['total_size_mb']:.1f} MB){gitignore_info}[/]", + subtitle_align="right" + )) + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command == "db": + if not mcp_args: + # Show current database or list all + if mcp_manager.mode == "database" and mcp_manager.selected_db_index is not None: + db = mcp_manager.databases[mcp_manager.selected_db_index] + console.print(f"[bold cyan]Currently using database #{mcp_manager.selected_db_index + 1}: {db['name']}[/]") + console.print(f"[dim]Path: {db['path']}[/]") + console.print(f"[dim]Tables: {', '.join(db['tables'])}[/]") + else: + console.print("[bold yellow]Not in database mode. Use '/mcp db ' to select a database.[/]") + + # Also show hint to list + console.print("\n[dim]Use '/mcp db list' to see all databases[/]") + continue + + if mcp_args == "list": + result = mcp_manager.list_databases() + + if result['success']: + if result['count'] == 0: + console.print("[bold yellow]No databases configured.[/]") + console.print("\n[bold cyan]Add a database with: /mcp add db ~/app/data.db[/]") + continue + + table = Table( + "No.", "Name", "Tables", "Size", "Status", + show_header=True, + header_style="bold magenta" + ) + + for db_info in result['databases']: + number = str(db_info['number']) + name = db_info['name'] + table_count = f"{db_info['table_count']} tables" + size = f"{db_info['size_mb']:.1f} MB" + + if db_info.get('warning'): + status = f"[red]{db_info['warning']}[/red]" + else: + status = "[green]✓[/green]" + + table.add_row(number, name, table_count, size, status) + + console.print(Panel( + table, + title="[bold green]MCP Databases[/]", + title_align="left", + subtitle=f"[dim]Total: {result['count']} database(s) | Use '/mcp db ' to select[/]", + subtitle_align="right" + )) + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + # Switch to database mode + try: + db_num = int(mcp_args) + result = mcp_manager.switch_mode("database", db_num) + + if result['success']: + db = result['database'] + console.print(f"\n[bold green]✓ {result['message']}[/]") + console.print(f"[dim cyan]Tables: {', '.join(db['tables'])}[/]") + console.print(f"\n[bold cyan]Available tools:[/] inspect_database, search_database, query_database") + console.print(f"\n[bold yellow]You can now ask questions about this database![/]") + console.print(f"[dim]Examples:[/]") + console.print(f" 'Show me all tables'") + console.print(f" 'Search for records mentioning X'") + console.print(f" 'How many rows in the users table?'") + console.print(f"\n[dim]Switch back to files: /mcp files[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + except ValueError: + console.print(f"[bold red]Invalid database number: {mcp_args}[/]") + console.print(f"[bold yellow]Use '/mcp db list' to see available databases[/]") + continue + + elif mcp_command == "files": + result = mcp_manager.switch_mode("files") + + if result['success']: + console.print(f"[bold green]✓ {result['message']}[/]") + console.print(f"\n[bold cyan]Available tools:[/] read_file, list_directory, search_files") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + elif mcp_command == "gitignore": + if not mcp_args: + if mcp_manager.server: + status = "enabled" if mcp_manager.server.respect_gitignore else "disabled" + pattern_count = len(mcp_manager.server.gitignore_parser.patterns) + console.print(f"[bold blue].gitignore filtering: {status}[/]") + console.print(f"[dim cyan]Loaded {pattern_count} pattern(s) from .gitignore files[/]") + else: + console.print("[bold yellow]MCP is not enabled. Enable with /mcp enable[/]") + continue + + if mcp_args.lower() == "on": + result = mcp_manager.toggle_gitignore(True) + if result['success']: + console.print(f"[bold green]✓ {result['message']}[/]") + console.print(f"[dim cyan]Using {result['pattern_count']} .gitignore pattern(s)[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + elif mcp_args.lower() == "off": + result = mcp_manager.toggle_gitignore(False) + if result['success']: + console.print(f"[bold green]✓ {result['message']}[/]") + console.print("[bold yellow]Warning: All files will be visible, including those in .gitignore[/]") + else: + console.print(f"[bold red]❌ {result['error']}[/]") + else: + console.print("[bold yellow]Usage: /mcp gitignore on|off[/]") + continue + + elif mcp_command == "status": + result = mcp_manager.get_status() + + if result['success']: + status_color = "green" if result['enabled'] else "red" + status_text = "Active ✓" if result['enabled'] else "Inactive ✗" + + table = Table( + "Property", "Value", + show_header=True, + header_style="bold magenta" + ) + + table.add_row("Status", f"[{status_color}]{status_text}[/{status_color}]") + + # Mode info + mode_info = result['mode_info'] + table.add_row("Current Mode", mode_info['mode_display']) + if 'database' in mode_info: + db = mode_info['database'] + table.add_row("Database Path", db['path']) + table.add_row("Database Tables", ', '.join(db['tables'])) + + if result['uptime']: + table.add_row("Uptime", result['uptime']) + + table.add_row("", "") + table.add_row("[bold]Configuration[/]", "") + table.add_row("Allowed Folders", str(result['folder_count'])) + table.add_row("Databases", str(result['database_count'])) + table.add_row("Total Files Accessible", str(result['total_files'])) + table.add_row("Total Size", f"{result['total_size_mb']:.1f} MB") + table.add_row(".gitignore Filtering", result['gitignore_status']) + if result['gitignore_patterns'] > 0: + table.add_row(".gitignore Patterns", str(result['gitignore_patterns'])) + + if result['enabled']: + table.add_row("", "") + table.add_row("[bold]Tools Available[/]", "") + for tool in result['tools_available']: + table.add_row(f" ✓ {tool}", "") + + stats = result['stats'] + if stats['total_calls'] > 0: + table.add_row("", "") + table.add_row("[bold]Session Stats[/]", "") + table.add_row("Total Tool Calls", str(stats['total_calls'])) + + if stats['reads'] > 0: + table.add_row("Files Read", str(stats['reads'])) + if stats['lists'] > 0: + table.add_row("Directories Listed", str(stats['lists'])) + if stats['searches'] > 0: + table.add_row("File Searches", str(stats['searches'])) + if stats['db_inspects'] > 0: + table.add_row("DB Inspections", str(stats['db_inspects'])) + if stats['db_searches'] > 0: + table.add_row("DB Searches", str(stats['db_searches'])) + if stats['db_queries'] > 0: + table.add_row("SQL Queries", str(stats['db_queries'])) + + if stats['last_used']: + try: + last_used = datetime.datetime.fromisoformat(stats['last_used']) + time_ago = datetime.datetime.now() - last_used + + if time_ago.seconds < 60: + time_str = f"{time_ago.seconds} seconds ago" + elif time_ago.seconds < 3600: + time_str = f"{time_ago.seconds // 60} minutes ago" + else: + time_str = f"{time_ago.seconds // 3600} hours ago" + + table.add_row("Last Used", time_str) + except: + pass + + console.print(Panel( + table, + title="[bold green]MCP Status[/]", + title_align="left" + )) + else: + console.print(f"[bold red]❌ {result['error']}[/]") + continue + + else: + console.print("[bold cyan]MCP (Model Context Protocol) Commands:[/]\n") + console.print(" [bold yellow]/mcp enable[/] - Start MCP server") + console.print(" [bold yellow]/mcp disable[/] - Stop MCP server") + console.print(" [bold yellow]/mcp status[/] - Show comprehensive MCP status") + console.print("") + console.print(" [bold cyan]FILE MODE (default):[/]") + console.print(" [bold yellow]/mcp add [/] - Add folder for file access") + console.print(" [bold yellow]/mcp remove [/] - Remove folder") + console.print(" [bold yellow]/mcp list[/] - List all folders") + console.print(" [bold yellow]/mcp files[/] - Switch to file mode") + console.print(" [bold yellow]/mcp gitignore [on|off][/] - Toggle .gitignore filtering") + console.print("") + console.print(" [bold cyan]DATABASE MODE:[/]") + console.print(" [bold yellow]/mcp add db [/] - Add SQLite database") + console.print(" [bold yellow]/mcp db list[/] - List all databases") + console.print(" [bold yellow]/mcp db [/] - Switch to database mode") + console.print(" [bold yellow]/mcp remove db [/]- Remove database") + console.print("") + console.print("[dim]Use '/help /mcp' for detailed command help[/]") + console.print("[dim]Use '/help mcp' for comprehensive MCP guide[/]") + continue + + # ============================================================ + # ALL OTHER COMMANDS + # ============================================================ + if user_input.lower() == "/retry": if not session_history: console.print("[bold red]No history to retry.[/]") @@ -1094,6 +3611,7 @@ def chat(): console.print("[bold green]Retrying last prompt...[/]") app_logger.info(f"Retrying prompt: {last_prompt[:100]}...") user_input = last_prompt + elif user_input.lower().startswith("/online"): args = user_input[8:].strip() if not args: @@ -1129,6 +3647,7 @@ def chat(): else: console.print("[bold yellow]Usage: /online on|off (or /online to view status)[/]") continue + elif user_input.lower().startswith("/memory"): args = user_input[8:].strip() if not args: @@ -1154,6 +3673,7 @@ def chat(): else: console.print("[bold yellow]Usage: /memory on|off (or /memory to view status)[/]") continue + elif user_input.lower().startswith("/paste"): optional_prompt = user_input[7:].strip() @@ -1205,7 +3725,7 @@ def chat(): console.print(f"[bold red]Error processing clipboard content: {e}[/]") app_logger.error(f"Clipboard processing error: {e}") continue - + elif user_input.lower().startswith("/export"): args = user_input[8:].strip().split(maxsplit=1) if len(args) != 2: @@ -1244,6 +3764,7 @@ def chat(): console.print(f"[bold red]Export failed: {e}[/]") app_logger.error(f"Export error: {e}") continue + elif user_input.lower().startswith("/save"): args = user_input[6:].strip() if not args: @@ -1256,6 +3777,7 @@ def chat(): console.print(f"[bold green]Conversation saved as '{args}'.[/]") app_logger.info(f"Conversation saved as '{args}' with {len(session_history)} messages") continue + elif user_input.lower().startswith("/load"): args = user_input[6:].strip() if not args: @@ -1292,6 +3814,7 @@ def chat(): console.print(f"[bold green]Conversation '{conversation_name}' loaded with {len(session_history)} messages.[/]") app_logger.info(f"Conversation '{conversation_name}' loaded with {len(session_history)} messages") continue + elif user_input.lower().startswith("/delete"): args = user_input[8:].strip() if not args: @@ -1331,6 +3854,7 @@ def chat(): console.print(f"[bold red]Conversation '{conversation_name}' not found.[/]") app_logger.warning(f"Delete failed for '{conversation_name}' - not found") continue + elif user_input.lower() == "/list": conversations = list_conversations() if not conversations: @@ -1359,6 +3883,7 @@ def chat(): console.print(Panel(table, title=f"[bold green]Saved Conversations ({len(conversations)} total)[/]", title_align="left", subtitle="[dim]Use /load or /delete to manage conversations[/]", subtitle_align="right")) app_logger.info(f"User viewed conversation list - {len(conversations)} conversations") continue + elif user_input.lower() == "/prev": if not session_history or current_index <= 0: console.print("[bold red]No previous response.[/]") @@ -1369,6 +3894,7 @@ def chat(): console.print(Panel(md, title=f"[bold green]Previous Response ({current_index + 1}/{len(session_history)})[/]", title_align="left")) app_logger.debug(f"Viewed previous response at index {current_index}") continue + elif user_input.lower() == "/next": if not session_history or current_index >= len(session_history) - 1: console.print("[bold red]No next response.[/]") @@ -1379,6 +3905,7 @@ def chat(): console.print(Panel(md, title=f"[bold green]Next Response ({current_index + 1}/{len(session_history)})[/]", title_align="left")) app_logger.debug(f"Viewed next response at index {current_index}") continue + elif user_input.lower() == "/stats": credits = get_credits(API_KEY, OPENROUTER_BASE_URL) credits_left = credits['credits_left'] if credits else "Unknown" @@ -1395,6 +3922,7 @@ def chat(): console.print(f"[bold red]⚠️ {warning_text}[/]") app_logger.warning(f"Warnings in stats: {warning_text}") continue + elif user_input.lower().startswith("/middleout"): args = user_input[11:].strip() if not args: @@ -1409,6 +3937,7 @@ def chat(): else: console.print("[bold yellow]Usage: /middleout on|off (or /middleout to view status)[/]") continue + elif user_input.lower() == "/reset": try: confirm = typer.confirm("Reset conversation context? This clears history and prompt.", default=False) @@ -1496,7 +4025,6 @@ def chat(): if 1 <= choice <= len(filtered_models): selected_model = filtered_models[choice - 1] - # Apply default online mode if model supports it if supports_online_mode(selected_model) and DEFAULT_ONLINE_MODE == "on": online_mode_enabled = True console.print(f"[bold cyan]Selected: {selected_model['name']} ({selected_model['id']})[/]") @@ -1634,7 +4162,6 @@ def chat(): set_config('log_max_size_mb', str(new_size_mb)) LOG_MAX_SIZE_MB = new_size_mb - # Reload logging configuration immediately app_logger = reload_logging_config() console.print(f"[bold green]Log size limit set to {new_size_mb} MB and applied immediately.[/]") @@ -1715,6 +4242,14 @@ def chat(): DEFAULT_MODEL_ID = get_config('default_model') memory_status = "Enabled" if conversation_memory_enabled else "Disabled" memory_tracked = len(session_history) - memory_start_index if conversation_memory_enabled else 0 + + mcp_status = "Enabled" if mcp_manager.enabled else "Disabled" + mcp_folders = len(mcp_manager.allowed_folders) + mcp_databases = len(mcp_manager.databases) + mcp_mode = mcp_manager.mode + gitignore_status = "enabled" if (mcp_manager.server and mcp_manager.server.respect_gitignore) else "disabled" + gitignore_patterns = len(mcp_manager.server.gitignore_parser.patterns) if mcp_manager.server else 0 + table = Table("Setting", "Value", show_header=True, header_style="bold magenta", width=console.width - 10) table.add_row("API Key", API_KEY or "[Not set]") table.add_row("Base URL", OPENROUTER_BASE_URL or "[Not set]") @@ -1728,6 +4263,9 @@ def chat(): table.add_row("Current Model", "[Not set]" if selected_model is None else str(selected_model["name"])) table.add_row("Default Online Mode", "Enabled" if DEFAULT_ONLINE_MODE == "on" else "Disabled") table.add_row("Session Online Mode", "Enabled" if online_mode_enabled else "Disabled") + table.add_row("MCP Status", f"{mcp_status} ({mcp_folders} folders, {mcp_databases} DBs)") + table.add_row("MCP Mode", f"{mcp_mode}") + table.add_row("MCP .gitignore", f"{gitignore_status} ({gitignore_patterns} patterns)") table.add_row("Max Token", str(MAX_TOKEN)) table.add_row("Session Token", "[Not set]" if session_max_token == 0 else str(session_max_token)) table.add_row("Session System Prompt", session_system_prompt or "[Not set]") @@ -1767,22 +4305,24 @@ def chat(): if user_input.lower() == "/clear" or user_input.lower() == "/cl": clear_screen() DEFAULT_MODEL_ID = get_config('default_model') - token_value = session_max_token if session_max_token != 0 else " Not set" + token_value = session_max_token if session_max_token != 0 else "Not set" console.print(f"[bold cyan]Token limits: Max= {MAX_TOKEN}, Session={token_value}[/]") console.print("[bold blue]Active model[/] [bold red]%s[/]" %(str(selected_model["name"]) if selected_model else "None")) if online_mode_enabled: console.print("[bold cyan]Online mode: Enabled (web search active)[/]") + if mcp_manager.enabled: + gitignore_status = "on" if (mcp_manager.server and mcp_manager.server.respect_gitignore) else "off" + mode_display = "Files" if mcp_manager.mode == "files" else f"DB #{mcp_manager.selected_db_index + 1}" + console.print(f"[bold cyan]MCP: Enabled (Mode: {mode_display}, {len(mcp_manager.allowed_folders)} folders, {len(mcp_manager.databases)} DBs, .gitignore {gitignore_status})[/]") continue if user_input.lower().startswith("/help"): args = user_input[6:].strip() - # If a specific command is requested if args: show_command_help(args) continue - # Otherwise show the full help menu help_table = Table("Command", "Description", "Example", show_header=True, header_style="bold cyan", width=console.width - 10) # SESSION COMMANDS @@ -1793,50 +4333,117 @@ def chat(): ) help_table.add_row( "/clear or /cl", - "Clear the terminal screen for a clean interface. You can also use the keycombo [bold]ctrl+l[/]", + "Clear terminal screen. Keyboard shortcut: [bold]Ctrl+L[/]", "/clear\n/cl" ) help_table.add_row( - "/help [command]", - "Show this help menu or get detailed help for a specific command.", - "/help\n/help /model" + "/help [command|topic]", + "Show help menu or detailed help. Use '/help mcp' for MCP guide.", + "/help\n/help mcp" ) help_table.add_row( "/memory [on|off]", - "Toggle conversation memory. ON sends history (AI remembers), OFF sends only current message (saves cost).", + "Toggle conversation memory. ON sends history, OFF is stateless (saves cost).", "/memory\n/memory off" ) help_table.add_row( "/next", - "View the next response in history.", + "View next response in history.", "/next" ) help_table.add_row( "/online [on|off]", - "Enable/disable online mode (web search) for current session. Overrides default setting.", + "Enable/disable online mode (web search) for session.", "/online on\n/online off" ) help_table.add_row( "/paste [prompt]", - "Paste plain text/code from clipboard and send to AI. Optional prompt can be added.", - "/paste\n/paste Explain this code" + "Paste clipboard text/code. Optional prompt.", + "/paste\n/paste Explain" ) help_table.add_row( "/prev", - "View the previous response in history.", + "View previous response in history.", "/prev" ) help_table.add_row( "/reset", - "Clear conversation history and reset system prompt (resets session metrics). Requires confirmation.", + "Clear history and reset system prompt (requires confirmation).", "/reset" ) help_table.add_row( "/retry", - "Resend the last prompt from history.", + "Resend last prompt.", "/retry" ) + # MCP COMMANDS + help_table.add_row( + "[bold yellow]━━━ MCP (FILE & DATABASE ACCESS) ━━━[/]", + "", + "" + ) + help_table.add_row( + "/mcp enable", + "Start MCP server for file and database access.", + "/mcp enable" + ) + help_table.add_row( + "/mcp disable", + "Stop MCP server.", + "/mcp disable" + ) + help_table.add_row( + "/mcp status", + "Show mode, stats, folders, databases, and .gitignore status.", + "/mcp status" + ) + help_table.add_row( + "/mcp add ", + "Add folder for file access (auto-loads .gitignore).", + "/mcp add ~/Documents" + ) + help_table.add_row( + "/mcp add db ", + "Add SQLite database for querying.", + "/mcp add db ~/app.db" + ) + help_table.add_row( + "/mcp list", + "List all allowed folders with stats.", + "/mcp list" + ) + help_table.add_row( + "/mcp db list", + "List all databases with details.", + "/mcp db list" + ) + help_table.add_row( + "/mcp db ", + "Switch to database mode (select DB by number).", + "/mcp db 1" + ) + help_table.add_row( + "/mcp files", + "Switch to file mode (default).", + "/mcp files" + ) + help_table.add_row( + "/mcp remove ", + "Remove folder by path or number.", + "/mcp remove 2" + ) + help_table.add_row( + "/mcp remove db ", + "Remove database by number.", + "/mcp remove db 1" + ) + help_table.add_row( + "/mcp gitignore [on|off]", + "Toggle .gitignore filtering (respects project excludes).", + "/mcp gitignore on" + ) + # MODEL COMMANDS help_table.add_row( "[bold yellow]━━━ MODEL COMMANDS ━━━[/]", @@ -1845,16 +4452,16 @@ def chat(): ) help_table.add_row( "/info [model_id]", - "Display detailed info (pricing, modalities, context length, online support, etc.) for current or specified model.", + "Display model details (pricing, capabilities, context).", "/info\n/info gpt-4o" ) help_table.add_row( "/model [search]", - "Select or change the current model for the session. Shows image and online capabilities. Supports searching by name or ID.", + "Select/change model. Shows image and online capabilities.", "/model\n/model gpt" ) - # CONFIGURATION COMMANDS + # CONFIGURATION help_table.add_row( "[bold yellow]━━━ CONFIGURATION ━━━[/]", "", @@ -1862,56 +4469,56 @@ def chat(): ) help_table.add_row( "/config", - "View all current configurations, including limits, credits, and history.", + "View all configurations.", "/config" ) help_table.add_row( "/config api", - "Set or update the OpenRouter API key.", + "Set/update API key.", "/config api" ) help_table.add_row( - "/config costwarning [value]", - "Set the cost warning threshold. Alerts when response exceeds this cost (in USD).", + "/config costwarning [val]", + "Set cost warning threshold (USD).", "/config costwarning 0.05" ) help_table.add_row( "/config log [size_mb]", - "Set log file size limit in MB. Older logs are rotated automatically. Takes effect immediately.", + "Set log file size limit. Takes effect immediately.", "/config log 20" ) help_table.add_row( "/config loglevel [level]", - "Set log verbosity level. Valid levels: debug, info, warning, error, critical. Takes effect immediately.", - "/config loglevel debug\n/config loglevel warning" + "Set log verbosity (debug/info/warning/error/critical).", + "/config loglevel debug" ) help_table.add_row( - "/config maxtoken [value]", - "Set stored max token limit (persisted in DB). View current if no value provided.", + "/config maxtoken [val]", + "Set stored max token limit.", "/config maxtoken 50000" ) help_table.add_row( "/config model [search]", - "Set default model that loads on startup. Shows image and online capabilities. Doesn't change current session model.", + "Set default startup model.", "/config model gpt" ) help_table.add_row( "/config online [on|off]", - "Set default online mode for new model selections. Use '/online on|off' to override current session.", + "Set default online mode for new models.", "/config online on" ) help_table.add_row( "/config stream [on|off]", - "Enable or disable response streaming.", + "Enable/disable response streaming.", "/config stream off" ) help_table.add_row( "/config url", - "Set or update the base URL for OpenRouter API.", + "Set/update base URL.", "/config url" ) - # TOKEN & SYSTEM COMMANDS + # TOKEN & SYSTEM help_table.add_row( "[bold yellow]━━━ TOKEN & SYSTEM ━━━[/]", "", @@ -1919,21 +4526,21 @@ def chat(): ) help_table.add_row( "/maxtoken [value]", - "Set temporary session token limit (≤ stored max). View current if no value provided.", + "Set temporary session token limit.", "/maxtoken 2000" ) help_table.add_row( "/middleout [on|off]", - "Enable/disable middle-out transform to compress prompts exceeding context size.", + "Enable/disable prompt compression for large contexts.", "/middleout on" ) help_table.add_row( "/system [prompt|clear]", - "Set session-level system prompt to guide AI behavior. Use 'clear' to reset.", + "Set session system prompt. Use 'clear' to reset.", "/system You are a Python expert" ) - # CONVERSATION MANAGEMENT + # CONVERSATION MGMT help_table.add_row( "[bold yellow]━━━ CONVERSATION MGMT ━━━[/]", "", @@ -1941,31 +4548,31 @@ def chat(): ) help_table.add_row( "/delete ", - "Delete a saved conversation by name or number (from /list). Requires confirmation.", + "Delete saved conversation (requires confirmation).", "/delete my_chat\n/delete 3" ) help_table.add_row( - "/export ", - "Export conversation to file. Formats: md (Markdown), json (JSON), html (HTML).", - "/export md notes.md\n/export html report.html" + "/export ", + "Export conversation (md/json/html).", + "/export md notes.md" ) help_table.add_row( "/list", - "List all saved conversations with numbers, message counts, and timestamps.", + "List saved conversations with numbers and stats.", "/list" ) help_table.add_row( "/load ", - "Load a saved conversation by name or number (from /list). Resets session metrics.", + "Load saved conversation.", "/load my_chat\n/load 3" ) help_table.add_row( "/save ", - "Save current conversation history to database.", + "Save current conversation.", "/save my_chat" ) - # MONITORING & STATS + # MONITORING help_table.add_row( "[bold yellow]━━━ MONITORING & STATS ━━━[/]", "", @@ -1973,12 +4580,12 @@ def chat(): ) help_table.add_row( "/credits", - "Display credits left on your OpenRouter account with alerts.", + "Display OpenRouter credits with alerts.", "/credits" ) help_table.add_row( "/stats", - "Display session cost summary: tokens, cost, credits left, and warnings.", + "Display session stats: tokens, cost, credits.", "/stats" ) @@ -1990,17 +4597,17 @@ def chat(): ) help_table.add_row( "@/path/to/file", - "Attach files to messages: images (PNG, JPG, etc.), PDFs, and code files (.py, .js, etc.).", - "Debug @script.py\nSummarize @document.pdf\nAnalyze @image.png" + "Attach files: images (PNG, JPG), PDFs, code files.", + "Debug @script.py\nAnalyze @image.png" ) help_table.add_row( "Clipboard paste", - "Use /paste to send clipboard content (plain text/code) to AI.", - "/paste\n/paste Explain this" + "Use /paste to send clipboard content.", + "/paste\n/paste Explain" ) help_table.add_row( "// escape", - "Start message with // to send a literal / character (e.g., //command sends '/command' as text, not a command)", + "Start with // to send literal / character.", "//help sends '/help' as text" ) @@ -2012,7 +4619,7 @@ def chat(): ) help_table.add_row( "exit | quit | bye", - "Quit the chat application and display session summary.", + "Quit with session summary.", "exit" ) @@ -2020,7 +4627,7 @@ def chat(): help_table, title="[bold cyan]oAI Chat Help (Version %s)[/]" % version, title_align="center", - subtitle="💡 Tip: Commands are case-insensitive • Use /help for detailed help • Memory ON by default • Use // to escape / • Visit: https://iurl.no/oai", + subtitle="💡 Commands are case-insensitive • Use /help for details • Use /help mcp for MCP guide • Memory ON by default • MCP has file & DB modes • Use // to escape / • Visit: https://iurl.no/oai", subtitle_align="center", border_style="cyan" )) @@ -2029,10 +4636,11 @@ def chat(): if not selected_model: console.print("[bold yellow]Select a model first with '/model'.[/]") continue - # Process file attachments with PDF support - content_blocks = [] + + # ============================================================ + # PROCESS FILE ATTACHMENTS + # ============================================================ text_part = user_input - file_attachments = [] for match in re.finditer(r'@([^\s]+)', user_input): file_path = match.group(1) expanded_path = os.path.expanduser(os.path.abspath(file_path)) @@ -2049,7 +4657,6 @@ def chat(): with open(expanded_path, 'rb') as f: file_data = f.read() - # Handle images if mime_type and mime_type.startswith('image/'): modalities = selected_model.get("architecture", {}).get("input_modalities", []) if "image" not in modalities: @@ -2060,7 +4667,6 @@ def chat(): content_blocks.append({"type": "image_url", "image_url": {"url": f"data:{mime_type};base64,{b64_data}"}}) console.print(f"[dim green]✓ Image attached: {os.path.basename(expanded_path)} ({file_size / 1024:.1f} KB)[/]") - # Handle PDFs elif mime_type == 'application/pdf' or file_ext == '.pdf': modalities = selected_model.get("architecture", {}).get("input_modalities", []) supports_pdf = any(mod in modalities for mod in ["document", "pdf", "file"]) @@ -2072,7 +4678,6 @@ def chat(): content_blocks.append({"type": "image_url", "image_url": {"url": f"data:application/pdf;base64,{b64_data}"}}) console.print(f"[dim green]✓ PDF attached: {os.path.basename(expanded_path)} ({file_size / 1024:.1f} KB)[/]") - # Handle code/text files elif (mime_type == 'text/plain' or file_ext in SUPPORTED_CODE_EXTENSIONS): text_content = file_data.decode('utf-8') content_blocks.append({"type": "text", "text": f"Code File: {os.path.basename(expanded_path)}\n\n{text_content}"}) @@ -2105,12 +4710,28 @@ def chat(): console.print("[bold red]Prompt cannot be empty.[/]") continue - # Build API messages with conversation history if memory is enabled + # ============================================================ + # BUILD API MESSAGES + # ============================================================ api_messages = [] if session_system_prompt: api_messages.append({"role": "system", "content": session_system_prompt}) + # Add database context if in database mode + if mcp_manager.enabled and mcp_manager.mode == "database" and mcp_manager.selected_db_index is not None: + db = mcp_manager.databases[mcp_manager.selected_db_index] + db_context = f"""You are currently connected to SQLite database: {db['name']} +Available tables: {', '.join(db['tables'])} + +You can: +- Inspect the database schema with inspect_database +- Search for data across tables with search_database +- Execute read-only SQL queries with query_database + +All queries are read-only. INSERT/UPDATE/DELETE are not allowed.""" + api_messages.append({"role": "system", "content": db_context}) + if conversation_memory_enabled: for i in range(memory_start_index, len(session_history)): history_entry = session_history[i] @@ -2125,10 +4746,12 @@ def chat(): api_messages.append({"role": "user", "content": message_content}) - # Get effective model ID with :online suffix if enabled + # Get effective model ID effective_model_id = get_effective_model_id(selected_model["id"], online_mode_enabled) - # Build API params + # ======================================================================== + # BUILD API PARAMS WITH MCP TOOLS SUPPORT + # ======================================================================== api_params = { "model": effective_model_id, "messages": api_messages, @@ -2138,6 +4761,27 @@ def chat(): "X-Title": APP_NAME } } + + # Add MCP tools if enabled + mcp_tools_added = False + mcp_tools = [] + if mcp_manager.enabled: + if supports_function_calling(selected_model): + mcp_tools = mcp_manager.get_tools_schema() + if mcp_tools: + api_params["tools"] = mcp_tools + api_params["tool_choice"] = "auto" + mcp_tools_added = True + + # IMPORTANT: Disable streaming if MCP tools are present + if api_params["stream"]: + api_params["stream"] = False + app_logger.info("Disabled streaming due to MCP tool calls") + + app_logger.info(f"Added {len(mcp_tools)} MCP tools to request (mode: {mcp_manager.mode})") + else: + app_logger.debug(f"Model {selected_model['id']} doesn't support function calling - MCP tools not added") + if session_max_token > 0: api_params["max_tokens"] = session_max_token if middle_out_enabled: @@ -2148,17 +4792,44 @@ def chat(): history_messages_count = len(session_history) - memory_start_index if conversation_memory_enabled else 0 memory_status = "ON" if conversation_memory_enabled else "OFF" online_status = "ON" if online_mode_enabled else "OFF" - app_logger.info(f"API Request: Model '{effective_model_id}' (Online: {online_status}), Prompt length: {len(text_part)} chars, {file_count} file(s) attached, Memory: {memory_status}, History sent: {history_messages_count} messages, Transforms: middle-out {'enabled' if middle_out_enabled else 'disabled'}, App: {APP_NAME} ({APP_URL}).") + + if mcp_tools_added: + if mcp_manager.mode == "files": + mcp_status = f"ON (Files, {len(mcp_manager.allowed_folders)} folders, {len(mcp_tools)} tools)" + elif mcp_manager.mode == "database": + db = mcp_manager.databases[mcp_manager.selected_db_index] + mcp_status = f"ON (DB: {db['name']}, {len(mcp_tools)} tools)" + else: + mcp_status = f"ON ({len(mcp_tools)} tools)" + else: + mcp_status = "OFF" + + app_logger.info(f"API Request: Model '{effective_model_id}' (Online: {online_status}, MCP: {mcp_status}), Prompt length: {len(text_part)} chars, {file_count} file(s) attached, Memory: {memory_status}, History sent: {history_messages_count} messages.") - # Send and handle response - is_streaming = STREAM_ENABLED == "on" + # Send request + is_streaming = api_params["stream"] if is_streaming: console.print("[bold green]Streaming response...[/] [dim](Press Ctrl+C to cancel)[/]") if online_mode_enabled: console.print("[dim cyan]🌐 Online mode active - model has web search access[/]") console.print("") else: - console.print("[bold green]Thinking...[/]", end="\r") + thinking_msg = "Thinking" + if mcp_tools_added: + if mcp_manager.mode == "database": + thinking_msg = "Thinking (database tools available)" + else: + thinking_msg = "Thinking (MCP tools available)" + console.print(f"[bold green]{thinking_msg}...[/]", end="\r") + if online_mode_enabled: + console.print(f"[dim cyan]🌐 Online mode active[/]") + if mcp_tools_added: + if mcp_manager.mode == "files": + gitignore_status = "on" if (mcp_manager.server and mcp_manager.server.respect_gitignore) else "off" + console.print(f"[dim cyan]🔧 MCP active - AI can access {len(mcp_manager.allowed_folders)} folder(s) with {len(mcp_tools)} tools (.gitignore {gitignore_status})[/]") + elif mcp_manager.mode == "database": + db = mcp_manager.databases[mcp_manager.selected_db_index] + console.print(f"[dim cyan]🗄️ MCP active - AI can query database: {db['name']} ({len(db['tables'])} tables, {len(mcp_tools)} tools)[/]") start_time = time.time() try: @@ -2169,8 +4840,150 @@ def chat(): app_logger.error(f"API Error: {type(e).__name__}: {e}") continue + # ======================================================================== + # HANDLE TOOL CALLS (MCP FUNCTION CALLING) + # ======================================================================== + tool_call_loop_count = 0 + max_tool_loops = 5 + + while tool_call_loop_count < max_tool_loops: + wants_tool_call = False + + if hasattr(response, 'choices') and response.choices: + message = response.choices[0].message + + if hasattr(message, 'tool_calls') and message.tool_calls: + wants_tool_call = True + tool_calls = message.tool_calls + + console.print(f"\n[dim yellow]🔧 AI requesting {len(tool_calls)} tool call(s)...[/]") + app_logger.info(f"Model requested {len(tool_calls)} tool calls") + + tool_results = [] + for tool_call in tool_calls: + tool_name = tool_call.function.name + + try: + tool_args = json.loads(tool_call.function.arguments) + except json.JSONDecodeError as e: + app_logger.error(f"Failed to parse tool arguments: {e}") + tool_results.append({ + "tool_call_id": tool_call.id, + "role": "tool", + "name": tool_name, + "content": json.dumps({"error": f"Invalid arguments: {e}"}) + }) + continue + + args_display = ', '.join(f'{k}="{v}"' if isinstance(v, str) else f'{k}={v}' for k, v in tool_args.items()) + console.print(f"[dim cyan] → Calling: {tool_name}({args_display})[/]") + app_logger.info(f"Executing MCP tool: {tool_name} with args: {tool_args}") + + try: + result = asyncio.run(mcp_manager.call_tool(tool_name, **tool_args)) + except Exception as e: + app_logger.error(f"MCP tool execution error: {e}") + result = {"error": str(e)} + + if 'error' in result: + result_content = json.dumps({"error": result['error']}) + console.print(f"[dim red] ✗ Error: {result['error']}[/]") + app_logger.warning(f"MCP tool {tool_name} returned error: {result['error']}") + else: + # Display appropriate success message based on tool + if tool_name == 'search_files': + count = result.get('count', 0) + console.print(f"[dim green] ✓ Found {count} file(s)[/]") + elif tool_name == 'read_file': + size = result.get('size', 0) + truncated = " (truncated)" if result.get('truncated') else "" + console.print(f"[dim green] ✓ Read file ({size} bytes{truncated})[/]") + elif tool_name == 'list_directory': + count = result.get('count', 0) + truncated = " (limited to 1000)" if result.get('truncated') else "" + console.print(f"[dim green] ✓ Listed {count} item(s){truncated}[/]") + elif tool_name == 'inspect_database': + if 'table' in result: + console.print(f"[dim green] ✓ Inspected table: {result['table']} ({result['row_count']} rows)[/]") + else: + console.print(f"[dim green] ✓ Inspected database ({result['table_count']} tables)[/]") + elif tool_name == 'search_database': + count = result.get('count', 0) + console.print(f"[dim green] ✓ Found {count} match(es)[/]") + elif tool_name == 'query_database': + count = result.get('count', 0) + truncated = " (truncated)" if result.get('truncated') else "" + console.print(f"[dim green] ✓ Query returned {count} row(s){truncated}[/]") + else: + console.print(f"[dim green] ✓ Success[/]") + + result_content = json.dumps(result, indent=2) + app_logger.info(f"MCP tool {tool_name} succeeded") + + tool_results.append({ + "tool_call_id": tool_call.id, + "role": "tool", + "name": tool_name, + "content": result_content + }) + + api_messages.append({ + "role": "assistant", + "content": message.content, + "tool_calls": [ + { + "id": tc.id, + "type": "function", + "function": { + "name": tc.function.name, + "arguments": tc.function.arguments + } + } for tc in tool_calls + ] + }) + api_messages.extend(tool_results) + + console.print("\n[dim cyan]💭 Processing tool results...[/]") + app_logger.info("Sending tool results back to model") + + followup_params = { + "model": effective_model_id, + "messages": api_messages, + "stream": False, # Keep streaming disabled for follow-ups + "http_headers": { + "HTTP-Referer": APP_URL, + "X-Title": APP_NAME + } + } + + if mcp_tools_added: + followup_params["tools"] = mcp_tools + followup_params["tool_choice"] = "auto" + + if session_max_token > 0: + followup_params["max_tokens"] = session_max_token + + try: + response = client.chat.send(**followup_params) + tool_call_loop_count += 1 + app_logger.info(f"Follow-up request successful (loop {tool_call_loop_count})") + except Exception as e: + console.print(f"[bold red]Error getting follow-up response: {e}[/]") + app_logger.error(f"Follow-up API error: {e}") + break + + if not wants_tool_call: + break + + if tool_call_loop_count >= max_tool_loops: + console.print(f"[bold yellow]⚠️ Reached maximum tool call depth ({max_tool_loops}). Stopping.[/]") + app_logger.warning(f"Hit max tool call loop limit: {max_tool_loops}") + response_time = time.time() - start_time + # ======================================================================== + # PROCESS FINAL RESPONSE + # ======================================================================== full_response = "" if is_streaming: try: @@ -2194,7 +5007,7 @@ def chat(): continue else: full_response = response.choices[0].message.content if response.choices else "" - console.print(f"\r{' ' * 20}\r", end="") + console.print(f"\r{' ' * 50}\r", end="") if full_response: if not is_streaming: @@ -2204,7 +5017,6 @@ def chat(): session_history.append({'prompt': user_input, 'response': full_response}) current_index = len(session_history) - 1 - # Process metrics usage = getattr(response, 'usage', None) input_tokens = usage.input_tokens if usage and hasattr(usage, 'input_tokens') else 0 output_tokens = usage.output_tokens if usage and hasattr(usage, 'output_tokens') else 0 @@ -2215,9 +5027,8 @@ def chat(): total_cost += msg_cost message_count += 1 - app_logger.info(f"Response: Tokens - I:{input_tokens} O:{output_tokens} T:{input_tokens + output_tokens}, Cost: ${msg_cost:.4f}, Time: {response_time:.2f}s, Online: {online_mode_enabled}") + app_logger.info(f"Response: Tokens - I:{input_tokens} O:{output_tokens} T:{input_tokens + output_tokens}, Cost: ${msg_cost:.4f}, Time: {response_time:.2f}s, Tool loops: {tool_call_loop_count}") - # Per-message metrics display if conversation_memory_enabled: context_count = len(session_history) - memory_start_index context_info = f", Context: {context_count} msg(s)" if context_count > 1 else "" @@ -2225,12 +5036,18 @@ def chat(): context_info = ", Memory: OFF" online_info = " 🌐" if online_mode_enabled else "" - console.print(f"\n[dim blue]📊 Metrics: {input_tokens + output_tokens} tokens | ${msg_cost:.4f} | {response_time:.2f}s{context_info}{online_info} | Session: {total_input_tokens + total_output_tokens} tokens | ${total_cost:.4f}[/]") + mcp_info = "" + if mcp_tools_added: + if mcp_manager.mode == "files": + mcp_info = " 🔧" + elif mcp_manager.mode == "database": + mcp_info = " 🗄️" + tool_info = f" ({tool_call_loop_count} tool loop(s))" if tool_call_loop_count > 0 else "" + console.print(f"\n[dim blue]📊 Metrics: {input_tokens + output_tokens} tokens | ${msg_cost:.4f} | {response_time:.2f}s{context_info}{online_info}{mcp_info}{tool_info} | Session: {total_input_tokens + total_output_tokens} tokens | ${total_cost:.4f}[/]") - # Cost and credit alerts warnings = [] if msg_cost > COST_WARNING_THRESHOLD: - warnings.append(f"High cost alert: This response exceeded configurable threshold ${COST_WARNING_THRESHOLD:.4f}") + warnings.append(f"High cost alert: This response exceeded threshold ${COST_WARNING_THRESHOLD:.4f}") credits_data = get_credits(API_KEY, OPENROUTER_BASE_URL) if credits_data: warning_alerts = check_credit_alerts(credits_data) diff --git a/requirements.txt b/requirements.txt index 8a0a132..ea6105b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,38 +1,26 @@ -anyio==4.11.0 -beautifulsoup4==4.14.2 -charset-normalizer==3.4.4 -click==8.3.1 -docopt==0.6.2 -h11==0.16.0 -httpcore==1.0.9 -httpx==0.28.1 -idna==3.11 -latex2mathml==3.78.1 -loguru==0.7.3 -markdown-it-py==4.0.0 -markdown2==2.5.4 -mdurl==0.1.2 -natsort==8.4.0 -openrouter==0.0.19 -packaging==25.0 -pipreqs==0.4.13 -prompt-toolkit==3.0.52 -Pygments==2.19.2 -pyperclip==1.11.0 -python-dateutil==2.9.0.post0 -python-magic==0.4.27 -PyYAML==6.0.3 -requests==2.32.5 -rich==14.2.0 -shellingham==1.5.4 -six==1.17.0 -sniffio==1.3.1 -soupsieve==2.8 -svgwrite==1.4.3 -tqdm==4.67.1 -typer==0.20.0 -typing-extensions==4.15.0 -urllib3==2.5.0 -wavedrom==2.0.3.post3 -wcwidth==0.2.14 -yarg==0.1.10 +# oai.py v2.1.0-beta - Core Dependencies +anyio>=4.11.0 +charset-normalizer>=3.4.4 +click>=8.3.1 +h11>=0.16.0 +httpcore>=1.0.9 +httpx>=0.28.1 +idna>=3.11 +markdown-it-py>=4.0.0 +mdurl>=0.1.2 +openrouter>=0.0.19 +packaging>=25.0 +prompt-toolkit>=3.0.52 +Pygments>=2.19.2 +pyperclip>=1.11.0 +requests>=2.32.5 +rich>=14.2.0 +shellingham>=1.5.4 +sniffio>=1.3.1 +typer>=0.20.0 +typing-extensions>=4.15.0 +urllib3>=2.5.0 +wcwidth>=0.2.14 + +# MCP (Model Context Protocol) +mcp>=1.25.0 \ No newline at end of file