My Dive into Local LLMs, Part 2: Taming Personal Finance with Homegrown AI (and Why Privacy Matters)

Key Takeaways:

• Transform your local LLM setup into a practical personal finance analyzer
• Build a privacy-first solution that keeps sensitive financial data on your machine
• Learn batch processing strategies for handling large transaction datasets
• Get working code to create your own AI financial assistant

Prerequisites

• Completed setup from Part 1 (Ollama installed, GPU configured)
• Basic Python knowledge
• Ubuntu/Linux system with NVIDIA GPU (8GB+ VRAM)
• A healthy paranoia about cloud services handling your financial data

If you read my last article, "My Dive into Local LLMs, Part 1: From Alexa Curiosity to Homegrown AI," you know I've been on a bit of a journey, diving headfirst into the world of local Large Language Models (LLMs) on my trusty Ubuntu machine. That initial curiosity, spurred by my work on the Alexa team, quickly turned into a fascination with the raw power and flexibility of running AI right on your own hardware. But beyond the sheer "cool factor" of getting Llama 3 to hum on my GPU, I started thinking about practical applications – problems in my daily life where this homegrown AI could actually make a difference.

That's when personal finance popped into my head. Now, before you mentally flag me for suggesting you feed your bank statements to an AI, hear me out. We're bombarded with cloud-based financial tools, and while convenient, they often come with a lingering question: Where exactly is my data going and what are they doing with it? For something as sensitive as personal finances, data privacy isn't just a buzzword; it's paramount. This is where the local LLM truly shines, offering a compelling alternative to cloud-dependent solutions.

My goal wasn't to replace my bank's app or become a financial wizard overnight. It was about creating a secure, private sandbox where I could interact with my financial data using an LLM, analyze trends, categorize expenses, and even get insights, all without ever sending sensitive information outside my local network. Think of it as your personal financial co-pilot, air-gapped and under your direct control.

The "Why" Beyond Curiosity: Security and Data Sovereignty

Working in tech, especially with products like Alexa, you gain an immense appreciation for robust security protocols and data handling. But you also become acutely aware of the inherent trade-offs when data leaves your immediate control. For personal finances, I wanted zero compromise.

The benefits of going local are compelling:

No Cloud, No Problem: The biggest win here is that my financial data, parsed and analyzed by the LLM, never touches a third-party server. It lives and breathes on my machine, exactly where I want it. This inherently reduces the attack surface and eliminates concerns about third-party data breaches affecting my sensitive financial records.

Total Data Control: I decide what data goes into the model, when it's processed, and how it's stored (or not stored). There's no hidden telemetry, no opaque processing. It's truly my data, my AI.

Experimentation Without Fear: Want to try a new categorization scheme? Or ask a hypothetical "what if" about your spending habits? You can do so freely, knowing there's no commercial interest in your data, just your own analytical curiosity.

This level of data sovereignty felt like a natural evolution from my initial "local LLM for fun" experiments. It transformed a technical sandbox into a genuinely useful, privacy-centric application.

Technical Setup: Building on Part 1

In Part 1, I walked through getting Ollama running on my Ubuntu machine with GPU acceleration. If you haven't read that yet, the TL;DR is: install Ollama, get your NVIDIA drivers sorted, install the NVIDIA Container Toolkit, and you're golden. That foundation is exactly what we need for this finance project.

Current Setup (carried over from Part 1):

• Ubuntu 22.04 with that same trusty machine
• NVIDIA RTX 3080 (though any 8GB+ VRAM card will work)
• Ollama configured with GPU support
• 32GB RAM (overkill for this, but hey, future-proofing)

New Additions for Finance Work:

• Python 3.8+ with pandas, matplotlib
• About 50GB free space for transaction data and outputs
• Your bank's CSV export (most banks offer this)

Performance Update: Remember how in Part 1, running Phi-3 on CPU was "deliberate" (read: painfully slow)? With GPU acceleration and Llama 3 8B, I'm getting:

• ~25-30 tokens/second for standard inference
• ~40-50 tokens/second with 4-bit quantization
• Context window: 8,192 tokens (about 6,000 words)

For financial analysis, this speed difference is crucial – nobody wants to wait 5 minutes for transaction categorization!

From Raw Data to Actionable Insights: The Local LLM Pipeline

Getting this up and running involved a few key steps, building on the foundation I laid in the previous article. The core challenge was feeding structured financial data (like transaction lists) to an LLM designed for natural language, and then coaxing useful, structured responses back.

Here's a high-level overview of the process I devised:

Step 1: Data Export and Pre-processing With Python

Most banks allow you to export transactions as CSV files. This was my starting point. It's usually a messy CSV with dates, descriptions, amounts, and sometimes cryptic merchant names. This is where the heavy lifting happens.

import pandas as pd
import json
from datetime import datetime
import re

def clean_merchant_name(merchant):
    """Standardize merchant names for consistency"""
    # Remove common suffixes and clean up
    merchant = re.sub(r'#\d+', '', merchant)  # Remove store numbers
    merchant = re.sub(r'\*\d+', '', merchant)  # Remove transaction IDs
    merchant = merchant.strip().upper()
    
    # Map common variations
    merchant_map = {
        'AMZN': 'AMAZON',
        'AMAZONCOM': 'AMAZON',
        'AMAZON.COM': 'AMAZON',
        'STARBUCKS': 'STARBUCKS',
        'SBUX': 'STARBUCKS',
        'WHOLEFDS': 'WHOLE FOODS',
        'WHOLE FOODS MARKET': 'WHOLE FOODS'
    }
    
    for pattern, replacement in merchant_map.items():
        if pattern in merchant:
            return replacement
    
    return merchant

def preprocess_transactions(csv_path):
    """Load and clean transaction data"""
    df = pd.read_csv(csv_path)
    
    # Standardize column names
    df.columns = [col.lower().strip() for col in df.columns]
    
    # Clean merchant names
    if 'description' in df.columns:
        df['clean_merchant'] = df['description'].apply(clean_merchant_name)
    
    # Convert to LLM-friendly format
    transactions = []
    for _, row in df.iterrows():
        transaction = {
            "date": row['date'],
            "merchant": row.get('clean_merchant', row.get('description', 'Unknown')),
            "amount": float(row['amount']),
            "original_description": row.get('description', '')
        }
        transactions.append(transaction)
    
    return transactions

Step 2: Prompting the Local LLM

With Ollama running Llama 3 (my current favorite for its balance of performance and capability on local hardware), I crafted prompts. This wasn't just "summarize this." It was highly structured, often using few-shot learning.

import requests
import json
from typing import List, Dict, Optional

class LocalFinanceLLM:
    def __init__(self, model="llama3:8b", base_url="http://localhost:11434"):
        self.model = model
        self.base_url = base_url
        self.categories = [
            "Food & Dining", "Transportation", "Entertainment", 
            "Utilities", "Groceries", "Shopping", "Healthcare",
            "Income", "Rent/Mortgage", "Insurance", "Education",
            "Personal Care", "Miscellaneous"
        ]
    
    def query_llm(self, prompt: str, max_retries: int = 3) -> Optional[str]:
        """Query the local LLM with retry logic"""
        url = f"{self.base_url}/api/generate"
        headers = {"Content-Type": "application/json"}
        data = {
            "model": self.model,
            "prompt": prompt,
            "stream": False,
            "temperature": 0.1  # Low temperature for consistency
        }
        
        for attempt in range(max_retries):
            try:
                response = requests.post(url, headers=headers, json=data, timeout=30)
                response.raise_for_status()
                return response.json()['response'].strip()
            except requests.exceptions.RequestException as e:
                if attempt == max_retries - 1:
                    print(f"Error querying Ollama after {max_retries} attempts: {e}")
                    return None
        
        return None
    
    def categorize_transaction(self, transaction: Dict) -> str:
        """Categorize a single transaction"""
        # Build a focused prompt with examples
        prompt = f"""Categorize this transaction into exactly one category.

Categories: {', '.join(self.categories)}

Examples:
- "UBER TECHNOLOGIES" → Transportation
- "WALMART GROCERY" → Groceries
- "NETFLIX.COM" → Entertainment
- "PACIFIC GAS & ELECTRIC" → Utilities

Transaction: {transaction['merchant']} for ${abs(transaction['amount'])}

Category:"""
        
        response = self.query_llm(prompt)
        
        # Validate response
        if response and any(cat.lower() in response.lower() for cat in self.categories):
            for cat in self.categories:
                if cat.lower() in response.lower():
                    return cat
        
        return "Miscellaneous"
    
    def batch_categorize(self, transactions: List[Dict], batch_size: int = 10) -> List[Dict]:
        """Process transactions in batches to respect context limits"""
        categorized = []
        
        for i in range(0, len(transactions), batch_size):
            batch = transactions[i:i + batch_size]
            
            # Create a batch prompt
            batch_prompt = f"""Categorize these transactions. 
Categories: {', '.join(self.categories)}

Respond with a JSON array where each item has 'index' and 'category'.

Transactions:
"""
            for idx, trans in enumerate(batch):
                batch_prompt += f"{idx}: {trans['merchant']} - ${abs(trans['amount'])}\n"
            
            batch_prompt += "\nJSON Response:"
            
            response = self.query_llm(batch_prompt)
            
            # Parse response and fallback to individual processing if needed
            try:
                results = json.loads(response)
                for item in results:
                    idx = item['index']
                    batch[idx]['category'] = item['category']
            except:
                # Fallback to individual processing
                for trans in batch:
                    trans['category'] = self.categorize_transaction(trans)
            
            categorized.extend(batch)
            print(f"Processed {len(categorized)}/{len(transactions)} transactions...")
        
        return categorized

Step 3: Post-processing and Output

The LLM's raw text responses needed to be parsed back into a structured format. For instance, if it returned "Category: Groceries," my Python script would extract "Groceries" and update my internal data model. From there, I could generate simple text reports, or even pipe it into a local plotting library (like Matplotlib) to visualize spending trends.

def analyze_spending(categorized_transactions: List[Dict], llm: LocalFinanceLLM) -> Dict:
    """Analyze spending patterns and generate insights"""
    
    # Calculate category totals
    category_totals = {}
    for trans in categorized_transactions:
        if trans['amount'] < 0:  # Only expenses
            cat = trans.get('category', 'Miscellaneous')
            category_totals[cat] = category_totals.get(cat, 0) + abs(trans['amount'])
    
    # Sort by amount
    sorted_categories = sorted(category_totals.items(), key=lambda x: x[1], reverse=True)
    
    # Generate natural language summary
    summary_prompt = f"""Analyze these spending patterns and provide 3 key insights:

Monthly Spending by Category:
"""
    for cat, amount in sorted_categories[:5]:
        summary_prompt += f"- {cat}: ${amount:.2f}\n"
    
    summary_prompt += """
Provide actionable insights about spending habits. Be specific and practical."""
    
    insights = llm.query_llm(summary_prompt)
    
    return {
        'category_totals': dict(sorted_categories),
        'top_categories': sorted_categories[:3],
        'insights': insights,
        'total_spending': sum(category_totals.values())
    }

def find_recurring_transactions(transactions: List[Dict], llm: LocalFinanceLLM) -> List[Dict]:
    """Identify recurring subscriptions and payments"""
    
    # Group by merchant and look for patterns
    merchant_groups = {}
    for trans in transactions:
        merchant = trans['merchant']
        if merchant not in merchant_groups:
            merchant_groups[merchant] = []
        merchant_groups[merchant].append(trans)
    
    # Find recurring patterns
    recurring = []
    for merchant, trans_list in merchant_groups.items():
        if len(trans_list) >= 2:  # At least 2 occurrences
            amounts = [abs(t['amount']) for t in trans_list]
            # Check if amounts are consistent (within 10% variance)
            if amounts and max(amounts) - min(amounts) < 0.1 * max(amounts):
                recurring.append({
                    'merchant': merchant,
                    'amount': sum(amounts) / len(amounts),
                    'frequency': len(trans_list),
                    'transactions': trans_list
                })
    
    return sorted(recurring, key=lambda x: x['amount'] * x['frequency'], reverse=True)

Step 4: Privacy-Preserving Output

For security, I never store the full enriched dataset. Instead, I generate reports and immediately dispose of sensitive data:

import matplotlib.pyplot as plt
from datetime import datetime
import os

def generate_report(analysis_results: Dict, output_dir: str = "./reports"):
    """Generate a privacy-conscious financial report"""
    
    # Create output directory
    os.makedirs(output_dir, exist_ok=True)
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    
    # Generate text report
    report_path = os.path.join(output_dir, f"finance_report_{timestamp}.txt")
    with open(report_path, 'w') as f:
        f.write("Personal Finance Analysis Report\n")
        f.write("=" * 40 + "\n\n")
        
        f.write("Top Spending Categories:\n")
        for cat, amount in analysis_results['top_categories']:
            f.write(f"  - {cat}: ${amount:.2f}\n")
        
        f.write(f"\nTotal Monthly Spending: ${analysis_results['total_spending']:.2f}\n")
        
        f.write("\nAI-Generated Insights:\n")
        f.write(analysis_results['insights'] + "\n")
    
    # Generate visualization
    if analysis_results['category_totals']:
        plt.figure(figsize=(10, 6))
        categories = list(analysis_results['category_totals'].keys())[:8]
        amounts = [analysis_results['category_totals'][cat] for cat in categories]
        
        plt.bar(categories, amounts)
        plt.xlabel('Category')
        plt.ylabel('Amount ($)')
        plt.title('Spending by Category')
        plt.xticks(rotation=45)
        plt.tight_layout()
        
        chart_path = os.path.join(output_dir, f"spending_chart_{timestamp}.png")
        plt.savefig(chart_path)
        plt.close()
    
    print(f"Report generated: {report_path}")
    
    # Important: Don't persist sensitive data
    return report_path

Challenges and My "Aha!" Moments

This wasn't just a straight line from idea to execution. There were, as always, a few bumps:

Prompt Engineering is King (Still): Just like with cloud LLMs, getting the right output from a local model is all about the prompt. I spent a surprising amount of time refining prompts to ensure consistent categorization and accurate summaries. Little things, like asking the LLM to "respond only with the category" or providing clear examples, made a huge difference.

Data Consistency is Crucial: Financial data is notoriously messy. Dealing with variations in merchant names ("Starbucks," "STARBUCKS #123," "SBUX") required robust pre-processing rules. The LLM can help infer, but it's not a magic bullet for truly dirty data.

Performance vs. Accuracy: While Llama 3 on my GPU is fast, processing months or years of transactions still takes time. It forced me to optimize my scripting and consider how to effectively batch requests to the LLM without overwhelming its context window. For very specific, small-scale tasks, a smaller model like Phi-3 might even be faster if its capabilities are sufficient.

The "Human-in-the-Loop" Factor: This isn't about fully automating my finances. It's about augmentation. The LLM suggests categories or flags anomalies, but the final decision and interpretation always rest with me. It's a powerful assistant, not a replacement for my own judgment.

Security Considerations

Running this locally doesn't mean ignoring security:

1. Encrypted Storage: Process data in memory and store results encrypted
2. Access Control: Run the service on localhost only
3. Data Minimization: Don't persist full transaction details after processing
4. Regular Cleanup: Automated deletion of temporary files

# Example: Secure temporary file handling
import tempfile
import shutil

def process_with_cleanup(csv_path):
    temp_dir = tempfile.mkdtemp()
    try:
        # Process data
        results = run_analysis(csv_path, temp_dir)
        return results
    finally:
        # Always cleanup
        shutil.rmtree(temp_dir)

Practical Results and Next Steps

After running this for three months, I've discovered:

• Subscription creep: Found $47/month in forgotten subscriptions
• Category drift: What I thought was "groceries" included 30% restaurant spending
• Patterns: Thursday is my peak spending day (post-work socializing)

My next enhancements include:

• Building a local Streamlit dashboard for easier interaction
• Implementing budget forecasting based on historical patterns
• Adding investment portfolio analysis (also local-only)
• Exploring smaller models like Phi-3 for simple categorization tasks

Why This Matters for Developers

For us developers, this isn't just a cool personal project; it's a tangible demonstration of several critical concepts:

Edge AI Potential: It showcases how powerful AI models can be deployed and utilized at the "edge" – on personal devices, within a local network – opening doors for privacy-preserving applications across various domains, not just finance. Think local healthcare data analysis, personal content moderation, or even enhanced smart home automation that keeps your data truly yours.

Data Privacy by Design: This project reinforced the importance of building systems with privacy as a foundational principle. By keeping data local, we side-step many of the complex compliance and security challenges associated with cloud data storage.

The Power of Open Source: The entire stack – Ollama, Llama 3 (or other open-source models), Python – is built on open-source technologies. This democratization of AI tools is what truly enables projects like this, allowing anyone with the right hardware to experiment and innovate.

Thinking Beyond the API Call: While cloud APIs are convenient, understanding the underlying mechanics of running LLMs locally gives you a deeper appreciation for their resource demands, performance characteristics, and the art of prompt engineering when resources are constrained.

The Road Ahead

This personal finance sandbox is still evolving. My next steps involve:

More Sophisticated Prompting: Exploring more advanced prompt engineering techniques to get richer insights, perhaps even asking for budgeting suggestions based on historical data.

Local UI Integration: Building a simple web-based UI (maybe with Streamlit or Flask) that runs locally, allowing for easier interaction and visualization without touching the cloud.

Integrating Other Local Data: Perhaps pulling in investment data (again, locally!) to get a holistic view of my financial health.

Remember my RAG aspirations from Part 1? This finance project is actually perfect for that – imagine feeding it your bank's terms and conditions or tax documentation to get personalized insights!

Getting Started

Ready to build your own private financial AI? Here's your quickstart (assuming you've got Ollama from Part 1):

1. Make sure Ollama is running: ollama list (should show your models)
2. Pull Llama 3 if you haven't: ollama pull llama3:8b
3. Clone the example code: [GitHub repository link]
4. Export your bank transactions as CSV
5. Run: python finance_analyzer.py --csv your_transactions.csv

Wrapping Up

If you're a developer who values privacy and is curious about pushing the boundaries of what's possible with local AI, I highly encourage you to give this a try. It's a rewarding experience that not only enhances your technical skills but also offers a significant personal benefit. The journey of building your own AI co-pilot, securely and privately, is one well worth taking.

Just like my initial dive into local LLMs was about curiosity, this finance project proved that we can build genuinely useful, privacy-preserving tools with the tech. The question isn't whether we can—it's what we'll build next.

What are your thoughts? Have you experimented with local LLMs for personal data? Share your experiences in the comments below!

Resources

• Ollama Documentation
• Llama 3 Model Card
• Privacy-Preserving ML Techniques
• Github Repo of Actual Project