Jubin Soni, FBCS

CORE

Senior Software Engineer at Yahoo

San Francisco, US

Joined May 2025

https://jubinsoni.com/newsletter

About

Jubin Soni is a Senior Software Engineer with 14+ years building scalable systems, real-time data pipelines, and AI platforms for technology and media companies. He is a Fellow of BCS, The Chartered Institute for IT, an IEEE Senior Member, and a Full Member of Sigma Xi, with research published in IEEE venues and a book with Manning Publications. He has spoken at P99 CONF and APIdays on distributed systems, serverless architectures, and applied ML.

Stats

Reputation:	4082
Pageviews:	135.0K
Articles:	55
Comments:	3

Expertise

Microservices

Articles

Microsoft Foundry Tool Search: Your Agent Pays a Tax on Every Tool It Never Calls

A practical look at why large AI agents waste tokens on unused tool schemas, and how tool retrieval and pinning can cut cost and improve selection.

August 7, 2026

· 1,469 Views

Spark Performance Deep Dive on Databricks: Shuffle Tuning, Skew Handling, and Z-Ordering With Delta Lake + Unity Catalog

Learn Spark performance tuning on Databricks with shuffle optimization, skew handling, AQE, broadcast joins, and Delta Lake Z-Ordering best practices.

July 31, 2026

· 1,260 Views

Building Production-Grade Semantic Search With GPT-5 and Microsoft Foundry, From Scratch

Learn key concepts like embeddings, vector search, chunking, hybrid search, semantic ranking, and how these pieces fit into a scalable, enterprise-grade AI application.

July 24, 2026

· 2,284 Views

API Facade vs. Orchestration vs. Eventing, Now With AI in the Loop

Discover architectural patterns for integrating AI into enterprise systems without sacrificing governance, reliability, observability, or loose coupling.

July 13, 2026

· 2,912 Views · 1 Like

Implementing Zero-Trust Networking and Identity for Microsoft Foundry Agents

This article explains how to implement a zero-trust security architecture for Microsoft Foundry agents by combining identity-first authentication.

July 13, 2026

· 1,848 Views

Building Evaluation, Cost Governance, and Observability for a Multi-Agent System in Microsoft Foundry

This article walks through building a production-ready multi-agent AI system using Microsoft's AI stack, focusing on the operational capabilities.

July 13, 2026

· 1,957 Views

Azure Databricks vs Microsoft Fabric: An Honest Guide to When to Use What

Azure Databricks and Microsoft Fabric overlap, but they're built for different priorities. Databricks for data engineering, ML, open-source, and Spark workloads.

July 9, 2026

· 1,908 Views

Building Production-Grade Delta Lake Pipelines With Apache Spark on Databricks

This article walks through building a modern Databricks pipeline using the Medallion Architecture, explains Delta Lake's transaction log and ACID guarantees.

July 8, 2026

· 1,566 Views · 1 Like

Azure Databricks for Scalable MLOps and Feature Engineering With Apache Spark, Delta Lake, and MLflow

A practical guide to feature engineering at scale with Azure Databricks, covering distributed data processing with Spark and reliable storage with Delta Lake.

July 6, 2026

· 1,288 Views

Building an AI Agent That Responds to Real-Time Events With AWS Bedrock, Kinesis, DynamoDB, and S3

Build an AI agent that processes real-time events with Amazon Bedrock and a serverless AWS architecture powered by Kinesis, DynamoDB, and S3.

July 3, 2026

· 2,204 Views · 1 Like

Real-Time AI Feature Engineering With Spark Structured Streaming and Databricks Feature Store

How Spark Structured Streaming and the Databricks Feature Store work together to build point-in-time-correct features from Kafka events to streaming transformations.

July 2, 2026

· 2,278 Views

Apache Spark Query Optimization on Databricks: Catalyst, AQE, and Photon Engine

Spark query performance on Databricks is driven by a multi-layer optimization stack: Catalyst transforms SQL into optimized execution plans.

July 2, 2026

· 2,047 Views

Fine-Tuning LLMs at Scale With Databricks MLflow and Spark

Learn how Databricks, Apache Spark, MLflow, and Hugging Face Transformers work together to create an end-to-end fine-tuning platform.

June 30, 2026

· 1,429 Views

Connect Existing Data to AI Retrieval: How to Build Production-Ready Search Without Rebuilding Core Systems

Step-by-step tutorial building AI retrieval over existing data systems using a thin layer, covering workflow design, indexing, evaluation, and RAG pipeline.

June 23, 2026

· 2,340 Views · 1 Like

I Built a VS Code Extension to Debug Azure AI Foundry Agents Without Leaving My Editor

Free VS Code extension for Azure AI Foundry agent traces into your editor as an interactive timeline — see tool calls, token costs, and conversation replays.

June 23, 2026

· 1,799 Views · 1 Like

Amazon CodeWhisperer to Q Developer to Kiro: The Rise of Agentic Coding

If you’re a backend engineer working with AWS and curious about how we went from autocomplete-style AI to agentic, this one breaks down the architecture shifts.

June 18, 2026

· 2,388 Views

Building a Vector Index in Azure AI Search: HNSW, Profiles, and RAG Retrieval

Use Azure AI Search as your RAG vector store. Build a Python example: define an HNSW vector index, upload embeddings, and run k-NN queries.

June 15, 2026

· 1,300 Views

Amazon OpenSearch Vector Search Explained for RAG Systems

Use Amazon OpenSearch k-NN as your RAG vector store. Build a small Python example: create the index, embed docs, search by meaning.

June 9, 2026

· 1,488 Views

Amazon Quick: AWS's Agentic Workspace, Explained for Engineers

A technical deep dive into Amazon Quick — how it works, how it connects to your tools via MCP, and where it sits in the AWS agent stack.

June 9, 2026

· 4,319 Views

S3 Vectors: How to Build a RAG Without a Vector Database

You don't need a vector database anymore. S3 Vectors gives you native vector storage + similarity search, serverless, at up to 90% less cost

May 19, 2026

· 1,718 Views

Run Gemma 4 on Your Laptop: A Hands-On Guide to Google's Latest Open Multimodal LLM

Gemma 4 — architecture deep-dive, thinking mode, function calling, audio + image input, and a small project — no cloud, no API keys.

May 19, 2026

· 4,014 Views · 2 Likes

The Agent Protocol Stack: MCP vs. A2A vs. AG-UI

Three protocols are shaping how AI agents interact with tools, other agents, and users. Here's what each one does, how they fit together, and when to reach for which.

May 15, 2026

· 3,424 Views

AWS Kiro: The Agentic IDE That Makes Specs the Unit of Work

You describe a feature, and Kiro generates structured requirements.md, design.md, and tasks.md files first, then implements against them.

May 13, 2026

· 3,496 Views · 2 Likes

I Gave Gemini 3 My Worst Legacy Code — Here’s What Happened

Feeding legacy code to Gemini 3 reveals key refactoring lessons: prioritize modularity, decouple logic, and use AI to automate unit tests.

May 7, 2026

· 3,055 Views · 1 Like

Engineering LLMOps: Building Robust CI/CD Pipelines for LLM Applications on Google Cloud

Master LLMOps on GCP by automating prompt evaluation, model deployment, and monitoring with Cloud Build and Vertex AI for robust AI apps.

May 5, 2026

· 2,131 Views

5 Ways Azure AI Search Enhances Enterprise RAG Architectures

Azure AI Search enhances RAG through hybrid search, semantic reranking, and enterprise-grade security for scalable production AI apps.

April 30, 2026

· 3,678 Views

What AWS Kiro Matters for Agentic Development

AWS Kiro is a high-speed communication fabric for AI agents, enabling sub-millisecond coordination and persistent memory management.

April 29, 2026

· 2,426 Views

65% of Enterprises Will Deploy Agentic AI by 2027: A Deep Technical Analysis of Readiness

Agentic AI is the next frontier for enterprises. This guide covers technical architectures, multi-agent design, and deployment readiness.

April 28, 2026

· 3,122 Views

Architecting Autonomous Agents: A Deep Dive into Azure AI Foundry Agent Service

Build enterprise-ready AI agents using Azure AI Foundry Agent Service with integrated tools, state management, and robust security.

April 27, 2026

· 2,130 Views

Gemini + Veo: A Deep Dive into Google’s High-Fidelity Video Generation Pipeline

A deep dive into Google's video generation stack, combining Gemini's reasoning with Veo's cinematic 1080p video production pipeline.

April 23, 2026

· 2,431 Views

Trend Reports

Trend Report

Cognitive Databases, Intelligent Data

No longer passive storage and query engines, databases are becoming active, intelligent participants in how modern systems interpret, connect, and act on data. As AI moves deeper into production and enterprises adopt generative and agentic architectures, the database layer is being reshaped to support semantic search, contextual retrieval, and real-time decision-making. Vector databases, semantic indexing, and AI-driven optimization are changing how developers work with both structured and unstructured data, while the line between transactional and analytical systems continues to fade under hybrid workload demands.This report examines these industry shifts in practical terms, exploring how relational, NoSQL, vector, and multi-model systems are coming together to support AI-native applications. Our research, guest thought leadership, and practitioner insights look at how teams are bringing vector search into production, updating architectures for AI workloads, and redesigning data pipelines around semantic and contextual intelligence.

Comments

AWS Kiro: The Agentic IDE That Makes Specs the Unit of Work

May 20, 2026 · Jubin Soni, FBCS

Really appreciate that, Georgi, thanks for taking the time to read it!

Enhancing SQL Server Performance with Query Store and Intelligent Query Processing

Apr 14, 2026 · arvind toorpu

Great article! thanks for sharing

Mastering Azure Kubernetes Service: The Ultimate Guide to Scaling, Security, and Cost Optimization

Apr 14, 2026 · Jubin Soni, FBCS

Thanks, appreciate it! Great question, AKS autoscaling behaves similarly to self-managed Kubernetes since it uses the same HPA + Cluster Autoscaler components. HPA reacts quickly to spikes, but node scaling still has some lag due to provisioning time. AKS mainly makes this more reliable and easier to manage, but for sharp bursts, you may still need buffer capacity or tools like KEDA.