Senior Software Engineer · Microsoft · Bangalore, India

I work a layer beneath most software.

Query engines, cluster schedulers, compression formats — the parts of a system nobody thinks about until they're the reason something is slow. I've spent the past four years building them at Microsoft, for platforms enterprises run their data on.

Read the work ↓ Curriculum vitae ↗

Now: Fabric Spark query engine
Since: 2022
Based: Bangalore, India
Writes: Rust, C++, Scala, Go

Experience

Microsoft Sept 2026 — Present Microsoft Fabric · Spark query engine

Senior Software Engineer

Making Spark stop recomputing what it already knows.

I led the research, design and development of materialized views and incremental view maintenance inside the Spark query engine that powers Microsoft Fabric. Medallion-architecture pipelines re-derive the same tables endlessly; incremental view maintenance updates only what actually changed. It is the kind of capability that decides whether a data platform is competitive at the top of the market.

Spark
Scala
Query optimization
Distributed systems

Microsoft Sept 2024 — Sept 2026 Azure HDInsight on AKS

Software Engineer II

In-place upgrades for a big data PaaS.

Big data clusters are notoriously hard to upgrade without downtime, and most managed platforms simply don't try. I engineered in-place upgrades for HDInsight on AKS so customers could absorb the latest open-source features and fixes with minimal disruption to running workloads — a rare capability in the market, and one that unblocked enterprise onboarding that had stalled on exactly this requirement.

Kubernetes
Azure
Go

Microsoft July 2022 — Sept 2024 Azure HDInsight on AKS

Software Engineer I

A custom Kubernetes scheduler for zone-resilient Kafka.

Kafka's durability guarantees mean very little if every broker lands in the same availability zone. I designed and implemented zone-aware broker placement by writing a custom Kubernetes scheduler, which let key enterprise customers run zone-resilient Kafka on the platform for the first time.

Core platform, held to 99.9%.

I wrote several of the platform's foundational components — cluster provisioning and the gateway layer — and held them to a 99.9% reliability bar. Every cluster on the platform passes through this path.

Governance, end to end.

I built end-to-end provisioning for Apache Ranger clusters, the component that governs everything else: row-level policies, topic restrictions, and access control across Trino and Kafka. Governance is rarely the exciting part, and it is routinely the thing an enterprise deal hinges on.

Kubernetes
Kafka
Trino
Ranger
Go
Azure

Microsoft Jan 2022 — July 2022

Software Engineer Intern

Shipping the replacement while the original was on fire.

I built a low-cost alternative to the platform's serverless scaling infrastructure as an intern project. When the legacy scaling system failed in production, I took data from test regions to leadership and argued for deploying mine instead. It shipped, ended the outage's impact on customers, and permanently replaced the old system — cutting a substantial recurring maintenance cost in every region.

Serverless
Autoscaling
Azure

SmokeTrees May 2020 — July 2020

Full-stack intern

Two products in Flutter and FastAPI: a digital marketplace for BearLyfe, a Hong Kong wellbeing company, and a home salon-booking app with payments for fabsalons in India.

Flutter
FastAPI
PostgreSQL

Open source

Things I build when nobody's asking.

C++
Compression

hybridzip

A model-aware data compression platform in C++. Compression ratio is bounded by how well you can predict the next symbol, so hybridzip makes the model a first-class, swappable part of the format instead of something welded into the codec.

Rust
WASI

mitsuha

A job execution engine built on WASI — sandboxed, portable units of compute without dragging an entire container runtime along.

C++
Memory

rainman

A lightweight hierarchical memory manager for C++. Ownership arranged as a tree, so releasing a parent releases everything beneath it.

bitio

A small, fast bit-level I/O library for C++ — the primitive you end up rewriting badly every time you build a codec.

Java
Neo4j

lucy

An OGM for Neo4j that makes graph models behave like ordinary objects.

Rust
Axum

vish.bio

This site. Rust, Axum and Askama compiled to a single static binary that serves its own assets.

Toolkit

The list, since people ask.

Systems & languages

Rust
C++
Scala
Go
Java
Python
C#
TypeScript
Dart

Distributed & data

Spark
Kafka
Trino
Ranger
Query engines
System design
Big data platforms

Infrastructure

Kubernetes
Docker
Azure
Linux
Networking

Interfaces

React
Next
Nuxt
Flutter
Electron

Also

OpenCL / GPGPU
TensorFlow

Elsewhere

Open to interesting conversations.

Especially about query engines, schedulers, compression, or anything where the interesting part lives below the API.

Email: vishaals2000@gmail.com
GitHub: supercmmetry
LinkedIn: vishaal-selvaraj
Telegram: supercmmetry
Curriculum vitae: PDF