Distributed Systems, Infrastructure & Data Platforms

Ujjawal Kumar

At Salesforce, I lead HBase and Phoenix work across migration, backup, recovery and security.

I am a Lead Member of Technical Staff. My work spans projects such as moving live databases between Kubernetes platforms, redesigning backups for object storage, designing per-tenant encryption at rest for backups and tracing a production Java service's missing memory to the native allocator. Outside the database platform, I build retrieval tools over engineering knowledge.

Portrait of Ujjawal Kumar

01 / Selected work

Selected engineering work

Long-form notes on the problem, the constraints, what I tried, and the evidence behind the final decision.

01.1

Platform migration

Petabyte-scale database migration with near-zero downtime

A petabyte-scale HBase and Phoenix fleet moved from its own Amazon EKS clusters to an internal managed Kubernetes platform while tenants kept writing, and each environment was validated table by table before cutover. I led the team and the design: a historical copy alongside live replication, cutover and rollback, and validation that could tell real divergence from TTL expiry and compaction.

A delete can reach the new cluster before the row it deletes. If compaction runs in between, the copy brings the row back.

Read the writeup
01.2

Cloud object storage

Moving backups to S3 without the rename penalty

Changing the output committer brought an S3 backup benchmark from 4h 25m down to 1h 05m, close to its HDFS time. The jobs had slowed badly after moving from HDFS to S3; I benchmarked them and traced the cost to the commit protocol, where every rename is a copy. We moved over without bulk-copying the existing backups.

Tripling the cluster cut total time by only about a quarter, because job commit was a serial copy on one thread.

Read the writeup
01.3

Applied cryptography

Per-tenant encryption at rest for HBase backups

Customer-key encryption stopped at HBase's store files, so backups were protected only by disk-level encryption. My design extends per-tenant keys to backups, starting with a separate chunk file for each encrypted tenant, keeps restores parallel, and lets one tenant's key be destroyed while every other tenant still restores.

Destroying one tenant's key removes its data from every retained backup without rewriting anything, which is why each key covers only one tenant's chunks.

Read the writeup
01.4

Production debugging

Finding 16 GiB of memory the JVM couldn't account for

One environment variable cut a Java service's resident memory from about 32 GiB to 10 GiB in a controlled test. The service used far more physical memory than its heap limit, and the JVM's own accounting couldn't say where the rest went. I traced it below the JVM, to the native allocator.

pmap showed 767 glibc malloc arenas, a count that grows with the host's CPUs. The fix was MALLOC_ARENA_MAX=4.

Read the writeup
01.5

Stream processing

One backup job on MapReduce, Spark and Flink

A Flink proof of concept for continuous backup targets a recovery point of minutes instead of the gap between batch runs, and its output matched the MapReduce backup of the same time range. I built a Spark proof of concept as an intern and later co-mentored the Flink one, which uses two-phase commit. The writeup compares commit, failure handling and recovery point on each engine.

With five-minute windows, the recovery point is bounded by the window plus two checkpoint intervals, and the job only catches up if a scan fits inside the window.

Read the writeup

03 / Research

Automating cryptographic fault analysis.

Peer-reviewed publication

An automated framework for exploitable fault identification in block ciphers

A fault attack corrupts a cipher during execution and uses the changed output to narrow the secret key. I co-developed a framework that uses cipher simulations to find which injected faults lead to a practical attack, then estimates the work needed to recover the key. On AES and PRESENT it recovered most known DFA attacks without implementing each one, and covered more attack classes than earlier automated approaches.

Sayandeep Saha, Ujjawal Kumar, Debdeep Mukhopadhyay, and Pallab Dasgupta · Journal of Cryptographic Engineering, 2019

04 / Career

Experience

Salesforce

2018 — Present

Jun 2023 — Present

Lead Member of Technical Staff

Technical direction for HBase/Phoenix data management, migration, backup architecture, and a 10-engineer team.

Aug 2021 — May 2023

Senior Member of Technical Staff

Object-storage backup architecture, cloud enablement, distributed-storage reliability, and mentorship.

Jun 2019 — Jul 2021

Member / Associate Member of Technical Staff

Automated restore and backup migration, Kubernetes delivery, monitoring, and production recovery.

May — Jun 2018

Intern

Ported the tenant-chunked backup job from MapReduce to Spark as a proof of concept, including the reduce phase for salted tables and a Hadoop task context rebuilt inside each Spark task.

Education

Indian Institute of Technology Kharagpur

B.Tech., Computer Science and Engineering · Full Ride Scholarship

2015 — 2019

May — Jun 2017

Research Intern

Built an automated way to find exploitable faults in block ciphers from simulation data; the work became the fault-analysis paper.

05 / Contact

Contact

Email is the best way to reach me. You can also find my work on GitHub and my experience on LinkedIn.