Microsoft (Consulting)

Systems R&D Consultant · 2021–2025
Microsoft Research & Cloud Operations + Innovation

Between 2021 and 2025, I served as an embedded consultant on 5 projects across two distinct divisions of Microsoft — Microsoft Research (MSR) and Cloud Operations + Innovation (CO+I) — working at the intersection of human-centered design, future-of-work, spatial computing, and AI/ML infrastructure research.

All engagements operated under NDA. The following reflects only what has been publicly disclosed or is otherwise non-confidential.

Project 1 — Hybrid Meetings & the Future of Work

Microsoft Research. Publicly announced as Project Malta.

The Challenge

As hybrid work reshaped the modern workplace, MSR set out to answer a critical question:

How do you ensure that remote and in-person participants feel equally present, represented, and heard?

The answer is harder than it sounds. Research has consistently shown that human communication depends on subtle physical cues — eye gaze, body orientation, gestures, and turn-taking signals — that we process largely unconsciously in face-to-face settings. Conventional video calls strip most of these away. Remote participants struggle to know who is looking at them, when it is their turn to speak, or whether they are even being seen. Meanwhile, co-located groups form natural coalitions, making decisions and building rapport that remote attendees can only partially follow.

This is fundamentally an accessibility and equity problem. The technology existed to connect people — but it wasn't designed around how people actually communicate. Project Malta tackled this challenge head-on, and it is precisely the kind of problem that draws me in:

Human factors research, inclusive design, and emerging technology all have to work together to make a meaningful difference.

My Role and Approach

I brought a human-centered design lens to a complex sociotechnical problem, where technology alone wasn't enough. Equity, perception, and embodied presence had to be designed for deliberately.

My contribution started with mapping the full design space — identifying every meaningful way a hybrid meeting participant could be represented, from a simple audio icon to a life-size telepresence robot to a fully embodied VR avatar. This user representation spectrum became a shared reference for the team, making it possible to reason clearly about the tradeoffs between different modalities in terms of presence, agency, and social legibility.

To structure the technical and design challenges more rigorously, I developed a “4D” framework that mapped all 48 possible configurations across three axes — Embodiment, Space ownership, and Interface type — with a fourth dimension of implementation feasibility encoded with three color levels. I built this visualization in Shapes XR and presented it alongside different interaction scenarios using AfterNow Prez, our own immersive presentation platform, allowing the research team to explore the design space spatially and collaboratively rather than through static slides.

The through-line connecting all of this to my broader practice is the same one that runs through my PhD work in rehabilitation technology and my approach to inclusive design generally:

The people most excluded by default assumptions are often the ones who reveal what the design is really missing.

Technologies: Microsoft Teams, AfterNow Prez, Unity, Shapes XR, Azure Kinect, WebRTC

Read the Project Malta article Access the research paper

Projects 2 & 3 — AR Training for Data Center Operations

Microsoft Cloud Operations + Innovation

Data centers are mission-critical infrastructure, but training technicians to diagnose and repair server hardware at scale presents a genuine dilemma:

How do you prepare people for high-stakes repairs without exposing them to the risks of equipment damage, personal safety, or the pressure of real-world downtime?

Traditional hands-on training is expensive, slow, and risky — both for the technician and the equipment. Simulation offers a safer alternative, but most training simulations are abstract and disconnected from the embodied, spatial nature of actual hands-on repair work. A technician needs to understand not just the steps, but the feel of the work — the spatial relationships between components, the force needed to remove a part, the sequence of dependencies that keep the system from breaking.

I approached this as a problem of embodied learning — designing interactions that let trainees build genuine task fluency through spatially grounded experience, not just by reading steps from a screen.

My first step was deep domain research into the training tasks: understanding the exact sequence of steps, the dependencies between components, the physical constraints that govern the work, and the safety requirements technicians must follow. That included mapping PPE protocols — gloves, protective eyewear, no loose clothing — as real UX constraints, not afterthoughts. Getting those details right was essential to building a training experience that reflected actual working conditions and could be trusted by the trainees and their supervisors.

POC Project

The POC Phase combined a tabletop projected prototype with Azure Kinect body tracking. I designed the interaction model and step-by-step guidance system to respond to the user's physical position and actions in real time, giving feedback that was spatially grounded rather than abstract. A green-screen recording of me performing specific tasks was projected as an instructional overlay — keeping the human reference frame at the center of the experience.

MVP Project

Once we secured shareholder buy-in, the MVP Phase 2 expanded on this with a 3D physical prototype and a second Kinect camera for object tracking using retroreflective markers. I redesigned the interaction model around direct object manipulation, adding more animations that reinforced the physical logic of each task. The result felt less like a simulation and more like a guided practice environment.

Reporting and Delivery

Throughout development we delivered regular live-streamed updates, incorporated feedback iteratively, and concluded with an on-site installation and validation session with stakeholders and selected end users — a critical human-centered step that grounded the design in real-world conditions.

Technologies: Azure Kinect, Unity, Reality Shader, Projection mapping, 3D printing

Projects 4 & 5 — AI/ML Research: Photonics & Next Gen AI Infrastructure

Microsoft Cloud Operations + Innovation

This was a departure from typical human-centered design into pure applied systems R&D — but it drew on the same computer vision expertise built across the earlier Microsoft projects, and the same instinct to find elegant solutions under real-world constraints.

Modern AI training at data center scale is expensive — in compute, in time, and in energy. A core bottleneck is the all-reduce operation: the serial process by which GPUs distribute, sum, and redistribute data during parallel training. The bigger the model, the more this bottleneck hurts.

What if you could bypass that entirely — performing computation through light rather than through circuits?

Project 4 — Point-to-Point Optical Communication

In our push to accomplish optical P2P communication between servers, I developed the visual encoding scheme, calibration interface, and resolution optimization for a system that transmitted data between machines using projected light — conceptually similar to QR codes but optimized for maximum bandwidth. We implemented a self-correcting error protocol similar to TCP/IP, ensuring transmissions completed without missing packets. Upon completion, we demonstrated the system on-site at Microsoft, successfully encoding, transmitting, and decoding various data formats — text, video, 3D models, and ZIP files.

If data can travel as light, what else can computation do with it?

Project 5 — Optical All-Reduce for Parallel Computing

Building on Project 4, we used overlapping gaming projectors to perform light-based computation — where four GPU data streams, projected simultaneously, add together through the natural properties of light. I developed a custom encoding scheme, a significantly more complex multi-projector calibration system accounting for geometric distortion and brightness variation, and an error assessment protocol to determine the limits of reliable encoding density. CUDA kernels were used throughout to maximize parallelism. We demonstrated the system on-site at Microsoft across multiple sessions, inviting stakeholders and researchers to review our work.

Multiple research papers from these projects are currently in progress or pending publication.

Technologies: C++, Python, CUDA, NCCL, OpenCV, high-speed cameras and monitors

Reflection

Working across four distinct Microsoft engagements gave me a rare vantage point: from the human experience of hybrid collaboration, to the physical training environment of data center operations, to the computational frontier of AI/ML infrastructure.

It reinforced my conviction that the most meaningful technology work happens at the intersection of rigorous engineering and genuine human need.

Testimonials

I had the pleasure of working with Jeff on the design and prototyping of an AR product. Jeff solves problems with a systems view, thinking of how the overall design will be impacted even by what appear to be tiny tweaks. He always came with his sleeves rolled up, ready to take on challenges as we navigated uncharted waters.

I relied on his and the AfterNow team's impressive expertise in Mixed Reality R&D. The product exceeded my expectations. I would not hesitate to engage him again. On a personal note, he always seemed to be in a good mood and his optimism lifted the whole project.

Teresa Nick — Microsoft CO+I

Jeff is innovative and his critical thinking skills and Augmented Reality expertise allowed us to identify solutions to our user needs in record time. Jeff developed the program from ideation to deployment.

The results were excellent, and our users were ready to interact with the tools developed from day one. The solution Jeff built was of the utmost quality, and since the program was deployed there have been zero bugs or issues. I highly recommend Jeff as a partner and leader for technology and innovation.

Kimberly M — Microsoft CO+I