Where Mixing Stops Being Mechanical

Most of mixing, when you watch yourself do it, is clicking and dragging.

I noticed this on a session a few weeks ago. I was three hours in, working on a vocal chain, and I stopped to count what my hands had actually been doing. Opening plugins. Sweeping a frequency. Toggling a mute. Pulling a fader down two dB and putting it back. Hitting play, hitting stop. Comparing one preset to another. The hand was busy. The ears were doing something else, sometimes alongside the hand and sometimes not.

That raised a question I couldn't answer by thinking about it. How much of mixing is the hand doing mechanical work, and how much is the hand actually part of the listening? I'd have given a confident answer if asked. I just wouldn't have trusted it.

This is how I tend to work. When a question about my own process is worth answering and I can't get to it by introspection, I build the apparatus that forces an answer. It's the same problem-solver instinct that pushes me toward tools rather than theory. Building wasn't the point. Building was the fastest way to find the line.

So I spent a day on it. The setup is plain: a small macOS app reads the webcam, sends the hand position to the system cursor, and lets a pinch act as a click. Open palm engages tracking. Pinch with thumb and index drags. Fist disengages. It's mouse emulation, nothing clever — every DAW already speaks mouse. The build ↗ is on GitHub if you want to see it.

Then I sat down with a session I'd already mixed, undid the moves I could undo, and tried to redo them with my hands in the air.

What worked

Plugin parameter sweeps worked. Reaching for a knob, grabbing it, moving it across its range — that motion translates almost directly. In some cases it felt more natural than a mouse, because a mouse forces a sweep through a tiny horizontal patch of desk and the hand just moves where it wants to.

Transport worked. Play, stop, loop, jump to a marker. The hand is fine at hitting a target.

Mute and solo toggles worked. Bypassing a plugin to check it in and out worked. Broad gain rides — the kind where you're shaping a section's energy in three or four dB moves — worked.

These are mechanical motions. The decision is already made before the hand moves. The hand's only job is to deliver the input. Whether the input arrives via mouse, trackpad, MIDI controller, or webcam doesn't change anything about the work.

Where it broke

Fader rides under one dB. Anything where I was making a half-dB push on a vocal phrase and then immediately reaching back to undo half of it. The tracking wasn't bad. My hand wasn't bad. The motion itself doesn't survive being abstracted.

Fast A/B comparisons broke. Toggling between two reverb settings while listening for which one sits better in the mix — the hand needs to be exactly where I expect it, every time, with no latency in my own proprioception. Webcam tracking adds a few milliseconds and a small spatial uncertainty. That's enough to break the loop.

Anything where I was listening hard while moving broke. The fine fader work, the surgical EQ where you're nudging a band by half a dB and listening for the artifact to disappear, the moment-by-moment level adjustments on a busy section. These aren't mechanical motions. They're sensory loops. The body is part of the listening — the small adjustments are coming from somewhere closer to the ear than to the conscious decision.

I'd have told you, before doing this, that I knew where that line was. I didn't. I had it in roughly the right place but I'd put far more of mixing on the craft side than belongs there. The mechanical half is bigger than I thought.

What this is actually about

The interesting result isn't that you can mix with your hands. You can, awkwardly, for the parts that don't matter. The interesting result is the line itself — and the fact that the mechanical half of mixing is where most of the time goes.

Loading sessions. Naming tracks. Setting up routing. Inserting the same plugin chain on the same kind of source for the hundredth time. Bouncing stems. Conforming to delivery specs. None of it is craft. All of it eats hours. That's automation in mastering territory, and it's the same shape of problem that produced ChapterPass — a mechanical step that didn't need a human, sitting between the human and the work that did.

Hand tracking is the wrong tool for that, obviously. The point of the experiment wasn't to ship gesture control. The point was to find out, with my hands, which parts of the work I'd been quietly assuming were craft and which parts weren't. Once you know the line, you stop defending the wrong half.

The build is at github.com/giovicordova/hand-tracking. v1 is mouse emulation only — plugin parameters and transport, no fader automation, no MIDI. Whether it goes beyond v1 depends on whether the question keeps being interesting. Right now the answer it gave me is more useful than the tool itself.