Steering a served model at compiled speed

How the vllm-lens fork compares with upstream and with vLLM

Figure 1:

Step 1: steering no longer slows eager mode

Figure 2:

Step 2: what compilation adds

Figure 3:

Step 3: the remaining gap

Figure 4:

Full results

Speed of every case.

Memory and load time of every mode.