GPU Sold

How we test local AI on a used RTX 3090

Tested Nov 16, 2026 on a used RTX 3090 24 GB (300 W limit), llama.cpp b12023, CUDA build

Every number in this section was measured on one NVIDIA RTX 3090 24 GB (used) with llama.cpp b12023, CUDA build. We publish measured values only. Nothing is estimated or copied from other sites.

The rig

Test machine
PartUsed in our tests
GPUNVIDIA RTX 3090 24 GB (used)
Power limit300 W
CPUIntel Core i9-11900K
System RAM32 GB DDR4
OSUbuntu 24.04
Runtimellama.cpp b12023, CUDA build
TestedNov 16, 2026

What we measure

Speed per model

Prompt processing and generation speed for one chat, with the whole model on the GPU.

llama-bench -ngl 99 -fa 1 -p 512 -n 128 -r 2; Q4 weights; f16 KV cache; batch 1

Largest context that fits

We start the server at a context size and step up until it no longer loads.

llama-server -ngl 99 -fa on -np 1 -c N -ctk/-ctv f16|q8_0; 'fits' = server started and allocated the full context; next size up ran out of VRAM

Speed deep into a conversation

Generation speed with the context already filled to a given depth.

llama-bench -d 0,8192,32768 -p 512 -n 128

Models bigger than 24 GB

Mixture-of-experts models with the expert weights of some layers kept in system RAM.

llama-bench -ngl 99 -fa 1 --n-cpu-moe N

Several chats at once

Total and per-chat generation speed with several requests running together.

llama-batched-bench -npp 512 -ntg 128 -npl 1,2,4,8 -fa on (per-chat = total/parallel)

Power

Board power reported by the card.

average board power from nvidia-smi over the whole benchmark window (prefill + decode)

Units

tok/s is tokens per second as reported by llama.cpp, not wall-clock time. 1K of context is 1,024 tokens. VRAM is shown in the same GB the card’s 24 GB is quoted in (1 GB = 1,024 MiB). File sizes are the download size of the model file.

What we do not test yet

Output quality: these pages measure speed, memory and power, not how good the answers are. Other GPUs: only the RTX 3090 so far, so we do not publish numbers for cards we have not run.

How often

Results are updated when we re-test. Each page shows its test date.

Local AI test method: used RTX 3090 | GPU Sold