Using a Second GPU in Strata: Layer Split vs. Helper Cache on RTX 5080 + 5060 Ti

What the second card handles depends on the method, and in every configuration tested on this PC prompt processing was slower than with one card

What we measured here is a setup where a second GPU with less compute than the main GPU is added over a narrow link such as an external x4 connection, and the model files do not fit in VRAM, so many of the experts sit in system RAM. The RTX 5080 in this PC and the RTX 5060 Ti attached externally over OCuLink are one such pair; we have not checked whether another pair with the same characteristics gives the same results. The Qwen3.8-Flash-Next IQ3_S we ran is distributed as two files totaling about 84GB (the download size), which does not fit even in the combined VRAM of both cards (16GB each). Strata keeps the experts themselves in system RAM and computes with frequently used ones loaded into the GPU's expert cache (an area in VRAM that holds experts). The official benchmark records include a case where, in a setup in which nearly all of the experts in use fit in the caches of two fast cards, prompt processing with a hand-picked split was faster than a single 5080. The official docs also say that adding cards is worthwhile when the experts in use do not fit on the fast card, but the 5080 + 3090 benchmark records were not able to test that case. This PC differs from the official measurement in that the model files do not fit in the VRAM of both cards, but we have not checked whether it counts as the case where the experts in use do not fit (for caveats about the figures and baselines, see the "Differences from the official measurements" section).

We tried two ways of using the second card: layer split, where the model's layers are divided between the two cards, and the helper cache (helper expert cache), where the second card holds additional experts and helps the 5080. What we looked at was what changes in prompt processing (prefill, the stage that reads the input in bulk) and in generation (decode, the stage that outputs one token at a time). Layer split involves a value called K. K is the number of the layer at which the later card takes over (the first card handles layers 0 to K-1), and it is separate from the input length.

How to read the Strata versions

• 0.1.39 (October 6, 2026): measured the speed of layer split (auto and K=32) and the helper cache (without and with the optimization option --remote-expert-opt; what it does is described in the "Three ways to use a second GPU in Strata" section).
• 0.1.40 (same day): checked answer pass/fail and the total time for long inputs with the helper cache. Layer split was not measured on 0.1.40. We measured 0.1.40 as of October 6, not the later 0.1.40.1 to 0.1.40.4.
• 0.1.41 (October 9, 2026): because the way automatic layer split divides the layers changed, we re-measured only the speed of automatic layer split and a single 5080. K=32 and the helper cache were not measured on 0.1.41.

We did not test the peer tier (the method that uses direct GPU-to-GPU transfers), two GPUs of the same model, an x16 connection, Linux, or setups with three or more cards.

With an input of about 32,000 tokens, every two-card configuration took longer to first character (median) than a single 5080.

• On 0.1.41, automatic layer split (K=2, 5060 Ti first) took 32.7 seconds, several times as long as a single 5080 (9.02 seconds).
• On 0.1.39, automatic layer split (K=47, 5080 first) took 62.1 seconds, K=32 (5080 first) 15.2 seconds, and the helper cache 13.1 seconds without optimization and 13.3 seconds with it, all longer than a single 5080 (8.86 seconds).

On 0.1.40 we looked at the total time per request rather than time to first character. With an input of about 240,000 tokens, the helper cache (with optimization) took 112.4 seconds against 82.0 seconds for a single 5080, and most of that was the prompt processing time recorded by the engine (prompt_ms). A table putting layer split and the helper cache side by side on the same criteria is in the "How to use the second card" section.

Also, on Windows, in the configurations that used the second card, the part of the expert area in system RAM that the GPU can read directly, without going through a staging area, dropped from 46GiB with a single 5080 to 7GiB. We did not take measurements with the 8GiB limit changed, so we do not know whether this is the cause of the slowdown. Details are in the "On Windows, using a second card reduces the system RAM the GPU can read directly" section.

In a separate test using a different quantization, we tried five different layer splits.

Three ways to use a second GPU in Strata

Based on Strata's official docs (SECOND_GPU.md and MULTI_GPU.md in v0.1.41), there are three ways to use a second GPU. The official docs do not themselves count them as "three." For how to write the option values, we defer to those two documents.

• Layer split (--layer-split): places the layers across the two cards.
• Helper cache (--expert-cache-device1): puts additional experts on the second card so it can help the first.
• Peer tier (--peer-device): uses direct GPU-to-GPU transfers (P2P or NVLink). On 0.1.41 it can run without P2P, but only for sharing prompt processing.

According to the official docs, the peer tier cannot be combined with either layer split or the helper cache.

As MULTI_GPU.md explains it, in layer split every card holds a copy of the dense weights, the conversation state (KV cache), a buffer for prompt processing, and the expert cache for the layers it handles, and the last card also holds the output head and the lookahead layers. The experts themselves are kept once in system RAM, and any expert that is on no card is computed by the CPU. Data passes between cards only once per verify window (one pass that verifies a batch of drafted lookahead tokens), and NVLink or P2P is said to be unnecessary.

For the helper cache, according to SECOND_GPU.md, the first card keeps the dense layers, the KV cache, the MTP (the layer that looks ahead and drafts the next few tokens; the main model verifies the draft) and its own expert cache as before, and the second card holds additional experts. Prompt processing (prefill) is done by the first card only, and the second card helps with generation (including verifying the lookahead). P2P is said to be unnecessary here as well.

The helper cache has an optimization option (--remote-expert-opt). SECOND_GPU.md says the engine default is off, and that from 0.1.39b the setup (the step that generates Strata's launch options) adds it to configurations with two or more cards. With the optimization on, the second card sums the results of the experts it computed and returns them as a single vector per token, and it takes over frequently used experts that were being computed on the CPU, swapping out less-used ones. Because the order of addition changes, the official docs do not claim the output is bit-identical. It only takes effect when the helper cache is in use; layer split runs as before even with it set. The 0.1.41 MULTI_GPU.md says that since the option does nothing in a layer-split-only configuration, the setup no longer adds it there.

The layer split configurations in this article (auto and K=32 on 0.1.39, auto on 0.1.41) were launched with --remote-expert-opt, but if the official description holds, it has no effect on layer split, so they are not a with/without optimization comparison. With and without optimization was measured separately, in the helper cache configuration.

The table below summarizes the assignments shown in the startup logs. The "Prompt chunk" column is the number of tokens of input read at a time; it is covered in detail in the "Layer split: auto gave the 5060 Ti the fewest possible layers" section. The "Strata" column shows the version measured.

Configuration	Strata	Layers on the 5080	What the 5060 Ti handles	5080 expert cache	5060 Ti expert cache	Prompt chunk
Single RTX 5080	0.1.41	All 48 layers	None	3752 (7.17GiB)	-	8192 tokens
Layer split, auto (K=2)	0.1.41	2–47 and output head	Layers 0–1 (first)	3880 (7.44GiB)	1024 (1.76GiB)	2048 tokens
Single RTX 5080	0.1.39	All 48 layers	None	3762 (7.19GiB)	-	8192 tokens
Layer split, auto (K=47)	0.1.39	0–46 (first)	Layer 47 and output head	4533 (8.58GiB)	512 (1.27GiB)	768 tokens
Layer split, K=32	0.1.39	0–31 (first)	Layers 32–47 and output head	4795 (8.85GiB)	4093 (8.22GiB)	8192 tokens
Helper cache (without / with optimization)	0.1.39	All 48 layers	Additional experts	3832 (7.32GiB)	7548 (14.28GiB)	8192 tokens

There are 48 layers, counted from 0. The expert cache counts are numbers of (layer, expert) pairs. With auto on 0.1.41 the card order was swapped and the 5060 Ti went first. The helper cache without and with optimization had identical assignments, so they share one row. The 5080 also drives the display and uses VRAM for that as well. In the 5060 Ti expert cache column, automatic layer split puts 512 to 1024 entries on it, K=32 puts 4093, and the helper cache puts 7548, so how much goes onto the second card varies widely with the method and the split.

The 5060 Ti is attached externally over OCuLink at PCIe 4.0 x4. The system-RAM-to-GPU transfer rate measured at startup was in the 40GB/s range for the 5080 and 7.0GB/s for the 5060 Ti. We did not run a measurement that changed only the link speed, so we also do not know whether this difference affected the results. Running nvidia-smi topo -p2p r on this PC on October 9, 2026 showed P2P reads between the 5080 and the 5060 Ti as NS (not supported). According to the 0.1.41 release notes, GeForce pairs without P2P can now also use the peer tier's prompt sharing through a transfer area in system RAM. It is off by default, and in the official example generation is 4–5% slower and the low-order bits of the answer change. We did not test the peer tier over either path in this article.

Comparing automatic layer split with a single 5080 on 0.1.41

These are the results of sending the same inputs to a single 5080 and to automatic layer split (K=2, 5060 Ti first) on 0.1.41. The conditions are in the "Test machine and measurement method" section.

Configuration	Time to first character (approx. 1,000)	Time to first character (approx. 4,000)	Time to first character (approx. 32,000)	Prompt processing (approx. 32,000)	Generation (approx. 32,000)
Single RTX 5080	1.24 seconds	1.60 seconds	9.02 seconds	3,522 tok/s	82.6 tok/s
Layer split, auto (K=2)	2.32 seconds	5.29 seconds	32.7 seconds	969 tok/s	57.0 tok/s

At every length from about 1,000 to about 32,000 tokens, the single 5080 reached the first character sooner, and the gap widened as the input got longer.

Generation speed (median of the engine's records) was lower too: at about 1,000 and about 4,000 tokens, layer split ran at 68.5 and 64.7 tok/s against 76.8 and 73.2 tok/s for the single 5080, and the median was lower for layer split at all three lengths. At about 1,000 and about 4,000, the ranges of the 5 runs overlapped with the single 5080, so we cannot say there is a difference. At about 32,000, even the ranges did not overlap (layer split 53.8–68.3 tok/s, single 5080 73.8–90.6 tok/s). Response content was not scored in this measurement.

The following note concerns only the single 5080's value at about 1,000 tokens; the results at about 4,000 and about 32,000 are unaffected. According to the 0.1.41 release notes, starting with 0.1.41, when a single NVIDIA GPU processes a prompt chunk of less than 1,024 tokens, the CPU handles some of the experts by default (answers can change slightly; it can be turned off with STRATA_PREFILL_CPU_SHARE=0; it does not apply to layer split). One of the five single-5080 runs at about 1,000 tokens (1,002 tokens) falls under this.

Why auto chose this split is covered in the "Layer split: auto gave the 5060 Ti the fewest possible layers" section.

Five configurations compared on 0.1.39

These are the results for a single 5080 and four two-card configurations on 0.1.39.

Configuration	Time to first character (approx. 1,000)	Time to first character (approx. 4,000)	Time to first character (approx. 32,000)	Prompt processing (approx. 32,000)	Generation (approx. 32,000)
Single RTX 5080	1.22 seconds	1.50 seconds	8.86 seconds	3,621 tok/s	78.5 tok/s
Layer split, auto (K=47)	2.89 seconds	8.64 seconds	62.1 seconds	511 tok/s	68.3 tok/s
Layer split, K=32	3.01 seconds	4.09 seconds	15.2 seconds	2,103 tok/s	73.6 tok/s
Helper cache, without optimization	2.19 seconds	2.74 seconds	13.1 seconds	2,444 tok/s	74.5 tok/s
Helper cache, with optimization	2.13 seconds	2.69 seconds	13.3 seconds	2,400 tok/s	81.5 tok/s

All four two-card configurations took longer to first character than the single 5080 at every length: about 1,000, about 4,000 and about 32,000.

For generation speed (median of the engine's records), the ranges of the 5 runs for all four two-card configurations overlapped with the single 5080 at about 1,000, about 4,000 and about 32,000, so we cannot say there is a difference. Looking at the medians alone, the helper cache with optimization was higher than the single 5080 at all three lengths (81.5 tok/s vs. 78.5 tok/s at about 32,000), and auto (K=47, 5080 first) was lower at all of them. The helper cache without optimization and K=32 were higher at about 1,000 and about 4,000 and lower at about 32,000.

The 0.1.39 table and the table in "Comparing automatic layer split with a single 5080 on 0.1.41" differ in version and measurement date. They were measured on the same PC with the same launch options and the same procedure (how inputs were built, number of runs, output limit), but not as an interleaved comparison. Median time to first character at about 32,000 was 62.1 seconds for auto on 0.1.39 (K=47, 5080 first) and 32.7 seconds on 0.1.41 (K=2, 5060 Ti first); for the single 5080 it was 8.86 seconds on 0.1.39 and 9.02 seconds on 0.1.41. We have not identified which change between the versions made the difference.

The way the helper cache is handled changed in 0.1.40, so read the helper cache values in the "Five configurations compared on 0.1.39" table as 0.1.39 values. The check on 0.1.40 is in the "Checking the helper cache on 0.1.40: same pass rate, and long inputs still took longer than with one card" section.

Layer split: auto gave the 5060 Ti the fewest possible layers

How auto chooses the split

K is the number of the layer at which the later card takes over: the first card handles layers 0 to K-1 and the next card handles K onward (a different number from input lengths such as "about 32,000 tokens"). The 0.1.41 MULTI_GPU.md explains that with the split set to automatic (auto), Strata tries every possible split, counts experts more heavily the more often they are used according to the record of expert usage (expert profile), and picks the split that fits the most in the caches.

In the 0.1.41 source (generate.cpp), however, with two cards it tries K in order starting from 2, predicts for each split the time one generation step takes (one pass that verifies a batch of several lookahead tokens; "decode window" in the source's terms), and picks the shortest. The prediction includes each card's time per layer and the cost of experts that do not fit in the cache. That cost is multiplied by a factor derived from each card's link speed measured at startup, so the slower a card's link, the larger the estimate. This prediction targets the time of one generation step and does not directly predict prompt processing time. It does, however, skip splits under which prompt processing cannot start. Among splits with the same prediction, it picks the one where the most heavily loaded card's time (number of layers × time per layer) is smaller. If no split can start, it drops only the can-it-start condition and chooses again. This criterion is worded differently from "fits the most in the caches" in MULTI_GPU.md. The layer split benchmark record in the same repository (the bench README) also says that it predicts the time of one generation step for each split and picks the fastest, and states that it chose K=22 on that benchmark PC. The predicted values appear in the startup log, so you can check them on your own machine.

From 0.1.41, automatic layer split puts the faster card (SM count × maximum clock) last. According to MULTI_GPU.md, this is because the last card handles the output head, the lookahead layers and verification, and the prompt chunks also wait on it; it gives an example in which prompt processing of an input of about 6,000 tokens on an RTX 4070 Ti SUPER + RTX 5060 Ti pair took either 50 seconds or 15 seconds depending on the order. Setting "gpu_order": "as_given" keeps the order you specified. According to the release notes, the automatic split now also looks at each card's PCIe link.

0.1.41 result: the 5060 Ti went first and handled only layers 0–1

With auto on 0.1.41 on this PC, the card order was swapped (card order 1,0 in the server log): the 5060 Ti went first and handled layers 0–1, and the 5080 handled layers 2–47 and the output head (K=2). The startup log estimated the 5060 Ti at 0.78ms per layer with a cost factor for experts not in the cache ("miss cost" in the log) of ×2.85, and the 5080 at 0.33ms per layer with a factor of ×1.00 (the factor is 20GB/s divided by the link speed measured at startup, and it is never set below 1), and reported that of 46 possible splits, K=2 had the shortest prediction (27.0ms). This prediction compares two-card splits with each other and does not include a comparison with a single 5080. The prediction being the shortest and the measured result being slower than one card are not in conflict. K=2 is the lower end of the range the source tries with two cards, and it is the split that gives the first card the least work.

In this case, all of the experts of layers 0–1 fit in the 5060 Ti's expert cache (one layer has 512 experts). The prompt chunk was 2048 tokens, and no warning appeared.

0.1.39 result: the 5080 went first, and the 5060 Ti got only the last layer

0.1.39 has no logic for swapping the card order, so even with auto the order stayed as specified (5080 first). K=47 was chosen, and the 5060 Ti got only the last layer (layer 47) and the output head. The 5060 Ti still had free VRAM. The 0.1.39 startup log only shows the predicted time of one generation step (25.9ms) on the K=47 line and does not say how many splits it was the shortest of. We have not checked the 0.1.39 source for how it chooses.

On this launch, a warning appeared in the log. In short, it said the prompt chunk had been reduced from 8192 tokens to 768 tokens because the 5060 Ti's cache was too small. As a breakdown, it said that reading in 8192-token chunks would require borrowing 512 entries from the 5060 Ti's cache, but the 5060 Ti's cache had only 512 entries, and the rules are to keep 128 entries and to lend at most 85%. The warning also said prompt processing would be slower than a single 5080, but that is the engine's expectation, not a measurement of the contribution on this PC. Even with free VRAM, the warning shrank the chunk on the assumption that the prompt buffer is borrowed from the cache allotment. MULTI_GPU.md explains that in layer split every card has a buffer for prompt processing; we have not checked how that relates to this warning.

The next line of the same warning suggested that if the assignment leaves the card with this little cache, it can be used as a helper expert cache instead of as one stage of a layer split, and showed how to drop layer split and add --expert-cache-device1. For the helper cache measurements in this article we set the value to auto (--expert-cache-device1 auto).

With the split fixed at K=32 on 0.1.39 (5080 first), the 5060 Ti handled layers 32–47 and the output head, and the prompt chunk stayed at 8192 tokens. Even so, time to first character was longer than a single 5080. We did not run a measurement that changed only the chunk size, so we have not identified the cause, including whether the smaller chunk contributed to the slowdown. K=32 is one fixed split we tried, and we did not try other values.

K=2 on 0.1.41 is the lower end of the range the source tries, and K=47 on 0.1.39 is just the last layer, so on both versions the 5060 Ti got the fewest layers possible. Note that the CUDA numbers Strata assigns follow the reordered order, so on this 0.1.41 launch CUDA0 is the 5060 Ti and CUDA1 is the 5080; on 0.1.39 it is the other way around.

Differences from the official measurements

The layer split measurements in MULTI_GPU.md and the benchmark record it refers to were taken on a community member's PC (Windows 11, Ryzen 9 9950X3D) with an RTX 5080 (x16) and an RTX 3090 (PCIe 4.0 x4, about 6GB/s), using the Coder model (IQ1_M, experts about 23GB) with a context of about 32,000 tokens. At K=18 or more, 98–100% of the experts in use fit in the two cards' caches. The official docs summarize that with the best hand-picked split (K=26), prompt processing of inputs from about 16,000 to about 28,000 tokens was 18–20% faster than a single 5080. However, the official table lists the single 5080 at about 16,000 in two rows, 1,726 tok/s and 2,017 tok/s. K=26's 2,039 tok/s is 18% above the 1,726 from the same series of runs in which K was varied, and about 1% above the 2,017 from a separate run. At about 28,000, K=26 was 2,357 tok/s and the single 5080 was 1,970–1,974 tok/s. For generation, the official summary is that at K=26–28 story text was about the same and code was 4–7% faster (compared against the single-5080 row measured with three repeated generation runs; calculating from the table values, K=26 alone is about 4%). Auto (K=22) in the official table was 2,037 tok/s at about 16,000 tokens (single 5080: 1,726–2,017) and 2,073 tok/s at about 28,000 tokens (single 5080: 1,970–1,974).

The differences from this PC are the model size (the IQ3_S in this article totals about 84GB in files and does not fit in the VRAM of both cards), the second card, and the context length. The second card's link is PCIe 4.0 x4 class in both cases. The benchmark record itself says that the case of a large model whose experts mostly do not fit on the two fast cards could not be tested with that PC's system RAM (62GB).

MULTI_GPU.md gives an example where adding a much slower RTX 2080 Ti as a third card to a 5080 + 3090 setup that could already hold the model made things slower, and says to remove such slow cards. That concerns a third card; it is not advice to remove the slower card from a two-card setup. The same document also says adding cards is worthwhile when the experts in use do not fit on the faster card, but the 5080 + 3090 benchmark records were not able to test that case. This PC is a setup where the model files do not fit in the VRAM of both cards, but we have not checked whether it counts as the case where the experts in use do not fit on the faster card. Here, the second card did not make prompt processing faster. This is a result from one machine and one pair of cards, and it does not refute the official assessment.

How to check on your own PC

1. Look at the assignment at launch. Check the card order on the card order line of the server log (this line appears only when the order is swapped; it was not in this article's 0.1.39 auto log), the split and prediction on the layer split auto: K= line of the startup log, and the prompt chunk on the prompt chunk auto: line.
2. Send inputs of the same length to a one-card launch and a two-card launch, and compare time to first character.
3. If automatic layer split is slower than one card, fix K and compare with the same input. The only K we fixed in this article was K=32 on 0.1.39 (5080 first), so we cannot say which K is best. What K means depends on which card is first, so when you fix it, also note which card you put first when comparing.

Where the logs end up depends on how you launch Strata. For this article's measurements, we wrote the server and engine output to separate files and read those. With auto on 0.1.39 the engine warned when the chunk shrank from 8192 to 768; with auto on 0.1.41 it was 2048 from the start and no warning appeared. On this PC we also tried switching to the helper cache and fixing K=32.

Helper cache: prompt processing stays on the 5080, and the 5060 Ti helps with generation

With the helper cache, prompt processing (prefill) is done by the 5080 alone, and the 5060 Ti helps with generation. On this PC with 0.1.39, the 5060 Ti held 7548 additional expert entries, and the prompt chunk also stayed at 8192 tokens. The 5080's expert cache was almost the same as with a single 5080.

Even so, time to first character was longer than with a single 5080, and turning the optimization on or off made little difference (figures in the "Five configurations compared on 0.1.39" table). We have not identified why it got slower even though prompt processing is done by the 5080 alone. One candidate is that on Windows, using a second card leaves the GPU able to read less system RAM directly than the 46GiB it can with a single 5080 (the "On Windows, using a second card reduces the system RAM the GPU can read directly" section), but we did not take measurements with that limit changed.

On Windows, using a second card reduces the system RAM the GPU can read directly

The "directly readable" part here means the part of the experts in system RAM that has been set up so the GPU can read it as is over PCIe. The rest is first copied to a staging area and then sent to the GPU. On this PC the area holding the experts is the same size, about 47GiB, with a single 5080 and in the two-card configurations; going to two cards does not shrink the area itself. The 8GiB limit when a second card is used on Windows applies equally to layer split and to the helper cache.

According to MULTI_GPU.md, under the Windows (and WSL2) driver model (WDDM), when a second card is used, the part of the expert area the GPU can read directly is limited to 8GiB, and the rest is streamed through the staging area. The Linux driver has no such limit, and from 0.1.31 Strata makes the whole area directly readable by the GPU. As background, the official docs cite a case from when this limit also applied on Linux, in which an RTX 4090 + RTX 3060 layer split lost two-thirds of its prompt processing speed. That is a Linux case, not a figure showing how much the limit matters on Windows.

In this PC's logs, the directly readable part was 46GiB with a single 5080 (on both 0.1.39 and 0.1.41; the GiB figures in the log are truncated to whole numbers, and with one card this is the whole area) and 7GiB in every configuration that used the second card (the four 0.1.39 configurations and auto on 0.1.41). The logs for the two-card configurations say it was limited to 8GiB because of multi-GPU under WDDM, or remote experts.

To check on your own machine, look for the line cudaHostRegister limited to 8 GiB in the startup log. If this line appears on Windows, you can read it as the directly readable part being reduced in that environment too. We did not try the setting that changes the limit (STRATA_ARENA_PIN_GIB) in this article.

Checking the helper cache on 0.1.40: same pass rate, and long inputs still took longer than with one card

In 0.1.40, one thing about how the helper cache is handled changed. According to SECOND_GPU.md, a problem was fixed where the set of experts the first card reads over PCIe also included experts the helper cache already held, and on a different machine (two RX 6900 XT cards) the time the first card waits per verification pass dropped substantially. So we checked the helper cache separately on 0.1.40. Read the helper cache speeds measured on 0.1.39 as 0.1.39 values; they were not compared with 0.1.40 under the same conditions.

First, these are the results of giving the same 8 questions 5 times each with the thinking setting at medium.

Configuration	Passed	Median time	Runs with answer text identical to the single 5080
Single RTX 5080	40/40	5.71 seconds	(baseline)
Helper cache, without optimization	40/40	5.95 seconds	25/40
Helper cache, with optimization	40/40	6.05 seconds	25/40

All three configurations passed all 40 runs. The answer text was identical, wording included, to the single 5080's answer on the corresponding run in 25 of 40 runs both without and with optimization; the rest passed even though the wording differed. The median time combines 8 questions whose times range from about 2 seconds to about 60 seconds, so we do not read anything into differences of 0.1 to 0.3 seconds. The two helper cache configurations were launches that ran only the 8 questions × 5 times; for the single 5080 we used the 40 medium runs of the same 8 questions from a launch that ran 16 questions × 4 thinking settings. These 8 questions are a check that answers do not break when the second card is added, not a test large enough to measure differences in answer quality.

Examples of the questions are shown below (the originals are in English; the table gives a summary). The 8 questions are 4 answered in JSON (identifying the culprit from three people's statements, merging overlapping intervals, an order total, and a tax calculation) and 4 that fix a broken Python function (reading a config file, aggregating logs, merging intervals, and a tax-inclusive total); the code questions were graded by running the fixed code against tests for the function.

Question	Description	Correct answer
Identify the culprit from three statements (JSON)	A: "The culprit is B." B: "The culprit is not B." C: "The culprit is not A." Only one of them is telling the truth, and there is one culprit	A
Merge overlapping intervals (JSON)	From [5,8], [1,3], [3,6], [10,11] and [10,12], merge the intervals that overlap or touch, and sort them by start	[1,8], [10,12]
Tax calculation (JSON)	Pre-tax 105 yen (10% tax) and 1031 yen (8% tax). Round each item's tax half up to a whole yen, and give the tax and the total	Tax 11 yen and 82 yen, total 1229 yen
Fix the tax-inclusive total function (code)	Fix a function that rounds tax with the built-in round so that it rounds per item (0.5 rounds up) without floating-point errors	Returns 1229 for 105 yen and 1031 yen (also checked with 4 other values)

We looked at what the second card did using counters the engine reports per request. For the 40 helper cache runs, the medians of the 5060 Ti counters divided by each request's number of output tokens are as follows.

Configuration	Experts computed on the 5060 Ti (per token)	Data returned to the first card (per token)	Time the host waited for the 5060 Ti (per token)
Helper cache, without optimization	87.9	0.86 MiB	0.31 ms
Helper cache, with optimization	318.7	0.54 MiB	1.58 ms

With optimization, the 5060 Ti computed more experts and returned less data to the first card. On the other hand, the time the host (CPU) waited for the 5060 Ti's results went up. The returned data is the amount counted by the engine, not a measurement of the amount actually sent over PCIe or of bandwidth. We cannot tell how each of these changes affected the result, and the median time showed no clear difference either, so this measurement gave no clear basis for choosing whether to use the optimization.

For long inputs, we gave a task of finding values embedded in an input built from code documentation, 5 times each with thinking off. Total time is from the start to the end of one request, and prompt processing time is the prompt_ms recorded by the engine; both are medians. The helper cache was measured with optimization.

Configuration	Total time (approx. 32,000)	Prompt processing time (approx. 32,000)	Total time (approx. 240,000)	Prompt processing time (approx. 240,000)
Single RTX 5080	10.3 seconds	9.47 seconds	82.0 seconds	80.4 seconds
Helper cache (with optimization)	14.7 seconds	13.9 seconds	112.4 seconds	110.8 seconds

At both about 32,000 and about 240,000 tokens, the helper cache took longer than a single 5080, and most of the time was prompt processing. The median numbers of tokens actually sent were 30,990 for about 32,000 and 238,988 for about 240,000. These results are part of a separate measurement that recorded how Windows uses memory. The about-32,000 values are from runs with the recording turned off and the about-240,000 values from runs with it on; comparing both at about 32,000, the difference was around 0.1 seconds, which does not affect the comparison. The inputs were built differently from those in the 0.1.39 and 0.1.41 speed measurements, so these results are not compared side by side with those tables.

How to use the second card

The table in this section comes from one PC and one pair of cards and is no basis for ranking the methods. Each cell pairs its value with the single 5080's value on the same version.

What you want to know	Layer split, auto	Layer split, K=32	Helper cache
Time to first character with about 32,000 tokens of input (median)	0.1.41: 32.7 seconds vs. 9.02 seconds for a single 5080. 0.1.39: 62.1 seconds vs. 8.86 seconds	0.1.39: 15.2 seconds vs. 8.86 seconds	0.1.39: 13.1 seconds without optimization and 13.3 seconds with it, vs. 8.86 seconds
Total time with about 240,000 tokens of input (0.1.40, median)	Not measured	Not measured	112.4 seconds (with optimization) vs. 82.0 seconds for a single 5080
Generation speed (median)	Lower than a single 5080 on both 0.1.39 and 0.1.41 (the 5-run ranges did not overlap only at about 32,000 on 0.1.41)	0.1.39: higher than a single 5080 at about 1,000 and about 4,000, lower at about 32,000 (the 5-run ranges overlapped at every length)	0.1.39 with optimization was about the same as or slightly higher than a single 5080, and the ranges overlapped
Answer pass/fail	Not measured	Not measured	All 40 runs passed on the 8 questions on 0.1.40, as with a single 5080
Log lines to check	card order (appears only when the order is swapped) / layer split auto: K= / prompt chunk auto:	layer split: CUDA1 runs layers / prompt chunk auto:	additional experts

Under these conditions, we saw no benefit from the second card in shortening time to first character for long inputs. At about 32,000 tokens on 0.1.39, the gap to the single 5080 (8.86 seconds) was smallest for the helper cache (13.1 seconds without optimization, 13.3 seconds with it), next for K=32 (15.2 seconds), and largest for auto (K=47, 62.1 seconds). This order reflects the size of the gaps for this one pair and does not show which method is better.

In separate tests without Strata, we ran 27B–35B class LLMs entirely in the combined VRAM of two 16GB cards.

Test machine and measurement method

The 0.1.41 speeds (October 9, 2026) were measured with Strata 0.1.41, Qwen3.8-Flash-Next IQ3_S, and a maximum context of 262,144 tokens. The KV cache was int8, with 32,768 tokens' worth kept in VRAM, and the VRAM headroom setting was 1200MiB. With thinking off, temperature 0 and a 256-token output limit, we sent inputs of about 1,000, about 4,000 and about 32,000 tokens built from code documentation 5 times each and took the median. Prompt processing and generation speeds come from the engine's records.

The 0.1.39 speeds (October 6, 2026) were measured with the same conditions and inputs as 0.1.41 apart from the version. The measurement command was the same except for the output path and labels, but for the single 5080 on 0.1.39 only, we also measured about 64,000 and about 128,000 in the same launch, after about 1,000, about 4,000 and about 32,000. The table shows the about-1,000 to about-32,000 results, which were measured before those.

The 0.1.40 helper cache check (October 6, 2026) used Strata 0.1.40, IQ3_S and a maximum context of 262,144 tokens. The 8-question check used thinking set to medium, temperature 0 and an 8192-token output limit; all three configurations had the 5080's expert cache matched at 3742 entries, and the helper cache's 5060 Ti held 7548 entries (14.28GiB). The long-input runs used thinking off and a 256-token output limit, with the 5080's expert cache matched at 3827 entries in both configurations.

Every measurement started the server once per configuration and measured back to back; configurations were not interleaved. The speed measurements used a short 256-token output limit, so we do not read anything into small differences in generation speed.

The 5060 Ti is connected over OCuLink at PCIe 4.0 x4.

The measured values in this article were taken on our test machine with these settings: Strata 0.1.39 and 0.1.40 on October 6, 2026, and 0.1.41 on October 9, 2026.

Summary

In the configurations tested on this PC, prompt processing was slower than a single 5080 on all of 0.1.39, 0.1.40 and 0.1.41. Automatic layer split chose K=47 (5080 first) on 0.1.39 and K=2 (5060 Ti first) on 0.1.41, and on both versions it gave the 5060 Ti the fewest layers possible (on 0.1.41, the two layers at the lower end of the range it tries; on 0.1.39, the last layer). All of these are results under the conditions of running 0.1.39–0.1.41 on this PC, where a card of different performance was added over a narrow link. By goal, the results under these conditions were as follows.

• Shorter time to first character: no benefit was seen in the configurations tried. With the helper cache too, time to first character on 0.1.39 and total time for long inputs on 0.1.40 were longer than a single 5080.
• Faster generation: for the helper cache on 0.1.39, the 5-run ranges overlapped with the single 5080, so we could not say there was a difference. Automatic layer split was lower than a single 5080 by median (at about 32,000 on 0.1.41, even the ranges did not overlap).
• Checking that answers do not break: the helper cache on 0.1.40 passed all 40 runs of the 8-question check, as did the single 5080 (the check is small).

When you add a second card or update the version, check the assignment logs using the steps in "How to check on your own PC" (the line to use for checking the helper cache is in the last row of the table in "How to use the second card"), then compare with a one-card launch using inputs of the same length.

Measurements with a second GPU added on the Strata 0.2 series, and how to choose a model based on the amount of system RAM and 12GB or 16GB of VRAM, are in "Strata models and memory requirements".

We also tested installing and using Strata on 0.2.10, including whether to start with the desktop app or Terminal, and launching, updating and uninstalling.

References

• Strata (GitHub)
• Strata v0.1.41 docs/MULTI_GPU.md (layer split)
• Strata v0.1.41 src/program/generate.cpp (how automatic layer split chooses the split)
• Strata v0.1.41 docs/SECOND_GPU.md (helper cache and peer tier)
• Strata v0.1.41 bench/results/2026-09-29-layer-split/README.md (official layer split benchmark record)
• Strata v0.1.41 (release)
• Strata v0.1.40 (release)
• Strata v0.1.39 (release)
• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Hugging Face)