Knobs System

All addresses in this page apply to ptxas v13.0.88 (CUDA 13.0). Other versions will differ.

The knobs system is ptxas's internal configuration mechanism -- a separate layer beneath the public CLI flags that exposes 1,294 tuning parameters to NVIDIA developers. Every significant compiler heuristic (register allocation thresholds, scheduling priorities, pass enable/disable, peephole rules) has a corresponding knob. The system is shared with cicc via a common header (generic_knobs_impl.h) but ptxas instantiates it twice: once for the DAG scheduler pipeline (99 knobs) and once for the OCG (Optimizing Code Generator) backend (1,195 knobs). All knob names are stored ROT13-encoded in the binary, a lightweight obfuscation that prevents casual strings discovery while being trivially reversible.

The knobs infrastructure lives primarily in two address regions: 0x6F0000--0x6F8000 (DAG knob instantiation, shared with the Mercury SASS pipeline) and 0x797000--0x7A2000 (OCG knob instantiation, the larger set). Both regions are compiled from the same template in generic_knobs_impl.h.

Architecture

                  ┌──────────────────────────────────────────┐
                  │            KnobsInit (sub_79D990)        │
                  │  Called once from global init sub_662920  │
                  └─────┬──────────┬──────────┬──────────────┘
                        │          │          │
              ┌─────────▼──┐  ┌───▼──────┐  ┌▼───────────────┐
              │ ReadKnobsFile│  │ -knob CLI│  │ PTX pragma     │
              │ sub_79D070   │  │ parsing  │  │ (unless        │
              │ [knobs] fmt  │  │          │  │ DisablePragma) │
              └─────────┬───┘  └───┬──────┘  └┬───────────────┘
                        │          │           │
                        ▼          ▼           ▼
              ┌─────────────────────────────────────────────┐
              │       ParseKnobsString (sub_79B530)         │
              │  Handles WHEN=, INJECTSTRING, ~-delimited   │
              └──────────────────┬──────────────────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │   GetKnobIndex           │
                    │   sub_6F0820 (DAG)       │
                    │   sub_79B240 (OCG)       │
                    │   ROT13 decode + lookup  │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │   ParseKnobValue         │
                    │   sub_6F7360 (DAG)       │
                    │   sub_79F540 (OCG)       │
                    │   Type-specific parsing  │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────▼────────────┐
                    │   Runtime knob array     │
                    │   72 bytes per slot      │
                    │   Accessed by index      │
                    └──────────────────────────┘

ROT13 Name Obfuscation

Every knob name in the binary is stored as a ROT13-encoded string. The GetKnobIndex function decodes each character inline during comparison, without ever materializing the cleartext name in memory. The decode is combined with a case-insensitive tolower() comparison against the user-supplied query.

The inline ROT13 decode from sub_6F0820:

// For each character in the stored ROT13 name:
char c = stored_name[i];
if ((unsigned char)((c & 0xDF) - 65) <= 12)
    c += 13;                   // A-M (or a-m) -> N-Z (or n-z)
else if ((unsigned char)((c & 0xDF) - 78) < 13)
    c -= 13;                   // N-Z (or n-z) -> A-M (or a-m)
// Then compare case-insensitively:
if (tolower(query_char) != tolower(c))
    goto mismatch;

The & 0xDF trick converts lowercase to uppercase before range-checking, so both 'a'-'m' and 'A'-'M' hit the first branch. Non-alphabetic characters pass through unchanged. This means knob names like SchedNumBB_Limit with underscores and digits are handled correctly -- only the alphabetic portion rotates.

To reverse-engineer knob names from the binary: extract the ROT13 strings from the knob definition table (64-byte stride at the table base pointer), apply ROT13, and you get the cleartext name.

Knob Descriptor Layout

Each knob is described by a 64-byte entry in the knob definition table. The table is an array at (knob_state + 16) with count at (knob_state + 24).

Offset  Size  Field
──────  ────  ─────────────────────────────────────
+0      8     name_ptr          Pointer to ROT13-encoded primary name
+8      8     name_len          Length of primary name
+16     1     type_tag          Knob type (OKT_* enum, 1-12)
+17     7     (padding)
+24     16    (reserved)
+40     8     alias_ptr         Pointer to ROT13-encoded alias name
+48     8     alias_len         Length of alias name
+56     8     (reserved)
──────  ────
        64    Total

Both primary and alias names are checked during lookup. A knob matches if either its primary name or alias decodes to the query string (case-insensitive). The alias mechanism allows backward-compatible renaming of knobs across toolkit versions.

Knob Value Layout

Runtime knob values are stored in a flat array of 72-byte slots at (knob_state + 72 * index). The slot layout depends on the type:

Offset  Size  Field
──────  ────  ─────────────────────────────────────
+0      1     type_tag          Runtime type (0=unset, 1-10)
+1      7     (padding)
+8      8     value / pointer   Primary value (int32, int64, float, double, or pointer)
+16     8     list_begin        For list types: first element pointer
+24     8     list_sentinel     For list types: sentinel node
+32     4     aux_value         Secondary value (e.g., int-range high bound)
+36     4     (padding)
+40     8     list_tail         For list types: last element pointer
+48     8     list_head         For list types: head pointer
+56     4     element_count     For list types: number of elements
+60     4     (padding)
+64     8     allocator         Arena allocator pointer for list/range types
──────  ────
        72    Total

The type tag at runtime differs from the definition-table type tag. The definition type drives parsing; the runtime type reflects what was actually stored:

Per-Type Slot Usage (confirmed from decompilation)

Types 1, 2, 3, 4, 5, 7, 8 -- scalar types using only bytes +0 through +15:

Type 1 (int32):      +0 = 0x01, +8 = int32 value (4 bytes)
Type 2 (float):      +0 = 0x02, +8 = float value (4 bytes, upper 4 undefined)
Type 3 (double):     +0 = 0x03, +8 = double value (8 bytes)
Type 4 (boolean):    +0 = 0x04  (no payload -- presence = true)
Type 5 (string):     +0 = 0x05, +8 = char* pointer (8 bytes, NOT owned)
Type 7 (budget):     +0 = 0x07, +8 = int32 primary, +12 = int32 secondary
Type 8 (int-range):  +0 = 0x08, +8 = int32 low, +12 = int32 high

Types 6 and 9 -- doubly-linked list types using the full 72 bytes:

+0:   byte   type tag (6 or 9)
+8:   ptr    next pointer (initially 0)
+16:  ptr    → slot+24 (sentinel backward link)
+24:  ptr    → slot+8 (sentinel forward link)
+32:  int64  (unused, set to 0)
+40:  ptr    tail of list
+48:  ptr    head of list
+56:  int32  element count (starts at 2 for sentinel nodes)
+64:  ptr    arena allocator (for node allocation)

Each list node is 24 bytes, allocated from the arena at +64:

Type 6 node: [next(8), prev(8), string_ptr(8)]
Type 9 node: [next(8), prev(8), opcode_id(4) | int_value(4)]

Type 10 -- dynamic growable array:

+0:   byte   = 0x0A
+8:   ptr    arena allocator
+16:  ptr    array base (int32 elements, grown via sub_6EFD20)
+24:  int32  element count (initialized to 0xFFFFFFFF = -1; first insert sets to 0)

The array grows by calling sub_6EFD20(slot+8, count+2) before each insertion, which reallocates if capacity is exceeded. Elements are 4-byte int32 values stored contiguously starting at the base pointer.

Knob Type System

The definition-table type tag (at descriptor offset +16) determines how ParseKnobValue interprets the value string. There are 10 logical knob types with 1,294 total registrations:

The INT type (616 knobs, 47.6%) dominates. These control thresholds, limits, and numeric heuristic parameters across the entire compiler. BDGT (budget) knobs (88) are semantically similar to INT but carry a secondary field used for budget-tracking in cost models. The 100 DBL knobs control floating-point heuristic weights (scheduling priorities, cost ratios, etc.).

Definition-Type to Runtime-Type Mapping

The definition-table type tag drives parsing; ParseKnobValue writes a different runtime type tag into the 72-byte slot. The mapping is not 1:1 -- several definition types collapse into the same runtime type, and compound types undergo a pre-initialization phase before the main parse:

Types 11 and 12 are aliases: type 11 shares the exact handler with type 8 (both produce runtime type 5), and type 12 shares parsing logic with type 5 but its pre-switch initializes the allocator from the knob state object instead of inline.

ParseKnobValue Dispatch Algorithm

ParseKnobValue (sub_79F540, source lines 435--551 of generic_knobs_impl.h) implements a two-phase dispatch. The first switch pre-initializes compound types; the second switch parses the value string.

Phase 1 -- Pre-initialization (compound types only):

// v15 = definition type tag at (knob_descriptor + 16)
// v14 = runtime slot at (knob_state[9] + 72 * index)
switch (v15) {
case 9:   // OKT_WHEN -> runtime type 6
    KnobValueReset(v14);
    v14[0] = 6;
    // Initialize doubly-linked list with two sentinel nodes:
    //   +8  = 0 (next), +16 -> +24, +24 -> +8 (circular sentinels)
    //   +40 = tail, +48 = head, +56 = count (starts at 2)
    //   +64 = allocator from knob_state[1]
    break;

case 10:  // OKT_OPCODE_STR_LIST -> runtime type 9
    KnobValueReset(v14);
    v14[0] = 9;
    // Same linked-list initialization as case 9
    break;

case 12:  // OKT_ILIST variant -> runtime type 10
    KnobValueReset(v14);
    v14[0] = 10;
    *(ptr*)(v14 + 16) = NULL;           // growable array base
    *(ptr*)(v14 + 8)  = allocator;      // from knob_state[1]
    *(int32*)(v14 + 24) = 0xFFFFFFFF;   // sentinel count (-1)
    break;
}

Phase 2 -- Value parsing (all types):

Type 1 (OKT_NONE, boolean): No value string needed. Stores runtime type 4 (boolean true). Presence alone indicates the knob is set.

Type 2 (OKT_INT, integer): Calls sub_6F71D0(value, NULL) -- a strtol wrapper with base 0, which auto-detects decimal, hex (0x prefix), and octal (0 prefix). Stores runtime type 1, value at slot+8 as int32.

Type 3 (OKT_BDGT, budget): Same integer parsing as type 2. Stores runtime type 7 with the primary value at slot+8 and the secondary (budget counter) at slot+12 zeroed. Cost models decrement the secondary field as optimization budget is consumed.

Type 4 (OKT_IRNG, integer range): Parses "low..high" format with these edge cases:

"100..200"    -> low=100,  high=200        Standard range
"100.."       -> low=100,  high=0x7FFFFFFF  Open upper bound
"..200"       -> low=0x80000000, high=200   Open lower bound
".."          -> low=0x80000000, high=0x7FFFFFFF  Full range
"42"          -> low=42, high=42            Degenerate (single value)
""            -> error "Empty integer range value"

The .. separator is detected by checking *endptr == '.' && endptr[1] == '.'. Default bounds are INT_MIN (0x80000000) and INT_MAX (0x7FFFFFFF). Stores runtime type 8 with low at slot+8, high at slot+12.

Type 5 (OKT_ILIST, integer list): Parses comma-separated integers. Validation requires each element to start with a digit or -. Uses a growable array (runtime type 10) at slot+16, grown via sub_6EFD20(slot+8, count+2) before each insertion. Elements are 4-byte int32 values stored contiguously. Example: "1,2,3,4" produces a 4-element array.

Type 6 (OKT_FLOAT, float): Calls sscanf(value, "%f", &result). Stores runtime type 2, value at slot+8 as a 4-byte IEEE 754 single. Returns error "Invalid floating point value" if sscanf does not return 1.

Type 7 (OKT_DBL, double): Calls sscanf(value, "%lf", &result). Stores runtime type 3, value at slot+8 as an 8-byte IEEE 754 double. Returns error "Invalid double value" if sscanf does not return 1.

Type 8/11 (OKT_STR, string): Both handled identically. Stores runtime type 5 with a direct pointer copy: *(char**)(slot+8) = value. The string is NOT duplicated -- the pointer references the original buffer, so the caller must ensure the string's lifetime exceeds the knob's.

Type 9 (OKT_WHEN, when-condition): Pre-switch already initialized the linked list (runtime type 6). Allocates a 24-byte node via the allocator's vtable (allocator_vtable[3](allocator, 24)). Node layout: [next_ptr(8), prev_ptr(8), string_ptr(8)]. The condition string pointer is stored at node+16. Nodes are inserted at the tail of the doubly-linked list. Error if value is NULL; empty string is permitted.

Type 10 (OKT_OPCODE_STR_LIST, value-pair list): Pre-switch already initialized the linked list (runtime type 9). Parsing loop:

1. Call vtable+40 to split the next comma-delimited token into opcode name and integer value strings
2. If opcode name is NULL: error "Empty opcode string" (line 520)
3. If integer value is NULL: error "Empty integer value" (line 522)
4. Parse integer via strtol(nptr, 0, 10) (base 10 only, unlike OKT_INT)
5. Resolve opcode name to internal ID via vtable+56 (SASS opcode table lookup)
6. Allocate 24-byte node: [next(8), prev(8), opcode_id(4) | int_value(4)]
7. Insert into linked list; loop until input exhausted

Format: "FADD,3,FMUL,2" produces two nodes: (FADD_id, 3) and (FMUL_id, 2). The opcode resolution uses the same 11,240-byte opcode recognition table as the peephole optimizer.

Type 12 (OKT_ILIST variant, opcode list): Pre-switch already initialized the growable array (runtime type 10). Parsing loop:

1. Call vtable+64 to extract the next comma-delimited opcode name
2. Resolve to internal ID via vtable+56
3. Grow array via sub_6EFD20(slot+8, count+2)
4. Store opcode ID as int32 in the array

Format: "FADD,FMUL,IADD3" -- opcode names only, no integers. Each is resolved to its internal opcode ID.

Default: Error "Invalid knob type" (line 551).

Parse Error Messages

ParseKnobValue (sub_79F540 / sub_6F7360) produces these diagnostic strings on parse failure:

All errors carry source attribution: generic_knobs_impl.h with a line number and function name ("GetKnobIndex", "ParseKnobValue", "ReadKnobsFile"). Error constructors: sub_79CDB0 (simple format string) and sub_79AED0 (format with knob name and value context).

Setting Knobs

Method 1: -knob CLI Flag

ptxas -knob SchedNumBB_Limit=100 -knob DisableCSE=1 input.ptx -o output.cubin

Multiple -knob flags accumulate. Each is parsed by KnobsInit (sub_79D990) during startup. The knob name is looked up via GetKnobIndex, then the value is parsed according to the knob's type.

Method 2: Knobs File

A knobs file is a plain-text file with a required [knobs] section header:

; Comments or metadata can appear before the header.
; ReadKnobsFile ignores everything until [knobs] is found.
[knobs]
SchedNumBB_Limit=100
DisableCSE=1
RegAllocBudget=5000
; WHEN= syntax is also supported inside the file:
WHEN=SH=0xDEADBEEF;SchedNumBB_Limit=200

ReadKnobsFile (sub_79D070, source lines 1060--1090 of generic_knobs_impl.h) processes the file:

1. fopen(path, "r")                               line ~1060
2. fseek(file, 0, SEEK_END)                        line 1075
3. size = ftell(file)                               line 1075
4. fseek(file, 0, SEEK_SET)                         line 1075
5. buffer = allocator->vtable[2](allocator, size+1) (heap alloc)
6. bytes = fread(buffer, 1, size, file)             line 1070
7. buffer[bytes] = '\0'                             (null-terminate)
8. marker = strstr(buffer, "[knobs]")               line 1065
9. if (!marker) error "Knobs header not found"
10. content = marker + 7                            (skip "[knobs]")
11. vtable[4](result, knob_state, content, 0)       (parse callback)
12. fclose(file)                                    line 1085

Key implementation details:

• Entire file read at once. The file is fseek/ftell-measured, then fread into a single buffer of size + 1 bytes. No line-by-line streaming.
• strstr-based header detection. The [knobs] marker is located via strstr, so it can appear anywhere in the file -- not necessarily on the first line. Everything before it (comments, version metadata, other INI sections) is silently ignored.
• Parsing starts at marker+7. Exactly 7 characters ([knobs]) are skipped. The parse callback is ParseKnobsString (sub_79B530), which processes newline-delimited key=value pairs. The ~ separator and WHEN= conditional syntax are supported.
• Result/Expected monad. Every I/O operation has a corresponding error path. Errors are accumulated via sub_79A3D0 (ErrorChainAppend) and propagated through a tagged result object. Multiple errors from a single file are chained, not short-circuited.

Error strings with source line numbers:

Method 3: PTX Pragma

Knobs can be set from PTX source via .pragma directives, unless the DisablePragmaKnobs knob is set. The pragma string is copied into a temporary buffer and parsed by ParseKnobsString (sub_79B530), following the same key=value syntax.

Method 4: WHEN= Conditional Overrides

The most powerful mechanism allows setting knobs conditionally, based on shader hash or instruction hash. The override string uses ~ (tilde) as a record separator:

WHEN=SH=0xDEADBEEF;SchedNumBB_Limit=200~WHEN=IH=0x12345;DisableCSE=1

ParseKnobsString (sub_79B530) recognizes these prefixes (case-insensitive):

• WHEN= -- conditional knob application
• SH= -- match by shader hash (decimal, hex with 0x, or range with ..)
• IH= -- match by instruction hash
• K= -- direct knob setting (no condition)
• INJECTSTRING -- special directive terminated by ;; (double semicolon)

The full conditional override system is parsed by ParseKnobOverrides (sub_79C210), which iterates a linked list of override entries at knob_state + 68904. Each entry carries the condition (hash match criterion) and the knob assignment to apply when matched.

Hash matching uses FNV-1a (magic 0x811C9DC5, prime 16777619) for the per-function override table lookup at ctx+120 → +1128. See IsPassDisabledFull (sub_7992A0).

Priority Order

When the same knob is set by multiple mechanisms, the last write wins. KnobsInit (sub_79D990) processes sources in this order:

1. Environment variable overrides (getenv)
2. Knobs file (if specified via -knobs-file or equivalent)
3. -knob CLI flags
4. PTX pragma knobs (applied per-function at compile time)
5. WHEN= conditional overrides (applied per-function when hash matches)

Later sources override earlier ones for the same knob index.

Two Instantiations: DAG and OCG

The knob system is a C++ template instantiated twice with different knob definition tables:

DAG Knobs (sub_6F0820)

The DAG (Directed Acyclic Graph) scheduler knob table contains 99 entries. These control the Mercury SASS pipeline: instruction expansion, WAR hazard handling, scoreboard configuration, and the decode/expand/opex pipeline stages.

DAG knobs referenced in the binary include knob indices 8 and 17 (pipeline options in sub_6F52F0), 16 (WAR generation options in sub_6FBC20), and 743/747 (expansion options in sub_6FFDC0).

OCG Knobs (sub_79B240)

The OCG (Optimizing Code Generator) knob table contains 1,195 entries -- the vast majority of all knobs. These control the optimization passes, register allocation, instruction scheduling, and code generation.

OCG knob indices referenced across the codebase include: 185 (pass-disable string, offset 13320), 294 (epilogue instruction count, used in tepid scheduling), 487 (LoopMakeSingleEntry enablement), 956-957 (shader hint settings at offsets 68832/68904).

Knob State Object

The master knob state object is constructed by KnobInit (sub_7A0C10):

Offset    Size    Field
────────  ──────  ──────────────────────────────
+0        8       vtable pointer (off_21C0738)
+8        8       arena allocator
+16       8       knob definition table pointer
+24       8       knob count
+32       40      (zero-initialized control fields)
+72       var     knob value array (72 * count bytes)
+80       4       max knob index (initially 0xFFFFFFFF)
+88       16      DUMP_KNOBS_TO_FILE path (growable string)

The vtable at off_21C0738 provides virtual methods for knob access:

• vtable+72: IsKnobSet(index) -- check if a knob has a value
• vtable+152: GetKnobIntValue(index) -- retrieve int32 value
• And others for bool, string, double retrieval

Knob Access Helpers

Throughout the codebase, knobs are accessed by index via small helper functions:

Access is O(1) by index -- no hash lookup or name comparison at runtime. The GetKnobIndex name-to-index translation happens only during initialization.

Pass Disable Mechanism

The knobs system provides a string-based pass disable mechanism through knob index 185 (OCG offset 13320). The string contains +-delimited pass names:

-knob DisablePhases=LoopMakeSingleEntry+SinkCodeIntoBlock

Two check functions consult this string:

IsPassDisabled (sub_799250)

Simple version. Reads the disable flag byte at ctx+13320:

• If byte == 0: no pass-disable configured, returns false
• If byte == 5: string pointer at ctx+13328, performs substring match via sub_6E1520 (strcasestr-like)

Called from 16+ sites across the codebase: sub_78B430 (LoopMakeSingleEntry), sub_78DB70 (SinkCodeIntoBlock), sub_8236B0, sub_8D0640, sub_8F45E0, and others.

IsPassDisabledFull (sub_7992A0)

Full version with per-function overrides. First checks a per-function hash table at ctx+120 → +1128 using FNV-1a on the function identifier. If the function has a specific override entry, reads the disable string from there. Otherwise falls back to the global disable string at ctx+72 → +13320.

// FNV-1a hash for per-function lookup
uint32_t hash = 0x811C9DC5;
for (each byte b in function_id)
    hash = 16777619 * (hash ^ b);
uint32_t bucket = hash & (table_size - 1);

The + character is used as a delimiter between alternative phase names in the disable string, allowing "phaseA+phaseB" to match either name.

NamedPhases Parser (sub_798B60)

Parses a comma-separated list of name=value pairs into parallel arrays (max 256 entries). Used by KnobsInitFromEnv (sub_79C9D0) to process environment variable-based knob overrides.

Input:  "knob1=value1,knob2=value2,knob3=value3"
Output: names[256], values[256], full_strings[256]

Knob Categories

The 1,294 knobs cluster into functional categories. Prefix analysis of decoded knob names reveals these major groups:

Notable Individual Knobs

Selected knobs referenced by address in the binary:

Register Allocation Knobs (87 knobs, indices 613--699)

The register allocator is the most heavily parameterized subsystem in ptxas. Its 87 knobs span indices 613 through 699 in the OCG knob table, registered in ctor_005 at addresses 0x4197F0--0x41B2E0. The knobs cluster into seven functional sub-categories. All names decoded from ROT13 strings at 0x21B9730--0x21BA6C0.

A. Spill Cost Model (26 knobs)

The spill guidance engine (sub_96D940, 84 KB) uses these knobs to compute per-candidate spill costs. The model multiplies hardware-specific latency and resource metrics by configurable scale factors, then applies threshold-based activation logic.

The cost model uses three register-count tiers (low/medium/high), each with independent scale factors (664, 665, 661). The tier boundaries are set by the heuristic knobs (663, 660). Latency scales (670, 671) multiply the estimated stall cycles, while resource scales (672, 675, 676) multiply memory bandwidth consumption. The MIO-heavy threshold (673) triggers a separate cost path when the basic block is already saturated with memory operations.

B. Rematerialization (11 knobs)

Rematerialization recomputes values instead of spilling them. The allocator treats remat as a first-class spill alternative with its own budget and candidate ordering.

Knob 650 (RegAllocRematDisableRange) is unique as the only IRNG-type knob in the set, accepting "lo..hi" to disable rematerialization for a range of instruction indices -- a debugging aid for bisecting remat-related miscompiles.

C. Pre-Assignment / MAC (8 knobs)

MAC (Machine-level Allocation with Constraints) pre-assigns physical registers to high-priority operands before the main Fatpoint allocator runs. Entry: sub_94A020 (331 lines).

D. Coalescing (3 knobs)

Register coalescing eliminates unnecessary register-to-register copies by merging live ranges.

E. Performance-Difference Backoff (5 knobs)

Progressive constraint relaxation: on retry iteration N, if the performance difference exceeds a limit, constraints relax between the begin and end iterations.

F. Register Target Selection (13 knobs)

The target selection phase determines how many physical registers to aim for -- the occupancy/performance tradeoff. More registers per thread means fewer warps can execute concurrently.

The four "Slack" knobs (688--691) fine-tune lower register limits for specific architectural features that have minimum register requirements: MMA (matrix multiply), TCGEN (tensor core generation), SPARSIFY (structured sparsity), DECOMPRESS (decompression).

G. General Allocation Control (12 knobs)

H. Miscellaneous (8 knobs)

Scheduling Knobs (89 knobs, indices 229--978)

The instruction scheduler is the second most heavily parameterized subsystem after register allocation. Its 89 knobs span two contiguous blocks (indices 738--811 for the core Sched* set, and 569--574 for the PostSched* set) plus 11 scattered entries for scheduling-adjacent features. All names decoded from ROT13 strings at 0x21B6CB0--0x21BE100, registered in ctor_005 at code addresses 0x411FF0--0x420A00.

The knobs control every aspect of the list scheduler: how latencies are modeled, which functional units are treated as busy, how aggressively cross-block motion is attempted, and how register pressure feedback loops interact with the priority function. Three Blackwell-era SchedResBusy* knobs (QMMA at 964, OMMA at 977, MXQMMA at 978) sit outside the main block because they were appended in a later toolkit version for new MMA unit types.

A. Resource Busy Overrides (28 knobs)

The SchedResBusy* knobs override the hardware-profile resource busy times for individual functional units. Each knob sets the number of cycles the named unit is considered occupied after issuing an instruction to it. When unset, the scheduler uses the value from the latency model's per-SM hardware profile. Setting a SchedResBusy* knob to 0 effectively makes the unit appear always free to the scheduler.

Two knobs accept string values instead of integers: SchedResBusyOp and SchedResBusyMachineOpcode take a string identifying a specific opcode or machine opcode to override, enabling per-instruction busy-time tuning.

The five HMMA variants (790--793) correspond to different tensor core shapes: HMMA16 for 16-wide half-precision, HMMA1688 for the 16x8x8 tile used on Volta/Turing, HMMA16816 for the 16x8x16 tile used on Ampere+, and HMMA32 for 32-wide half-precision operations. IMMA (794) handles integer tensor operations (INT8/INT4).

B. Latency Overrides (12 knobs)

These override the default latency values the scheduler uses for dependency edges. The SchedRead* prefix indicates read-after-write latencies; the SchedTex* and SchedLDS* variants target texture and shared-memory operations specifically.

C. Register Pressure Feedback (8 knobs)

The scheduler's priority function incorporates register pressure awareness through these knobs. They control how aggressively the scheduler tries to reduce live register count: SchedMaxRTarget sets the target register count, while the SchedMaxRLive* knobs define slack bands around that target. SchedReduceIncLimit* throttles how quickly the scheduler increases its pressure-reduction efforts.

D. Cross-Block Scheduling (8 knobs)

Cross-block motion allows the scheduler to move instructions across basic block boundaries for better latency hiding. These knobs control the scope and cost limits of cross-block speculation.

E. Texture Batching (7 knobs)

Texture operations have high latency, so the scheduler groups them into batches to maximize memory-level parallelism. These knobs control batch formation and target selection.

F. Dependency Modeling (6 knobs)

These control how the scheduler builds and refines the dependency graph between instructions.

G. Post-Scheduler (6 knobs)

The post-scheduler runs after register allocation (phase 103) and adjusts the schedule to account for actual register assignments. It primarily inserts stall cycles and adjusts issue delays.

H. Ordering and Preservation (5 knobs)

These control whether the scheduler preserves the original instruction order (from the optimizer or PTX source) versus reordering freely.

I. Scoreboard (4 knobs)

The hardware scoreboard tracks instruction completion. These knobs tune how the scheduler predicts scoreboard occupancy to avoid stalls.

Note: SbXBlock appears in both cross-block (D) and scoreboard (I) categories because it serves both purposes -- it controls whether the scoreboard state propagates across block boundaries, which is a prerequisite for cross-block scheduling correctness.

J. MMA Coupling (3 knobs)

Matrix multiply-accumulate instructions on certain architectures share functional unit resources. These knobs control how the scheduler models coupled execution.

K. Scheduler Model (4 knobs)

These control how the scheduler models the hardware pipeline and instruction movement costs.

L. Budget, Scale, and Control (7 knobs)

General scheduling control knobs covering budgets, loop iteration estimates, the master disable switch, and validation.

Disable Switches (75 knobs)

The disable switches are boolean knobs that turn off specific passes, optimizations, or workarounds. All 75 knobs containing "Disable" were decoded from ROT13 strings at 0x21BDE30--0x21BFA10. Nearly all are OKT_NONE (boolean) type -- setting them with no value or any value disables the corresponding feature. The single exception is RegAllocRematDisableRange, which is OKT_IRNG and accepts a "lo..hi" instruction index range.

The bare Disable knob at 0x21BE860 appears to be a master pass-disable switch. SchedDisableAll is the master scheduler disable. DisablePragmaKnobs prevents PTX .pragma directives from setting knobs -- a meta-level control that protects the knob system itself.

A. Workaround (WAR) Switches (9 knobs)

These disable hardware or compiler bug workarounds. Each War_SW* knob corresponds to an NVIDIA internal bug tracker ID. Disabling a WAR reverts to the unpatched behavior -- useful for bisecting whether a WAR is causing a regression.

B. Memory and Addressing (11 knobs)

These control address computation, memory access conversion, and shared-memory optimizations.

C. Register Allocation and Uniform Registers (9 knobs)

These control uniform register (UR) file usage, live range management, and remat-related disable ranges.

D. Loop Optimization (6 knobs)

E. Code Motion and Scheduling (6 knobs)

F. Vectorization (4 knobs)

G. Predication and Branching (4 knobs)

H. Synchronization and Barriers (2 knobs)

I. Dead Code and Store Elimination (2 knobs)

J. Control Flow Merging (5 knobs)

K. Pass Control (2 knobs)

L. Sanitizer (3 knobs)

These control the address sanitizer instrumentation for different memory spaces. When the sanitizer is active, these knobs can selectively disable checking for one space while keeping the others.

M. Floating Point (2 knobs)

N. Miscellaneous (10 knobs)

Rematerialization Knobs (35 knobs)

Rematerialization knobs control the three dedicated remat pipeline phases (Phase 28: SinkRemat, Phase 69: OriDoRemat) and the cost model that decides whether recomputing a value is cheaper than keeping it live in a register. These are separate from the 12 RegAlloc*Remat* knobs documented above in section B, which control allocator-integrated rematerialization. The distinction matters: allocator-integrated remat fires during register allocation itself (sub_93AC90), while these knobs tune the standalone pre-allocation and post-predication remat passes.

The 35 knobs split into two contiguous blocks in the descriptor table plus one outlier:

• Remat* (27 knobs, indices 702--728): Late rematerialization (Phase 69) and shared cost model
• SinkRemat* (8 knobs, indices 824--831): Early sink+remat (Phase 28)

A. Remat Enable/Disable (5 knobs)

Knob 711 (RematEnable) is the master switch. When zeroed via -knob RematEnable=0, Phase 69 skips its core loop entirely. Knob 710 (RematEarlyEnable) independently controls Phase 54's mode flag write (ctx+1552 = 4). Knob 726 (RematStressTest) is a debug-only boolean that forces every candidate to be rematerialized regardless of profitability -- useful for stress-testing correctness.

B. Remat Cost Model (10 knobs)

These 10 knobs parameterize the remat profitability function (sub_90B790). The cost model computes remat_cost = instruction_cost * factor and compares against register savings. The DBL-typed knobs (8 of 10) are floating-point multipliers that allow fine-grained tuning. The texture-specific knobs (727, 728) implement a throttle: when the ratio of texture instructions exceeds the threshold, the register target is scaled to avoid excessive register use that would harm texture unit throughput.

C. Register Pressure Control (5 knobs)

The register target is the pressure level below which rematerialization becomes profitable. RematRegTargetFactor (723) scales the occupancy-derived target. RematRegTargetTrialLimit (724) caps the binary-search iterations in the target-finding loop. RematMaxRegCount (718) is a hard ceiling -- if current pressure exceeds this value, the remat pass operates in aggressive mode.

D. Instruction and Code Limits (2 knobs)

RematCostInstLimit (707) prevents the cost model from analyzing extremely large remat sequences. RematInflationSlack (715) limits how many extra instructions rematerialization may introduce before the pass backs off.

E. Placement Control (4 knobs)

These knobs control where rematerialized instructions are placed relative to their uses. RematMinDistance (720) ensures remat is not attempted for short live ranges where the original definition is close enough. RematPlacementLookback (721) limits how far back the placement algorithm scans when searching for a profitable insertion point.

F. Remat Budget (1 knob)

BDGT-typed knobs carry a primary value and a secondary counter. The budget is decremented as each remat decision is committed. When exhausted (secondary reaches zero), the pass stops processing further candidates. This provides a deterministic cap on compile-time cost.

G. SinkRemat (Phase 28) Knobs (8 knobs, indices 824--831)

Phase 28's SinkRemat pass (entry: sub_913A30, core: sub_A0F020) sinks instructions closer to their uses and marks remat candidates. Knob 827 (SinkRematEnable) is the master switch. The distance knobs (828, 829) prevent unprofitable micro-sinks. The penalty factors (830, 831) make the cost model more conservative for predicated instructions and for instructions with multiple cross-block uses, where sinking may duplicate code along multiple paths.

Related Knob Outside the Remat Block

This knob sits in the general MOV-weight family (indices 474--476) rather than the Remat block. It tunes how MOV instructions contribute to the scheduling cost model's remat profitability calculation. When the remat candidate is a MOV chain, this weight determines the per-MOV cost used to decide whether rematerialization beats keeping the value live.

DUMP_KNOBS_TO_FILE

The DUMP_KNOBS_TO_FILE environment variable triggers a full dump of all knob values to a file. Checked during KnobInit (sub_7A0C10) via getenv("DUMP_KNOBS_TO_FILE"):

char* dump_path = getenv("DUMP_KNOBS_TO_FILE");
if (dump_path) {
    size_t len = strlen(dump_path);
    // Store into SSO string at knob_state+88..104
}

The path is stored in a small-string-optimized (SSO) buffer at knob_state offsets +88 through +104:

Offset  Size  Field
──────  ────  ─────────────────────────────────────
+88     8     data pointer (or first 8 inline bytes if len <= 15)
+96     8     string length
+104    8     capacity (or remaining inline bytes)

Paths of 15 bytes or fewer are stored inline without heap allocation. Longer paths allocate via the arena allocator at knob_state+8. The dump is produced later during compilation -- KnobInit only stores the path; the actual file write happens after all knobs are resolved.

This is the primary mechanism for discovering which knobs exist and what their current values are. Setting it produces a text file with all 1,294 knob names and their resolved values.

Error Handling

The knob system uses structured error descriptors (96 bytes each) allocated from an arena:

Offset  Size  Field
──────  ────  ─────────────────────────────────────
+0      8     formatted message string pointer
+8      8     message length
+16     8     source file path pointer
+24     8     source file path length
+32     8     line number
+40     8     function name pointer
+48     48    (additional context fields)

Two error constructor functions:

Errors propagate through a tagged result: bit 0 of *(result + 16) is set on error, cleared on success. The GetKnobIndex return protocol:

// Success:
*(byte*)(result + 16) &= ~1;    // clear error bit
*(int32*)(result) = knob_index;  // store index

// Failure:
*(byte*)(result + 16) |= 1;     // set error bit
*(result + 0..15) = error_desc;  // store error descriptor

KnobValue Lifecycle

Construction

KnobValue::Destroy (sub_797790) resets a 72-byte value slot before writing a new value. It switches on the type tag:

Deep Copy

KnobValue::CopyFrom (sub_7978F0) handles deep copy of value slots, switching on type to properly duplicate linked lists and allocated buffers.

KnobInit (sub_7A0C10) constructs a new knob state object by allocating 72 * count bytes for the value array, then deep-copying each slot from a source state if one exists.

Function Map

Reimplementation Notes

To reimplement the knobs system:

1. Define the knob table as a compile-time array of descriptors (name, alias, type). No need for ROT13 -- that is purely obfuscation. Use an enum for knob indices so call sites reference KNOB_SchedNumBB_Limit instead of magic index 294.

2. Parse order matters. Process sources in the documented priority order (env, file, CLI, pragma, WHEN). Last-write-wins semantics.

3. The WHEN= system is the complex part. You need FNV-1a hashing of function identifiers and a per-function override table. The hash table at ctx+120 → +1128 uses open addressing with linear probing.

4. Budget knobs (OKT_BDGT) are just integers with a secondary tracking field. The secondary starts at 0 and is used by cost models to track how much "budget" remains during optimization.

5. Int-range knobs (OKT_IRNG) use .. as the range separator: "100..200" means [100, 200]. Missing bounds default to INT_MIN (0x80000000) / INT_MAX (0x7FFFFFFF).

6. The opcode-string-list type (OKT_OPCODE_STR_LIST) carries pairs of (opcode_name, integer). The opcode name is resolved to an internal opcode ID via the SASS opcode table. Used for per-instruction tuning overrides.

Cross-References

• CLI Options -- public command-line flags, the user-facing layer above knobs
• Optimization Levels -- O-levels set specific knob presets
• DUMPIR & NamedPhases -- DUMPIR knob and phase-level dump control
• Phase Manager -- pass disable mechanism consumes the DisablePhases knob
• Scheduling Algorithm -- consumes Sched* knobs
• Allocator Architecture -- consumes RegAlloc* knobs
• Mercury Encoder -- consumes Mercury* knobs and DAG knob table