Optimization Levels

cicc v13.0 supports four standard optimization levels (O0 through O3) and three fast-compile tiers (Ofcmin, Ofcmid, Ofcmax). These are mutually exclusive with the custom --passes= interface. The pipeline name is selected in the new-PM driver sub_226C400 and assembled by sub_12E54A0. The full optimization pipeline builder is sub_12DE330, with tier-specific insertion handled by sub_12DE8F0.

Pipeline Name Selection

The new-PM driver at sub_226C400 selects a pipeline name string based on boolean flags in the config struct:

Selection logic in sub_226C400 (lines 828--874):

if (O1_flag)       -> "nvopt<O1>"
else if (O2_flag)  -> "nvopt<O2>"
else if (O3_flag)  -> "nvopt<O3>"
else if (fc_len == 3) {
  if (fc == "max") -> "nvopt<Ofcmax>"
  if (fc == "mid") -> "nvopt<Ofcmid>"
  if (fc == "min") -> "nvopt<Ofcmin>"
}
else               -> "nvopt<O0>"

Combining -O# with --passes= is an error:

"Cannot specify -O#/-Ofast-compile=<min,mid,max> and --passes=/--foo-pass, use -passes='default<O#>,other-pass'"

The pipeline name is passed to sub_2277440 (new-PM text parser), which constructs the actual PassManager. The nvopt prefix is registered as a pipeline element in sub_225D540 (new PM) and sub_12C35D0 (legacy PM), with vtables at 0x4A08350 / 0x49E6A58.

Fast-Compile Level Encoding

The fast-compile level is stored as an integer at offset 1640 (or 1648 in the clone) of the compilation context:

Any other value produces: "libnvvm : error: -Ofast-compile called with unsupported level".

When level=1, the flag is forwarded to the optimizer phase as a pass argument and then the level is reset to 0 at offset 1640 (so it becomes normal O-level optimization). When level=2 (max), the optimizer arg string -Ofast-compile=max is appended. When level=3 (mid), -Ofast-compile=mid is appended. When level=4 (min), -Ofast-compile=min is appended.

Tier Summary

Pipeline Architecture: Tier 0 + Tiers 1/2/3

O1/O2/O3 share a common pipeline construction path. The key insight is that optimization happens in layers:

1. Tier 0 (sub_12DE330): The full base pipeline of ~40 passes. Fires for ALL of O1, O2, and O3 when opts[4224] (optimization-enabled) is set.
2. Tier 1 (sub_12DE8F0(PM, 1, opts)): Additional passes gated by opts[3528]. Fires for O1, O2, and O3.
3. Tier 2 (sub_12DE8F0(PM, 2, opts)): Additional passes gated by opts[3568]. Fires for O2 and O3 only.
4. Tier 3 (sub_12DE8F0(PM, 3, opts)): Additional passes gated by opts[3608]. Fires for O3 only.

The tier control fields in the NVVMPassOptions struct at 4512 bytes:

The assembler loop in sub_12E54A0 (lines 481--553) iterates over the plugin/external pass list at opts[4488]. Each entry has a phase_id; when the phase_id exceeds a tier's threshold, that tier fires:

for each entry in opts[4488..4496]:
  phase_id = entry[8..12]
  if (opts[4224] && phase_id > opts[4228]):
    sub_12DE330(PM, opts)   // Tier 0
    opts[4224] = 0          // one-shot
  if (opts[3528] && phase_id > opts[3532]):
    sub_12DE8F0(PM, 1, opts) // Tier 1
    opts[3528] = 0
  if (opts[3568] && phase_id > opts[3572]):
    sub_12DE8F0(PM, 2, opts) // Tier 2
    opts[3568] = 0
  if (opts[3608] && phase_id > opts[3612]):
    sub_12DE8F0(PM, 3, opts) // Tier 3
    opts[3608] = 0
  AddPass(PM, entry->createPass())

After the loop, any remaining unfired tiers fire unconditionally.

Tier 0: Full Base Pipeline (sub_12DE330)

sub_12DE330 at 0x12DE330 is called for all O1/O2/O3 compilations. It constructs the ~40-pass base pipeline:

After sub_12DE330 returns, opts[4224] is cleared (one-shot).

Tiers 1/2/3: Phase-Specific Sub-Pipeline (sub_12DE8F0)

sub_12DE8F0 at 0x12DE8F0 is a single function called with tier in {1, 2, 3}. The tier value is stored into qword_4FBB410 (phase tracker). When tier==3 and qword_4FBB370 byte4 is 0, the feature flags are set to 6 (enabling advanced barrier opt + memory space opt gates).

The following table lists every pass in sub_12DE8F0 with its tier-dependent guard condition. A pass runs only when ALL conditions in its Guard column are satisfied.

O1 vs O2 vs O3: Complete Diff

The three O-levels differ through exactly five mechanisms. Every pass that is NOT listed here runs identically at all three levels.

1. Tier guard: tier!=1 (O2/O3 only)

These passes are present in sub_12DE8F0 but skip when tier==1 (O1):

At O1, the base pipeline (Tier 0) already includes one instance of LoopVectorize with sub_19B73C0(2,-1,-1,-1,-1,-1,-1) -- width 2, all thresholds at -1 (unlimited). The tier!=1 guard blocks a SECOND, more aggressive vectorization pass with SM-dependent parameters.

2. Tier guard: tier==3 (O3 only)

These passes run exclusively at O3:

sub_1C46000 (NVVMLateOpt) is the most significant O3-exclusive pass. It runs only when !opts[360] (not disabled) and only at tier==3. This is a dedicated NVIDIA optimization pass that performs additional transformations after the main pipeline is complete.

3. Feature flag qword_4FBB370 escalation

When tier==3 and qword_4FBB370 byte4 is 0, the function sets qword_4FBB370 = 6 (binary 110). This enables two feature gates:

• Advanced barrier optimization (bit 1)
• Memory space optimization extensions (bit 2)

These gates affect behavior in downstream passes that read qword_4FBB370, such as sub_12EC4F0 (the machine pass pipeline executor).

4. LoopVectorize/SLP parameter differences

sub_19B73C0 is called with different parameters depending on context:

The 7 parameters to sub_19B73C0 control:

• arg1: Vector width factor (2 at Tier 0, tier at higher tiers)
• arg2..arg7: Thresholds for cost model, trip count, and SLP width. Value -1 means unlimited/auto; value 0 means conservative/disabled.

At O2, sub_19B73C0(2, ...) provides moderate vectorization. At O3, sub_19B73C0(3, ...) increases the vector width factor, enabling wider SIMD exploration. The SM-architecture-dependent parameters are resolved at runtime based on the target GPU.

5. CGSCC iteration count

sub_1A62BF0 is the CGSCC (Call Graph SCC) pass manager factory. The first argument is the pipeline extension point / iteration count:

O1/O2/O3 all use 1-iteration CGSCC in their shared Tier 0 pipeline. The iteration count differences appear in the fast-compile and language-specific paths, not between O-levels.

Complete O-Level Comparison Matrix

O0 Pipeline (Minimal)

When no O-level flag is set and no fast-compile level is active, the assembler falls through to LABEL_159 which calls:

sub_1C8A4D0(0)   -- NVVMFinalCleanup or similar minimal pass

Then the common tail at LABEL_84 adds:

1. MemorySpaceOpt (conditional, skipped at O0 since opts[3488] is typically unset)
2. sub_1CEBD10() -- NVVMFinal / cleanup
3. sub_1654860(1) -- VerifierPass
4. sub_12DFE00() -- Codegen pass setup

The O0 pipeline does NOT call sub_12DE330 or sub_12DE8F0. It runs only the infrastructure passes (TargetLibraryInfo, TargetTransformInfo, BasicAA, AssumptionCacheTracker, ProfileSummaryInfo) plus minimal canonicalization.

Ofcmax Pipeline (Fastest Compile)

Ofcmax bypasses the full pipeline entirely. It forces two optimizer flags:

• -lsa-opt=0 (disables LSA optimization)
• -memory-space-opt=0 (disables MemorySpaceOpt pass)

This forcing happens in BOTH sub_9624D0 (line 1358--1361) and sub_12CC750 (line 2025--2079). The condition is:

if (!compare(lsa_opt_flag, "0") || fc_level == 2):
  append("-lsa-opt=0")
  append("-memory-space-opt=0")

Additionally, when fc_level == 2 AND lsa_opt is NOT already "0", the libnvvm path also injects -lsa-opt=0, mem2reg, -memory-space-opt=0.

The minimal pass sequence:

Ofcmid Pipeline (Medium)

Ofcmid runs ~25--30 passes without forcing LSA or MemorySpaceOpt off. The pass sequence from sub_12E54A0 (lines 814--861):

Key differences from the O1+ pipeline: Ofcmid uses 5-iteration CGSCC (vs 1 at O1+), includes NVVMPeephole/Peephole2 early, uses conservative LoopVectorize parameters (3,-1,-1,0,0,-1,0) with some thresholds zeroed, and skips NVVMDivergenceLowering, SpeculativeExecution, NVVMBranchDist, NVVMRematerialization, and the entire tier sub-pipeline.

Ofcmin Pipeline (Closest to Full Optimization)

Ofcmin takes the same path as Ofcmid through LABEL_297 in sub_12E54A0 but with the v238 flag set differently, enabling more aggressive settings. The pipeline is essentially the Ofcmid sequence with:

• More aggressive loop optimizer thresholds
• Additional CGSCC framework passes
• Closer parameter alignment to the O2 full pipeline

Ofcmin does NOT force -lsa-opt=0 or -memory-space-opt=0. Like Ofcmid, it still skips the tier 1/2/3 sub-pipeline entirely, keeping compile time lower than O1.

Post-Optimization Common Tail

Regardless of pipeline tier, sub_12E54A0 always appends at LABEL_84 (lines 640--653):

sub_12DFE00 (codegen dispatch) reads the optimization level from opts[200] to determine codegen aggressiveness. When opts[200] > 1, full dependency tracking is enabled across all codegen passes.

Always-Added Analysis Passes

Before any optimization, the pipeline assembler inserts (lines 396--420):

These five passes run at ALL optimization levels including O0.

NVVMPassOptions Offset-to-Guard Map

The passes gated by NVVMPassOptions boolean flags (opts struct at 4512 bytes). Slot defaults from sub_12D6300:

Codegen Optimization Level Propagation

The -optO and -llcO flags propagate the optimization level to the backend code generator. In sub_12E54A0 (lines 1451--1460):

if (lsa_opt == "0" && some_flag == "1"):
  append("-optO<level>")
  append("-llcO2")

The codegen dispatch sub_12DFE00 reads opts[200] (the integer optimization level):

• opts[200] == 0: Minimal codegen (no dependency tracking)
• opts[200] >= 1: Standard codegen
• opts[200] >= 2: Full dependency tracking enabled (v121 = true)

Cross-References

• NVVMPassOptions System -- complete 222-slot struct layout
• Pipeline Pass Registration -- 526-pass registration table
• Optimizer Architecture -- two-phase model, AddPass mechanism
• CLI Flags -- -O#, -Ofc=, --passes= routing
• Knobs Reference -- all 1496 cl::opt knobs
• Concurrent Compilation -- Phase I/II threading model