Bitcode Reader/Writer

CICC v13.0 contains the complete LLVM 20.0.0 bitcode serialization infrastructure -- reader, writer, metadata loader, module summary IO, and the full intrinsic upgrader -- spread across two address ranges. The 0x9F0000--0xA2FFFF range hosts a first copy of the bitcode reader/writer core used by the standalone libNVVM pipeline, while the 0x1500000--0x157FFFF range hosts the primary copy used by the two-phase compilation path. Both copies are structurally identical LLVM BitcodeReader.cpp and BitcodeWriter.cpp compiled at different link addresses. The reader is stock upstream LLVM 20.0.0 with no NVIDIA modifications to the deserialization logic itself. The writer, however, contains a single critical NVIDIA change: it stamps "LLVM7.0.1" as the bitcode producer identification string rather than the true "LLVM20.0.0", preserving backward compatibility with the NVVM IR ecosystem.

The bitcode subsystem sits at the boundary between all pipeline stages. The standalone pipeline validates magic bytes on entry, the module linker reads bitcode from separate compilation objects, the two-phase orchestrator serializes per-function bitcode blobs between Phase I and Phase II, and the NVVM container wraps bitcode payloads in a proprietary envelope. Every bitcode load also runs the intrinsic upgrader -- a 700+ KB AutoUpgrade subsystem that includes roughly 240 KB of effectively-dead x86 intrinsic renaming tables.

Key Facts

Bitcode Format Basics

LLVM bitcode uses two magic signatures. The pipeline validates both at module load time:

If neither signature matches, the pipeline sets *error_code = 9 ("invalid bitcode") and aborts. The wrapper format is more common in practice -- nvcc generates wrapper-format .bc files that embed the raw stream at an offset specified in the wrapper header. The wrapper header is 20 bytes:

struct BitcodeWrapperHeader {
    uint32_t magic;       // 0x42, 0x43, 0xC0, 0xDE
    uint32_t version;     // wrapper version (0)
    uint32_t offset;      // byte offset to raw bitcode within file
    uint32_t size;        // size of raw bitcode in bytes
    uint32_t cpu_type;    // target CPU type (0 for NVPTX)
};

After magic validation, the bitstream enters the block-structured reader. LLVM bitcode is organized into nested blocks, each identified by a block ID. The reader uses abbreviation tables (defined in BLOCKINFO blocks) to decode records within each block efficiently using variable-bit-rate (VBR) encoding.

An epoch check runs after magic validation: "Incompatible epoch: Bitcode '<X>' vs current: '<Y>'". This ensures the bitcode was produced by a compatible LLVM generation.

Bitcode Reader

Module Parser (sub_1505110, 60 KB)

The top-level entry reads MODULE_BLOCK records from the bitcode stream. It processes:

• Global variable declarations and definitions
• Function declarations (bodies are deferred for lazy materialization)
• Calling conventions and comdat groups
• Module-level metadata, type tables, and value symbol tables
• Data layout and target triple strings

Error strings: "Invalid calling convention ID", "Invalid function comdat ID", "Invalid global variable comdat ID", "Invalid type for value".

parseFunctionBody (sub_151B070 / sub_9F2A40)

The function body parser is the largest single reader function. The standalone copy sub_9F2A40 is 185 KB (5,706 decompiled lines) with 174 error string references. The primary copy sub_151B070 is 123 KB. Both decode the same FUNCTION_BLOCK records:

• 57 FUNC_CODE instruction record types (switch cases 1--65), covering every LLVM IR opcode: INST_BINOP, INST_CAST, INST_GEP, INST_SELECT, INST_CMP, INST_RET, INST_BR, INST_SWITCH, INST_INVOKE, INST_CALL (opcode 85), INST_UNREACHABLE, INST_PHI, INST_ALLOCA, INST_LOAD, INST_STORE, INST_ATOMICRMW, INST_CMPXCHG, INST_FENCE, INST_EXTRACTVAL, INST_INSERTVAL, INST_LANDINGPAD, INST_RESUME, INST_CLEANUPPAD, INST_CATCHPAD, INST_CATCHSWITCH, INST_CALLBR, INST_FREEZE, and others.
• 4 nested sub-blocks: constants (0xB), metadata (0xE), use-list order (0x10), operand bundles (0x12).
• 53 unique error strings including: "Alignment value is too large", "Invalid record", "Invalid record: Unsupported version of DISubrange", "METADATA_NAME not followed by METADATA_NAMED_NODE".

For each INST_CALL record (opcode 85), the reader calls into the AutoUpgrade machinery to rename deprecated intrinsics. This is the hook that triggers the 700+ KB x86 upgrader on every call instruction -- even though the upgrader's x86 branches are dead code for NVPTX targets.

Pseudocode for the top-level body parse loop:

Error parseFunctionBody(Function *F) {
    SmallVector<uint64_t, 64> Record;
    while (true) {
        BitstreamEntry Entry = Stream.advance();
        switch (Entry.Kind) {
        case BitstreamEntry::Error:
            return error("Malformed block");
        case BitstreamEntry::EndBlock:
            return resolveForwardRefs();
        case BitstreamEntry::SubBlock:
            switch (Entry.ID) {
            case CONSTANTS_BLOCK_ID:  // 0xB
                parseConstants(); break;
            case METADATA_BLOCK_ID:   // 0xE
                parseMetadataAttachment(); break;
            case USELIST_BLOCK_ID:    // 0x10
                parseUseListBlock(); break;
            case OPERAND_BUNDLE_TAGS_BLOCK_ID: // 0x12
                parseOperandBundleTags(); break;
            }
            break;
        case BitstreamEntry::Record:
            unsigned Code = Stream.readRecord(Entry.ID, Record);
            switch (Code) {
            case FUNC_CODE_INST_BINOP: /* ... */ break;
            case FUNC_CODE_INST_CAST:  /* ... */ break;
            // ... 55 more cases ...
            case FUNC_CODE_INST_CALL:
                // Parse callee, args, calling convention
                // If callee is intrinsic:
                //   UpgradeIntrinsicFunction(callee, &newCallee);
                //   if (newCallee) UpgradeIntrinsicCall(CI, newCallee);
                break;
            }
        }
    }
}

Lazy Materialization (sub_1503DC0, 13 KB)

Function bodies are not parsed eagerly. The module parser records each function's byte offset in the bitcode stream, and materializeFunctions seeks to that position on demand. Error strings: "Could not find function in stream", "Expect function block", "Expect SubBlock", "Trying to materialize functions before seeing function blocks". The two-phase compilation exploits this by materializing individual functions for per-function Phase II optimization.

Bitstream Infrastructure

Bitcode Writer

writeModule (sub_1538EC0, 58 KB)

The top-level writer serializes an entire Module to a bitcode stream. It orchestrates sub-writers in a fixed order:

1. Enumerate all values via ValueEnumerator (sub_15467B0, 23 KB)
2. Write identification block (with producer string -- see next section)
3. Write MODULE_BLOCK header
4. Write type table (sub_1530240, 12 KB)
5. Write attribute groups (sub_152F610, 8 KB)
6. Write global variables
7. Write function declarations
8. For each defined function: writeFunction (sub_1536CD0, 40 KB)
9. Write metadata (sub_1531F90, 27 KB) + metadata records (sub_15334D0, 8 KB)
10. Write value symbol table (sub_1533CF0, 16 KB)
11. Write named metadata / comdat records (sub_15311A0, 14 KB)
12. If ThinLTO: write module summary (sub_1535340, 26 KB)

writeFunction (sub_1536CD0, 40 KB)

Writes one FUNCTION_BLOCK containing all instructions, each encoded via writeInstruction (sub_1528720, 27 KB). Instructions are encoded as (opcode, operand_ids...) records where operand IDs are relative to the value table. The writer uses abbreviations for compact encoding of common instruction patterns.

Value Enumeration

Before writing, the ValueEnumerator assigns a dense numeric ID to every value in the module. This is the reverse of what the reader does (mapping IDs back to Values).

Writer Function Map

Producer String Hack

This is the single most important NVIDIA deviation in the bitcode subsystem. Two global constructors cooperate to set the producer identification string:

ctor_036 at 0x48CC90 (544 bytes): Reads LLVM_OVERRIDE_PRODUCER from the environment. If unset, falls back to the string "20.0.0" (the true LLVM version). Stores the result in the global qword_4F837E0. Also registers disable-bitcode-version-upgrade (cl::opt<bool>).

ctor_154 at 0x4CE640 (215 bytes): Also reads LLVM_OVERRIDE_PRODUCER. Falls back to "7.0.1". Stores into a separate global.

When writeModule (sub_1538EC0) writes the IDENTIFICATION_BLOCK, it emits the string "LLVM7.0.1" as the producer. This is assembled from the prefix "LLVM" plus the version string "7.0.1" loaded from the ctor_154 global.

The consequence is that any tool reading CICC's output bitcode (including older libNVVM, nvdisasm, or third-party NVVM IR consumers) sees producer "LLVM7.0.1" and interprets the bitcode as LLVM 7.x-era IR. Internally, the IR is LLVM 20.0.0 -- all modern instruction opcodes, metadata formats, and type encodings are present. The producer string is purely a compatibility marker that tells downstream tools which NVVM IR version spec to apply, not the actual LLVM version.

Why 7.0.1 specifically: NVVM IR 2.0 was defined against LLVM 7.0.1. The NVVM toolchain ecosystem (libNVVM, nvcc's device compilation pipeline) standardized on this version string as the "NVVM IR format identifier." Upgrading the producer string would require coordinated changes across the entire CUDA toolkit and all consumers.

// Pseudocode for producer string initialization
static const char *producer_version;

void ctor_036() {  // at 0x48CC90
    const char *env = getenv("LLVM_OVERRIDE_PRODUCER");
    if (!env) env = "20.0.0";  // true LLVM version
    global_4F837E0 = env;
    // Also registers: -disable-bitcode-version-upgrade (cl::opt<bool>)
}

void ctor_154() {  // at 0x4CE640
    const char *env = getenv("LLVM_OVERRIDE_PRODUCER");
    if (!env) env = "7.0.1";   // NVVM IR compat marker
    producer_version = env;
}

// In writeModule (sub_1538EC0):
void writeIdentificationBlock(BitstreamWriter &Stream) {
    Stream.EnterSubblock(IDENTIFICATION_BLOCK_ID);
    // Writes: "LLVM" + producer_version → "LLVM7.0.1"
    Stream.EmitRecord(IDENTIFICATION_CODE_STRING, "LLVM");
    Stream.EmitRecord(IDENTIFICATION_CODE_EPOCH, CurrentEpoch);
    Stream.ExitBlock();
}

Reimplementation note: A reimplementation must write "LLVM7.0.1" as the producer for compatibility with the existing NVVM ecosystem. Setting LLVM_OVERRIDE_PRODUCER to a different value will change the embedded string. The disable-bitcode-version-upgrade flag controls whether the reader's AutoUpgrade logic activates for version-mismatched bitcode.

X86 AutoUpgrade -- Why to Skip It

The intrinsic upgrader is the single largest code mass in the entire cicc binary. Two functions dominate:

Total intrinsic upgrader code: approximately 1.4 MB across all copies and helpers.

The x86 portion (roughly 1.0 MB) handles SSE/SSE2/SSE4.1/SSE4.2/SSSE3, AVX2, AVX-512 (mask operations, conversions, FMA variants), and ARM NEON patterns (^arm\.neon\.vld, ^arm\.neon\.vst). These branches are functionally dead for NVPTX -- no CUDA program will ever contain an @llvm.x86.sse2.padds.b intrinsic. However, the code is NOT unreachable in the CFG sense: the reader calls UpgradeIntrinsicFunction on every intrinsic name, the function does a string-prefix match, and falls through the x86/ARM branches without matching. The x86 code paths simply never activate.

Reimplementation guidance: You can safely exclude the x86 and ARM AutoUpgrade tables (sub_A8A170, the x86 portions of sub_A939D0, and the ARM patterns in sub_15644B0). The NVVM-relevant upgraders must be preserved:

Stripping the x86 upgrader saves approximately 1.0 MB of binary size and significant reverse-engineering effort, with zero functional impact on GPU compilation.

Metadata Reader

MetadataLoader::parseOneMetadata (sub_A09F80, 121 KB)

The metadata reader handles 42 distinct metadata record types in a single switch statement. Each case constructs one metadata node:

• DI metadata nodes: DISubprogram, DIFile, DICompileUnit, DIVariable, DILocation, DIType, DIExpression, DISubrange, DIEnumerator, DIGlobalVariableExpression, DIModule, DINamespace, DITemplateTypeParameter, DITemplateValueParameter, DICompositeType, DIDerivedType, DIBasicType, DILexicalBlock, DILexicalBlockFile, DILabel, DIImportedEntity, DIMacro, DIMacroFile, DICommonBlock, DIGenericSubrange, DIStringType, DIArgList
• LLVM metadata nodes: MDTuple, MDString, named metadata
• NVVM annotations: nvvm.annotations (parsed as named metadata carrying per-kernel attributes)

The function is called from parseMetadataBlock (sub_1518180, 29 KB), which reads the block structure, and parseFunctionMetadata (sub_1520420, 32 KB), which processes function-level metadata attachments.

Value materialization (sub_A10370, 33 KB) handles forward references in metadata. When a metadata node references a value that hasn't been parsed yet, the materializer resolves it once the value becomes available.

Module Summary Serialization

Two pairs of functions handle ThinLTO module summary IO:

Summary Writer (sub_1535340, 26 KB)

Writes the MODULE_STRTAB_BLOCK and GLOBALVAL_SUMMARY_BLOCK into the bitcode stream. For each function/alias/global:

• Encodes the GUID hash (64-bit FNV-1a on the mangled name)
• Writes call graph edges with hotness annotations
• Writes reference edges (global value references)
• For ThinLTO: writes module path strings, type test GUIDs

Error string: "Unexpected anonymous function when writing summary".

The NVIDIA-extended summary fields (import priority, complexity budget, kernel bit, CUDA attributes) are written by the NVModuleSummary builder into the standard summary records via additional flag bits and extended record fields.

Summary Reader (sub_150B5F0, 63 KB)

Reads the summary index from bitcode. Handles GUID hashes, function/alias summaries, module paths. Error strings: "Alias expects aliasee summary", "Invalid hash length", "Invalid Summary Block: version expected", "Malformed block".

Summary Writer (standalone copy) (sub_A2D2B0, 48 KB)

A second copy of the summary/metadata writer exists at 0xA2D2B0 in the standalone pipeline's address range.

NVVM IR Version Validation

CICC gates bitcode acceptance on two version checks:

Module-Level Version Gate (sub_157E370, 7 KB)

After parsing the module, this function reads the "nvvmir.version" named metadata node. The metadata contains a pair of integers (major, minor). The check enforces:

major == 3  AND  minor <= 2

If the check fails, the function calls sub_16BD130 which emits "Broken module found, compilation aborted!" and terminates compilation. If the module passes the version check, it proceeds to sub_166CBC0 (verifyModule [MEDIUM confidence] -- identification based on call position after bitcode parsing and before optimization, consistent with LLVM's standard verify-after-parse pattern, but no diagnostic string directly confirms the function name) for structural IR verification, then sub_15ACB40 for post-verification processing.

A second instance at sub_12BFF60 (9 KB) in the standalone pipeline performs the same check with additional llvm.dbg.cu debug info presence validation.

Environment Override (NVVM_IR_VER_CHK)

The NVVM_IR_VER_CHK environment variable controls whether version validation runs at all:

The check is: if (!env || strtol(env, NULL, 10) != 0) then enforce version. This means any non-zero numeric string also enables the check. Only the literal string "0" disables it.

Two verifier instances exist:

• sub_12BFF60 at 0x12BFF60 (standalone pipeline)
• sub_2259720 at 0x2259720 (second instance, possibly duplicate link unit)

Configuration

Environment Variables

cl::opt Flags

Differences from Upstream LLVM

Function Map

Reader (primary, 0x1500000--0x1522000)

Reader (standalone copy, 0x9F0000--0xA20000)

Writer (0x1525000--0x1549000)

Intrinsic Upgrader (0xA80000--0xABFFFF + 0x1560000--0x1580000)

NVVM Version / Producer

Value Enumeration (0x1540000--0x1549000)

Cross-References

• NVVM Container -- wraps bitcode in the proprietary transport format
• LTO & Module Optimization -- consumes bitcode from separate compilation objects
• NVModuleSummary Builder -- extends module summary with CUDA-specific fields; serialized by sub_1535340
• Two-Phase Compilation -- serializes/deserializes per-function bitcode between phases
• Pipeline Entry -- magic byte validation on bitcode input
• Environment Variables -- LLVM_OVERRIDE_PRODUCER, NVVM_IR_VER_CHK
• Binary Layout -- address range context for reader/writer clusters