ELF/Cubin Output

All addresses in this page apply to ptxas v13.0.88 (CUDA 13.0). Other versions will differ.

After all per-kernel SASS encoding completes, ptxas enters the ELF output phase -- the final stage of the compilation pipeline. This phase transforms the accumulated per-kernel SASS bytes, relocation metadata, constant bank data, shared memory layouts, and debug information into a complete NVIDIA CUBIN file. The CUBIN is a standard ELF container with NVIDIA-proprietary extensions: machine type EM_CUDA (0xBE), non-standard ELF class bytes, CUDA-specific section types, and a rich per-entry metadata system called EIATTR. The output pipeline is a custom implementation with no libelf dependency -- ptxas constructs every byte of the ELF from scratch, including headers, section tables, symbol tables, string tables, relocations, and program headers.

The output phase handles three binary kinds: SASS (raw resolved SASS, legacy default), Mercury (SM 75--99 default), and Capsule Mercury (SM 100+ default, supporting deferred finalization). All three produce a valid CUBIN ELF; the difference is whether the .text sections contain final SASS bytes or Mercury-encoded streams that a downstream finalizer resolves at link or load time.

Pipeline Overview

The ELF output pipeline runs as a single-threaded sequence after all per-kernel OCG passes have completed (multi-kernel compilation may be parallel, but ELF emission is serialized). The flow is orchestrated by sub_612DE0, which reads compilation flags, constructs the ELFW central object, then drives 11 phases to produce the final .cubin or .o file.

sub_446240 (compilation driver -- "real main")
  |  all per-kernel OCG passes complete
  v
sub_612DE0 (cubin entry, 47KB)
  |  reads: deviceDebug, lineInfo, optLevel, IsCompute, IsPIC
  |  establishes setjmp/longjmp error recovery
  |  writes "Cuda compilation tools, release 13.0, V13.0.88"
  |         "Build cuda_13.0.r13.0/compiler.36424714_0"
  |
  |-- Phase 1: ELFW construction
  |     sub_1CB53A0 -- create 672-byte ELFW object, 7 standard sections
  |
  |-- Phase 2: Per-kernel section creation
  |     sub_1CB42D0 x N -- .text.<func>, .rela.text.<func> (one per kernel)
  |     sub_1CB3570 x 44 -- .nv.constant0.<func>, .nv.shared.<func>, etc.
  |
  |-- Phase 3: Call graph analysis
  |     sub_1CBB920 -- recursion detection (DFS)
  |     sub_1CBC090 -- dead function elimination
  |     sub_1CBE1B0 -- .nv.callgraph section builder
  |
  |-- Phase 4: Symbol fixup
  |     sub_1CB2CA0 -- renumber symbols after dead code elimination
  |     sub_1C99BB0 -- remap .symtab_shndx extended indices
  |
  |-- Phase 5: Memory allocation
  |     sub_1CABD60 -- assign addresses: shared, constant, local memory
  |     sub_1CA92F0 -- shared memory interference graph
  |     sub_1CA6890 -- constant bank deduplication
  |
  |-- Phase 6: nvinfo/EIATTR generation
  |     sub_1CC9800 -- build .nv.info.<func> sections (EIATTR attributes)
  |     sub_1CC8950 -- propagate barrier/register counts across call graph
  |
  |-- Phase 7: Symbol table construction
  |     sub_1CB68D0 -- build .symtab, handle SHN_XINDEX overflow
  |
  |-- Phase 8: Section layout
  |     sub_1C9DC60 -- compute file offsets with alignment padding
  |
  |-- Phase 9: Relocation resolution
  |     sub_1CD48C0 -- resolve all R_CUDA_* relocations
  |     sub_1CD5920 -- write .nv.resolvedrela sections
  |
  |-- Phase 10: Capsule Mercury embedding (SM 100+ only)
  |     sub_1C9B110 -- create .nv.merc.* section namespace
  |     sub_1CA2E40 -- clone memory-space sections into merc namespace
  |     sub_1C9C300 -- build 328-byte capsule descriptors per function
  |
  |-- Phase 11: Final assembly & write
  |     sub_1C9F280 -- master ELF emitter (assemble complete CUBIN)
  |     sub_1CD13A0 -- file serializer (write to disk)
  |
  v
OUTPUT: .cubin / .o file

Custom ELF Emitter

ptxas builds the entire ELF output without libelf. The custom implementation spans approximately 20 functions in the 0x1C99--0x1CD6 address range (~300 KB of binary code). At the center is the ELFW ("ELF world") object -- a 672-byte structure that owns all sections, symbols, and string tables for a single compilation unit.

ELFW Object Layout

The ELFW constructor sub_1CB53A0 allocates 672 bytes from the pool allocator sub_424070, creates a dedicated "elfw memory space" pool (4,096-byte initial allocation), writes the ELF header, and initializes 7 mandatory sections:

ELF Header

The ELF header uses standard structure with NVIDIA-specific overrides:

Standard ELF uses ELFCLASS32 = 1 and ELFCLASS64 = 2. CUDA cubins use non-standard values '3' (0x33) and 'A' (0x41), which the CUDA driver recognizes as the cubin signature. Any tool using standard libelf will reject these as invalid, which is one reason ptxas uses a custom emitter.

The e_flags field packs the SM architecture version in the low byte (e.g., 100 for sm_100) along with flags for relocatable vs. executable mode and address size. The mask 0x7FFFBFFF (clears bits 14 and 31) is applied during finalization to strip internal control flags that must not appear in the output cubin.

Section Generation

Each kernel/function produces a set of ELF sections. For a program with N entry functions and M device functions, the CUBIN contains on the order of 4*(N+M) sections minimum. The section creator sub_1CB3570 (44 call sites) handles the generic case; sub_1CB42D0 is specialized for .text.<funcname> code sections.

Per-Kernel Sections

For each kernel entry my_kernel, the output pipeline generates:

Additional per-kernel sections are generated as needed:

Global Sections

Constant Banks

CUDA supports up to 18 numbered constant banks (0--17) plus 6 named banks:

Section Ordering

During finalization, sections are sorted into 8 priority buckets that determine their order in the output ELF:

Section file offsets are assigned by sub_1C9DC60, walking the sorted list with alignment padding:

uint64_t offset = elf_header_size;
for (int i = 0; i < section_count; i++) {
    section_t* sec = sorted_sections[i];
    if (sec->sh_addralign > 1)
        offset = (offset + sec->sh_addralign - 1) & ~(sec->sh_addralign - 1);
    sec->sh_offset = offset;
    offset += sec->sh_size;
}

Two section types receive special treatment during layout: .nv.constant0 (address assigned by the OCG constant bank allocator, not the layout calculator) and .nv.reservedSmem (address assigned by the shared memory master allocator sub_1CABD60).

EIATTR Metadata

Each kernel's .nv.info.<funcname> section contains a sequence of EIATTR (Entry Information Attribute) records. These encode per-kernel metadata that the CUDA driver reads at launch time to configure the hardware correctly. The EIATTR builder is sub_1CC9800 (14,764 binary bytes, 90 KB decompiled, 51 callees) -- one of the largest functions in the output pipeline.

EIATTR Encoding

Each EIATTR record is a TLV (type-length-value) structure:

+--------+--------+------------------+
| type   | length | value            |
| 2 bytes| 2 bytes| length bytes     |
+--------+--------+------------------+

The type field is a 16-bit EIATTR code. The length field specifies the payload size. The value is type-dependent (scalar, array, or structured).

EIATTR Catalog

ptxas v13.0.88 defines 98 EIATTR codes. The critical ones that every cubin emitter must produce:

Additional EIATTR codes for textures/surfaces, barriers, cooperative groups, tensor cores, and hardware workarounds:

Barrier and Register Count Propagation

The EIATTR builder runs a cross-function propagation pass via sub_1CC8950 (2,634 bytes). When a device function uses barriers or a high register count, these requirements must propagate upward to the entry kernel that calls it:

"regcount %d for %s propagated to entry %s"
"Creating new EIATTR_NUM_BARRIERS and moving barcount %d
    from section flags of %s to nvinfo for entry symbol %s"
"Propagating higher barcount %d to the section flags
    of %s of entry symbol %s"

This ensures that the CUDA driver allocates sufficient barriers and registers for the entry kernel, accounting for all callees.

Relocation Processing

The relocation system handles symbol resolution for branch targets, constant bank references, function descriptors, texture/surface bindings, and address computations. The master relocation resolver is sub_1CD48C0 (4,184 binary bytes, 22 KB decompiled). ptxas defines 117 CUDA-specific relocation types (R_CUDA_NONE through R_CUDA_NONE_LAST).

Relocation Categories

Resolution Logic

The resolver performs these operations for each relocation entry:

1. Alias resolution -- redirect relocations from alias symbols to their targets ("change alias reloc %s to %s")
2. Dead function filtering -- skip relocations on eliminated functions ("ignore reloc on dead func %s")
3. UFT/UDT pseudo-relocation -- handle __UFT_OFFSET, __UFT_CANONICAL, __UDT_OFFSET, __UDT_CANONICAL synthetic symbols
4. PC-relative validation -- ensure branch targets are in the same section ("PC relative branch address should be in the same section")
5. YIELD-to-NOP conversion -- convert YIELD instructions to NOP when forward progress requirements prevent yielding
6. Unified reloc replacement -- convert type 103 (unified) to type 1 (absolute) for final resolution
7. Address computation -- compute final patched value from symbol address + addend

Output relocation sections (.nv.resolvedrela) are written by sub_1CD5920.

Debug Information

When --device-debug or --generate-line-info is active, ptxas generates DWARF debug sections. The debug subsystem spans 0x1CBF--0x1CC9 and includes parsers, emitters, and dumpers for the standard DWARF sections plus NVIDIA-specific debug extensions.

Debug Sections

Key debug infrastructure functions:

The --suppress-debug-info option (sub_432A00) disables debug section generation even when debug flags are present.

Capsule Mercury Output Path

For SM 100+ targets (Blackwell, Jetson Thor, consumer RTX 50-series), the default output mode is Capsule Mercury. In this mode, the CUBIN ELF contains two layers of content: standard CUBIN sections and a parallel set of .nv.merc.* sections carrying Mercury-encoded instruction streams plus all metadata needed for deferred finalization.

Mercury-Specific Sections

Capsule Mercury Construction

The capmerc path is integrated into the master ELF emitter. The sequence:

1. sub_1C9B110 (23 KB decompiled) creates the .nv.merc.* section namespace
2. sub_1CA2E40 (18 KB decompiled) clones constant/global/shared/local sections into the merc namespace, creating .nv.merc.rela relocation sections
3. sub_1C9C300 (24 KB decompiled) processes .nv.capmerc<funcname> sections. Constructs a 328-byte capsule descriptor per function containing: Mercury-encoded instruction stream, relocation metadata, KNOBS configuration snapshot, and function-level metadata (register counts, barriers, shared memory usage)
4. sub_1CA3A90 (45 KB decompiled) merges sections that have both merc and non-merc copies
5. sub_1C99BB0 (25 KB decompiled) remaps section indices after merc section insertion

Off-Target Finalization

The cubin entry sub_612DE0 implements a "fastpath optimization" for off-target finalization:

"[Finalizer] fastpath optimization applied for off-target %u -> %u finalization"

When a capmerc binary compiled for SM X is finalized for SM Y (within the same family), the fastpath patches the Mercury instruction stream directly without full recompilation. The compatibility checker sub_60F290 determines whether fastpath is safe based on instruction set compatibility, register file layout, and memory model.

Self-Check

The --self-check option performs roundtrip verification: generate capmerc, reconstitute SASS from the capsule, and compare section-by-section. The verifier uses a Flex SASS lexer (sub_720F00, 64 KB) and a comparator (sub_729540, 35 KB). Error string: "Failure of '%s' section in self-check for capsule mercury".

Multi-Kernel Output

A typical CUDA program compiles multiple kernels and device functions into a single CUBIN. The output pipeline handles this through per-function section isolation, combined with cross-function analysis for call graph construction, barrier propagation, and dead code elimination.

Per-Function Section Layout

Each entry function and each device function gets its own .text section (the -ffunction-sections pattern). This enables:

• Function-level dead code elimination -- sub_1CBC090 removes .text, .rela.text, .nv.info, and .nv.constant0 sections for unreachable functions
• Linker granularity -- nvlink can select individual functions from relocatable objects
• Driver loading -- the CUDA runtime can load individual kernels by name

Call Graph Construction

The call graph builder (sub_1CBE1B0) emits a .nv.callgraph section that encodes inter-function call edges. This section is present only in relocatable object mode (-c). The recursion detector (sub_1CBB920) performs a DFS traversal with manual 9-level unrolling, emitting "recursion at function %d" for each cycle found.

Dead functions are eliminated by sub_1CBC090:

"dead function %d(%s)"
"removed un-used section %s (%d)"   (x8 -- once per section type)
"function %d(%s) has address taken but no call to it"

Memory Allocation Across Kernels

The master section allocator sub_1CABD60 (11,856 binary bytes, 67 KB decompiled, 69 callees) assigns addresses to all memory-space sections across all kernels. It runs a multi-pass algorithm:

1. Global shared allocation -- shared variables visible to multiple kernels
2. Per-entry shared memory -- shared variables private to each kernel
3. Extern shared handling -- dynamically-sized shared memory (extern __shared__)
4. Reserved shared memory -- runtime reservations (.nv.reservedSmem.begin, .nv.reservedSmem.cap, .nv.reservedSmem.offset0, .nv.reservedSmem.offset1)
5. Local memory -- per-thread spill storage
6. Constant bank merging -- merges constant bank data across kernels, with deduplication (sub_1CA6890: "found duplicate value 0x%x, alias %s to %s")

The shared memory allocator sub_1CA92F0 (2,804 bytes) builds an interference graph for shared objects and performs group allocation for non-overlapping variables.

SHN_XINDEX Overflow

Large CUDA programs can exceed the ELF 65,280-section limit (SHN_LORESERVE = 0xFF00). Each kernel generates at minimum 4 sections (.text, .rela.text, .nv.info, .nv.constant0), so a program with 16,000+ kernels triggers the overflow mechanism:

1. e_shnum = 0 in the ELF header (signals overflow)
2. Section header [0].sh_size = real section count
3. e_shstrndx = SHN_XINDEX (0xFFFF)
4. Section header [0].sh_link = real .shstrtab index
5. Symbol st_shndx = SHN_XINDEX when real index >= 0xFF00
6. .symtab_shndx entries hold the actual section indices

This is standard ELF overflow handling, and it is production-critical -- sub_1CB68D0 checks for it with "overflow number of sections %d".

Key Functions

Cross-References

• Custom ELF Emitter -- deep dive into ELFW object, header construction, section management, file serialization
• Section Catalog & EIATTR -- complete inventory of section types and EIATTR attribute encoding
• Relocations & Symbols -- relocation resolution, UFT/UDT management, symbol table details
• Debug Information -- DWARF generation and .debug_* section handling
• Capsule Mercury & Finalization -- capmerc packaging format, off-target finalization, self-check
• Mercury Encoder -- Mercury instruction encoding (phases 117--122) that feeds the ELF emitter
• SASS Code Generation -- the upstream per-kernel compilation that produces SASS bytes
• Pipeline Overview -- where the ELF phase fits in the full PTX-to-SASS flow