Token Kind Table

Every token produced by cudafe++'s lexer carries a 16-bit token kind stored in the global word_126DD58. There are exactly 357 token kinds, numbered 0 through 356, with names indexed from a read-only string pointer table at off_E6D240 in the .rodata segment. A parallel 357-entry byte array at byte_E6C0E0 maps each token kind to an operator-name index, used by the initialize_opname_kinds routine (sub_588BB0) to populate the operator name display table at qword_126DE00. A boolean stop-token table at qword_126DB48 + 8 (357 entries) marks which token kinds are valid synchronization points for error recovery in skip_to_token (sub_6887C0).

Token kind assignment follows a block scheme established by the EDG 6.6 frontend: operators and punctuation occupy the lowest range, followed by alternative tokens (C++ digraphs and named operators), C89 keywords, C99/C11 extensions, MSVC keywords, core C++ keywords, compiler internals, type-trait intrinsics, and finally the newest C++23/26 and extended-type additions at the top. CUDA-specific additions from NVIDIA occupy three dedicated slots (328--330) within the type-trait block, plus additional entries in the extended range. This ordering reflects the historical accretion of the C and C++ standards: each new standard appended its keywords at the end rather than filling gaps.

Key Facts

Token Kind Ranges

Complete Token Table

Operators and Punctuation (0--51)

These tokens are produced directly by the character-level scanner sub_679800 (scan_token). Multi-character operators are resolved by dedicated scanning functions in the 0x67ABB0--0x67BAB0 range.

Alternative Tokens and Digraphs (52--76)

C++ alternative tokens (ISO 14882 clause 5.5) and C/C++ digraphs. These are registered during keyword_init (sub_5863A0) via sub_749600 when in C++ mode (dword_126EFB4 == 2).

C89 Keywords (77--108)

Always registered unconditionally. These form the base keyword set present in every compilation mode.

C99/C11/C23 Keywords (109--131)

Gated on the C standard version at dword_126EF68 (values: 199901 = C99, 201112 = C11, 202311 = C23).

MSVC Keywords (132--136)

Gated on dword_126EFB0 (Microsoft extensions enabled, language mode 2/MSVC).

C++ Core Keywords (137--199)

Gated on C++ mode (dword_126EFB4 == 2). Some keywords within this range were added in C++11 through C++23 and have additional standard-version gates.

Compiler Internal Tokens (200--206)

These tokens are used internally by the preprocessor and the token cache. They never appear in user-visible diagnostics.

Type Trait Intrinsics (207--327)

These are compiler intrinsic keywords that implement the C++ type traits (from <type_traits>) without requiring template instantiation. They are registered during keyword_init with C++ standard version gating -- earlier traits (C++11) are always available in C++ mode, while newer traits (C++20, C++23, C++26) require the corresponding standard version at dword_126EF68. Some traits are MSVC-specific (gated on dword_126EFB0) or Clang-specific (gated on qword_126EF90).

The complete list of type-trait intrinsics, organized alphabetically within each sub-category:

Unary Type Predicates

EDG Internal Keywords (272--283)

These are not user-facing keywords. They are injected by the EDG frontend into synthesized declarations for built-in types, throw specifications, and vector types.

More Type Predicates and Binary Traits (284--327)

NVIDIA CUDA Type Traits (328--330)

Three NVIDIA-specific type-trait intrinsics occupy dedicated token kinds. These are registered during keyword_init when GPU mode is active (dword_106C2C0 != 0) and participate in the same token classification pipeline as all other type traits. They are used internally by the CUDA frontend to detect extended lambda closure types during device/host separation.

When extended lambdas are disabled, these traits are predefined as macros expanding to false:

// Fallback definitions in preprocessor preamble:
#define __nv_is_extended_device_lambda_closure_type(X) false
#define __nv_is_extended_host_device_lambda_closure_type(X) false
#define __nv_is_extended_device_lambda_with_preserved_return_type(X) false

Extended Types and Recent Additions (331--356)

These are the newest token kinds, added for extended floating-point types (ISO/IEC TS 18661-3) and recent C++23/26 features.

Token Cache

The token cache provides lookahead, backtracking, and macro-expansion replay for C++ parsing. Tokens are stored in a linked list of cache entries, each 80--112 bytes depending on payload.

Cache Entry Layout

Cache Entry Kinds

Eight discriminator values select the payload interpretation at offset +40:

Cache Management Globals

Cache Operations

Keyword Registration

All keywords are registered during frontend initialization by sub_5863A0 (keyword_init / fe_translation_unit_init, 1,113 lines, in fe_init.c). The function calls sub_7463B0 (enter_keyword) for each keyword, passing the numeric token kind and the keyword string. GNU double-underscore variants (e.g., __asm and __asm__ for asm) are registered via sub_585B10 (enter_gnu_keyword), which automatically generates both __name and __name__ forms from a single root. Alternative tokens are registered via sub_749600.

Version Gating Architecture

Registration is conditional on a set of global configuration flags established during CLI processing:

Registration Pattern

The pseudocode below shows the version-gated registration pattern reconstructed from sub_5863A0:

void keyword_init(void) {
    // C89 keywords -- always registered
    enter_keyword(77, "auto");
    enter_keyword(78, "break");
    enter_keyword(79, "case");
    // ... all C89 keywords ...
    enter_keyword(108, "while");

    // C99 keywords -- gated on C99+ standard
    if (c_standard_version >= 199901) {
        enter_keyword(119, "restrict");
        enter_keyword(120, "_Bool");
        enter_keyword(121, "_Complex");
        enter_keyword(122, "_Imaginary");
    }

    // C11 keywords
    if (c_standard_version >= 201112) {
        enter_keyword(184, "_Static_assert");
        enter_keyword(247, "_Alignof");
        enter_keyword(248, "_Alignas");
        enter_keyword(260, "_Noreturn");
        enter_keyword(262, "_Generic");
        enter_keyword(263, "_Atomic");
        enter_keyword(194, "_Thread_local");
    }

    // C++ mode keywords
    if (language_mode == 2) {  // C++ mode
        enter_keyword(137, "bool");
        enter_keyword(138, "true");
        enter_keyword(139, "false");
        enter_keyword(140, "wchar_t");
        enter_keyword(150, "catch");
        enter_keyword(151, "class");
        // ... all C++ core keywords ...
        enter_keyword(183, "typename");

        // Alternative tokens (C++ only)
        enter_alt_token(52, "and", /*len*/3);
        enter_alt_token(53, "or", 2);
        enter_alt_token(64, "and_eq", 6);
        // ... all alternative tokens ...

        // C++11 keywords
        if (cpp_standard_version >= 201103) {
            enter_keyword(244, "constexpr");
            enter_keyword(185, "decltype");
            enter_keyword(237, "nullptr");
            enter_keyword(126, "char16_t");
            enter_keyword(127, "char32_t");
            enter_keyword(184, "static_assert");
            enter_keyword(194, "thread_local");
        }

        // C++20 keywords
        if (cpp_standard_version >= 202002) {
            enter_keyword(245, "consteval");
            enter_keyword(246, "constinit");
            enter_keyword(267, "co_yield");
            enter_keyword(268, "co_return");
            enter_keyword(269, "co_await");
            enter_keyword(294, "requires");
            enter_keyword(295, "concept");
        }
    }

    // GNU extensions -- gated on gnu_extensions flag
    if (gnu_extensions) {
        enter_gnu_keyword(187, "__extension__");
        enter_gnu_keyword(186, "__auto_type");
        enter_gnu_keyword(142, "__attribute");
        enter_keyword(117, "__builtin_offsetof");
        enter_keyword(143, "__builtin_types_compatible_p");
        enter_keyword(239, "__int128");
        // ... all GNU extensions ...
    }

    // MSVC extensions
    if (microsoft_extensions) {
        enter_keyword(132, "__declspec");
        enter_keyword(133, "__int8");
        enter_keyword(134, "__int16");
        enter_keyword(135, "__int32");
        enter_keyword(136, "__int64");
    }

    // Type traits (C++11+, ~60 traits)
    if (language_mode == 2) {
        enter_keyword(207, "__is_class");
        enter_keyword(208, "__is_enum");
        // ... all type traits through 327 ...
    }

    // CUDA type traits (GPU mode)
    if (gpu_mode) {
        enter_keyword(328, "__nv_is_extended_device_lambda_closure_type");
        enter_keyword(329, "__nv_is_extended_host_device_lambda_closure_type");
        enter_keyword(330, "__nv_is_extended_device_lambda_with_preserved_return_type");
    }

    // Extended float types (GNU)
    if (gnu_extensions) {
        enter_keyword(331, "_Float32");
        enter_keyword(332, "_Float32x");
        enter_keyword(333, "_Float64");
        enter_keyword(334, "_Float64x");
        enter_keyword(335, "_Float128");
    }

    // Post-keyword init: scope setup, builtin registration
    // ...
}

GNU Double-Underscore Registration

sub_585B10 (enter_gnu_keyword, assert at fe_init.c:698) implements the pattern where a single keyword name is registered in two or three forms:

• If name starts with _: registers name as-is and __name__ (e.g., _Bool stays, plus ___Bool__ if applicable)
• Otherwise: registers __name and __name__ (e.g., asm produces __asm and __asm__)

The function uses a stack buffer of 49 characters maximum (name + 5 <= 0x31), prepends __ (encoded as 0x5F5F in little-endian), copies the name, and appends __ with a null terminator. Both variants call sub_7463B0 (enter_keyword) with the same token kind.

Operator Name Table

The operator name display table at qword_126DE00 maps operator kinds to printable names for diagnostics and error messages. It is populated by sub_588BB0 (initialize_opname_kinds) during fe_wrapup.c initialization.

The initialization loop iterates all 357 entries of byte_E6C0E0 (operator-to-name index), mapping each non-zero entry to the corresponding string from off_E6D240 (the token name table). Two special cases are hardcoded:

Additionally, the array positions for new[] and delete[] are hardcoded separately, since these operator names do not correspond to single tokens.

The routine validates that all entries in the range qword_126DE08 through qword_126DF80 (the 48 operator name slots) are non-null, and panics with "initialize_opname_kinds: bad init of opname_names" if any gap is found.

Token State Globals

When a token is produced by the lexer, the following globals are populated:

Cross-References

• Lexer & Tokenizer -- full lexer subsystem documentation, architecture, and function map
• Pipeline Overview -- keyword registration during initialization
• Entry Point & Initialization -- keyword_init in the startup sequence
• Global Variable Index -- all global addresses referenced here
• Template Engine -- template argument scanning and >> disambiguation
• CUDA Lambda Overview -- NVIDIA type-trait token usage in lambda transforms
• Attribute System Overview -- CUDA attribute handling at token level
• EDG Source File Map -- lexical.c and fe_init.c binary layout