Token: Implement 12-byte struct with TokenKind enum

[pdaxt]

Component

Foundation / Token

Task

Implement Token struct (12 bytes) and TokenKind enum (1 byte).

Acceptance Criteria

• TokenKind enum with ~80 variants (fits in u8)
• Keywords: fn, let, if, else, while, return, etc.
• Operators: +, -, *, /, ==, !=, &&, ||, etc.
• Literals: Int, Float, String, Char
• Special: Ident, Eof, Error
• Token struct: kind: TokenKind, span: Span
• Size = exactly 12 bytes

Verify

assert_eq!(size_of::<TokenKind>(), 1);
assert_eq!(size_of::<Token>(), 12);

Security Testing (REQUIRED)

• Create adversarial tests (token_attack.rs)
• Test boundary conditions (max enum values)
• Test invalid state creation attempts
• Verify invariants hold in both debug AND release mode
• Document any unsafe usage and its safety guarantees

Ref: ADR-004, FOUNDATION.md

[pdaxt]

Already implemented in current codebase.