Skip to content

CLAUDE.md

Latest commit

 

History

History
205 lines (138 loc) · 14 KB

CLAUDE.md

File metadata and controls

205 lines (138 loc) · 14 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Apache IoTDB is a time series database for IoT data. It uses a distributed architecture with ConfigNodes (metadata/coordination) and DataNodes (storage/query). Data is stored in TsFile columnar format (separate repo: https://github.com/apache/tsfile). Current version is 2.0.11-SNAPSHOT.

Build Commands

# Full build (skip tests)
mvn clean package -pl distribution -am -DskipTests

# Build a specific module (e.g., datanode)
mvn clean package -pl iotdb-core/datanode -am -DskipTests

# Run unit tests for a specific module
mvn clean test -pl iotdb-core/datanode

# Run a single test class
mvn clean test -pl iotdb-core/datanode -Dtest=ClassName

# Run a single test method
mvn clean test -pl iotdb-core/datanode -Dtest=ClassName#methodName

# Build with Chinese log & error messages
mvn clean package -pl distribution -am -DskipTests -P with-zh-locale

# Format code (requires JDK 17+; auto-skipped on JDK <17)
mvn spotless:apply

# Format code in integration-test module
mvn spotless:apply -P with-integration-tests

# Check formatting without applying
mvn spotless:check

Integration Tests

Integration tests live in integration-test/ (not included in default build). They require the with-integration-tests profile:

# Build template-node first (needed once, or after code changes)
mvn clean package -DskipTests -pl integration-test -am -P with-integration-tests

# Run tree-model ITs (simple: 1 ConfigNode + 1 DataNode)
mvn clean verify -DskipUTs -pl integration-test -am -P with-integration-tests

# Run tree-model ITs (cluster: 1 ConfigNode + 3 DataNodes)
mvn clean verify -DskipUTs -pl integration-test -am -PClusterIT -P with-integration-tests

# Run table-model ITs (simple)
mvn clean verify -DskipUTs -pl integration-test -am -PTableSimpleIT -P with-integration-tests

# Run table-model ITs (cluster)
mvn clean verify -DskipUTs -pl integration-test -am -PTableClusterIT -P with-integration-tests

# Run a single IT class (tree-model simple, 1C1D)
mvn clean verify -DskipUTs -Dit.test=ClassName -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -P with-integration-tests

# Run a single IT class (tree-model cluster, 1C3D)
mvn clean verify -DskipUTs -Dit.test=ClassName -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PClusterIT -P with-integration-tests

# Run a single IT class (table-model simple, 1C1D)
mvn clean verify -DskipUTs -Dit.test=ClassName -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableSimpleIT -P with-integration-tests

# Run a single IT class (table-model cluster, 1C3D)
mvn clean verify -DskipUTs -Dit.test=ClassName -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableClusterIT -P with-integration-tests

# Run a single test method within an IT class (use ClassName#methodName)
mvn clean verify -DskipUTs -Dit.test=ClassName#methodName -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableSimpleIT -P with-integration-tests

When verifying a new feature, only run the specific IT classes/methods that were added or modified in the current branch — do not run all ITs.

To run integration tests from IntelliJ: enable the with-integration-tests profile in Maven sidebar, then run test cases directly.

Code Style

  • Apache License header: Every new file committed to the repository MUST include the Apache License 2.0 header. Use the comment syntax appropriate for the file type (e.g., <!-- --> for .md, /* */ for .java, # for .properties/.py). JSON files are exempt (no comment syntax). Refer to existing files in the repo for the exact format.
  • Spotless with Google Java Format (GOOGLE style). Import order: org.apache.iotdb, blank, javax, java, static.
  • Checkstyle is also configured (see checkstyle.xml at project root).
  • Java source/target level is 17 (compiled with maven.compiler.release=17 on JDK 17+).

Architecture

Node Types

  • ConfigNode (iotdb-core/confignode): Manages cluster metadata, schema regions, data regions, partition tables. Coordinates via Ratis consensus.
  • DataNode (iotdb-core/datanode): Handles data storage, query execution, and client connections. The main server component.
  • AINode (iotdb-core/ainode): Python-based node for AI/ML inference tasks.

Dual Data Model

IoTDB supports two data models operating on the same storage:

  • Tree model: Traditional IoT hierarchy (e.g., root.ln.wf01.wt01.temperature). SQL uses path-based addressing.
  • Table model (relational): SQL table semantics. Grammar lives in iotdb-core/relational-grammar/. Query plan code under queryengine/plan/relational/.

Key DataNode Subsystems (iotdb-core/datanode)

  • queryengine: SQL parsing, planning, optimization, and execution.
    • plan/parser/ - ANTLR-based SQL parser
    • plan/statement/ - AST statement nodes
    • plan/planner/ - Logical and physical planning (tree model: TreeModelPlanner, table model: under plan/relational/)
    • plan/optimization/ - Query optimization rules
    • execution/operator/ - Physical operators (volcano-style iterator model)
    • execution/exchange/ - Inter-node data exchange
    • execution/fragment/ - Distributed query fragment management
  • storageengine: Write path, memtable, flush, WAL, compaction, TsFile management.
    • dataregion/ - DataRegion lifecycle, memtable, flush, compaction
    • dataregion/wal/ - Write-ahead log
    • buffer/ - Memory buffer management
  • schemaengine: Schema (timeseries metadata) management.
  • pipe: Data sync/replication framework (source -> processor -> sink pipeline).
  • consensus: DataNode-side consensus integration.
  • subscription: Client subscription service for streaming data changes.

Consensus (iotdb-core/consensus)

Pluggable consensus protocols: Simple (single-node), Ratis (Raft-based), IoT Consensus (optimized for IoT writes). Factory pattern via ConsensusFactory.

Protocol Layer (iotdb-protocol/)

Thrift IDL definitions for RPC between nodes. Generated sources are produced automatically during build. Sub-modules: thrift-commons, thrift-confignode, thrift-datanode, thrift-consensus, thrift-ainode.

Client Libraries (iotdb-client/)

  • session/ - Java Session API (primary client interface)
  • jdbc/ - JDBC driver
  • cli/ - Command-line client
  • client-cpp/, client-go/, client-py/ - Multi-language clients
  • service-rpc/ - Shared Thrift service definitions

API Layer (iotdb-api/)

Extension point interfaces: udf-api (user-defined functions), trigger-api (event triggers), pipe-api (data sync plugins), external-api, external-service-api.

IDE Setup

After mvn package, right-click the root project in IntelliJ and choose "Maven -> Reload Project" to add generated source roots (Thrift and ANTLR).

Generated source directories that need to be on the source path:

  • **/target/generated-sources/thrift
  • **/target/generated-sources/antlr4

Common Pitfalls

Build

  • Missing Thrift compiler: The local machine may not have the thrift binary installed. Running mvn clean package -pl <module> -am -DskipTests will fail at the iotdb-thrift module. Workaround: To verify your changes compile, use mvn compile -pl <module> (without -am or clean) to leverage existing target caches.
  • Pre-existing compilation errors in unrelated modules: The datanode module may have pre-existing compile errors in other subsystems (e.g., pipe, copyto) that cause mvn clean test -pl iotdb-core/datanode -Dtest=XxxTest to fail during compilation. Workaround: First run mvn compile -pl iotdb-core/datanode to confirm your changed files compile successfully. If the errors are in files you did not modify, they are pre-existing and do not affect your changes.
  • Single-module / no-clean compiles hide cross-module and i18n errors: mvn compile -pl <module> (the thrift workaround above) builds only that module against the installed jars in ~/.m2 and the existing target/ classes; an incremental mvn test-compile without clean reuses stale target/*.class. If you add/rename a *Messages constant (or any cross-module API) in module B but ~/.m2/target/ still holds B's old artifact, module A referencing the new symbol compiles fine against the stale class — the real cannot find symbol only surfaces in CI or a clean build. Workaround: for i18n / cross-module edits, verify with a full-reactor compile (no -pl, no clean): mvn test-compile -DskipTests and mvn test-compile -DskipTests -P with-zh-locale. Capture the maven exit code directly (mvn ... ; echo $?), not via mvn ... && echo OK — set -e does NOT fail on a command inside a && list, so a && chain can print a false success.

i18n (Chinese Messages)

The project uses compile-time i18n via the build-helper-maven-plugin. The property i18n.locale (default: en) controls which source directory is added: src/main/i18n/${i18n.locale}. Activating -P with-zh-locale sets i18n.locale=zh, swapping English message constant classes for Chinese ones. Each module that participates has both src/main/i18n/en/ and src/main/i18n/zh/ directories containing Java classes with identical structure but different string literals.

Rule — never inline a user-facing English string literal. Every string a user, operator, or log reader will see MUST be a public static final String constant in the module's *Messages class (e.g. DataNodeQueryMessages), referenced as MessagesClass.CONSTANT — never a "raw literal". This applies to: LOGGER/log.info|warn|error|debug|trace(...), throw new …Exception(…), super(…) in exception constructors, resp.setMessage(…), the message arg of requireNonNull/checkArgument/checkState/checkNotNull, and the template of String.format(…). A raw literal is invisible to the zh build and renders English under -P with-zh-locale — that is the bug this rule prevents. (toString() debug output is exempt.) Reviewers and AI agents must reject/fix any new code that introduces such a literal.

Naming a new constant (deterministic, matches existing files): <PREFIX>_<NORMALIZED>_<HASH> where PREFIX is LOG_ for LOGGER/log calls, MESSAGE_ for setMessage(...), and EXCEPTION_ for throw new …(…), super(…), and the message arg of requireNonNull/checkArgument/checkState/checkNotNull (the three prefixes the codebase already uses — ~LOG_/MESSAGE_/EXCEPTION_); NORMALIZED is the English value uppercased with non-alphanumeric runs collapsed to _ and %s/%d/{} replaced by ARG; HASH is the first 8 hex chars of md5(englishValue), uppercased — compute with python3 -c "import hashlib;print(hashlib.md5(b'the english').hexdigest()[:8].upper())". Add the constant to both the en and zh files under the same name (zh gets a natural Chinese translation; preserve every %s/%d/{} exactly; keep config keys, SQL keywords, class/method names, and product terms like TsFile/DataRegion/Consensus/ConfigNode in English inside the Chinese string). Reuse an existing same-named constant instead of duplicating. Pure-punctuation fragments (. , , , : , ., etc.) are exempt — inline them, do not create a constant (punctuation does not localize, and a hashed . constant hurts readability). A single message that is built from several pieces should be ONE constant holding the full template, not several fragments concatenated at the call site.

Verify before commit:

  • mvn spotless:apply -pl <module> (the PostToolUse formatter does not add the i18n import; spotless does).
  • Compile both locales: mvn test-compile -DskipTests AND mvn test-compile -P with-zh-locale -DskipTests (the .github/workflows/zh-locale-compile.yml CI enforces the zh build).
  • The en and zh files must stay in key parity (same constant names) with identical format-specifier counts; a raw-literal scan of main java must return nothing.

Code Style

  • Always run mvn spotless:apply after editing Java files: Spotless runs spotless:check automatically during the compile phase. Format violations cause an immediate BUILD FAILURE. Make it a habit to run mvn spotless:apply -pl <module> right after editing, not at the end. For files under integration-test/, add -P with-integration-tests.
  • Always run black and isort after editing Python files under iotdb-core/ainode/: The AINode Code Style Check CI runs black --check . and isort --check-only --profile black . on that directory. Run cd iotdb-core/ainode && black . && isort --profile black . before committing. Requires pip install black==25.1.0 isort==6.0.1.
  • Gson version compatibility: JsonObject.isEmpty() / JsonArray.isEmpty() may not be available in the Gson version used by this project. Use size() > 0 instead and add a comment explaining why.

Git Commit

  • Do NOT add Co-Authored-By trailer to commit messages.

Security

Security model: SECURITY.md

Agents that scan this repository should consult SECURITY.md and the threat model it links before reporting issues.