`--parallelism_config_sp_seq_length_is_variable False` parses as True

[MicroMilo]

System Info

Repository: huggingface/accelerate
Branch: main
Commit inspected: 2278ebbfd5ee958452f344774d3b7191808a70b9
Area: accelerate launch CLI parser / parallelism_config / DeepSpeed sequence parallelism

This is a static CLI parser bug. I could not run `accelerate env` in the scanning
environment because torch is not installed there, but the issue reproduces with
standard-library argparse before any distributed runtime is involved.

Information

• The official example scripts
• My own modified scripts

Tasks

• One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
• My own task or dataset (give details below)

Reproduction

--parallelism_config_sp_seq_length_is_variable is documented as a boolean-like CLI option where False has a meaningful runtime behavior:

• if True, sequence length may vary between batches;
• if False, parallelism_config_sp_seq_length should match the batch sequence length.

However, the parser currently declares the option with type=bool:

accelerate/src/accelerate/commands/launch.py

Lines 964 to 968 in 2278ebb

	parallelism_config_args.add_argument(
	"--parallelism_config_sp_seq_length_is_variable",
	type=bool,
	default=True,
	help="If `True` will work with a sequence length that may change between batches, in which case `parallelism_config_sp_seq_length` value can be set to anything divisible by sp size or remain unset. If `False` then `parallelism_config_sp_seq_length` needs to match the batch's sequence length dimension. The default is `True`.",

The parsed value is then serialized into the runtime environment:

accelerate/src/accelerate/utils/launch.py

Lines 421 to 423 in 2278ebb

	if args.parallelism_config_sp_size > 1:
	current_env[prefix + "SP_SEQ_LENGTH"] = str(args.parallelism_config_sp_seq_length)
	current_env[prefix + "SP_SEQ_LENGTH_IS_VARIABLE"] = str(args.parallelism_config_sp_seq_length_is_variable)

The runtime config later reads that env var:

accelerate/src/accelerate/utils/dataclasses.py

Lines 2256 to 2259 in 2278ebb

	if self.sp_seq_length_is_variable is None:
	self.sp_seq_length_is_variable = (
	os.environ.get("PARALLELISM_CONFIG_SP_SEQ_LENGTH_IS_VARIABLE", "true").lower() == "true"
	)

Minimal reproduction of the parser behavior:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument(
    "--parallelism_config_sp_seq_length_is_variable",
    type=bool,
    default=True,
)

args = parser.parse_args([
    "--parallelism_config_sp_seq_length_is_variable",
    "False",
])

print(type(args.parallelism_config_sp_seq_length_is_variable).__name__)
print(args.parallelism_config_sp_seq_length_is_variable)

Actual output:

bool
True

A natural accelerate launch command that would hit this is:

accelerate launch \
  --use_fsdp \
  --fsdp_version 2 \
  --use_parallelism_config \
  --parallelism_config_sp_size 2 \
  --parallelism_config_sp_seq_length 128 \
  --parallelism_config_sp_seq_length_is_variable False \
  train.py

Because bool("False") is True in Python, the user-provided False is silently converted to True. prepare_extend_env_parallelism_config then writes "True" to PARALLELISM_CONFIG_SP_SEQ_LENGTH_IS_VARIABLE, so DeepSpeedSequenceParallelConfig enables variable sequence length even though the user selected fixed sequence length.

This also skips the fixed-length validation path:

accelerate/src/accelerate/utils/dataclasses.py

Lines 2261 to 2265 in 2278ebb

	if not self.sp_seq_length_is_variable and self.sp_seq_length is None:
	if "PARALLELISM_CONFIG_SP_SEQ_LENGTH" not in os.environ:
	raise ValueError(
	"when `sp_seq_length_is_variable` is `False` `sp_seq_length` must be provided either through the constructor or the environment variable PARALLELISM_CONFIG_SP_SEQ_LENGTH"
	)

Expected behavior

Passing:

--parallelism_config_sp_seq_length_is_variable False

should either:

1. parse to a false value and serialize PARALLELISM_CONFIG_SP_SEQ_LENGTH_IS_VARIABLE=False, or
2. be rejected as an invalid boolean spelling.

It should not silently parse to True.

A focused fix would be to avoid type=bool for this CLI argument and use the existing textual boolean conversion helper, such as str_to_bool, or paired store_true / store_false flags.

Suggested tests:

• parser test: --parallelism_config_sp_seq_length_is_variable False yields a false value;
• env bridge test: the launch env contains PARALLELISM_CONFIG_SP_SEQ_LENGTH_IS_VARIABLE=False for the same CLI input.

[lokeshps321]

Hi I want to help with the CLI parsing issue. I can try to figure out what is going on and send in a fix. Can you please give the CLI parsing issue to me to work on? I will take a look, at the CLI parsing issue and do my best to fix it.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.