Skip to content

fix: return views from shape ops, unify error handling across engines - #349

Merged
devcrocod merged 1 commit into
developfrom
fix/135-252-257-shape-views-and-error-handling
Sep 7, 2026
Merged

devcrocod merged 1 commit into
developfrom
fix/135-252-257-shape-views-and-error-handling

Conversation

@devcrocod

Copy link
Copy Markdown
Collaborator

Closes #135, closes #252, closes #257.

Shape ops return views (#135, #252)

reshape, squeeze, unsqueeze and expandDims copied whenever the array was not consistent.
They now compute strides for the new shape over the existing buffer — NumPy's rule — and copy only
when no such strides exist, e.g. transpose().reshape(6), where axes genuinely have to be merged.
One internal helper, reshapeStrides, backs all of them.

Bugs found on the way:

  • squeeze returned wrong data. It passed the new shape without strides, so they were recomputed
    as if the array were packed. On a (2,3,4) array, a[0 until 2, 1 until 2, 0 until 4].squeeze()
    gave [4,5,6,7,8,9,10,11] instead of [4,5,6,7,16,17,18,19] — silently, no exception.
  • expandDims on D4 and expandNDims ignored their axis argument — both called unsqueeze()
    with no axes, so they only changed the static type.
  • squeeze() threw on an all-ones shape, mk.ndarrayOf(5) included. Multik has no rank-0 arrays,
    so it now keeps one axis and returns [1] — what squeeze(0) on the same array already returned.
  • NPY write let complex through: the guard read dtype != ComplexFloat || dtype != ComplexFloat,
    so ComplexDouble reached an unchecked cast to NDArray<out Number, *>.
  • CSV write rejected D2, while writeCSV has an explicit D2 branch and read accepts D1 and D2.

Error handling (#257)

The engines disagreed: a singular matrix threw ArithmeticException in KEEngine and a bare
Exception in NativeEngine. multik-default resolves the engine at runtime, so the same user code
behaved differently per platform. Both now throw ArithmeticException.

Also: 20 bare throw Exception → 0 (four had empty messages); 29 message-less
UnsupportedOperationException() → 0, each now naming the operation and dtype; LAPACK info < 0
reports an argument the wrapper passed, so it is IllegalStateException rather than
IllegalArgumentException — and solveCommon had no info < 0 check at all, returning garbage on a
negative code; squeeze/unsqueeze validate axes instead of letting ArrayIndexOutOfBoundsException
escape from MutableList.add. Policy written down in CLAUDE.md.

Behaviour changes for the release notes

Writes go through to the source where reshape/unsqueeze/expandDims used to return a copy.

was now where
Exception ArithmeticException native inv/solve/eig: singularity, non-convergence
IllegalStateException ArithmeticException native svd non-convergence
IllegalStateException IllegalArgumentException native svd NaN input, DataType.of, toSlice()
IllegalArgumentException IllegalStateException LAPACK info < 0
Exception IllegalArgumentException npy/csv read/write, Number.toPrimitiveType
IllegalStateException shape [1] squeeze() on an all-ones shape
IllegalArgumentException success write of a D2 array to CSV
success IllegalArgumentException write of a ComplexDouble array to NPY

Testing

2207 tests, 0 failures on JVM, Native (macosArm64) and iOS simulator
(iosSimulatorArm64) for multik-core, multik-kotlin and multik-openblas. JS and WASM test tasks are
disabled in the build (#247), so those targets are compile-only here.

New tests: ReshapeTest (view vs copy, reshapeStrides rules, squeeze/unsqueeze axes),
ErrorHandlingTest, IOErrorsTest, plus mirrored KELinAlgErrorTest / NativeLinAlgErrorTest
pinning the cross-engine exception contract.

apiCheck and korroCheck pass — the public API is unchanged, the new helpers are internal.

Checklist

  • Existing tests pass
  • New/updated tests for changed behavior
  • New/updated documentation if necessary

Copilot AI lite review requested due to automatic review settings September 7, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

expandDims for D1/D2/D3 still allows invalid axes to throw IndexOutOfBoundsException instead of the intended IllegalArgumentException contract.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR improves Multik’s ndarray shape-manipulation semantics to return views whenever possible (including for non-consistent layouts when stridable), and standardizes error/exception behavior across engines and IO paths so multik-default behaves consistently at runtime.

Changes:

  • Introduces reshapeStrides/reshapeTo to compute view strides for reshape/squeeze/unsqueeze/expandDims and copy only when required.
  • Unifies error handling across KE/OpenBLAS engines and IO (eliminates throw Exception, adds messages that name offending values, aligns LAPACK info handling).
  • Adds targeted regression/contract tests and updates user documentation to reflect view-vs-copy behavior changes.
File summaries
File Description
multik-openblas/src/nativeMain/kotlin/org/jetbrains/kotlinx/multik/openblas/stat/JniStat.kt Replaces generic Exception with UnsupportedOperationException for unsupported native median inputs.
multik-openblas/src/nativeMain/kotlin/org/jetbrains/kotlinx/multik/openblas/math/JniMath.kt Standardizes unsupported native argMin/argMax input handling via UnsupportedOperationException.
multik-openblas/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/openblas/linalg/NativeLinAlgErrorTest.kt Adds native-engine exception contract tests (singularity, shape mismatch).
multik-openblas/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/openblas/linalg/NativeLinAlgEx.kt Aligns LAPACK info handling and exception types/messages across native linalg operations.
multik-kotlin/src/commonTest/kotlin/org/jetbrains/kotlinx/multik_kotlin/linAlg/KELinAlgErrorTest.kt Adds KE-engine mirror tests to pin cross-engine exception contracts.
multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/math/KEMathEx.kt Replaces silent fallbacks/empty exceptions with check(...) + actionable messages.
multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/pluDecomposition.kt Improves unsupported dtype error message for PLU.
multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/LinAlgEx.kt Improves unsupported dtype error message for solve.
multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/eigen.kt Improves unsupported dtype error message for eig.
multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/dot.kt Improves unsupported dtype error messages for dot variants.
multik-default/src/wasmJsMain/kotlin/org/jetbrains/kotlinx/multik/default/DefaultEngineFactory.kt Makes missing native-engine selection on WASM a typed UnsupportedOperationException.
multik-default/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/default/DefaultEngineFactory.kt Uses IllegalArgumentException for missing engine type input on JVM.
multik-default/src/jsMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt Makes missing native-engine selection on JS a typed UnsupportedOperationException.
multik-default/src/iosMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt Makes missing native-engine selection on iOS a typed UnsupportedOperationException.
multik-default/src/desktopMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt Uses IllegalArgumentException for missing engine type input on desktop native.
multik-core/src/jvmTest/kotlin/org/jetbrains/kotlinx/multik/io/IOErrorsTest.kt Adds IO regression tests for complex NPY rejection, CSV D2 acceptance, and format errors.
multik-core/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/api/io/npy.kt Changes NPY dtype validation to require(dtype.isNumber()) and improves error messaging/KDoc.
multik-core/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/api/io/io.kt Standardizes IO validation/exceptions/messages for read/write across formats and dims.
multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/SliceTest.kt Updates base expectations to match new “shape ops return views” behavior.
multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/ReshapeTest.kt Adds comprehensive tests for view-vs-copy reshape rules, squeeze/unsqueeze correctness, and axis handling.
multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/ErrorHandlingTest.kt Pins error-handling policy and message requirements via unit tests.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt Switches expandDims/expandNDims to view-based reshape helpers and fixes axis plumbing.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/IteratingNDArray.kt Replaces generic exceptions with consistent UnsupportedOperationException + dtype-aware messages.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/Slice.kt Aligns toSlice() invalid-range behavior to IllegalArgumentException with clearer message.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/NDArray.kt Reworks reshape/squeeze/unsqueeze implementations to use reshapeTo and adds axis validation.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/MultiArrays.kt Updates public KDoc for reshape/squeeze/unsqueeze to reflect view/copy semantics and rank-0 policy.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/MemoryView.kt Improves unsupported typed-array access errors with dtype-specific messages.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/Internals.kt Adds reshapeStrides + reshapeTo and standardizes primitive conversion errors.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/DataType.kt Makes DataType.of(...) and code-to-type mapping throw IllegalArgumentException with clearer messages.
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/api/random.kt Improves unsupported bound-type error message for random generation.
docs/topics/userGuide/shape-manipulation.md Updates docs to describe reshape/view semantics and size-1 axis ops always being views.
docs/topics/userGuide/copies-and-views.md Updates “typical view ops” list and clarifies reshape behavior.
docs/topics/FAQ.md Revises reshape FAQ to reflect new view semantics and recommends deepCopy() when detaching is required.
CLAUDE.md Documents engine/error-handling policy and project conventions in a consolidated form.
Review details

Suppressed comments (2)

multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt:403

  • expandDims for D2 does not validate axis; invalid values currently throw IndexOutOfBoundsException from MutableList.add. For consistency with the new axis-validation policy (invalid arguments → IllegalArgumentException with a helpful message), add a require bounds check.
@JvmName("expandDimsD2")
public fun <T> MultiArray<T, D2>.expandDims(axis: Int): MultiArray<T, D3> =
    reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D3)

multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt:412

  • expandDims for D3 still relies on MutableList.add(axis, 1) without checking bounds, so bad axis values escape as IndexOutOfBoundsException. Add an explicit require to surface an IllegalArgumentException with a clear message (consistent with unsqueeze).
@JvmName("expandDimsD3")
public fun <T> MultiArray<T, D3>.expandDims(axis: Int): MultiArray<T, D4> =
    reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D4)
  • Files reviewed: 34/34 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 392 to +394
@JvmName("expandDimsD1")
public fun <T> MultiArray<T, D1>.expandDims(axis: Int): MultiArray<T, D2> {
val newShape = shape.toMutableList().apply { add(axis, 1) }.toIntArray()
// TODO(get rid of copying)
val newData = if (consistent) this.data else this.deepCopy().data
val newBase = if (consistent) this.base ?: this else null
val newOffset = if (consistent) this.offset else 0
return D2Array(newData, newOffset, newShape, dim = D2, base = newBase)
}
public fun <T> MultiArray<T, D1>.expandDims(axis: Int): MultiArray<T, D2> =
reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D2)
reshape, squeeze, unsqueeze and expandDims copied whenever the array was
not `consistent`. They now stride the new shape over the existing buffer
(NumPy's rule) and copy only when no such strides exist, so writes go
through to the source where a copy used to be returned.

Bugs found along the way:
- squeeze recomputed packed strides, so squeezing a strided sub-array
  silently returned the wrong elements
- expandDims on D4 and expandNDims ignored their axis argument
- squeeze of an all-ones shape threw "Shape can't be empty"; it now
  keeps one axis, like squeeze(0) already did
- the NPY guard named ComplexFloat twice, letting ComplexDouble through;
  write rejected D2 arrays that writeCSV handles and read accepts

Error handling: drop 20 bare `throw Exception` and 29 message-less
`UnsupportedOperationException()`. Singular matrices and non-convergence
now throw ArithmeticException in both engines, which disagreed before and
made behaviour depend on the engine multik-default picked. LAPACK
`info < 0` blames an argument the wrapper passed, so it is now
IllegalStateException. Policy written down in CLAUDE.md.

Closes #135, #252, #257
@devcrocod
devcrocod force-pushed the fix/135-252-257-shape-views-and-error-handling branch from c229a9b to 369d29b Compare September 7, 2026 15:17
@devcrocod
devcrocod merged commit 572c969 into develop Sep 7, 2026
2 checks passed
@devcrocod
devcrocod deleted the fix/135-252-257-shape-views-and-error-handling branch September 7, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistent error handling Eliminate unnecessary array copying in reshape and unsqueeze optimize reshape function

2 participants