Repository navigation
fix: return views from shape ops, unify error handling across engines - #349
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
expandDims for D1/D2/D3 still allows invalid axes to throw IndexOutOfBoundsException instead of the intended IllegalArgumentException contract.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR improves Multik’s ndarray shape-manipulation semantics to return views whenever possible (including for non-consistent layouts when stridable), and standardizes error/exception behavior across engines and IO paths so multik-default behaves consistently at runtime.
Changes:
- Introduces
reshapeStrides/reshapeToto compute view strides forreshape/squeeze/unsqueeze/expandDimsand copy only when required. - Unifies error handling across KE/OpenBLAS engines and IO (eliminates
throw Exception, adds messages that name offending values, aligns LAPACKinfohandling). - Adds targeted regression/contract tests and updates user documentation to reflect view-vs-copy behavior changes.
File summaries
| File | Description |
|---|---|
| multik-openblas/src/nativeMain/kotlin/org/jetbrains/kotlinx/multik/openblas/stat/JniStat.kt | Replaces generic Exception with UnsupportedOperationException for unsupported native median inputs. |
| multik-openblas/src/nativeMain/kotlin/org/jetbrains/kotlinx/multik/openblas/math/JniMath.kt | Standardizes unsupported native argMin/argMax input handling via UnsupportedOperationException. |
| multik-openblas/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/openblas/linalg/NativeLinAlgErrorTest.kt | Adds native-engine exception contract tests (singularity, shape mismatch). |
| multik-openblas/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/openblas/linalg/NativeLinAlgEx.kt | Aligns LAPACK info handling and exception types/messages across native linalg operations. |
| multik-kotlin/src/commonTest/kotlin/org/jetbrains/kotlinx/multik_kotlin/linAlg/KELinAlgErrorTest.kt | Adds KE-engine mirror tests to pin cross-engine exception contracts. |
| multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/math/KEMathEx.kt | Replaces silent fallbacks/empty exceptions with check(...) + actionable messages. |
| multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/pluDecomposition.kt | Improves unsupported dtype error message for PLU. |
| multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/LinAlgEx.kt | Improves unsupported dtype error message for solve. |
| multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/eigen.kt | Improves unsupported dtype error message for eig. |
| multik-kotlin/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/kotlin/linalg/dot.kt | Improves unsupported dtype error messages for dot variants. |
| multik-default/src/wasmJsMain/kotlin/org/jetbrains/kotlinx/multik/default/DefaultEngineFactory.kt | Makes missing native-engine selection on WASM a typed UnsupportedOperationException. |
| multik-default/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/default/DefaultEngineFactory.kt | Uses IllegalArgumentException for missing engine type input on JVM. |
| multik-default/src/jsMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt | Makes missing native-engine selection on JS a typed UnsupportedOperationException. |
| multik-default/src/iosMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt | Makes missing native-engine selection on iOS a typed UnsupportedOperationException. |
| multik-default/src/desktopMain/kotlin/org.jetbrains.kotlinx.multik.default/DefaultEngineFactory.kt | Uses IllegalArgumentException for missing engine type input on desktop native. |
| multik-core/src/jvmTest/kotlin/org/jetbrains/kotlinx/multik/io/IOErrorsTest.kt | Adds IO regression tests for complex NPY rejection, CSV D2 acceptance, and format errors. |
| multik-core/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/api/io/npy.kt | Changes NPY dtype validation to require(dtype.isNumber()) and improves error messaging/KDoc. |
| multik-core/src/jvmMain/kotlin/org/jetbrains/kotlinx/multik/api/io/io.kt | Standardizes IO validation/exceptions/messages for read/write across formats and dims. |
| multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/SliceTest.kt | Updates base expectations to match new “shape ops return views” behavior. |
| multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/ReshapeTest.kt | Adds comprehensive tests for view-vs-copy reshape rules, squeeze/unsqueeze correctness, and axis handling. |
| multik-core/src/commonTest/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/ErrorHandlingTest.kt | Pins error-handling policy and message requirements via unit tests. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt | Switches expandDims/expandNDims to view-based reshape helpers and fixes axis plumbing. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/IteratingNDArray.kt | Replaces generic exceptions with consistent UnsupportedOperationException + dtype-aware messages. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/Slice.kt | Aligns toSlice() invalid-range behavior to IllegalArgumentException with clearer message. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/NDArray.kt | Reworks reshape/squeeze/unsqueeze implementations to use reshapeTo and adds axis validation. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/MultiArrays.kt | Updates public KDoc for reshape/squeeze/unsqueeze to reflect view/copy semantics and rank-0 policy. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/MemoryView.kt | Improves unsupported typed-array access errors with dtype-specific messages. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/Internals.kt | Adds reshapeStrides + reshapeTo and standardizes primitive conversion errors. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/data/DataType.kt | Makes DataType.of(...) and code-to-type mapping throw IllegalArgumentException with clearer messages. |
| multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/api/random.kt | Improves unsupported bound-type error message for random generation. |
| docs/topics/userGuide/shape-manipulation.md | Updates docs to describe reshape/view semantics and size-1 axis ops always being views. |
| docs/topics/userGuide/copies-and-views.md | Updates “typical view ops” list and clarifies reshape behavior. |
| docs/topics/FAQ.md | Revises reshape FAQ to reflect new view semantics and recommends deepCopy() when detaching is required. |
| CLAUDE.md | Documents engine/error-handling policy and project conventions in a consolidated form. |
Review details
Suppressed comments (2)
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt:403
expandDimsfor D2 does not validateaxis; invalid values currently throwIndexOutOfBoundsExceptionfromMutableList.add. For consistency with the new axis-validation policy (invalid arguments →IllegalArgumentExceptionwith a helpful message), add arequirebounds check.
@JvmName("expandDimsD2")
public fun <T> MultiArray<T, D2>.expandDims(axis: Int): MultiArray<T, D3> =
reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D3)
multik-core/src/commonMain/kotlin/org/jetbrains/kotlinx/multik/ndarray/operations/Transformation.kt:412
expandDimsfor D3 still relies onMutableList.add(axis, 1)without checking bounds, so badaxisvalues escape asIndexOutOfBoundsException. Add an explicitrequireto surface anIllegalArgumentExceptionwith a clear message (consistent withunsqueeze).
@JvmName("expandDimsD3")
public fun <T> MultiArray<T, D3>.expandDims(axis: Int): MultiArray<T, D4> =
reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D4)
- Files reviewed: 34/34 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @JvmName("expandDimsD1") | ||
| public fun <T> MultiArray<T, D1>.expandDims(axis: Int): MultiArray<T, D2> { | ||
| val newShape = shape.toMutableList().apply { add(axis, 1) }.toIntArray() | ||
| // TODO(get rid of copying) | ||
| val newData = if (consistent) this.data else this.deepCopy().data | ||
| val newBase = if (consistent) this.base ?: this else null | ||
| val newOffset = if (consistent) this.offset else 0 | ||
| return D2Array(newData, newOffset, newShape, dim = D2, base = newBase) | ||
| } | ||
| public fun <T> MultiArray<T, D1>.expandDims(axis: Int): MultiArray<T, D2> = | ||
| reshapeTo(shape.toMutableList().apply { add(axis, 1) }.toIntArray(), D2) |
reshape, squeeze, unsqueeze and expandDims copied whenever the array was not `consistent`. They now stride the new shape over the existing buffer (NumPy's rule) and copy only when no such strides exist, so writes go through to the source where a copy used to be returned. Bugs found along the way: - squeeze recomputed packed strides, so squeezing a strided sub-array silently returned the wrong elements - expandDims on D4 and expandNDims ignored their axis argument - squeeze of an all-ones shape threw "Shape can't be empty"; it now keeps one axis, like squeeze(0) already did - the NPY guard named ComplexFloat twice, letting ComplexDouble through; write rejected D2 arrays that writeCSV handles and read accepts Error handling: drop 20 bare `throw Exception` and 29 message-less `UnsupportedOperationException()`. Singular matrices and non-convergence now throw ArithmeticException in both engines, which disagreed before and made behaviour depend on the engine multik-default picked. LAPACK `info < 0` blames an argument the wrapper passed, so it is now IllegalStateException. Policy written down in CLAUDE.md. Closes #135, #252, #257
c229a9b to
369d29b
Compare
Closes #135, closes #252, closes #257.
Shape ops return views (#135, #252)
reshape,squeeze,unsqueezeandexpandDimscopied whenever the array was notconsistent.They now compute strides for the new shape over the existing buffer — NumPy's rule — and copy only
when no such strides exist, e.g.
transpose().reshape(6), where axes genuinely have to be merged.One internal helper,
reshapeStrides, backs all of them.Bugs found on the way:
squeezereturned wrong data. It passed the new shape without strides, so they were recomputedas if the array were packed. On a
(2,3,4)array,a[0 until 2, 1 until 2, 0 until 4].squeeze()gave
[4,5,6,7,8,9,10,11]instead of[4,5,6,7,16,17,18,19]— silently, no exception.expandDimson D4 andexpandNDimsignored their axis argument — both calledunsqueeze()with no axes, so they only changed the static type.
squeeze()threw on an all-ones shape,mk.ndarrayOf(5)included. Multik has no rank-0 arrays,so it now keeps one axis and returns
[1]— whatsqueeze(0)on the same array already returned.dtype != ComplexFloat || dtype != ComplexFloat,so
ComplexDoublereached an unchecked cast toNDArray<out Number, *>.writeCSVhas an explicit D2 branch andreadaccepts D1 and D2.Error handling (#257)
The engines disagreed: a singular matrix threw
ArithmeticExceptioninKEEngineand a bareExceptioninNativeEngine.multik-defaultresolves the engine at runtime, so the same user codebehaved differently per platform. Both now throw
ArithmeticException.Also: 20 bare
throw Exception→ 0 (four had empty messages); 29 message-lessUnsupportedOperationException()→ 0, each now naming the operation and dtype; LAPACKinfo < 0reports an argument the wrapper passed, so it is
IllegalStateExceptionrather thanIllegalArgumentException— andsolveCommonhad noinfo < 0check at all, returning garbage on anegative code;
squeeze/unsqueezevalidate axes instead of lettingArrayIndexOutOfBoundsExceptionescape from
MutableList.add. Policy written down inCLAUDE.md.Behaviour changes for the release notes
Writes go through to the source where
reshape/unsqueeze/expandDimsused to return a copy.ExceptionArithmeticExceptioninv/solve/eig: singularity, non-convergenceIllegalStateExceptionArithmeticExceptionsvdnon-convergenceIllegalStateExceptionIllegalArgumentExceptionsvdNaN input,DataType.of,toSlice()IllegalArgumentExceptionIllegalStateExceptioninfo < 0ExceptionIllegalArgumentExceptionread/write,Number.toPrimitiveTypeIllegalStateException[1]squeeze()on an all-ones shapeIllegalArgumentExceptionwriteof a D2 array to CSVIllegalArgumentExceptionwriteof aComplexDoublearray to NPYTesting
2207 tests, 0 failures on JVM, Native (
macosArm64) and iOS simulator(
iosSimulatorArm64) for multik-core, multik-kotlin and multik-openblas. JS and WASM test tasks aredisabled in the build (#247), so those targets are compile-only here.
New tests:
ReshapeTest(view vs copy,reshapeStridesrules, squeeze/unsqueeze axes),ErrorHandlingTest,IOErrorsTest, plus mirroredKELinAlgErrorTest/NativeLinAlgErrorTestpinning the cross-engine exception contract.
apiCheckandkorroCheckpass — the public API is unchanged, the new helpers areinternal.Checklist