gemm.c file

Classes

struct vm_gemm_job: One parallel-for chunk: invoke fn(begin, end, ctx).
struct vm_gemm_ws: Thread-local pack workspace for A/B panels (grows on demand).
struct vm_gemm_batch_ctx: Context for a batched GEMM parallel-for range.

Enums

enum (anonymous) { VM_GEMM_MAX_THREADS = 16, VM_GEMM_THREAD_MIN_FLOPS = 131072, VM_GEMM_THREAD_MIN_PER_WORKER = 2, VM_GEMM_STRIDE_STACK = 32 }

Functions

void vm_gemm_set_threads(const int n): Cap or force the GEMM worker-thread budget.
static auto vm_gemm_hw_threads(void) -> int: Online logical CPU count for GEMM threading defaults.
auto vm_gemm_threads(void) -> int: Resolve the GEMM worker-thread budget.
static auto vm_gemm_pick_threads(const int batch, const int M, const int N, const int K) -> int: Choose how many workers to use for a batched GEMM.
static void vm_gemm_parallel_for(const int n, void(*)(int begin, int end, void*ctx) fn, void* ctx, const int threads): Run fn(begin, end, ctx) over [0, n) using up to threads workers.
static auto vm_gemm_min(const int a, const int b) -> int: Return the smaller of two integers (panel edge clamp).
static auto vm_panel_at(const vm_float_t* M, const int ld, const int r, const int c, const bool trans, const vm_layout_t layout) -> vm_float_t: Read one entry from A or B with optional transpose and layout.
static auto vm_panel_get_c(const vm_float_t* C, const int ldc, const int r, const int c, const vm_layout_t layout) -> vm_float_t: Read C(r, c) according to layout (no transpose).
static void vm_panel_set_c(vm_float_t* C, const int ldc, const int r, const int c, const vm_float_t v, const vm_layout_t layout): Write C(r, c) according to layout (no transpose).
void vm_gemm_ref(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout): Triple-loop reference GEMM.
auto vm_gemm_ukernel_scalar(vm_float_t* acc, const vm_float_t* Ap, const vm_float_t* Bp, const int K) -> VECMAT_SCALAR_API void: Scalar packed MR×NR micro-kernel.
static void vm_gemm_ensure_dispatch(void): Ensure runtime CPU feature detection has run (no-op without dispatch).
static void vm_gemm_ukernel_call(vm_float_t* acc, const vm_float_t* Ap, const vm_float_t* Bp, const int K): Invoke the GEMM micro-kernel for a packed MR×NR tile.
static void vm_gemm_pack_a(vm_float_t* Ap, const vm_float_t* A, const int lda, const int i0, const int k0, const int ib, const int kb, const bool transA, const vm_layout_t layout): Pack an A panel into contiguous MR-wide micro-panels (zero-padded).
static void vm_gemm_pack_b(vm_float_t* Bp, const vm_float_t* B, const int ldb, const int k0, const int j0, const int kb, const int jb, const bool transB, const vm_layout_t layout): Pack a B panel into contiguous NR-wide micro-panels (zero-padded).
static void vm_gemm_store_tile(vm_float_t* C, const int ldc, const vm_float_t* acc, const int i0, const int j0, const int mr, const int nr, const vm_float_t alpha, const vm_float_t beta, const vm_layout_t layout, const int op, const vm_float_t* bias): Write a micro-kernel tile into C with alpha/beta and optional epilogue.
static void vm_gemm_scale_c(vm_float_t* C, const int ldc, const int M, const int N, const vm_float_t beta, const vm_layout_t layout): Apply C = beta * C over an M×N panel (alpha==0 / empty product path).
static auto vm_gemm_alloc_pack(const size_t need) -> vm_float_t*: Allocate a 32-byte-aligned pack buffer for at least need elements.
static void vm_gemm_free_pack(vm_float_t* p): Free a pack buffer from vm_gemm_alloc_pack.
static auto vm_gemm_ws_fit(vm_float_t** buf, size_t* cap, const size_t need) -> vm_float_t*: Grow a pack buffer so it holds at least need elements.
static auto vm_gemm_pack_a_elems(const int ib, const int kb) -> size_t: Element count for an A pack covering ib rows and kb K-steps.
static auto vm_gemm_pack_b_elems(const int jb, const int kb) -> size_t: Element count for a B pack covering jb columns and kb K-steps.
static void vm_gemm_apply_packed(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* Bp, const int M, const int jb, const int kb, const int ic0, const int jc, const int kc, const vm_float_t alpha, const vm_float_t beta_panel, const bool transA, const vm_layout_t layout, const int last_k, const int op, const vm_float_t* bias): Multiply A panels against a pre-packed B panel and store into C.
static void vm_gemm_ex_body(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int op, const vm_float_t* bias): Core blocked GEMM with optional bias/ReLU epilogue.
void vm_gemm_ex(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int op, const vm_float_t* bias): C = alpha * op(A) * op(B) + beta * C with optional fused epilogue.
void vm_gemm(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout): C = alpha * op(A) * op(B) + beta * C for dense float panels.
static void vm_gemm_batch_shared_b(vm_float_t*const* C, const int ldc, const vm_float_t*const* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int begin, const int end): Batched GEMM with one shared B, packing B once per KC/NC panel.
static void vm_gemm_batch_range(const int begin, const int end, void* arg): Runs GEMM for batch indices [begin, end).
void vm_gemm_batch(vm_float_t*const* C, const int ldc, const vm_float_t*const* A, const int lda, const vm_float_t*const* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int batch): Batched GEMM: batch independent GEMMs with shared shape.
void vm_gemm_strided_batch(vm_float_t* C, const int ldc, const int strideC, const vm_float_t* A, const int lda, const int strideA, const vm_float_t* B, const int ldb, const int strideB, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int batch): Strided batched GEMM: problems live strideX elements apart.
void vm_im2col(vm_float_t* col, const int ld_col, const vm_float_t* img, const int n, const int c, const int h, const int w, const int kh, const int kw, const int pad_h, const int pad_w, const int stride_h, const int stride_w, const vm_layout_t layout): NCHW im2col into a GEMM-ready panel.

Variables

static int vm_gemm_thread_limit: GEMM thread budget: 0 = auto, 1 = serial, N = cap at N.
static int vm_gemm_env_threads: Cached VECMAT_GEMM_THREADS (0 = unset / invalid).
static VM_TLS vm_gemm_ws vm_gemm_tls

Function documentation

void vm_gemm_set_threads(const int n)

Cap or force the GEMM worker-thread budget.

Parameters
n	Thread limit (`0` = auto, `1` = serial, `N` = cap at N).

n > 0 sets a fixed limit used by vm_gemm_threads; n == 0 (or negative, treated as 0) restores auto selection via env / hardware.

static int vm_gemm_hw_threads(void)

Online logical CPU count for GEMM threading defaults.

Returns	Hardware thread count estimate (>= 1).

Uses GetSystemInfo on Windows and sysconf(_SC_NPROCESSORS_ONLN) elsewhere. Falls back to 1 if the probe fails or returns a non-positive value.

int vm_gemm_threads(void)

Resolve the GEMM worker-thread budget.

Returns	Requested thread count (>= 1 from hardware fallback when unset).

Order: vm_gemm_set_threads limit if > 0; else a positive integer from VECMAT_GEMM_THREADS (invalid values ignored); else online CPU count.

static int vm_gemm_pick_threads(const int batch, const int M, const int N, const int K)

Choose how many workers to use for a batched GEMM.

Parameters
batch	Number of independent GEMMs.
M	Rows of op(A)/C per problem.
N	Columns of op(B)/C per problem.
K	Inner-product length per problem.
Returns	Thread count to pass to `vm_gemm_parallel_for`.

Returns 1 for a single problem, insufficient items per worker, or tiny total FLOPs (spawn cost dominates). Otherwise takes vm_gemm_threads(), clamped to [1, batch] and VM_GEMM_MAX_THREADS.

static void vm_gemm_parallel_for(const int n, void()(int begin, int end, voidctx) fn, void* ctx, const int threads)

Run fn(begin, end, ctx) over [0, n) using up to threads workers.

Parameters
n	Iteration count (no-op if <= 0).
fn	Callback for a half-open subrange `[begin, end)`.
ctx	Opaque pointer passed through to `fn`.
threads	Requested worker count (clamped; <= 1 forces serial).

Splits the range into roughly equal chunks (capped by n and VM_GEMM_MAX_THREADS). Runs serial when threading is off, threads <= 1, or the range is tiny. Persistent pool workers handle extra chunks; the caller thread always runs the first chunk.

static int vm_gemm_min(const int a, const int b)

Return the smaller of two integers (panel edge clamp).

Parameters
a	First value.
b	Second value.
Returns	The minimum of `a` and `b`.

static vm_float_t vm_panel_at(const vm_float_t* M, const int ld, const int r, const int c, const bool trans, const vm_layout_t layout)

Read one entry from A or B with optional transpose and layout.

Parameters
M	Matrix base pointer.
ld	Leading dimension of M.
r	Logical row in op(M).
c	Logical column in op(M).
trans	If true, treat M as transposed.
layout	Row-major or column-major storage.
Returns	The value at the mapped location.

Logical indices (r, c) are in op(M) space; trans swaps them before applying row- or column-major addressing.

static vm_float_t vm_panel_get_c(const vm_float_t* C, const int ldc, const int r, const int c, const vm_layout_t layout)

Read C(r, c) according to layout (no transpose).

Parameters
C	Output matrix base pointer.
ldc	Leading dimension of C.
r	Row index.
c	Column index.
layout	Row-major or column-major storage.
Returns	C at (r, c).

static void vm_panel_set_c(vm_float_t* C, const int ldc, const int r, const int c, const vm_float_t v, const vm_layout_t layout)

Write C(r, c) according to layout (no transpose).

Parameters
C	Output matrix base pointer.
ldc	Leading dimension of C.
r	Row index.
c	Column index.
v	Value to store.
layout	Row-major or column-major storage.

void vm_gemm_ref(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout)

Triple-loop reference GEMM.

Parameters
C	Output matrix (M×N), updated in place.
ldc	Leading dimension of C.
A	Left input matrix; ignored when alpha == 0 or K <= 0.
lda	Leading dimension of A.
B	Right input matrix; ignored when alpha == 0 or K <= 0.
ldb	Leading dimension of B.
M	Number of rows of op(A) and C.
N	Number of columns of op(B) and C.
K	Inner product length (columns of op(A), rows of op(B)).
alpha	Scale factor for the A*B product.
beta	Scale factor for the existing C values (0 skips reading C).
transA	If true, use A^T; otherwise A.
transB	If true, use B^T; otherwise B.
layout	Memory layout for A, B, and C.

Useful for tests and tiny/fallback paths.

Computes C = alpha * op(A) * op(B) + beta * C. op(X) = X or X^T according to the matching transpose flag. Layout selects row-major or column-major indexing for A, B and C together.

VECMAT_SCALAR_API void vm_gemm_ukernel_scalar(vm_float_t* acc, const vm_float_t* Ap, const vm_float_t* Bp, const int K)

Scalar packed MR×NR micro-kernel.

Parameters
acc	Accumulator tile (MR×NR), updated in place.
Ap	Packed A panel columns (MR values per k).
Bp	Packed B panel rows (NR values per k).
K	Number of packed inner-dimension steps.

Updates acc[i + j*MR] += Ap[k*MR+i] * Bp[k*NR+j] for k in [0, K). Ap / Bp must already be packed with MR/NR padding.

static void vm_gemm_ensure_dispatch(void)

Ensure runtime CPU feature detection has run (no-op without dispatch).

When built with VECMAT_RUNTIME_DISPATCH, calls vm_cpu_init() so the selected GEMM micro-kernel is ready. Otherwise does nothing.

static void vm_gemm_ukernel_call(vm_float_t* acc, const vm_float_t* Ap, const vm_float_t* Bp, const int K)

Invoke the GEMM micro-kernel for a packed MR×NR tile.

Parameters
acc	Accumulator tile (MR×NR), updated in place.
Ap	Packed A micro-panel (MR values per k, MR-padded).
Bp	Packed B micro-panel (NR values per k, NR-padded).
K	Inner-dimension length for this tile.

With VECMAT_RUNTIME_DISPATCH, calls the CPU-selected kernel (vm_gemm_ukernel_); otherwise calls vm_gemm_ukernel_scalar.

static void vm_gemm_pack_a(vm_float_t* Ap, const vm_float_t* A, const int lda, const int i0, const int k0, const int ib, const int kb, const bool transA, const vm_layout_t layout)

Pack an A panel into contiguous MR-wide micro-panels (zero-padded).

Parameters
Ap	Destination pack buffer.
A	Source matrix A.
lda	Leading dimension of A.
i0	Starting row of the panel in op(A).
k0	Starting K index of the panel.
ib	Number of rows to pack.
kb	Number of K steps to pack.
transA	If true, read A as transposed.
layout	Row-major or column-major storage of A.

Layout is k-major within each MR block so the micro-kernel streams linearly.

static void vm_gemm_pack_b(vm_float_t* Bp, const vm_float_t* B, const int ldb, const int k0, const int j0, const int kb, const int jb, const bool transB, const vm_layout_t layout)

Pack a B panel into contiguous NR-wide micro-panels (zero-padded).

Parameters
Bp	Destination pack buffer.
B	Source matrix B.
ldb	Leading dimension of B.
k0	Starting K index of the panel.
j0	Starting column of the panel in op(B).
kb	Number of K steps to pack.
jb	Number of columns to pack.
transB	If true, read B as transposed.
layout	Row-major or column-major storage of B.

Layout is k-major within each NR block so the micro-kernel streams linearly.

static void vm_gemm_store_tile(vm_float_t* C, const int ldc, const vm_float_t* acc, const int i0, const int j0, const int mr, const int nr, const vm_float_t alpha, const vm_float_t beta, const vm_layout_t layout, const int op, const vm_float_t* bias)

Write a micro-kernel tile into C with alpha/beta and optional epilogue.

Parameters
C	Output matrix.
ldc	Leading dimension of C.
acc	MR×NR accumulator from the micro-kernel.
i0	Destination row origin in C.
j0	Destination column origin in C.
mr	Active rows in this tile (<= MR).
nr	Active columns in this tile (<= NR).
alpha	Scale for the accumulator.
beta	Scale for existing C (0 skips the read).
layout	Row-major or column-major storage of C.
op	Epilogue flags (`VM_GEMM_OP_*`); `VM_GEMM_OP_NONE` skips bias/ReLU.
bias	Optional length-N bias vector (used with `VM_GEMM_OP_BIAS`).

Computes C = alpha * acc + beta * C. When op != VM_GEMM_OP_NONE, also adds per-column bias and/or applies ReLU as requested. Callers pass VM_GEMM_OP_NONE on non-final K panels so the epilogue runs only once.

static void vm_gemm_scale_c(vm_float_t* C, const int ldc, const int M, const int N, const vm_float_t beta, const vm_layout_t layout)

Apply C = beta * C over an M×N panel (alpha==0 / empty product path).

Parameters
C	Output matrix to scale in place.
ldc	Leading dimension of C.
M	Number of rows.
N	Number of columns.
beta	Scale factor (0 zeros the panel).
layout	Row-major or column-major storage of C.

static vm_float_t* vm_gemm_alloc_pack(const size_t need)

Allocate a 32-byte-aligned pack buffer for at least need elements.

Parameters
need	Required element count.
Returns	Aligned buffer pointer, or NULL on allocation failure.

Rounds the byte size up to a multiple of 32 (minimum 32 bytes).

static void vm_gemm_free_pack(vm_float_t* p)

Free a pack buffer from vm_gemm_alloc_pack.

Parameters
p	Buffer pointer returned by `vm_gemm_alloc_pack` (or NULL).

static vm_float_t* vm_gemm_ws_fit(vm_float_t** buf, size_t* cap, const size_t need)

Grow a pack buffer so it holds at least need elements.

Parameters
buf	In/out pointer to the buffer (may be reallocated).
cap	In/out capacity in elements.
need	Required element count.
Returns	Pointer to a buffer with capacity >= need, or NULL on OOM.

Reuses *buf when *cap >= need; otherwise allocates a new aligned buffer, frees the old one, and updates *buf / *cap.

static size_t vm_gemm_pack_a_elems(const int ib, const int kb)

Element count for an A pack covering ib rows and kb K-steps.

Parameters
ib	Rows in the A panel.
kb	K-panel width.
Returns	Number of `vm_float_t` elements required (MR-padded).

static size_t vm_gemm_pack_b_elems(const int jb, const int kb)

Element count for a B pack covering jb columns and kb K-steps.

Parameters
jb	Columns in the B panel.
kb	K-panel width.
Returns	Number of `vm_float_t` elements required (NR-padded).

static void vm_gemm_apply_packed(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* Bp, const int M, const int jb, const int kb, const int ic0, const int jc, const int kc, const vm_float_t alpha, const vm_float_t beta_panel, const bool transA, const vm_layout_t layout, const int last_k, const int op, const vm_float_t* bias)

Multiply A panels against a pre-packed B panel and store into C.

Parameters
C	Output matrix.
ldc	Leading dimension of C.
A	Left matrix (or NULL to skip).
lda	Leading dimension of A.
Bp	Pre-packed B panel for this (jc, kc) block.
M	Rows of op(A)/C to process from `ic0`.
jb	Columns in the packed B panel.
kb	K width of the packed panel.
ic0	Row origin of this A/C slab in the full problem.
jc	Column origin in C / op(B).
kc	K origin used when packing A.
alpha	Scale for the product tiles.
beta_panel	Beta for this K panel (beta on first panel, else 1).
transA	If true, read A as transposed.
layout	Storage layout for A and C.
last_k	Nonzero if this is the final K panel (enable epilogue).
op	Epilogue flags applied on last K (`VM_GEMM_OP_*`).
bias	Optional bias vector for `VM_GEMM_OP_BIAS`.

Walks M in MC blocks, packs each A slab, runs the MR×NR micro-kernel, and writes tiles. Epilogue ops run only when last_k is set. On thread-local A-pack allocation failure, returns without updating C (no fallback).

static void vm_gemm_ex_body(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int op, const vm_float_t* bias)

Core blocked GEMM with optional bias/ReLU epilogue.

Parameters
C	Output matrix (M×N), updated in place.
ldc	Leading dimension of C.
A	Left input matrix; ignored when alpha == 0 or K <= 0.
lda	Leading dimension of A.
B	Right input matrix; ignored when alpha == 0 or K <= 0.
ldb	Leading dimension of B.
M	Number of rows of op(A) and C.
N	Number of columns of op(B) and C.
K	Inner product length.
alpha	Scale factor for the A*B product.
beta	Scale factor for existing C on the first K panel.
transA	If true, use A^T; otherwise A.
transB	If true, use B^T; otherwise B.
layout	Memory layout for A, B, and C.
op	Epilogue flags (`VM_GEMM_OP_NONE`, bias, ReLU, …).
bias	Optional length-N bias; used when `op` includes bias.

Packs B by NC×KC panels, then applies packed A×B tiles. On allocation failure falls back to vm_gemm_ref (without epilogue). When the product is empty (alpha==0 / missing inputs / K==0), scales C and still applies op.

void vm_gemm_ex(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int op, const vm_float_t* bias)

C = alpha * op(A) * op(B) + beta * C with optional fused epilogue.

Parameters
C	Output matrix (M×N), updated in place.
ldc	Leading dimension of C.
A	Left input matrix; ignored when alpha == 0 or K <= 0.
lda	Leading dimension of A.
B	Right input matrix; ignored when alpha == 0 or K <= 0.
ldb	Leading dimension of B.
M	Number of rows of op(A) and C.
N	Number of columns of op(B) and C.
K	Inner product length.
alpha	Scale factor for the A*B product.
beta	Scale factor for the existing C values.
transA	If true, use A^T; otherwise A.
transB	If true, use B^T; otherwise B.
layout	Memory layout for A, B, and C.
op	Epilogue flags (`VM_GEMM_OP_*`).
bias	Optional length-N bias vector (may be NULL).

Same blocked packed path as vm_gemm, plus last-K bias add and/or ReLU when requested via op.

void vm_gemm(vm_float_t* C, const int ldc, const vm_float_t* A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout)

C = alpha * op(A) * op(B) + beta * C for dense float panels.

Parameters
C	Output matrix (M×N), updated in place.
ldc	Leading dimension of C.
A	Left input matrix; ignored when alpha == 0 or K <= 0.
lda	Leading dimension of A.
B	Right input matrix; ignored when alpha == 0 or K <= 0.
ldb	Leading dimension of B.
M	Number of rows of op(A) and C.
N	Number of columns of op(B) and C.
K	Inner product length.
alpha	Scale factor for the A*B product.
beta	Scale factor for the existing C values.
transA	If true, use A^T; otherwise A.
transB	If true, use B^T; otherwise B.
layout	Memory layout for A, B, and C.

Blocked packed kernel with thread-local A/B workspaces. Equivalent to vm_gemm_ex with op == VM_GEMM_OP_NONE.

static void vm_gemm_batch_shared_b(vm_float_tconst C, const int ldc, const vm_float_tconst A, const int lda, const vm_float_t* B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int begin, const int end)

Batched GEMM with one shared B, packing B once per KC/NC panel.

Parameters
C	Array of `batch` output matrix pointers.
ldc	Leading dimension shared by every C[p].
A	Array of `batch` left matrix pointers (per-item A).
lda	Leading dimension shared by every A[p].
B	Shared right matrix for all batch items.
ldb	Leading dimension of B.
M	Rows of op(A) and C (same for all problems).
N	Columns of op(B) and C (same for all problems).
K	Inner product length (same for all problems).
alpha	Scale factor for each A*B product.
beta	Scale factor for each existing C[p].
transA	If true, use A^T for every problem.
transB	If true, use B^T for the shared B.
layout	Memory layout for all panels.
begin	First batch index (inclusive).
end	Last batch index (exclusive).

Heap-allocates a B pack for this batch index range. Each KC×NC panel of the shared B is packed once, then applied to every item in [begin, end). Each item still packs its own A via thread-local workspace. On pack allocation failure, falls back to per-item vm_gemm.

static void vm_gemm_batch_range(const int begin, const int end, void* arg)

Runs GEMM for batch indices [begin, end).

Parameters
begin	First batch index (inclusive).
end	Last batch index (exclusive).
arg	Pointer to a `vm_gemm_batch_ctx`.

void vm_gemm_batch(vm_float_tconst C, const int ldc, const vm_float_tconst A, const int lda, const vm_float_tconst B, const int ldb, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int batch)

Batched GEMM: batch independent GEMMs with shared shape.

Parameters
C	Array of `batch` pointers to output matrices (each M×N).
ldc	Leading dimension shared by every C[p].
A	Array of `batch` pointers to left matrices, or NULL.
lda	Leading dimension shared by every A[p].
B	Array of `batch` pointers to right matrices, or NULL.
ldb	Leading dimension shared by every B[p].
M	Number of rows of op(A) and C (same for all problems).
N	Number of columns of op(B) and C (same for all problems).
K	Inner product length (same for all problems).
alpha	Scale factor for each A*B product.
beta	Scale factor for each existing C[p].
transA	If true, use A^T for every problem; otherwise A.
transB	If true, use B^T for every problem; otherwise B.
layout	Memory layout for all A, B, and C panels.
batch	Number of independent GEMM problems.

A[p], B[p], C[p] are the p-th problem. If every B[p] aliases the same buffer, uses a shared-B pack path; otherwise runs per-item vm_gemm. The batch range is split across worker threads when the work is large enough.

void vm_gemm_strided_batch(vm_float_t* C, const int ldc, const int strideC, const vm_float_t* A, const int lda, const int strideA, const vm_float_t* B, const int ldb, const int strideB, const int M, const int N, const int K, const vm_float_t alpha, const vm_float_t beta, const bool transA, const bool transB, const vm_layout_t layout, const int batch)

Strided batched GEMM: problems live strideX elements apart.

Parameters
C	Base pointer for output matrices; problem p at C + p*strideC.
ldc	Leading dimension shared by every C panel.
strideC	Element stride between consecutive C problems.
A	Base pointer for left matrices, or NULL; problem p at A + p*strideA.
lda	Leading dimension shared by every A panel.
strideA	Element stride between consecutive A problems.
B	Base pointer for right matrices, or NULL; problem p at B + p*strideB.
ldb	Leading dimension shared by every B panel.
strideB	Element stride between B problems (0 means one shared B).
M	Number of rows of op(A) and C (same for all problems).
N	Number of columns of op(B) and C (same for all problems).
K	Inner product length (same for all problems).
alpha	Scale factor for each A*B product.
beta	Scale factor for each existing C panel.
transA	If true, use A^T for every problem; otherwise A.
transB	If true, use B^T for every problem; otherwise B.
layout	Memory layout for all A, B, and C panels.
batch	Number of independent GEMM problems.

Problem p uses A + p * strideA, B + p * strideB, C + p * strideC. When strideB == 0 (shared B), reuses the shared-B batch path.

void vm_im2col(vm_float_t* col, const int ld_col, const vm_float_t* img, const int n, const int c, const int h, const int w, const int kh, const int kw, const int pad_h, const int pad_w, const int stride_h, const int stride_w, const vm_layout_t layout)

NCHW im2col into a GEMM-ready panel.

Parameters
col	Destination matrix for unfolded patches.
ld_col	Leading dimension of `col` under `layout`.
img	Source image tensor in NCHW order.
n	Batch size (N).
c	Channel count (C).
h	Input height (H).
w	Input width (W).
kh	Kernel height.
kw	Kernel width.
pad_h	Top/bottom padding in pixels.
pad_w	Left/right padding in pixels.
stride_h	Vertical kernel stride (<= 0 treated as 1).
stride_w	Horizontal kernel stride (<= 0 treated as 1).
layout	Row-major or column-major storage for `col`.

img is N×C×H×W packed as ((n*C+c)*H+y)*W+x. Output has C*kH*kW rows and N*outH*outW columns (zero-filled off-image taps). outH/outW use standard floor division with the given pads and strides; non-positive stride_h / stride_w are treated as 1.