Conversation
Reusable benchmark suite package for TensorOperations.jl, living in benchmark/. Core pieces: pure-data contraction specs (AddSpec/TraceSpec/ ContractSpec/NetworkSpec) decoupled from an AbstractProvider interface so downstream packages can plug in their own tensor type; a category registry; analytical flop/byte cost model; and independent thread-count configuration that never pollutes a timed sample. Ships with generic pairwise/permutation/ trace categories. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mixed_precision: differing input/output element types (e.g. Float32 x Float32 -> Float64, mixed real/complex), exercising promote_add/ promote_contract. mps: the MPS/MPO DMRG effective-Hamiltonian motif (1-site and 2-site "theta" variants), swept over bond dimension D -- the highest-value tensor-network contraction pattern from the literature review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
benchmarks.jl builds SUITE for PkgBenchmark, applying thread config once via set_threads! before building it. run_benchmarks.jl/show_benchmarks.jl (ArgParse-based CLIs) drive PkgBenchmark runs and plot GFLOP/s-vs-size curves from the results. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers suite construction/execution for every category, resultstable's flop/byte joins, the network cost model, thread-config scoping, and ArrayProvider's reproducible-but-varying randtensor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
An all-power-of-two sweep (8,32,64,128,256) hides alignment/padding/ vectorization-boundary effects. Mix in off-by-one and arbitrary sizes (15, 63, 96, 200) for pairwise/permute/trace, and realistic (non-power- of-two) DMRG bond dimensions (100, 300) for mps. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
24 real contraction patterns (CCSD, CCSD(T), AO2MO, INTENSLI) sourced from the TCCG benchmark (github.com/HPAC/tccg), a source of real high-rank/irregular-permutation shapes rather than hand-picked ones. Sizes each index uniformly, mirroring TCCG's own approach. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Cut multi-paragraph file-header prose down to one or two lines across all source, script, and test files. No behavior change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Member
|
Yes... hahaha... YES!!! I'll try to take a look but this is exciting |
ctmrg: corner-growth step (C-T-T-a) of the boundary-MPS/CTMRG method for 2D PEPS, swept over environment bond chi, cost ~O(chi^3*D^6). trg: plaquette contraction (4-ring of rank-3 tensors from splitting neighboring TRG tensors), swept over bond chi, cost ~O(chi^6) -- confirmed numerically (6x chi -> ~45500x flops vs theoretical 46656x). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@__DIR__ followed directly by an infix operator is ambiguous macro-call syntax that Runic's parser rejects (base Julia's parser happens to accept it). Use joinpath(@__DIR__, "..") instead, which is also more portable (Windows CI runs this repo too). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
TCCG cases were structurally just ContractSpec, same as :pairwise -- no separate execution path or spec type justified a separate category. Tag each case's params.source (:synthetic/:tccg) instead, and add a casefilter predicate to build_suite to select either subset without a separate top-level category. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s tags Each synthetic pairwise shape now generates up to 4 label-order layouts (gemm_ready/a_permuted/b_permuted/both_permuted): gemm_ready reshapes directly to a BLAS call, the others interleave open/contracted labels so no reshape or transpose flag suffices -- forcing a real permutation, which is what actually separates StridedNative from StridedBLAS. Structurally-redundant layouts (e.g. shapes with <=1 contracted index) are deduplicated automatically. Replace the bespoke `casefilter` predicate with BenchmarkTools' own `@tagged` mechanism: each case is now wrapped as a tagged BenchmarkGroup (category + every Symbol-valued params entry, via the new `casetags`), so `suite[@tagged "tccg"]` or `suite[@tagged "pairwise" && "both_permuted"]` filter natively, including boolean combinations, on the built suite -- no bespoke filtering API to learn or maintain. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Wrap run_benchmarks.jl/show_benchmarks.jl's logic in a main(args) function marked @main (Julia 1.11+), so ARGS flow through the normal argument-parsing path instead of being read as a top-level side effect. Bumps the julia compat bound to 1.11 accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
:contract better matches TensorOperations.jl's own tensorcontract API naming than the more incidental "pairwise" description. mps/ctmrg/trg were all NetworkSpec-based motifs with no structural difference beyond which topology they encode -- merged into one :network category, distinguished via params.topic (:mps/:ctmrg/:trg), filterable through the same @tagged mechanism already used for :contract's synthetic/tccg split. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- BenchmarkCase now stores tags explicitly (computed once at construction) rather than re-deriving them via a separate casetags function; suite.jl uses case.tags directly. - within_memory_budget(spec, id) warns (naming id) when skipping an oversized case, so a sparse sweep is diagnosable, not silent. MAX_CASE_BYTES now scales with host memory (Sys.total_memory() ÷ 64) rather than a fixed 256 MiB. - lowering.jl's maketensors/execute tuples reordered to match tensorcopy!/tensortrace!/tensorcontract!'s own (C, A, ...) argument order. - AddSpec/TraceSpec/ContractSpec/NetworkSpec get compact einsum-style show methods, e.g. `ContractSpec: C[i,j] = A[i,k] * B[k,j] (dim=64)`. - resultstable now returns a DataFrame (added DataFrames dependency) instead of a bespoke ResultRow/Vector; show_benchmarks.jl updated to iterate via eachrow. - @main moved onto the function definition itself (function (@main)(args) ... end), the documented placement, rather than a trailing bare @main. - README: dropped the "(v1)" version qualifier, and removed the BenchmarkCase-vs-BenchmarkTools writeup in favor of the equivalent (already up to date) docstrings in registry.jl/suite.jl. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmark/package: pure-data contraction specs + a smallAbstractProviderinterface, so downstream packages (e.g. a symmetric-tensor package) can plug in their own tensor type and run the same suite.pairwise,permute,trace,mixed_precision,mps(DMRG effective-Hamiltonian motif). CTMRG/TRG/MERA are follow-ups, addable without a redesign.set_threads!/with_threads) is applied once at the process level, never per-sample.Test plan
Pkg.test()passes (18 tests)🤖 Generated with Claude Code