fix(value): [OBE-10735] consolidate and raise the array-index cap

[JuanMantica45]

Why

Supersedes #12, which is conflicting with main. Same substance, rebased and self-contained.

main declares the bound twice — MAX_ARRAY_INDEX = 32_768 (crud/mod.rs) enforces it, MAX_ARRAY_CAPACITY = 32_769 (crud/insert.rs) preallocates for it. Two constants for one invariant means they can drift. This collapses them into one pub(super) constant and raises it to 2^20, per @ajayshekar-s1's feedback on #7 that 32_768 could reject legitimate large-array use.

The cap stays, and it cannot be optimised away

The review question on #7 was whether the cap should exist at all. It has to, and not for the reason the original fix implied.

Assigning to index N materialises N + 1 elements. The null padding is observable VRL semantics, not a preallocation hint — length() sees it, and test_insert_array already asserts c[2] = 10 yields [5, null, 10]. So removing Vec::with_capacity would only trade one large allocation for amortised doubling; the array still ends up holding N + 1 elements. There is no version of this where an event-controlled index does not commit memory proportional to the index. Bounding it is the only fix that does not change the language.

2^20 bounds a single indexed write to ~42 MB at today's 40-byte Value — 32x more headroom than the original cap, while staying bounded rather than unbounded.

What changed

• One pub(super) const MAX_ARRAY_INDEX = 1_048_576 in crud/mod.rs; insert.rs derives its capacity bound as MAX_ARRAY_INDEX + 1 instead of redeclaring it. grep MAX_ARRAY_CAPACITY src/ now returns nothing.
• index.unsigned_abs() replaces (-index) as usize in the capacity calculation, which overflows on isize::MIN (no positive isize counterpart). insert_value already used unsigned_abs; this was the remaining call site. Found by @jsbalis1 in review of fix(security): prevent 9 panic/OOM vectors in VRL runtime (batch J) #7.
• test_value_size_is_pinned asserts size_of::<Value>() is 40. It is a drift detector, not a correctness assertion: the cap is justified in terms of memory (cap x size), so a new Value variant should force a human to re-check the budget rather than silently changing it.

Test plan

• cargo test --lib: 1761 passed, 0 failed.
• New/updated in crud::insert: paired accept-at-2^20 / reject-at-2^20+1 (both signs), isize::MIN no-panic, and the size_of pin. The accept-at-cap and isize::MIN tests both fail against main — the latter with attempt to negate with overflow at insert.rs:33.
• lib/tests fixture runner: new obe 10735 array index cap passes. Suite goes 762 -> 763 passed; the 2 parse_etld custom-PSL failures and 3 emit_metric clippy errors are present on unmodified main and are untouched here.

🤖 Generated with Claude Code

[jsbalis1]

Even for a rejection we're still creating a 42MB array here. so an out-of-range index still reserves the full capacity.
Same thing as above but for negative index, and worse since this isn't even a rejected case. Lets fix this

[JuanMantica45]

Good catch, fixed in 4e502bd.

Root cause: insert_value (crud/mod.rs) already validates the index against MAX_ARRAY_INDEX before allocating anything, and already does its own correctly-sized allocation once the range check passes (a push growth loop for positive indices, Self::with_capacity(len_required) for negative ones — which fully replaces *self). So the Vec::with_capacity(capacity) here was redundant at best and wasteful at worst:

• Out-of-range writes (either sign) still paid for the full ~42MB with_capacity before insert_value rejected them and returned None.
• In-range negative-index writes paid for it twice: this allocation was immediately discarded by insert_value's *self = extended.

Replaced the capacity computation with plain Vec::new() and let insert_value own the one allocation it actually needs. Added test_insert_beyond_max_array_index_does_not_preallocate, which asserts a rejected out-of-range write leaves array.capacity() == 0.

[JuanMantica45]

There is already validation for max capacity here

fn insert_value(&mut self, key: isize, value: Value) -> Option<Value> {
115:     let max_index = MAX_ARRAY_INDEX as isize;
116:     if !(-max_index..=max_index).contains(&key) {
117:         // TODO: VRL-side array-index assignment is currently infallible (see
118:         // compiler::expression::assignment::Target::insert), so we can't surface this as a
119:         // proper VRL runtime error without a larger change. Log it so it's at least
120:         // observable instead of a silent no-op.
121:         tracing::warn!(
122:             index = key,
123:             max_index = MAX_ARRAY_INDEX,
124:             "array index assignment out of range, write dropped"
125:         );
126:         return None;
127:     }

[JuanMantica45]

Incremented to give more headroom