Prepare v2.2.0 release - #147
Merged
Merged
Conversation
New append-only UnivariateFunctionType enum and UnivariateFunctionFactory (CreateFunction by type, CreateFromXElement dispatching on the element local name, and GetFunctionType for live instances), mirroring the LinkFunctionFactory and UnivariateDistributionFactory patterns. Concrete ToXElement instance methods with static FromXElement counterparts land on LinearFunction, PowerFunction, and TabularFunction - deliberately NOT on IUnivariateFunction: external implementors of the interface exist and net481 has no default interface members, so an interface member would be a breaking change. G17 invariant-culture doubles; ConfidenceLevel is runtime sampling state and never serializes; PowerFunction.Minimum derives from Xi and never serializes; TabularFunction embeds its UncertainOrderedPairedData via SaveToXElement. Round-trip and dispatch tests cover all three types and the factory guards.
Q(h) = sum over controls of 10^(log10 alpha_k) (h - h_k)^beta_k for h > h_k, zero at and below the main-channel cease-to-flow stage h1, with a log10-space Gaussian residual applied at the confidence level - the exact body and stochastic form of the RMC-BestFit rating-curve Predict. The parameter-vector layout is the BestFit layout [h1, log10a1, b1, ..., sigma] (3 segments + 1), so a fitted posterior ParameterSet applies directly through SetParameters; breakpoints must be strictly ordered and exponents non-negative (the monotone-rating constraint the numeric Brent inverse relies on). One segment degenerates to PowerFunction. Parity constants in the tests were evaluated independently from the closed form; serialization rides the function factory (Minimum derives from h1 and ConfidenceLevel is runtime state - neither serializes).
A weighted combination over IUnivariateFunction children: the pointwise weighted average, and a mixture whose single confidence-level draw selects a child by cumulative weight and re-scales the remainder as the child draw - deterministic composition sampling with no internal random source (the math behind composite risk input functions). Weights are the composite parameters (non-negative, sum to one); children own and re-validate their own parameters at evaluation, so the composite validity flag deliberately does not fold in the latent child-flag quirk where a deterministic two-argument LinearFunction reports invalid until first evaluation. Mean convention outside [0,1] evaluates the weighted average of child means in both modes; a multi-branch mixture reports nondeterministic even over deterministic children. Serialization embeds children through their own forms and reconstructs through the function factory (nesting supported).
A template function plus posterior ParameterSet draws, sampled by index (range-checked) or percentile (min(floor(u*N), N-1)) as fresh configured clones - pure by construction: the template is captured as an immutable serialized snapshot at construction and every sample re-materializes through the function factory before applying its parameter set, so concurrent realizations share no mutable state (the thread-safe alternative to mutating ConfidenceLevel/SetParameters inside Parallel.For hot loops; pinned by a parallel no-shared-mutation test). This is the vehicle for imported fitted functions (BestFit rating curves) carrying knowledge uncertainty into a simulation engine. Serialization embeds the template form plus every parameter set.
Both types reported parameter property names without scalar parameter values, so the inherited scalar-only ToXElement could not even complete - the X/probability tables and the kernel sample were lost entirely. Each now overrides ToXElement (tables and sample data as G17 invariant-culture lists, interpolation transforms, the probability sort order for exceedance-convention ladders, kernel type, bandwidth, and the optional per-sample weights) with a static FromXElement counterpart, wired into UnivariateDistributionFactory.CreateDistribution alongside the existing Mixture/CompetingRisks/PertPercentile special cases. Round-trip tests cover ascending and descending probability conventions, evaluation equality, weighted kernels, and the loud missing-table guards.
An optional Recorder callback (x, weight, f(x)) that fires only when an interval is ACCEPTED into the final composite rule - the design that makes a weight-aware adaptive quadrature work at all: a naive per-evaluation weight hand-off double-counts, because a rejected interval's 21 evaluations are superseded by its children's and must never carry measure. Each evaluation caches its interval's nodes and values (only while a recorder is attached; null leaves the integration path unchanged with zero overhead, pinned bit-for-bit), and acceptance flushes them with half-length-scaled Kronrod weights. Two structural identities follow and are pinned under forced deep subdivision: the recorded weights sum to exactly the integration domain's width (any double count would overshoot), and the weighted node sum reproduces the returned integral to floating-point reassociation - so a consumer can take loss-exceedance probability mass directly from the quadrature. Works identically through the whole-interval and stratified-bin forms; G10K21 nodes are strictly interior, so adjacent intervals never repeat an abscissa.
ConvolveDiscrete convolves point-mass (atom) distributions EXACTLY on a shared uniform lattice - the form the continuous-PDF pipeline cannot represent (a zero-inflation atom of a defective risk curve is a CDF jump with no density): atoms deposit with a moment-preserving two-node split, the mass vectors convolve by FFT, ringing floors at zero, and the total renormalizes to the product of the input totals. Pinned against exact enumeration: total mass, the convolved mean = sum of input means (exact by the split), interior cumulative probes, and the joint zero atom of two defective curves. The new logSpacedOutput overload keeps the existing linear-grid FFT pipeline byte-identical (false delegates to it directly) and re-reads the convolved CDF on a log-spaced ladder for order-of-magnitude supports, with a loud guard on non-positive support. The existing five continuous Convolve tests pass unchanged.
The upstream unit-test half of the engine-level gamma audit: a known
heavy-upper-tail integrand (21(1-p)^20, integral exactly one) must stay
unbiased at gamma in {1, 4, 10}, and the weights handed to the
integrand must sum to the domain volume per evaluation batch at every
gamma - a missing Jacobian would distort both by the measure change,
an order-of-magnitude error at gamma = 10. Seeded MersenneTwister with
the Sobol default off, so the runs are deterministic.
The pooled combination-enumeration path allocated its output lists and every indicator row per call - a per-evaluation cost in joint-risk hot loops. The new overload takes caller-owned output lists, refills existing row arrays of matching length in place (steady-state zero output allocations, pinned by reference-identity tests across calls), trims stale tails from larger prior calls, and returns TRUE when the inclusion-exclusion expansion converged early - surfacing the previously silent truncation of the deepest combinations behind the closing pseudo-row. The allocating out-parameter overload now delegates to it, so outputs are identical by construction (pinned for full and truncated enumerations). The sibling exclusive kernels (PositivelyDependentExclusive/ExclusivePCM/ExclusiveMVN) follow the same recipe when their call sites go pooled.
MVNDST is a randomized lattice rule -- it draws from MultivariateNormal's own generator to randomize its quadrature, so the returned probability carries a small stochastic error and two calls with different generator states do not agree bit-for-bit. That generator defaulted to `new MersenneTwister()`, whose parameterless constructor seeds from DateTime.UtcNow.Ticks. Every CDF and Interval evaluation above two dimensions was therefore irreproducible across runs. It stayed hidden because the error sits near the requested tolerance, so statistical tests absorb it. RMC-TotalRisk found it downstream: dependent competing-risks incidence curves run through Interval once per unit per hazard level, and a results hash over that path changed on every run. MVNUNI now defaults to the fixed DefaultMVNUNISeed, and CompetingRisks gains a PRNGSeed applied when it builds its multivariate normal, so callers deriving seeds from model content can tie the result to the model instead of to a shared constant. Both are documented, including MVNUNI's thread-safety constraint (MVNDST advances it, so a shared instance needs a clone per thread, as JointProbabilitiesMVN already does). Gates: build 0 warnings; 1943/1943 on net10.0, net9.0, net8.0 and net481.
The reductions over bootstrap replications merged thread-local partials through Tools.ParallelAdd, whose partitioning follows the scheduler. Results therefore varied in their last bits between runs and between machines -- including Estimate's mean curve, which feeds a monotonic filter that can admit a different number of interpolation knots from a last-bit difference. Every such reduction now splits its work into a fixed number of chunks, each summed sequentially and merged in chunk order, so the association is independent of the thread count: ExpectedProbabilities (both overloads), UncertaintyAnalysisResults.ProcessMeanCurve, the P0 proportions in BiasCorrectedQuantileCI and BCaQuantileCI, AccelerationConstants, StandardError, and Statistics.JackKnifeStandardError. Parallelism is retained throughout. Defects found while reworking the class: - ExpectedProbabilities summed over the successfully fitted distributions but divided by the full replication count, biasing the expectation toward zero in proportion to the failure rate. Both now exclude failures. - Fit failures were silent. Distributions() records FailedReplications, so a caller can tell when results rest on fewer samples than requested. - The NaN guards in BiasCorrectedQuantileCI and BCaQuantileCI were written `x != double.NaN`, which is always true in IEEE-754, so the intended rejection never ran. Harmless, since the following comparison is false for NaN, but now correct. - Estimate's quantile ladder accumulated Log10(previous + shift) + delta, so rounding compounded across up to a thousand bins. Each ordinate is now computed from the origin, matching ProcessMeanCurve. - Quantiles() and Probabilities() could not accept an existing set of bootstrapped distributions, so a caller wanting both paid for the whole bootstrap twice. - ComputeMinMaxQuantiles took a lock per distribution; it now merges thread-local extremes once per partition. - The jackknife in AccelerationConstants, StandardError, JackKnifeSample and JackKnifeStandardError copied the whole sample into a fresh list per point. Each chunk now refills one leave-one-out buffer. Adds two pins: Estimate is bit-identical across calls at the same seed, and ExpectedProbabilities is bit-identical when one arm is constrained to a single worker thread. Gates: build 0 warnings; 1945/1945 on net10.0, net9.0, net8.0 and net481.
The exclusive expansions read their combinations from a dense n-by-(2^n - 1) indicator matrix built by Factorial.AllCombinations. That matrix is the binding constraint on dimension long before the arithmetic is: it is 80 MB at twenty events and 3.3 GB at twenty-five, and it refuses to build at all past thirty. Callers therefore carried dimension caps that had nothing to do with the model. Factorial gains NextCombination and AllCombinationsLazy, which generate the same rows in the same order -- subset size ascending, then lexicographic -- allocating nothing per row and imposing no upper bound on n. Probability gains IndependentExclusiveLazy on top of them, with caller-owned output buffers and an ExclusiveEnumerationStatus distinguishing a completed expansion from one the inclusion-exclusion bracket closed early and one that stopped at a caller-supplied cap. Where the dense form runs, the lazy form emits bit-identical probabilities and identical indicator rows. On convergence it closes with the same half-gap row; at the cap it closes with the exact residual 1 - sum(emitted), which for independent events is the mass of everything not enumerated. The cap is a backstop, not the operating mechanism. The bracket cannot be tested before the third subset size, so the floor is n + C(n,2) + C(n,3), and at the failure probabilities a risk model carries it closes at or just after that: a forty-event expansion enumerates around 10^5 of its 10^12 combinations. Gates: build 0 warnings; 1951/1951 on net10.0, net9.0, net8.0 and net481.
The constructor refused more than twenty dimensions. Nothing structural required that: every internal array sizes from Dimensions, and the Sobol sequence supports 21,201. It was also stricter than the reference implementations -- GSL, Cuba and Lepage's vegas impose no dimension cap at all, and Lepage's largest documented example is itself twenty-dimensional. The algorithm already degrades the way those implementations rely on. The importance-sampling grid is separable, so it holds NumberOfBins x D bins rather than bins^D, and the stratification self-limits: strata per axis are (calls/2)^(1/D), which reaches one around fifteen dimensions, after which the run is pure adaptive importance sampling. At twenty dimensions with ten thousand calls it is already in that regime, so raising the guard changes nothing about how it behaves there. Kept as a guard rather than removed, so a runaway input still fails fast. The new test integrates the mean of 2*x_i over thirty dimensions. A product of the same factors would concentrate its mass in one corner and is hopeless at that dimension for any sample budget -- the curse of dimensionality rather than a property of the integrator -- which is worth knowing before reaching for high dimensions. Gates: build 0 warnings; 1955/1955 on net10.0, net9.0, net8.0 and net481.
With a recorder attached, every interval allocated two 21-element buffers to hold its nodes and values -- including the rejected intervals, which are the majority under adaptive refinement. That is a per-evaluation allocation introduced purely by observing the integration. The buffers now come from a per-slot pool grown on demand. Depth-first recursion means the only intervals alive at once are the two halves of the current interval and their ancestors, so a slot index of 2*level+1 and 2*level+2 is unique among live intervals and every subtree reuses the same slots after its sibling finishes. Nothing is allocated when no recorder is attached. Gates: build 0 warnings; 1955/1955 on all four TFMs, including the recorder invariants -- weights summing to the domain width and the weighted node sum reproducing the result -- under forced deep subdivision.
ProcessParameterSets left a null entry wherever a sampled distribution had failed to fit, while BootstrapAnalysis.ParameterSets filled the same slot with a NaN-valued set. ParameterSets is a public array that consumers index directly, so the two paths differed in whether a failed replication throws or propagates. Both now fill NaN, taking the parameter count from the parent distribution. Also merges the mean curve's min/max extremes once per worker instead of once per distribution. Min and max are order-independent, so the result is unchanged however the loop partitions.
The recorder must flush the frozen composite rule on budget and depth exhaustion, not only on success, with weights summing to the domain area and weighted values reproducing the result on every non-throwing outcome, and report nothing when the integrand throws. Consumers adopting the recorded mass as an exhaustive partition rely on exactly these behaviors.
Stabilize probabilities, support, L-moments, moments, modes, and quantile derivatives. Reject unsuccessful MLE results and add independent oracle regressions. Release build has zero warnings and errors; all 32 K4 tests pass on each target framework. Haden approved committing with the documented unrelated BOM HTTP 500 test failure.
Restores the pre-serialization assembly attribute Parallelize(Scope = ExecutionScope.ClassLevel). The serialization commit replaced it with DoNotParallelize alongside MaxCpuCount=1 and TestTfmsInParallel=false; the follow-up settings change reverted the latter two but left the assembly attribute, so every method ran single-threaded per test host. Measured on the full suite (Release, VSTest, 22 logical processors): net10.0 2,850/2,850 in 1m48s parallel vs 2m46s serial (-35%); net481 2,835/2,835 in 2m20s parallel vs 3m21s serial (-30%); zero failures under parallelism on this evidence, so no per-class DoNotParallelize pins are needed yet. The wall-clock runs were taken with the unstaged configuration-snapshot working-tree changes present, which affect a handful of guard tests at millisecond scale and no test results.
Adds DistributionSnapshot, an immutable bitwise capture of a distribution tree's mutable configuration whose equality implies an identical canonical configuration string: exact built-in leaves mirror their GetParameters scalar order (extended by the logarithm base on the log families and the physical-moment surface on LnNormal, which sit outside the flattened parameters), and exact Mixture/CompetingRisks nodes capture their flags, weights, seed, and correlation entries and recurse into children. Derived types and table-backed families defeat capture, so their owners retain the generic canonical-string path; a bitwise mismatch always falls back to the string comparison, which remains the deciding authority. CompetingRisks' WeibullConfiguration becomes DependentConfigurationCache around the shared snapshot, so the dependent arm stops rebuilding the canonical string on every call for any exact supported component set, and the derivative-step cache now also engages there (the step is a pure function of the pinned parameters). The fixed-support dependent fast arm stays gated on AllExactWeibull: extending DependentCDFCore reuse to other families could move boundary-stencil values and is left as a separate evidence-carrying change. Measured (2 LnNormal components, PerfectlyPositive, Release, per call): dependent CDF 31,057 B -> 136 B (-99.6%) and 4,030 ns -> 1,237 ns (-69%); LogPDF 62,362 B -> 464 B (-99.3%) and 8,995 ns -> 3,097 ns (-66%). All-Weibull behavior and results are unchanged. All 36 CompetingRisks family tests pass, and the full suites passed with these changes present: net10.0 2,850/2,850, net481 2,835/2,835.
…shot Mixture.RefreshCachedConfiguration serialized the full canonical configuration string on every call - recursively rendering every component to XML - and then discarded it whenever nothing had changed, which every InverseCDF call, CreateEmpiricalCDF, and moment getter paid. A published DistributionSnapshot now short-circuits the unchanged path bitwise, exactly the CompetingRisks pattern; a mismatch or an uncapturable component tree still builds and compares the string, which remains the deciding authority for cache invalidation, so incomplete capture can only cost a string comparison, never a wrong match. Measured (four LnNormal components over the empirical-CDF fast path, Release, per call): InverseCDF 62,512 B -> 928 B (-98.5%) and 11.6 us -> 2.8 us (-76%). CDF is unchanged (its cost is per-call validation, addressed separately). All 47 Mixture family tests pass; values are bit-identical - a snapshot match implies the identical canonical string.
Mixture evaluation validated every component on every call - allocating a parameter array per non-Normal component - and read its support bounds through capturing LINQ closures on every quantile clamp. A ValidationCertificate now publishes the exact bitwise snapshot that passed the full validator: a match skips re-validation without allocating, any mutation re-runs the verbatim validator with identical exceptions and precedence, and uncapturable component trees keep per-call validation unchanged. The certificate also carries lazily published support bounds (pure functions of the certified bits), and InverseCDF unifies its refresh and validation checks into one snapshot walk by reference identity with the refresh-published instance. The weight-log cache now serves LogCDF, LogCCDF, and interval probabilities as it already served LogPDF, hot loops index Length instead of LINQ Count(), and LnNormal exposes its stored physical-moment fields internally so snapshot capture pins the moment surface without evaluating it. The snapshot compare path is rewritten as inline per-family arms over the stored bits (capture keeps the shared cursor walk): the consuming libraries link this assembly's Debug build, whose minimal-optimization jitting keeps every helper call, so the hot path minimizes call count. Measured (four LnNormal components over the empirical-CDF fast path, Debug assembly, median of three, per call): InverseCDF 928 B -> 96 B and 2.8 us -> 0.79 us - now better than the pre-hardening baseline (144 B / 0.97 us) on both axes; CDF 160 B -> 0 B at wall parity with the prior state (the remaining gap to the pre-hardening direct sum is the retained log-sum-exp tail repair, deliberately unchanged). All 135 Mixture, CompetingRisks, and LnNormal family tests pass; values are bit-identical.
Adds the snapshot regression suite: a capture/compare round trip per supported family (pinning the capture and inline-compare switches together), a reflection sweep asserting every public settable double, int, bool, and enum property on a supported family flips the compare (the completeness guard for the per-family scalar lists), nested-grandchild invalidation through composite recursion, the derived-component capture refusal, and correlation-matrix entry participation including the bitwise-identical clone case. The sweep immediately caught GeneralizedPareto.Lambda - carried peaks-per-block metadata outside the flattened parameters - which is now captured in both arms so the uniform invariant holds with no exception list. Lambda does not enter evaluation or the canonical string, so the addition is strictness only; extra strictness can only force a string re-comparison, never a wrong match.
LogPearsonTypeIII allocated and validated a fresh PearsonTypeIII on every LogCDF, LogCCDF, CDF, and InverseCDF call. Following the existing static LogPDF precedent, the Pearson tail and quantile interiors are extracted as internal statics over already-validated parameters (the instance methods keep their guards and delegate verbatim), and the wrapper's hot sites call them directly - its own validation and support short-circuits already guarantee the preconditions, and both classes validate the identical constraint set, so the bypassed instance guards were unreachable. The statics compute the gamma shape once per call where the instance path re-evaluated the Alpha property expression; both evaluations of the same pure expression produce identical bits, so results are unchanged. Measured (Debug assembly, per call): wrapper CDF 56 B -> 0 B and 1,029 ns -> 934 ns; wrapper InverseCDF 56 B -> 0 B and 4,566 ns -> 4,358 ns. The uncertainty and fitting paths keep their per-call construction (cold by design). All 84 Pearson and log-Pearson family tests pass.
DistributionEndpointTail is consumed only by CompetingRisks and MixtureLogWeights only by Mixture; both internal helpers move into their consumers' source files verbatim. KappaFourBoundary and KappaExpectedInformation are shared kernels - the boundary transform serves KappaFour, GeneralizedLogistic, and GeneralizedNormal, and the expected-information integration serves KappaFour, GeneralizedExtremeValue, and GeneralizedLogistic - so their files move to the Base folder beside the other shared distribution numerics rather than into KappaFour. StandardErrorExtensions stays in its own file: it is a public extension API surface. Pure file organization; no code changes, both target frameworks build clean, and all 305 touched-family tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RMC.Numerics 2.2.0with assembly version2.2.0.0.2.1.5to2.2.1.RMC.Numerics 2.2.0is the latest stable package on NuGet.org.Summary
TimeSeries.SmoothedSeries.Upgrade Considerations
NUTS.AdaptMassMatrixdefaults totrue; setting it tofalseretains a supplied fixed metric.OptimizationStatus.LineSearchFailed.Validation
dotnet restoredotnet build -c Releasedotnet build Numerics/Numerics.csproj -c Release /p:Version=2.2.0VSTEST_CONNECTION_TIMEOUT=600, usingdotnet test -c Release --no-buildand complete affected-framework reruns.dotnet pack Numerics/Numerics.csproj -c Release /p:Version=2.2.0 --no-build -o ./packagesgit diff --checkBoth Release builds completed with zero warnings and errors.
These are complete passing framework runs. Initial combined runs encountered BOM HTTP 500
DatasourceErrorresponses. Every affected method/framework passed its exact isolated rerun; the full affected framework gates were then rerun successfully. No tests were excluded and no test settings or numerical behavior were changed during release preparation.