diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..bc8e7d0 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,97 @@ +# Codex repository instructions + +## Repository state + +`intervalNets` contains two certified enclosure pipelines: + +- interval and derivative enclosures with adaptive norm integration; +- polynomial-zonotope (PZ) propagation of neural-network values, Jacobians, and Hessians. + +The PZ core, `PZTwoJet`, affine and activation propagation, `model.eval_pz_twojet(...)`, PZ integration, and PZ norm routines already exist. A separate `model.eval_pz_value(...)` path propagates only function values and is the default for PZ \(L^2\) computation; do not reintroduce Jacobian or Hessian construction into that path. The scalable `model.eval_pz_onejet(...)` path is the default for PZ \(W^{1,2}\): it propagates a dependent Jacobian polynomial core plus a certified remainder for explicitly reduced terms, without constructing Hessians. Do not route order-one norms through `eval_pz_twojet(...)`. Do not treat the original two-jet blueprint as an unimplemented feature checklist. + +## Read the relevant specification first + +Inspect the existing implementation and tests before editing it. Use the document matching the task: + +- `docs/polynomial_zonotope_notation_and_terminology.tex`: canonical notation and vocabulary for domain noise, approximation noise, sparse polynomial supports, network layers, and two-jets; consult this before introducing new mathematical notation or terminology; +- `docs/polynomial_zonotope_twojet_recurrence.tex`: ground-truth initialization and affine/activation recurrences for direct propagation of value, Jacobian, and Hessian enclosures through an input polynomial zonotope; +- `docs/blueprints/pz_twojet_blueprint.tex`: mathematical design and historical implementation blueprint for PZ two-jets; +- `docs/affine_tanh_enclosures.tex`: certified affine enclosures for `tanh`, `tanh'`, and `tanh''`; +- `docs/certified_polynomial_zonotope_integration.tex`: geometric PZ integration and approximation-noise semantics; +- `docs/direct_integrated_twojet_squares.tex`: direct certified integration of squared PZ two-jets without constructing the squared integrand. +- `docs/dependency_preserving_graph_hilbert_integration.tex`: immutable graph moments, certified Hilbert compression, reverse-triangle bounds, and Neumann dual witnesses for scalable positive lower certificates. +- `docs/diagnostics_and_metrics_glossary.tex`: ground-truth metric definitions, aggregation rules, canonical CSV/JSON schemas, and mandatory mini- and medium-benchmark outputs; consult this before adding or changing benchmark diagnostics or output columns. + +The current source code and tests define the implemented public behavior. When a design document and the implementation differ, identify the discrepancy explicitly instead of silently changing semantics. + +## Important implementation invariants + +- Preserve rigorous enclosure guarantees and outward-rounding behavior. +- Preserve shared polynomial dependencies; do not silently replace them by intervals unless the relevant specification explicitly permits re-enclosure. +- Keep domain noise symbols \(\alpha\) distinct from approximation noise symbols \(\eta\), and use \(\varepsilon=(\alpha,\eta)\) for their combined vector. +- Use **approximation noise symbol** as the canonical term. Reserve **residual function** for the actual difference between a function and its approximation, and **approximation-error radius** for its certified coefficient \(\rho\). +- Under pointwise integration semantics, an approximation noise symbol may represent the residual function separately at each physical point; it is not one global symbolic value over the integration domain. Follow the semantics in `pz_integration.py`. +- Canonicalize equal exponent vectors and combine their coefficients before applying absolute values or interval collapse. This is required to preserve cancellations and reproduce the existing enclosure. +- Maintain tensor-valued coefficient support and the established shapes of `Y`, `J`, and `H`. +- Exploit Hessian symmetry only where the stored Hessian convention guarantees it. For a full symmetric Hessian, off-diagonal Frobenius contributions have weight two. +- Avoid changing public APIs or numerical semantics unless the task explicitly requires it. + +## PZ norm and direct-integration work + +For changes to certified PZ `L^2`, `W^{1,2}`, or `W^{2,2}` integration, read `docs/direct_integrated_twojet_squares.tex` in full and inspect: + +- `src/intervalnets/polynomial_zonotope.py`; +- `src/intervalnets/pz_integration.py`; +- `src/intervalnets/pz_norms.py`; +- `tests/test_polynomial_zonotope.py`; +- `tests/test_pz_integration.py`; +- `tests/test_pz_norms.py`. + +The direct-integration optimization must reproduce the current certified enclosure while avoiding materialization of the squared PZ integrand. Exploit unordered monomial-pair symmetry and Hessian symmetry, but still merge all contributions with the same retained pointwise-noise exponent before taking absolute values. Keep the existing explicit-square path available at least internally for regression comparisons until equivalence is well tested. + +Benchmark enclosure construction, norm-integrand construction, and integration separately. Final monomial count alone is not an adequate performance measure because sparse polynomial multiplication processes intermediate term pairs before canonicalization. + +All benchmark notebooks and reusable diagnostics must follow +`docs/diagnostics_and_metrics_glossary.tex`. Preserve its schema version, +exact canonical column orders, missing-value/status conventions, deterministic +row ordering, and distinction between absolute widths, local relative radii, +familywise global normalized radii, and relative norm widths. A medium +benchmark includes every mini-benchmark output plus the per-neuron activation +approximation table and `layer_normalized_radius_Y.csv`; do not silently omit +unimplemented required quantities. + +For schema version 1.2 and later, preserve both local relative radius and local +relative width in the prescribed columns, and populate the physical domain +volume plus the domain-volume-normalized lower endpoint, upper endpoint, and +width for every implemented squared and unsquared norm row. Domain-volume +normalization is a scale metric; relative norm width is the tightness metric. + +For value-only \(L^2\) performance work, use +`notebooks/pz_l2_value_benchmarks.ipynb`. The affine tanh enclosure keeps the +value support degree one, with one domain symbol per input coordinate and one approximation noise symbol per hidden neuron. Preserve this +independence when batching activation enclosures. + +For one-jet \(W^{1,2}\) performance work, use +`notebooks/pz_w12_polynomial_reduction_benchmarks.ipynb`. The scalable one-jet +retains a dependent Jacobian polynomial core and propagates a separate, +certified pointwise box only for terms explicitly removed by a reduction +policy. Do not replace the whole Jacobian by intervals. Available experimental +policies are top-k generator retention, degree-capped top-k retention, and a +sound coefficient-space PCA reduction with an explicitly bounded projection +remainder. Treat runtime, retained support, polynomial degree, reduction +remainder, and final enclosure width as joint diagnostics. + +## Development workflow + +1. Inspect the relevant source, tests, and specification. +2. Make the smallest coherent change. +3. Add focused regression tests, including cancellation and noise-kind edge cases. +4. Run targeted tests first, then the full suite: + + ```bash + pytest -q tests/test_polynomial_zonotope.py tests/test_pz_integration.py tests/test_pz_norms.py + pytest -q + ``` + +5. For performance work, report both correctness comparisons and timings on the same input. +6. Keep experimental notebook code thin; reusable logic belongs in `src/intervalnets/` and assertions belong in `tests/`. diff --git a/README.md b/README.md index 7303c2a..53e0d85 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,8 @@ 1. an overloaded `model.eval(interval)` pathway (enabled via `enable_interval_eval(...)`) for interval propagation through neural networks with outward-rounded arithmetic, including roundoff-aware bounds; 2. rigorous enclosure of Lebesgue/Lp norms over interval domains via `model.lpnorm(domain, p, iterations=...)`; -3. interval Jacobian enclosure via `model.eval_jacobian(domain)` and Sobolev-style norms via - `model.sobolev_norm(domain, p, iterations=...)`. +3. interval derivative enclosure via `model.eval_jacobian(domain)` / `model.eval_hessian(domain)` and Sobolev-style norms via + `model.sobolev_norm(domain, p, order=..., iterations=...)`. The current implementation follows the same interval-enclosure + adaptive-refinement strategy outlined in the preprint *Certified and accurate computation of function space norms of deep neural networks* (arXiv:2603.06431). @@ -31,14 +31,14 @@ always the tightest possible interval enclosure one could compute with more expe - interval propagation currently supports `nn.Sequential`, `nn.Flatten`, `nn.Linear`, `nn.ReLU`, `nn.Sigmoid`, `nn.Tanh`, `nn.Softplus`, `nn.LeakyReLU`, `nn.Softmax`, `nn.Identity`, plus `IntervalAdd`/`IntervalCat` branch combinators, - ReLU propagation preserves mathematically exact zero images (`[0, 0]`) for non-positive pre-activation intervals; only non-exact branches are outward-padded, - linear and Jacobian propagation are implemented with midpoint-radius matrix formulas for speed; this favors runtime performance over globally minimal box tightness, -- `model.eval(interval)`, `model.eval_jacobian(...)`, `model.lpnorm(...)`, and `model.sobolev_norm(...)` are attached through a single opt-in monkey patch (`enable_interval_eval()`), and the selected `enclosure_mode` is reused for both `model.eval(interval)` and sequential pre-activation propagation inside `model.eval_jacobian(...)`, +- `model.eval(interval)`, `model.eval_jacobian(...)`, `model.eval_hessian(...)`, `model.lpnorm(...)`, and `model.sobolev_norm(...)` are attached through a single opt-in monkey patch (`enable_interval_eval()`), and the selected `enclosure_mode` is reused for both `model.eval(interval)` and sequential pre-activation propagation inside derivative enclosures, - `enable_interval_eval(enclosure_mode="slope")` accepts `"box"` or `"slope"` (default: slope-aware affine relaxation for `nn.Sequential` chains of `nn.Linear` + `nn.ReLU`, with conservative fallback to `"box"` for unsupported layers), - slope mode is particularly useful for dependency-heavy patterns such as `Linear(rotation) -> ReLU -> Linear(rotation^{-1})`: plain box propagation can overestimate strongly, while slope-aware relaxations keep substantially tighter certified bounds, - for additional tightness, `interval_forward_refine(model, interval, enclosure_mode="slope", splits_per_dim=...)` subdivides the input box and hulls sub-box outputs (higher cost, tighter bounds), - `model.lpnorm(domain, p, iterations, theta=0.5)` and - `model.sobolev_norm(domain, p, iterations, theta=0.5)` use + `model.sobolev_norm(domain, p, order, iterations, theta=0.5)` use Dörfler-type bulk marking (with uncertainty indicators) and adaptive - bisection to return outward-rounded certified norm enclosures; rigorously constant boxes with zero Jacobian are skipped during Sobolev refinement. + bisection to return outward-rounded certified norm enclosures; rigorously constant boxes with zero Jacobian (and zero Hessian when `order=2`) are skipped during Sobolev refinement. - both norm routines accept optional `forward_refine_splits` / `forward_refine_max_cells` arguments to tighten per-box forward enclosures during integration. ## Quick start @@ -72,7 +72,8 @@ enable_interval_eval(enclosure_mode="slope") domain = IntervalTensor.from_bounds([0.0, 0.0], [1.0, 1.0]) lp_bounds = model.lpnorm(domain, p=2.0, iterations=8) -w1p_bounds = model.sobolev_norm(domain, p=2.0, iterations=8) +w1p_bounds = model.sobolev_norm(domain, p=2.0, order=1, iterations=8) +w2p_bounds = model.sobolev_norm(domain, p=2.0, order=2, iterations=8) ``` ### Option 2: run directly from the repo without installing @@ -91,6 +92,7 @@ sys.path.insert(0, str(repo_root / "src")) After that, `from intervalnets import ...` will work from the checkout as well. + ## Installation notes - the core `Interval` type uses only the Python standard library, @@ -101,7 +103,8 @@ For worked examples, see: - `notebooks/test_suite.ipynb` for quick feature checks and sanity tests, - `notebooks/reproduce_lp_w1p_experiments.ipynb` for reproducible certified - `L^p` and `W^{1,p}` experiments aligned with arXiv:2603.06431 (intentionally excluding `W^{2,p}`). + `L^p` and `W^{1,p}` experiments aligned with arXiv:2603.06431. +- `notebooks/pinn_aposteriori_square_poisson.ipynb` for a Poisson PINN example with certified residual and boundary terms using Hessian bounds and `W^{2,2}`-compatible tooling. ## Numerical experiment figures @@ -120,6 +123,14 @@ README. ![2D curves](notebooks/notebooks/artifacts/figure_cd_2d_curves.png) ![Local gap heatmaps](notebooks/notebooks/artifacts/figure_d_local_gap_heatmaps.png) +### PINN a-posteriori Poisson example + +![PINN Poisson results](notebooks/notebooks/artifacts/figure_pinn_results.png) + +Certified interval bounds can be propagated through the PINN residual and +boundary-condition terms, so the same pipeline can rigorously account for PDE +interior constraints **and** boundary conditions. + ## Reference - Johannes Gründler, Moritz Maibaum, Philipp Petersen, diff --git a/docs/API.md b/docs/API.md index 9451e12..2c85655 100644 --- a/docs/API.md +++ b/docs/API.md @@ -13,7 +13,7 @@ This document describes the public Python API exposed by `intervalnets` and how ## Core interval arithmetic (`Interval`) -`Interval` uses midpoint-radius internals to support fast affine interval propagation paths used by +`Interval` uses midpoint-radius internals to support fast matrix interval propagation paths used by the PyTorch backend. The library targets **certified enclosures with good throughput**, not globally optimal/tightest interval boxes for every operation. @@ -37,7 +37,7 @@ optimal/tightest interval boxes for every operation. ### Arithmetic operations - `+`, `-`, unary `-`, `*`, `/` are implemented with outward rounding. -- In affine-heavy code paths (e.g. linear layers and Jacobian composition), interval propagation is +- In matrix-heavy code paths (e.g. linear layers and Jacobian composition), interval propagation is optimized around midpoint-radius matrix formulas (`A x_mid ± |A| x_rad`). - Scalar division by an interval containing `0` raises `ZeroDivisionError`. - Vector interval division is intentionally not implemented and raises `NotImplementedError`. @@ -73,10 +73,22 @@ After this, every `torch.nn.Module` gets: - Interval forward propagation. - `model.lpnorm(domain: IntervalTensor, p: float, iterations: int = 0, theta: float = 0.5, forward_refine_splits: int = 1, forward_refine_max_cells: int = 256)` - Outward-rounded enclosure of the model `L^p` norm on a box domain. +- `model.lpnorm(..., method="pz")` + - Uses the polynomial-zonotope two-jet norm backend instead of the default interval backend. The default remains `method="interval"` for backward compatibility. - `model.eval_jacobian(domain: IntervalTensor)` - Interval enclosure of Jacobian matrix entries over the domain, using the same `enclosure_mode` selected when calling `enable_interval_eval(...)` for sequential pre-activation propagation. -- `model.sobolev_norm(domain: IntervalTensor, p: float, iterations: int = 0, theta: float = 0.5, forward_refine_splits: int = 1, forward_refine_max_cells: int = 256)` - - Enclosure of a first-order Sobolev-style norm (`|f|^p + |Df|^p`) over the domain. +- `model.eval_hessian(domain: IntervalTensor)` + - Interval enclosure of Hessian tensor entries over the domain (shape `(output_dim, input_dim, input_dim)`). +- `model.sobolev_norm(domain: IntervalTensor, p: float, order: int = 1, iterations: int = 0, theta: float = 0.5, forward_refine_splits: int = 1, forward_refine_max_cells: int = 256)` + - Enclosure of a Sobolev-style norm over the domain: + - `order=1`: `|f|^p + |Df|^p`, + - `order=2`: `|f|^p + |Df|^p + |D^2 f|^p`. +- `model.sobolev_norm(..., method="pz")` + - Uses the polynomial-zonotope two-jet backend while preserving the existing interval backend as the default. +- `model.pz_l2norm(domain: IntervalTensor, p: float = 2.0, iterations: int = 0, theta: float = 0.5, chebyshev_degree: int = 5, residual_subdivisions: int = 128, output: str = "interval")` + - Convenience alias for the PZ two-jet `L^2` norm backend. +- `model.pz_sobolev_norm(domain: IntervalTensor, p: float = 2.0, order: int = 1, iterations: int = 0, theta: float = 0.5, chebyshev_degree: int = 5, residual_subdivisions: int = 128, output: str = "interval")` + - Convenience alias for the PZ two-jet `W^{1,2}` or `W^{2,2}` norm backend. > Note: these methods are attached by monkey-patching `torch.nn.Module`. If patching is not desired in your application architecture, call `interval_forward(...)` directly for pure forward enclosure and avoid the norm/Jacobian helpers. @@ -130,6 +142,7 @@ Runs each branch on the same input interval and concatenates outputs. - Requires at least one branch. - Current interval backend supports 1D vector outputs and `dim in {0, -1}`. + ## Certified norm computation details `model.lpnorm(..., theta=0.5)` and `model.sobolev_norm(..., theta=0.5)` use adaptive box subdivision with Dörfler-type marking: @@ -144,7 +157,7 @@ Runs each branch on the same input interval and concatenates outputs. 6. Accumulate interval integral bounds over the resulting partition. 7. Clamp tiny negative roundoff artifacts to zero before taking the `1/p` power. -Sobolev refinement additionally skips boxes that are rigorously constant with an exactly zero Jacobian enclosure, avoiding unnecessary subdivision of derivative-inactive regions. +Sobolev refinement additionally skips boxes that are rigorously constant with an exactly zero Jacobian enclosure (and exactly zero Hessian enclosure for `order=2`), avoiding unnecessary subdivision of derivative-inactive regions. Important constraints: @@ -156,6 +169,36 @@ Important constraints: - `forward_refine_splits` must be an integer `>= 1` (default: `1`, meaning no extra per-box subdivision). - `forward_refine_max_cells` limits refinement combinatorics (default: `256`). +### Polynomial-zonotope norm backend + +The PZ norm API is exposed both as explicit methods and through `method="pz"`: + +```python +from intervalnets import IntervalTensor, enable_interval_eval, pz_l2norm, pz_sobolev_norm +from torch import nn + +enable_interval_eval() + +model = nn.Sequential(nn.Linear(2, 4), nn.Tanh(), nn.Linear(4, 1)) +box = IntervalTensor.from_bounds([0.0, -1.0], [1.0, 2.0]) + +l2_a = model.pz_l2norm(box) +l2_b = model.lpnorm(box, p=2.0, method="pz") +w12_a = model.pz_sobolev_norm(box, order=1) +w22_b = model.sobolev_norm(box, p=2.0, order=2, method="pz") + +# Top-level wrappers are also available when PyTorch is importable. +l2_c = pz_l2norm(model, box) +w12_c = pz_sobolev_norm(model, box, order=1) +``` + +PZ norm limitations in the initial implementation: + +- Only `p=2` is supported. Calls with another `p` raise `NotImplementedError`. +- Supported network layers match `eval_pz_twojet`: `nn.Sequential`, `nn.Linear`, `nn.Tanh`, `nn.Identity`, and already-flat `nn.Flatten`. +- Geometric integration initially supports affine interval-box cells via `PZIntegrationCell.from_affine_box(...)`. +- Approximation noise from tanh residual certification is intervalized after exact domain integration so pointwise residual symbols are not silently treated as globally shared polynomial variables. + ## Jacobian enclosure details `model.eval_jacobian(domain)` returns an `IntervalTensor` whose shape is `(output_dim, input_dim)` (stored as nested tuples). @@ -171,6 +214,23 @@ Layer derivatives currently implemented: For `nn.Sequential`, Jacobian enclosures are composed with interval matrix multiplication, and layer-input intervals are computed with the configured `enclosure_mode` (`"slope"` by default via `enable_interval_eval`). +## Hessian enclosure details + +`model.eval_hessian(domain)` returns an `IntervalTensor` with shape `(output_dim, input_dim, input_dim)` (stored as nested tuples). + +Layer Hessians currently implemented: + +- `nn.Linear` (exact zero Hessian), +- `nn.ReLU` (zero enclosure), +- `nn.Sigmoid`, +- `nn.Tanh`, +- `nn.Flatten`. + +For `nn.Sequential`, Hessian enclosures are composed with interval chain-rule terms: + +- Jacobian-weighted propagation of previous Hessians, +- plus local layer-Hessian curvature terms weighted by interval outer products of previous Jacobian rows. + Implementation note: Jacobian composition currently favors vectorized midpoint-radius interval matrix products for speed. This is conservative and efficient, but not necessarily the tightest enclosure that could be achieved with more expensive symbolic or optimization-based techniques. @@ -215,8 +275,8 @@ sob = model.sobolev_norm(box, p=2.0, iterations=6, forward_refine_splits=2, forw The package-level import surface in `intervalnets.__init__` is: -- Always: `Interval` -- When PyTorch is importable: `IntervalTensor`, `IntervalAdd`, `IntervalCat`, `enable_interval_eval`, `interval_forward`, `interval_forward_refine` +- Always: `Interval`, `PolynomialZonotope`, `PZTwoJet`, `TanhApproximation`, `compute_tanh_polynomial`, `certify_tanh_residual_subdivision`, `tanh_pz_scalar` +- When PyTorch is importable: `IntervalTensor`, `IntervalAdd`, `IntervalCat`, `enable_interval_eval`, `interval_forward`, `interval_forward_refine`, `pz_l2norm`, `pz_sobolev_norm`, `pz_twojet_forward` Prefer importing these from the top-level package for user-facing code: diff --git a/docs/affine_tanh_enclosures.tex b/docs/affine_tanh_enclosures.tex new file mode 100644 index 0000000..e0c9826 --- /dev/null +++ b/docs/affine_tanh_enclosures.tex @@ -0,0 +1,534 @@ +\documentclass[11pt]{article} + +\usepackage[T1]{fontenc} +\usepackage[utf8]{inputenc} +\usepackage{lmodern} +\usepackage[a4paper,margin=28mm]{geometry} +\usepackage{amsmath,amssymb,amsthm,mathtools} +\usepackage{booktabs} +\usepackage{array} +\usepackage{enumitem} +\usepackage{xcolor} +\usepackage{hyperref} +\usepackage{microtype} +\usepackage{listings} + +\hypersetup{ + colorlinks=true, + linkcolor=blue!50!black, + citecolor=blue!50!black, + urlcolor=blue!50!black +} + +\newtheorem{proposition}{Proposition} +\newtheorem{lemma}{Lemma} +\newtheorem{remark}{Remark} +\newtheorem{algorithm}{Algorithm} + +\newcommand{\R}{\mathbb{R}} +\newcommand{\E}{[-1,1]} +\newcommand{\sech}{\operatorname{sech}} +\newcommand{\artanh}{\operatorname{artanh}} +\newcommand{\cand}{\mathcal{C}} +\newcommand{\clamp}{\operatorname{clamp}} + +\lstdefinestyle{pseudo}{ + basicstyle=\ttfamily\small, + columns=fullflexible, + frame=single, + rulecolor=\color{black!25}, + backgroundcolor=\color{black!2}, + xleftmargin=0.5em, + xrightmargin=0.5em, + aboveskip=0.8em, + belowskip=0.8em, + keepspaces=true, + showstringspaces=false +} + +\title{Cheap Certified Affine Enclosures for \(\tanh\), \(\tanh'\), and \(\tanh''\)} +\author{Implementation note for polynomial-zonotope two-jet propagation} +\date{\today} + +\begin{document} +\maketitle + +\begin{abstract} +This note gives a noniterative routine for enclosing the hyperbolic tangent activation and its first two derivatives on a real interval by an affine function plus one symmetric approximation noise term. On an interval where the function is convex or concave, the construction reduces to the affine Chebyshev representation of Rump and Kashiwagi. If the interval crosses one or more curvature breakpoints, the same secant slope is retained and the exact residual range for that slope is obtained by evaluating a fixed finite set of explicitly computable stationary candidates. No optimization, root iteration, or branch-and-bound step is required. The resulting enclosure is directly suitable for polynomial-zonotope propagation: +\[ + f(Z)\subseteq pZ+q+\rho\eta, + \qquad \eta\in[-1,1]. +\] +\end{abstract} + +\tableofcontents + +\section{Affine representations} + +Let \(X=[a,b]\subset\R\), with \(a{\raggedright\arraybackslash}p{0.19\textwidth} p{0.34\textwidth} p{0.34\textwidth}} +\toprule +Function & Convex regions & Concave regions \\ +\midrule +\(\tanh x\) & \(( -\infty,0]\) & \([0,\infty)\) \\ +\(\tanh' x\) & \(( -\infty,-\tau]\cup[\tau,\infty)\) & \([ -\tau,\tau]\) \\ +\(\tanh''x\) & \(( -\infty,-\beta]\cup[0,\beta]\) & \([ -\beta,0]\cup[\beta,\infty)\) \\ +\bottomrule +\end{tabular} +\end{table} + +Thus the breakpoint sets relevant to a two-jet implementation are +\[ + B_0=\{0\},\qquad + B_1=\{-\tau,\tau\},\qquad + B_2=\{-\beta,0,\beta\}. +\] +These sets are useful for a fast dispatch test: if \([a,b]\) does not cross a breakpoint for the requested function, the ordinary convex/concave formula applies. The finite-candidate construction below is nevertheless valid in all cases and can be used as a unified implementation. + +\section{Closed-form candidates for \(\tanh\)} + +Let +\[ + f_0(x)=\tanh x, + \qquad A=\tanh a, + \qquad B=\tanh b, +\] +and choose +\begin{equation}\label{eq:p0} + p=\frac{B-A}{b-a}. +\end{equation} +Because \(f_0'(x)=1-\tanh^2x\), the stationary equation \(f_0'(x)=p\) is +\[ + 1-t^2=p. +\] +The only possible transformed stationary values are therefore +\begin{equation}\label{eq:t0candidates} + t_\pm=\pm\sqrt{1-p}. +\end{equation} +Retain a value \(t_\pm\) only if +\[ + t_\pm\in[A,B] +\] +and \(|t_\pm|<1\), then set +\[ + x_\pm=\artanh(t_\pm). +\] +The candidate set contains at most four points: +\begin{equation}\label{eq:C0} + \cand_0=\{a,b\}\cup\{x_+,x_-\text{ that lie in }(a,b)\}. +\end{equation} +Evaluate +\[ + r(x)=\tanh x-px +\] +on \(\cand_0\), then use \eqref{eq:qdelta-general}. + +\paragraph{Cost.} +Two endpoint evaluations are already needed for the secant slope. The mixed-curvature case adds one square root, up to two inverse hyperbolic tangents, and at most two additional residual evaluations. + +\section{Closed-form candidates for \(\tanh'\)} + +Let +\[ + f_1(x)=\tanh'(x)=1-t^2 +\] +and choose +\begin{equation}\label{eq:p1} + p=\frac{f_1(b)-f_1(a)}{b-a}. +\end{equation} +The stationary equation \(f_1'(x)=p\) becomes +\begin{equation}\label{eq:cubic} + -2t(1-t^2)=p, + \qquad\text{equivalently}\qquad + t^3-t-\frac{p}{2}=0. +\end{equation} +For a secant slope of \(f_1\), one has +\[ + |p|\le \max_x|f_1'(x)|=\frac{4}{3\sqrt{3}}, +\] +so the cubic has three real roots, counted with multiplicity. They are +\begin{equation}\label{eq:cubicroots} + t_k=\frac{2}{\sqrt{3}} + \cos\!\left( + \frac{1}{3}\arccos\!\left(\frac{3\sqrt{3}}{4}p\right) + -\frac{2\pi k}{3} + \right), + \qquad k=0,1,2. +\end{equation} +Due to rounding, the argument of \(\arccos\) should be clamped to \([-1,1]\) in floating-point code. + +Retain only roots satisfying +\[ + t_k\in[\tanh a,\tanh b]\cap(-1,1), +\] +remove numerical duplicates, and set +\[ + x_k=\artanh(t_k). +\] +The candidate set is +\begin{equation}\label{eq:C1} + \cand_1=\{a,b\}\cup\{x_k\in(a,b):k=0,1,2\}. +\end{equation} +Evaluate +\[ + r(x)=\sech^2x-px=(1-\tanh^2x)-px +\] +on \(\cand_1\), then apply \eqref{eq:qdelta-general}. + +\paragraph{Cost.} +The routine uses one \(\arccos\), three cosine evaluations, and at most three inverse hyperbolic tangents. This is fixed cost and requires no iterative cubic solver. + +\section{Closed-form candidates for \(\tanh''\)} + +Let +\[ + f_2(x)=\tanh''(x)=-2t+2t^3 +\] +and choose +\begin{equation}\label{eq:p2} + p=\frac{f_2(b)-f_2(a)}{b-a}. +\end{equation} +The stationary equation \(f_2'(x)=p\) is +\[ + -2+8t^2-6t^4=p. +\] +With \(u=t^2\), this becomes +\begin{equation}\label{eq:quadraticu} + 6u^2-8u+(p+2)=0. +\end{equation} +Hence +\begin{equation}\label{eq:uroots} + u_\pm=\frac{2\pm\sqrt{1-\frac{3}{2}p}}{3}. +\end{equation} +For each \(u_\pm\), retain it only if \(0\le u_\pm<1\). It then produces the possible transformed candidates +\begin{equation}\label{eq:t2candidates} + t=\pm\sqrt{u_\pm}. +\end{equation} +Retain only those \(t\)-values that lie in \([\tanh a,\tanh b]\), remove duplicates, and set +\[ + x=\artanh(t). +\] +The candidate set is +\begin{equation}\label{eq:C2} + \cand_2=\{a,b\}\cup\{\text{retained interior candidates}\}. +\end{equation} +Evaluate +\[ + r(x)=(-2\tanh x+2\tanh^3x)-px +\] +on \(\cand_2\), then apply \eqref{eq:qdelta-general}. + +\paragraph{Cost.} +At most one discriminant square root, four square roots, four inverse hyperbolic tangents, and four interior residual evaluations are required. + +\section{Unified algorithm} + +\begin{algorithm}[Noniterative affine enclosure]\label{alg:unified} +Given a function selector \(j\in\{0,1,2\}\), representing \(f_0=\tanh\), \(f_1=\tanh'\), or \(f_2=\tanh''\), and a nondegenerate interval \([a,b]\): +\begin{enumerate}[label=\arabic*.,leftmargin=2em] + \item Compute \(f_j(a)\), \(f_j(b)\), and + \[ + p=\frac{f_j(b)-f_j(a)}{b-a}. + \] + \item Form the finite transformed candidate set using: + \begin{itemize} + \item \eqref{eq:t0candidates} for \(j=0\), + \item \eqref{eq:cubicroots} for \(j=1\), + \item \eqref{eq:uroots}--\eqref{eq:t2candidates} for \(j=2\). + \end{itemize} + \item Filter candidates to \([\tanh a,\tanh b]\cap(-1,1)\), convert them by \(x=\artanh(t)\), retain only interior points, and append the endpoints \(a,b\). + \item Evaluate \(r(x)=f_j(x)-px\) at all retained candidates. + \item Set + \[ + q=\frac{\max r+\min r}{2}, + \qquad + \rho=\frac{\max r-\min r}{2}. + \] + \item Return \([[p,q,\rho]]\), with outward rounding or a final floating-point safety inflation if a mathematically rigorous machine enclosure is required. +\end{enumerate} +\end{algorithm} + +\subsection{Pseudocode} + +\begin{lstlisting}[style=pseudo] +function affine_tanh_jet_enclosure(j, a, b): + # j = 0: tanh, j = 1: tanh', j = 2: tanh'' + assert a <= b + + if a == b: + return p = 0, q = f_j(a), rho = 0 + + ta = tanh(a) + tb = tanh(b) + fa = value_from_t(j, ta) + fb = value_from_t(j, tb) + p = (fb - fa) / (b - a) + + T = empty list + + if j == 0: + d = max(0, 1 - p) # outward-safe in certified code + s = sqrt(d) + append_if_in_interval(T, +s, ta, tb) + append_if_in_interval(T, -s, ta, tb) + + else if j == 1: + z = clamp((3*sqrt(3)/4)*p, -1, 1) + theta = acos(z) / 3 + for k in {0,1,2}: + t = (2/sqrt(3))*cos(theta - 2*pi*k/3) + append_if_in_interval(T, t, ta, tb) + + else if j == 2: + d = max(0, 1 - (3/2)*p) + s = sqrt(d) + for u in {(2+s)/3, (2-s)/3}: + if 0 <= u < 1: + v = sqrt(u) + append_if_in_interval(T, +v, ta, tb) + append_if_in_interval(T, -v, ta, tb) + + remove_duplicates(T) + + R = [fa - p*a, fb - p*b] + for t in T: + if -1 < t < 1: + x = atanh(t) + if a < x < b: + append(R, value_from_t(j, t) - p*x) + + rmin = min(R) + rmax = max(R) + q = (rmax + rmin) / 2 + rho = (rmax - rmin) / 2 + + return p, q, rho + +function value_from_t(j, t): + if j == 0: return t + if j == 1: return 1 - t*t + if j == 2: return -2*t + 2*t*t*t +\end{lstlisting} + +\section{Numerical rigor and implementation details} + +The mathematics above assumes exact real arithmetic. A certified implementation must also account for roundoff. The following points are recommended. + +\begin{enumerate}[leftmargin=2em] + \item \textbf{Outward rounding.} Compute \(p\), all candidate values, residual values, and the final \(q,\rho\) using directed rounding or interval elementary functions. Alternatively, compute in ordinary floating point and inflate \(\rho\) by a rigorously derived rounding-error bound. + + \item \textbf{Candidate filtering.} Near a stationary point or curvature breakpoint, use tolerant or interval membership tests. It is safe to retain an extra candidate; omitting a genuine candidate is not safe. + + \item \textbf{Clamping.} Clamp the trigonometric cubic argument to \([-1,1]\) before calling \(\arccos\). In interval code, intersect its enclosure with \([-1,1]\). + + \item \textbf{Duplicate roots.} At discriminant-zero cases, formulas may return duplicate roots. Removing duplicates is only a performance optimization. + + \item \textbf{Endpoint equality.} Analytically, \(r_p(a)=r_p(b)\). In floating point, evaluate both and include both in the minimum and maximum. + + \item \textbf{Degenerate input.} If \(a=b\), one may return \(p=0\), \(q=f(a)\), and \(\rho=0\). If preserving the local linear part is desirable, another valid choice is \(p=f'(a)\), \(q=f(a)-pa\), \(\rho=0\). + + \item \textbf{Vectorization.} The formulas are branch-light and can be evaluated in batches across neurons. A curvature-region dispatch may avoid computing unnecessary candidates, but a unified finite-candidate routine is simpler and still constant-cost. +\end{enumerate} + +\section{Use in polynomial-zonotope two-jet propagation} + +Suppose a scalar preactivation polynomial zonotope \(Z\) is enclosed by \([a,b]\). For each of +\[ + f_0=\tanh,\qquad f_1=\tanh',\qquad f_2=\tanh'', +\] +compute a triple \([[p_j,q_j,\rho_j]]\). Then +\begin{align} + \tanh(Z)&\subseteq p_0Z+q_0+\rho_0\eta_0,\\ + \tanh'(Z)&\subseteq p_1Z+q_1+\rho_1\eta_1,\\ + \tanh''(Z)&\subseteq p_2Z+q_2+\rho_2\eta_2, +\end{align} +with fresh \(\eta_j\in[-1,1]\). + +These componentwise enclosures are sound and cheap. However, independent error symbols do not encode the fact that the three quantities arise from one common argument and satisfy algebraic relations such as +\[ + \tanh'(x)=1-\tanh^2x, + \qquad + \tanh''(x)=-2\tanh x+2\tanh^3x. +\] +A later implementation may exploit such cross-dependence, but this is not required for correctness of the componentwise two-jet enclosure. + +\section{Recommended dispatch rule} + +For each neuron and each partition cell: +\begin{enumerate}[leftmargin=2em] + \item Enclose the scalar preactivation polynomial zonotope by \([a,b]\). + \item Check whether \([a,b]\) lies inside a single curvature region from Table~\ref{tab:curvature}. + \item If it does, use the Rump--Kashiwagi formula with the unique closed-form stationary point. + \item If it crosses a curvature breakpoint, retain the same secant slope and evaluate the full fixed candidate set described above. + \item Propagate the affine part exactly through the polynomial-zonotope representation and add one fresh approximation noise term \(\rho\eta\). + \item Optionally use \(\rho\), or its contribution after subsequent linear layers, as a refinement score for adaptive partitioning. +\end{enumerate} + +The key design property is that both branches have bounded, explicit cost. No iterative approximation routine is called per neuron or per cell. + +\section{Summary of formulas} + +\begin{table}[ht] +\centering +\caption{Stationary candidates for the secant-slope residual. Here \(t=\tanh x\).} +\renewcommand{\arraystretch}{1.35} +\begin{tabular}{p{0.16\textwidth} p{0.31\textwidth} p{0.42\textwidth}} +\toprule +\(f(x)\) & Stationary equation \(f'(x)=p\) & Explicit transformed candidates \\ +\midrule +\(\tanh x\) +& \(1-t^2=p\) +& \(t=\pm\sqrt{1-p}\) \\ +\(\tanh' x\) +& \(t^3-t-p/2=0\) +& \(t_k=\frac{2}{\sqrt3}\cos\!\bigl(\frac13\arccos(\frac{3\sqrt3}{4}p)-\frac{2\pi k}{3}\bigr)\), \(k=0,1,2\) \\ +\(\tanh''x\) +& \(6t^4-8t^2+(p+2)=0\) +& \(u_\pm=\frac{2\pm\sqrt{1-\frac32p}}{3}\), then \(t=\pm\sqrt{u_\pm}\) \\ +\bottomrule +\end{tabular} +\end{table} + +For every row, retain only transformed candidates in \([\tanh a,\tanh b]\cap(-1,1)\), convert by \(x=\artanh(t)\), evaluate \(r(x)=f(x)-px\), and center its exact finite range according to \eqref{eq:qdelta-general}. + +\begin{thebibliography}{9} + +\bibitem{RumpKashiwagi2015} +S.~M. Rump and M.~Kashiwagi, +\newblock Implementation and improvements of affine arithmetic, +\newblock \emph{Nonlinear Theory and Its Applications, IEICE}, 2(3):1101--1119, 2011. +\newblock See especially Lemma~1 for the Min-Range and Chebyshev representations. + +\bibitem{MooreKearfottCloud2009} +R.~E. Moore, R.~B. Kearfott, and M.~J. Cloud, +\newblock \emph{Introduction to Interval Analysis}, +\newblock SIAM, 2009. + +\end{thebibliography} + +\end{document} diff --git a/docs/blueprints/pz_twojet_blueprint.tex b/docs/blueprints/pz_twojet_blueprint.tex new file mode 100644 index 0000000..7eca168 --- /dev/null +++ b/docs/blueprints/pz_twojet_blueprint.tex @@ -0,0 +1,948 @@ +\documentclass[11pt]{article} + +\usepackage[margin=1in]{geometry} +\usepackage{amsmath,amssymb,amsthm,mathtools} +\usepackage{enumitem} +\usepackage{hyperref} + +\title{Blueprint for Polynomial-Zonotope Two-Jet Enclosures in \texttt{intervalNets}} +\author{Technical implementation specification} +\date{\today} + +\newtheorem{definition}{Definition}[section] +\newtheorem{proposition}[definition]{Proposition} +\newtheorem{theorem}[definition]{Theorem} +\newtheorem{remark}[definition]{Remark} + +\newcommand{\R}{\mathbb{R}} +\newcommand{\N}{\mathbb{N}} +\newcommand{\PZ}{\mathrm{PZ}} +\newcommand{\eps}{\varepsilon} +\newcommand{\Id}{\mathrm{Id}} +\newcommand{\roundup}{\operatorname{round}_{\uparrow}} + +\begin{document} + +\maketitle + +\begin{abstract} +This document is a technical blueprint for adding polynomial-zonotope based forward-mode two-jet enclosure propagation to the PyTorch interval arithmetic repository \texttt{intervalNets}. The target networks are feedforward neural networks built from affine linear layers and componentwise hyperbolic tangent activations. Given an input polynomial zonotope, the implementation should compute rigorous polynomial-zonotope enclosures for the network output, Jacobian field, and Hessian field over the input set. The activation enclosure is based on a Remez/minimax polynomial approximation of \(\tanh\), with a certified residual error obtained by interval root isolation or interval residual bounding. +\end{abstract} + +\tableofcontents + +\section{Overview and implementation objective} + +The repository \texttt{intervalNets} already provides interval arithmetic and certified interval propagation through PyTorch models. The goal of this extension is different: instead of only propagating boxes, we want to propagate polynomial-zonotope enclosures of the two-jet field of a neural network. + +Let +\[ + \Phi:\R^{d_{\mathrm{in}}}\to\R^{d_{\mathrm{out}}} +\] +be a feedforward neural network built from affine linear layers and componentwise \(\tanh\) activations. Let the input set be represented by a polynomial zonotope +\[ + X(\eps) + = + c+\sum_{\alpha\in A}g_\alpha\eps^\alpha, + \qquad + \eps\in[-1,1]^p. +\] +Here \(c\in\R^{d_{\mathrm{in}}}\), \(g_\alpha\in\R^{d_{\mathrm{in}}}\), and +\[ + \eps^\alpha + = + \eps_1^{\alpha_1}\cdots\eps_p^{\alpha_p}. +\] +The represented input set is +\[ + Z_X + := + X([-1,1]^p) + = + \left\{X(\eps):\eps\in[-1,1]^p\right\}. +\] + +The implementation goal is to compute validated enclosures for +\[ + \Phi(x), + \qquad + D\Phi|_x, + \qquad + D^2\Phi|_x +\] +for all \(x\in Z_X\). Thus the output should consist of: +\begin{enumerate}[label=(\roman*)] + \item a vector-valued polynomial zonotope enclosing \(\Phi(Z_X)\); + \item a matrix-valued polynomial zonotope enclosing the Jacobian field \(D\Phi(Z_X)\); + \item a third-order tensor-valued polynomial zonotope enclosing the Hessian field \(D^2\Phi(Z_X)\). +\end{enumerate} + +This is not pointwise automatic differentiation. Pointwise automatic differentiation evaluates \(D\Phi|_x\) and \(D^2\Phi|_x\) for one fixed point \(x\). Here, the goal is set-valued and validated: +\[ + \forall x\in Z_X: + \qquad + \Phi(x)\in Y, + \quad + D\Phi|_x\in J, + \quad + D^2\Phi|_x\in H. +\] +The objects \(Y\), \(J\), and \(H\) should preserve polynomial dependence on shared noise variables whenever feasible. + +\section{Polynomial zonotope representation} + +\begin{definition}[Polynomial zonotope] +Let \(V\) be a finite-dimensional real vector space and let \(A\subset\N_0^p\) be a finite exponent set. A polynomial zonotope with coefficients in \(V\) is a set of the form +\[ + Z + = + \left\{ + c+\sum_{\alpha\in A}g_\alpha\eps^\alpha + : + \eps\in[-1,1]^p + \right\}, +\] +where \(c\in V\), \(g_\alpha\in V\), and +\[ + \eps^\alpha + = + \eps_1^{\alpha_1}\cdots\eps_p^{\alpha_p}. +\] +We write +\[ + Z=\PZ\bigl(c,\{(\alpha,g_\alpha)\}_{\alpha\in A}\bigr). +\] +\end{definition} + +The implementation should support polynomial zonotopes whose coefficients lie in +\[ + V=\R^n, + \qquad + V=\R^{m\times n}, + \qquad + V=\R^{m\times n\times n}. +\] +The same noise variables \(\eps_1,\ldots,\eps_p\) should be shared across vector-, matrix-, and tensor-valued polynomial zonotopes. This is essential for dependency preservation. + +\begin{remark}[Noise variables] +A polynomial zonotope stores dependence on uncertainty variables. If a new approximation error is introduced, for example from a certified \(\tanh\) approximation +\[ + \tanh(t)\in p(t)+\Delta[-1,1], +\] +the implementation should add a new noise variable \(\eta\in[-1,1]\). Nonlinear powers such as \(\eta^2\) and \(\eta^3\) should be preserved if the degree budget allows it. +\end{remark} + +\section{Core polynomial-zonotope arithmetic} + +Codex should implement a core class, tentatively named \texttt{PolynomialZonotope}. A polynomial zonotope should store: +\begin{enumerate}[label=(\roman*)] + \item the center coefficient \(c\); + \item a dictionary or structured tensor of exponent vectors \(\alpha\in\N_0^p\); + \item one coefficient tensor \(g_\alpha\) for each exponent vector; + \item metadata for the number of noise variables \(p\), coefficient shape, dtype, and device. +\end{enumerate} + +\subsection{Required arithmetic operations} + +\paragraph{Addition.} +For two polynomial zonotopes with the same coefficient shape, +\[ + Z_1=c_1+\sum_\alpha g_\alpha\eps^\alpha, + \qquad + Z_2=c_2+\sum_\alpha h_\alpha\eps^\alpha, +\] +define +\[ + Z_1+Z_2 + = + c_1+c_2+\sum_\alpha(g_\alpha+h_\alpha)\eps^\alpha. +\] +The implementation must merge equal exponent vectors. + +\paragraph{Scalar multiplication.} +For \(\lambda\in\R\), +\[ + \lambda Z + = + \lambda c+\sum_\alpha \lambda g_\alpha\eps^\alpha. +\] + +\paragraph{Linear maps.} +For \(A\in\R^{m\times n}\) and \(Z\in\PZ(\R^n)\), +\[ + AZ + = + Ac+\sum_\alpha A g_\alpha\eps^\alpha. +\] +For matrix-valued or tensor-valued coefficients, the corresponding contraction should be applied along the output dimension. + +\paragraph{Multiplication of scalar polynomial zonotopes.} +For scalar-valued polynomial zonotopes +\[ + P=a+\sum_\alpha p_\alpha\eps^\alpha, + \qquad + Q=b+\sum_\beta q_\beta\eps^\beta, +\] +define +\[ +\begin{aligned} + PQ + &= + ab + + + \sum_\beta a q_\beta\eps^\beta + + + \sum_\alpha b p_\alpha\eps^\alpha + + + \sum_{\alpha,\beta}p_\alpha q_\beta\eps^{\alpha+\beta}. +\end{aligned} +\] +Equal exponents must be merged. + +\paragraph{Scalar multiplication of vector-, matrix-, and tensor-valued polynomial zonotopes.} +If \(P\in\PZ(\R)\) and \(Z\in\PZ(V)\), define \(PZ\in\PZ(V)\) by the same convolution rule, replacing scalar products by scalar multiplication of \(V\)-valued coefficients. + +\paragraph{Tensor products.} +For Hessian propagation, the implementation needs products of the form +\[ + u\otimes u, +\] +where \(u\in\PZ(\R^{d_{\mathrm{in}}})\). If +\[ + u=c+\sum_\alpha u_\alpha\eps^\alpha, +\] +then +\[ +\begin{aligned} + u\otimes u + &= + c\otimes c + + + \sum_\alpha + \bigl(c\otimes u_\alpha+u_\alpha\otimes c\bigr)\eps^\alpha \\ + &\quad+ + \sum_{\alpha,\beta} + \bigl(u_\alpha\otimes u_\beta\bigr)\eps^{\alpha+\beta}. +\end{aligned} +\] +This produces a matrix-valued polynomial zonotope in \(\PZ(\R^{d_{\mathrm{in}}\times d_{\mathrm{in}}})\). + +\paragraph{Degree truncation and reduction.} +Polynomial multiplication causes representation growth. The implementation should support a configurable reduction operation +\[ + \mathrm{reduce}\bigl(Z;\texttt{max\_degree},\texttt{max\_terms}\bigr). +\] +Reduction must be inclusion-preserving. A safe baseline is: +\begin{enumerate}[label=(\alph*)] + \item keep selected low-degree or large-magnitude terms explicitly; + \item convert discarded terms to a box or independent error zonotope; + \item add new independent noise variables for discarded remainder components. +\end{enumerate} +If reduction is not implemented initially, Codex should allow an option \texttt{reduce=False} and raise a clear error when the representation becomes too large. + +\paragraph{Interval enclosure.} +A simple interval enclosure of +\[ + Z=c+\sum_\alpha g_\alpha\eps^\alpha +\] +is given componentwise by +\[ + Z + \subseteq + c+ + \left[ + -\sum_\alpha |g_\alpha|, + \sum_\alpha |g_\alpha| + \right], +\] +where absolute values and sums are taken componentwise. This enclosure is conservative but simple and dependency-safe. + +All interval bounds must be outward-rounded. If the current codebase has an interval tensor type with outward rounding, the polynomial-zonotope code should reuse it. + +\section{Two-jet object} + +\begin{definition}[Polynomial-zonotope two-jet object] +At layer \(\ell\), a two-jet enclosure object is a triple +\[ + \mathcal{J}^\ell=(Y^\ell,J^\ell,H^\ell), +\] +where +\[ + Y^\ell\in\PZ(\R^{n_\ell}), +\] +\[ + J^\ell\in\PZ(\R^{n_\ell\times d_{\mathrm{in}}}), +\] +and +\[ + H^\ell\in\PZ(\R^{n_\ell\times d_{\mathrm{in}}\times d_{\mathrm{in}}}). +\] +Here \(Y^\ell\) encloses the layer output, \(J^\ell\) encloses the Jacobian field with respect to the original physical input \(x\in\R^{d_{\mathrm{in}}}\), and \(H^\ell\) encloses the Hessian field with respect to \(x\). +\end{definition} + +Codex should implement this as a class, tentatively named \texttt{PZTwoJet}. It should contain: +\begin{verbatim} +class PZTwoJet: + Y: PolynomialZonotope # shape (n_layer,) + J: PolynomialZonotope # shape (n_layer, d_in) + H: PolynomialZonotope # shape (n_layer, d_in, d_in) +\end{verbatim} + +\subsection{Initialization} + +For an input polynomial zonotope \(X(\eps)\in\PZ(\R^{d_{\mathrm{in}}})\), initialize +\[ + Y^0=X(\eps), + \qquad + J^0=\Id_{d_{\mathrm{in}}}, + \qquad + H^0=0. +\] +Here \(J^0\) and \(H^0\) are constant polynomial zonotopes. The propagated jet is the jet of the network with respect to the physical input variable \(x\), evaluated over \(x\in Z_X\). It is not the jet with respect to the noise variables \(\eps\). Thus even if \(X(\eps)\) is nonlinear in \(\eps\), the correct initialization for the network map \(\Phi(x)\) is still +\[ + D_xx=\Id, + \qquad + D_x^2x=0. +\] + +\section{Affine layer propagation} + +Consider an affine layer +\[ + z=Ay+b, +\] +where \(A\in\R^{n_\ell\times n_{\ell-1}}\) and \(b\in\R^{n_\ell}\). The two-jet propagation is exact: +\[ + Y_z=AY+b, + \qquad + J_z=AJ, + \qquad + H_z=AH. +\] +In coordinates: +\[ + (Y_z)_i + = + b_i+\sum_j A_{ij}Y_j, +\] +\[ + (J_z)_{ia} + = + \sum_j A_{ij}J_{ja}, +\] +\[ + (H_z)_{iab} + = + \sum_j A_{ij}H_{jab}. +\] + +Implementation notes: +\begin{enumerate}[label=(\roman*)] + \item For \(Y\), this is ordinary matrix-vector multiplication on the coefficient tensors. + \item For \(J\), contract the layer weight matrix with the first axis of the matrix-valued coefficients. + \item For \(H\), contract the layer weight matrix with the first axis of the tensor-valued coefficients. + \item Bias \(b\) affects only the center of \(Y\), not \(J\) or \(H\). +\end{enumerate} + +\section{Tanh activation enclosure} + +Consider a scalar preactivation polynomial zonotope \(Z_i\in\PZ(\R)\). First compute an interval enclosure +\[ + I_i=[\ell_i,u_i] + \supseteq + Z_i([-1,1]^p). +\] +On \(I_i\), compute a rigorous polynomial approximation of \(\tanh\): +\[ + \forall t\in I_i: + \qquad + \tanh(t)\in p_i(t)+[-\Delta_i,\Delta_i]. +\] + +\subsection{Remez/minimax polynomial} + +The preferred polynomial \(p_i\) is a degree-\(q\) minimax or near-minimax polynomial: +\[ + p_i + \approx + \operatorname*{argmin}_{p\in\mathbb{P}_q} + \sup_{t\in I_i}|\tanh(t)-p(t)|. +\] +The implementation may compute \(p_i\) numerically using a Remez algorithm or a reliable approximation backend. The numerical Remez result is not by itself a proof. It only proposes a good polynomial. A separate validation step must certify the error. + +The degree \(q\) must be user-configurable, for example by an argument named \texttt{remez\_degree}. The implementation must not hard-code a first-degree approximation. Larger \(q\) typically decreases the certified approximation error \(\Delta_i\), but it also increases the polynomial degree and the number of monomials after evaluating \(p_i(Z_i)\). + +\subsection{Certified residual error} + +Define the residual +\[ + r_i(t)=\tanh(t)-p_i(t). +\] +We need a rigorous number \(\Delta_i\) such that +\[ + \Delta_i + \geq + \sup_{t\in I_i}|r_i(t)|. +\] +The preferred certification method is residual root isolation. Since maxima of \(|r_i|\) occur at endpoints or critical points of \(r_i\), compute +\[ + r_i'(t)=1-\tanh(t)^2-p_i'(t). +\] +Then: +\begin{enumerate}[label=(\roman*)] + \item isolate all roots of \(r_i'\) in \(I_i\) using interval Newton and interval bisection; + \item obtain small certified intervals \(C_{i,1},\ldots,C_{i,m}\) containing all critical points; + \item evaluate \(r_i\) with outward-rounded interval arithmetic on \([\ell_i,\ell_i]\), \([u_i,u_i]\), and every \(C_{i,k}\); + \item set \(\Delta_i\) to the upward-rounded maximum of the resulting absolute interval bounds. +\end{enumerate} +Formally, +\[ + \Delta_i + := + \roundup + \max\left\{ + |r_i(\ell_i)|, + |r_i(u_i)|, + \sup |r_i(C_{i,1})|, + \ldots, + \sup |r_i(C_{i,m})| + \right\}. +\] + +\subsection{Fallback residual certification} + +If root isolation is not implemented initially, use adaptive subdivision: +\[ + I_i=\bigcup_{k=1}^M I_{i,k}. +\] +On each subinterval, evaluate +\[ + r_i(I_{i,k}) + = + \tanh(I_{i,k})-p_i(I_{i,k}) +\] +using outward-rounded interval arithmetic. Then set +\[ + \Delta_i + := + \roundup + \max_k \sup |r_i(I_{i,k})|. +\] +This may be less sharp but is easier to implement and still rigorous if all interval operations are outward-rounded. + +\section{Derivative enclosures for \texorpdfstring{\(\tanh\)}{tanh} from the same approximation} + +Once the certified approximation +\[ + \tanh(t)\in p(t)+\Delta\eta, + \qquad + \eta\in[-1,1], +\] +is available, do not compute separate Remez approximations for \(\tanh'\) and \(\tanh''\). Instead use the recursive derivative identities. + +Define polynomials \(q_k\) by +\[ + q_0(y)=y, +\] +\[ + q_{k+1}(y)=(1-y^2)q_k'(y). +\] +Then +\[ + \tanh^{(k)}(t)=q_k(\tanh(t)). +\] +For the two-jet case: +\[ + q_1(y)=1-y^2, +\] +\[ + q_2(y)=-2y+2y^3. +\] +For neuron \(i\), introduce a fresh error noise variable \(\eta_i\in[-1,1]\) and define +\[ + S_i:=p_i(Z_i)+\Delta_i\eta_i. +\] +Then +\[ + S_i' := 1-S_i^2 +\] +is a polynomial-zonotope enclosure of \(\tanh'(Z_i)\), and +\[ + S_i'' := -2S_i+2S_i^3 +\] +is a polynomial-zonotope enclosure of \(\tanh''(Z_i)\). + +The implementation should preserve powers such as \(\eta_i^2\) and \(\eta_i^3\) whenever the degree budget allows it. Replacing these powers by new affine error variables is a reduction step, not the primary representation. + +\section{Componentwise tanh two-jet propagation} + +Let +\[ + y_i=\tanh(z_i) +\] +be the \(i\)-th scalar activation. Suppose the preactivation two-jet data are +\[ + Z_i, + \qquad + J_{z,i}\in\PZ(\R^{d_{\mathrm{in}}}), + \qquad + H_{z,i}\in\PZ(\R^{d_{\mathrm{in}}\times d_{\mathrm{in}}}). +\] +Let \(S_i\), \(S_i'\), and \(S_i''\) be the activation enclosures from the previous section. Then the forward-mode two-jet propagation is +\[ + Y_i=S_i, +\] +\[ + J_i=S_i'J_{z,i}, +\] +\[ + H_i=S_i''\,J_{z,i}\otimes J_{z,i}+S_i'H_{z,i}. +\] +In coordinates: +\[ + \partial_a y_i + = + S_i'\,\partial_a z_i, +\] +\[ + \partial_{ab}^2y_i + = + S_i''(\partial_a z_i)(\partial_b z_i) + + + S_i'\partial_{ab}^2z_i. +\] +Implementation notes: +\begin{enumerate}[label=(\roman*)] + \item \(S_i'\) is scalar-valued and multiplies the vector-valued polynomial zonotope \(J_{z,i}\). + \item \(S_i''\) is scalar-valued and multiplies the matrix-valued polynomial zonotope \(J_{z,i}\otimes J_{z,i}\). + \item \(S_i'H_{z,i}\) is scalar times matrix-valued polynomial zonotope. + \item The Hessian \(H_i\) should be symmetric in the last two indices. The implementation may either store the full matrix or store only the upper triangular part. The first implementation should store the full tensor for simplicity. +\end{enumerate} + +\section{Full network propagation algorithm} + +\subsection{Inputs} + +The main user-facing method should accept: +\begin{enumerate}[label=(\roman*)] + \item a PyTorch \texttt{nn.Sequential} model with supported layers; + \item an input polynomial zonotope \(X\); + \item a configurable Remez degree \(q\), for example \texttt{remez\_degree}; + \item a maximum polynomial degree or maximum number of terms; + \item root-isolation and residual-certification tolerances; + \item reduction options. +\end{enumerate} + +\subsection{Pseudocode} + +\begin{verbatim} +def eval_pz_twojet(model, X, chebyshev_degree, options): + # X is a PolynomialZonotope with shape (d_in,) + d_in = X.shape[0] + + Y = X + J = PolynomialZonotope.constant(identity(d_in), shape=(d_in, d_in)) + H = PolynomialZonotope.constant(zeros(d_in, d_in, d_in), + shape=(d_in, d_in, d_in)) + + jet = PZTwoJet(Y=Y, J=J, H=H) + + for layer in model: + if isinstance(layer, nn.Linear): + jet = pz_twojet_linear(layer, jet) + + elif isinstance(layer, nn.Tanh): + jet = pz_twojet_tanh(layer, jet, + chebyshev_degree=chebyshev_degree, + options=options) + + else: + raise NotImplementedError( + "Polynomial-zonotope two-jet propagation " + "does not yet support this layer." + ) + + if options.reduce: + jet = jet.reduce(options) + + return jet +\end{verbatim} + +\subsection{Tanh layer pseudocode} + +\begin{verbatim} +def pz_twojet_tanh(layer, jet, chebyshev_degree, options): + Z = jet.Y + Jz = jet.J + Hz = jet.H + + Ys = [] + Js = [] + Hs = [] + + for i in range(Z.output_dim): + Zi = Z.component(i) + Jzi = Jz.component(i) # shape (d_in,) + Hzi = Hz.component(i) # shape (d_in, d_in) + + Ii = Zi.interval_enclosure(outward=True) + + p_i = compute_chebyshev_tanh(Ii, degree=chebyshev_degree) + Delta_i = certify_tanh_residual(Ii, p_i, options) + + eta_i = new_noise_symbol() + + Si = evaluate_polynomial_on_pz(p_i, Zi) + Delta_i * eta_i + Sip = 1 - Si * Si + Sipp = -2 * Si + 2 * Si * Si * Si + + Yi = Si + Ji = Sip * Jzi + Hi = Sipp * tensor_outer(Jzi, Jzi) + Sip * Hzi + + Ys.append(Yi) + Js.append(Ji) + Hs.append(Hi) + + return PZTwoJet( + Y=stack_pz(Ys), + J=stack_pz(Js), + H=stack_pz(Hs) + ) +\end{verbatim} + +\section{Soundness theorem} + +\begin{theorem}[Soundness of polynomial-zonotope two-jet propagation] +Assume the following: +\begin{enumerate}[label=(\roman*)] + \item every polynomial-zonotope arithmetic operation used by the implementation is inclusion-preserving; + \item every reduction or re-enclosure step is inclusion-preserving; + \item for every tanh activation neuron \(i\), the certified approximation satisfies + \[ + \forall t\in I_i: + \qquad + \tanh(t)\in p_i(t)+[-\Delta_i,\Delta_i]; + \] + \item every scalar preactivation polynomial zonotope \(Z_i\) satisfies + \[ + Z_i([-1,1]^p)\subseteq I_i. + \] +\end{enumerate} +Then the final propagated objects +\[ + Y^L\in\PZ(\R^{d_{\mathrm{out}}}), + \qquad + J^L\in\PZ(\R^{d_{\mathrm{out}}\times d_{\mathrm{in}}}), + \qquad + H^L\in\PZ(\R^{d_{\mathrm{out}}\times d_{\mathrm{in}}\times d_{\mathrm{in}}}) +\] +satisfy +\[ + \forall x\in Z_X: + \qquad + \Phi(x)\in Y^L, +\] +\[ + D\Phi|_x\in J^L, +\] +and +\[ + D^2\Phi|_x\in H^L. +\] +\end{theorem} + +\begin{proof} +The proof is by induction over layers. At the input layer, +\[ + Y^0=X, + \qquad + J^0=\Id, + \qquad + H^0=0, +\] +so the statement is exact. + +For an affine layer \(z=Ay+b\), the chain rule gives +\[ + Dz=A\,Dy, + \qquad + D^2z=A\,D^2y. +\] +The implemented propagation +\[ + Y_z=AY+b, + \qquad + J_z=AJ, + \qquad + H_z=AH +\] +therefore gives exact enclosures, assuming polynomial-zonotope linear maps are implemented inclusion-preservingly. + +For a tanh layer, consider one neuron \(i\). By assumption, +\[ + \tanh(z_i)\in p_i(z_i)+\Delta_i\eta_i=:S_i. +\] +The identities +\[ + \tanh'(t)=1-\tanh(t)^2, +\] +\[ + \tanh''(t)=-2\tanh(t)+2\tanh(t)^3 +\] +imply that +\[ + S_i'=1-S_i^2 +\] +encloses \(\tanh'(z_i)\), and +\[ + S_i''=-2S_i+2S_i^3 +\] +encloses \(\tanh''(z_i)\), provided polynomial-zonotope multiplication is inclusion-preserving. The second-order chain rule gives +\[ + Dy_i=\tanh'(z_i)Dz_i, +\] +\[ + D^2y_i=\tanh''(z_i)Dz_i\otimes Dz_i+\tanh'(z_i)D^2z_i. +\] +Thus the implemented formulas +\[ + J_i=S_i'J_{z,i}, +\] +\[ + H_i=S_i''J_{z,i}\otimes J_{z,i}+S_i'H_{z,i} +\] +are valid enclosures. This proves the induction step. The conclusion follows after the last layer. +\end{proof} + +\section{Practical implementation plan for \texttt{intervalNets}} + +The following modules and classes are suggested. + +\subsection{\texttt{PolynomialZonotope}} + +A core file such as \path{src/intervalnets/pz.py} should define the class \texttt{PolynomialZonotope}. Responsibilities: +\begin{enumerate}[label=(\roman*)] + \item store center and monomial coefficients; + \item store exponent vectors; + \item support vector-, matrix-, and tensor-valued coefficients; + \item implement addition, scalar multiplication, multiplication, linear maps, tensor products, stacking, slicing, and component extraction; + \item provide interval enclosure through existing outward-rounded interval types; + \item provide optional reduction. +\end{enumerate} + +\subsection{\texttt{PZTwoJet}} + +A file such as \path{src/intervalnets/pz_twojet.py} should define \texttt{PZTwoJet}. Responsibilities: +\begin{enumerate}[label=(\roman*)] + \item carry \texttt{Y}, \texttt{J}, and \texttt{H}; + \item expose shape checks; + \item expose reduction; + \item expose conversion to interval enclosures for debugging and testing. +\end{enumerate} + +\subsection{Activation approximation} + +A file such as \path{src/intervalnets/activation_approx.py} should implement: +\begin{enumerate}[label=(\roman*)] + \item \texttt{compute\_remez\_tanh(interval, degree)}; + \item \texttt{certify\_tanh\_residual(interval, polynomial, options)}; + \item fallback adaptive interval residual certification. +\end{enumerate} + +\subsection{Residual certification} + +A file such as \path{src/intervalnets/residual_certification.py} should implement: +\begin{enumerate}[label=(\roman*)] + \item interval evaluation of \(r(t)=\tanh(t)-p(t)\); + \item interval evaluation of \(r'(t)=1-\tanh(t)^2-p'(t)\); + \item interval Newton or bisection root isolation for \(r'\); + \item upward-rounded residual maximum computation. +\end{enumerate} + +\subsection{Layer propagation} + +A file such as \path{src/intervalnets/pz_layers.py} should implement: +\begin{enumerate}[label=(\roman*)] + \item \texttt{pz\_twojet\_linear(layer, jet)}; + \item \texttt{pz\_twojet\_tanh(layer, jet, remez\_degree, options)}; + \item optional support dispatch for \texttt{nn.Sequential}. +\end{enumerate} + +\subsection{User-facing integration} + +The repository already uses opt-in monkey patching for interval evaluation. The new functionality should follow the same style. For example, one could add \texttt{enable\_pz\_jet\_eval()}. After enabling, a user should be able to call: +\begin{verbatim} +from intervalnets import PolynomialZonotope, enable_pz_jet_eval + +enable_pz_jet_eval() + +X = PolynomialZonotope.from_generators(center, generators, exponents) +jet = model.eval_pz_twojet(X, chebyshev_degree=5) + +Y = jet.Y +J = jet.J +H = jet.H +\end{verbatim} + +\section{Testing plan} + +\subsection{Unit tests for polynomial-zonotope arithmetic} + +Implement tests for: +\begin{enumerate}[label=(\roman*)] + \item addition with exponent merging; + \item scalar multiplication; + \item scalar polynomial-zonotope multiplication; + \item scalar times vector-valued polynomial zonotope; + \item scalar times matrix-valued polynomial zonotope; + \item tensor products; + \item stacking and component extraction; + \item reduction preserving enclosure. +\end{enumerate} +For each operation, compare against dense sampling over \(\eps\in[-1,1]^p\) as a sanity check. Sampling is not a proof but is useful for detecting implementation bugs. + +\subsection{Unit tests for interval enclosure} + +For random polynomial zonotopes \(Z\), verify numerically that sampled values lie inside the computed interval enclosure. Also test degenerate cases: +\begin{enumerate}[label=(\roman*)] + \item no generators; + \item zero generators; + \item repeated exponents; + \item high-degree monomials; + \item mixed tensor-valued coefficients. +\end{enumerate} + +\subsection{Tests for tanh residual certification} + +For random intervals \(I\) and degrees \(q\): +\begin{enumerate}[label=(\roman*)] + \item compute \(p\) by Remez or fallback approximation; + \item certify \(\Delta\); + \item verify by dense sampling that \(|\tanh(t)-p(t)|\leq\Delta\); + \item test wide, narrow, positive, negative, and symmetric intervals. +\end{enumerate} +The dense sampling check is only a sanity test; the actual guarantee comes from interval certification. + +\subsection{Tests against PyTorch autograd} + +For small networks and random input polynomial zonotopes: +\begin{enumerate}[label=(\roman*)] + \item compute \((Y,J,H)=\texttt{model.eval\_pz\_twojet}(X)\); + \item sample \(x=X(\eps)\); + \item compute \(\Phi(x)\), \(D\Phi|_x\), and \(D^2\Phi|_x\) using PyTorch autograd; + \item verify that the sampled pointwise values are contained in the corresponding polynomial-zonotope interval enclosures. +\end{enumerate} + +\subsection{Exact small-network tests} + +Use networks where the exact formulas are simple: +\[ + \Phi(x)=Ax+b, +\] +\[ + \Phi(x)=\tanh(ax+b), +\] +\[ + \Phi(x)=\tanh(Ax+b) +\] +in low dimensions. For affine networks, the Hessian enclosure must be exactly zero. + +\subsection{Shape tests} + +Verify that: +\[ + \texttt{Y.shape}=(d_{\mathrm{out}},), +\] +\[ + \texttt{J.shape}=(d_{\mathrm{out}},d_{\mathrm{in}}), +\] +\[ + \texttt{H.shape}=(d_{\mathrm{out}},d_{\mathrm{in}},d_{\mathrm{in}}). +\] +Also test batched or unsupported inputs explicitly and fail with clear error messages if batching is not supported initially. + +\subsection{Regression tests after reduction} + +If reduction is implemented, test that reduction never invalidates sampled containment. For randomly generated polynomial zonotopes, compare the interval enclosure before and after reduction: +\[ + Z_{\mathrm{before}}\subseteq Z_{\mathrm{after}}. +\] +This should hold as a set enclosure, at least as verified by interval bounds and dense sampling sanity checks. + +\section{Limitations and future extensions} + +\subsection{Representation growth} + +Polynomial multiplication causes rapid growth in the number of monomials. The two-jet propagation through tanh layers introduces additional noise variables and powers of these variables. Therefore reduction strategies are essential for deeper networks. + +\subsection{Dependency preservation versus re-enclosure} + +Keeping polynomial dependence is sharper but more expensive. Re-enclosing discarded terms by intervals or independent noise variables is cheaper but increases overestimation. The implementation should make this tradeoff explicit through user options. + +\subsection{Higher-order jets} + +The same strategy extends to higher-order jets. The activation derivatives can be generated recursively by +\[ + q_{k+1}(y)=(1-y^2)q_k'(y). +\] +However, higher-order tensors grow quickly. The first implementation should focus on the two-jet. + +\subsection{Other activations} + +The framework can be extended to other smooth activations if one can provide: +\begin{enumerate}[label=(\roman*)] + \item a certified polynomial approximation for the activation; + \item certified derivative enclosures, preferably derived algebraically from the activation enclosure. +\end{enumerate} + +\subsection{Chebyshev interpolation fallback} + +If Remez is unavailable or unstable, Chebyshev interpolation can be used to obtain a good polynomial candidate. The residual must still be certified rigorously by root isolation or interval residual subdivision. + +\subsection{Certified operator norm bounds} + +The final Hessian polynomial zonotope can be converted into certified interval tensor bounds. From these, one can derive scalar operator norm or Frobenius norm bounds. For example, if +\[ + \boldsymbol H_{iab}=[\ell_{iab},u_{iab}] +\] +is an interval enclosure of the Hessian tensor components, then +\[ + \left( + \sum_{i,a,b} + \max\{|\ell_{iab}|,|u_{iab}|\}^2 + \right)^{1/2} +\] +is a valid Frobenius-type upper bound, and hence also an upper bound for the corresponding operator norm. + +\section{Minimal implementation milestones} + +The recommended order of implementation is: +\begin{enumerate}[label=\textbf{M\arabic*.}] + \item Implement \texttt{PolynomialZonotope} with addition, scalar multiplication, multiplication, stacking, slicing, and interval enclosure. + \item Implement \texttt{PZTwoJet}. + \item Implement affine layer propagation. + \item Implement a simple tanh approximation with conservative interval residual subdivision. + \item Implement tanh two-jet propagation. + \item Add \texttt{model.eval\_pz\_twojet(...)} for \texttt{nn.Sequential}. + \item Add Remez/minimax polynomial generation with configurable degree \(q\). + \item Add residual root isolation for sharper certified \(\Delta\). + \item Add reduction strategies for polynomial-zonotope growth. + \item Add full test coverage and examples. +\end{enumerate} + +\section{Summary} + +This extension adds a validated forward-mode two-jet propagation engine to \texttt{intervalNets}. The core object is a triple +\[ + (Y,J,H), +\] +where \(Y\) encloses the network output, \(J\) encloses the Jacobian field, and \(H\) encloses the Hessian field over a polynomial-zonotope input set. Affine layers are propagated exactly. Tanh layers are handled by a certified polynomial approximation of \(\tanh\), together with algebraic derivative enclosures +\[ + \tanh'(t)=1-\tanh(t)^2, + \qquad + \tanh''(t)=-2\tanh(t)+2\tanh(t)^3. +\] +The implementation should preserve polynomial dependence on uncertainty variables whenever possible, while providing rigorous reduction and interval re-enclosure mechanisms to control growth. + +\end{document} diff --git a/docs/certified_polynomial_zonotope_integration.tex b/docs/certified_polynomial_zonotope_integration.tex new file mode 100644 index 0000000..277d50d --- /dev/null +++ b/docs/certified_polynomial_zonotope_integration.tex @@ -0,0 +1,594 @@ +\documentclass[11pt]{article} + +\usepackage[a4paper,margin=1in]{geometry} +\usepackage{amsmath,amssymb,amsthm,mathtools} +\usepackage{enumitem} +\usepackage{microtype} +\usepackage[hidelinks]{hyperref} + +\newtheorem{remark}{Remark} +\newtheorem{algorithmblock}{Algorithm} +\newcommand{\R}{\mathbb{R}} +\newcommand{\N}{\mathbb{N}} +\newcommand{\abs}[1]{\left\lvert #1\right\rvert} +\newcommand{\norm}[1]{\left\lVert #1\right\rVert} + +\title{Certified Integration over Polynomial-Zonotope Domains} +\author{} +\date{} + +\begin{document} +\maketitle + +\section{Setting and interpretation} +\label{sec:pz-integration} + +Let +\[ +\Xi=[-1,1]^s +\] +be a reference parameter box, and let +\[ +X:\Xi\to\R^n +\] +be a polynomial map of the form +\begin{equation} + X(\alpha) + = + x_0+\sum_{\lambda\in\Lambda_X} x_\lambda \alpha^\lambda, + \qquad + \alpha=(\alpha_1,\ldots,\alpha_s)\in\Xi, + \label{eq:pz-parametrization} +\end{equation} +where +\[ + x_0,x_\lambda\in\R^n, + \qquad + \alpha^\lambda + := + \prod_{j=1}^s \alpha_j^{\lambda_j}. +\] +The corresponding polynomial zonotope is +\[ + \mathcal{X}=X(\Xi). +\] + +Let \(f:\mathcal{X}\to\R\) be a scalar-valued function. Assume that a certified polynomial enclosure of \(f\) on \(\mathcal{X}\) has been computed: +\begin{equation} + f(x)=\widehat f(x)+r(x), + \qquad + \abs{r(x)}\leq \rho + \quad\text{for all }x\in\mathcal{X}, + \label{eq:polynomial-remainder-enclosure} +\end{equation} +where \(\widehat f:\R^n\to\R\) is a polynomial approximation and +\(\rho\geq0\) is its certified approximation-error radius. + +Equivalently, one may write the pointwise enclosure as +\begin{equation} + f(x)\in \widehat f(x)+\rho[-1,1]. + \label{eq:pointwise-noise} +\end{equation} +Equivalently, one may use an approximation noise symbol \(\eta_x\in[-1,1]\) +at each point and write +\[ + f(x)=\widehat f(x)+\rho\eta_x. +\] +The residual function \(r(x)=f(x)-\widehat f(x)\) satisfies +\(\abs{r(x)}\leq\rho\). The notation does not assert that one fixed scalar +value of \(\eta_x\) represents the residual simultaneously at every point. + +There are two different integrals associated with the parametrization \eqref{eq:pz-parametrization}: +\begin{enumerate}[label=\arabic*.] + \item the parameter-space integral + \[ + \int_\Xi f(X(\alpha))\,d\alpha, + \] + \item the geometric integral + \[ + \int_{\mathcal{X}} f(x)\,dx, + \] + where \(dx\) denotes \(n\)-dimensional Lebesgue measure. +\end{enumerate} +These integrals agree only in special cases. The geometric integral requires an appropriate Jacobian factor. + +\section{Exact integration of polynomials on the reference box} + +Let +\[ + q(\alpha)=\sum_{\lambda\in\Lambda} q_\lambda\alpha^\lambda +\] +be a polynomial on \(\Xi=[-1,1]^s\). Then +\begin{equation} + \int_\Xi q(\alpha)\,d\alpha + = + \sum_{\lambda\in\Lambda} q_\lambda\,\mu_\lambda, + \label{eq:box-polynomial-integration} +\end{equation} +where +\begin{equation} + \mu_\lambda + := + \int_\Xi \alpha^\lambda\,d\alpha + = + \prod_{j=1}^s\int_{-1}^1 t^{\lambda_j}\,dt. + \label{eq:box-moment} +\end{equation} +For every \(k\in\N_0\), +\begin{equation} + \int_{-1}^1 t^k\,dt + = + \begin{cases} + 0, & k\text{ odd},\\[1mm] + \displaystyle\frac{2}{k+1}, & k\text{ even}. + \end{cases} + \label{eq:one-dimensional-moment} +\end{equation} +Consequently, +\begin{equation} + \mu_\lambda + = + \begin{cases} + \displaystyle\prod_{j=1}^s \frac{2}{\lambda_j+1}, + & \lambda_j\text{ is even for every }j,\\[3mm] + 0, & \text{otherwise}. + \end{cases} + \label{eq:multivariate-moment} +\end{equation} +Thus polynomial integration over the reference box is a linear coefficient operation. No range enclosure of the polynomial is needed. + +For pointwise residual terms one also needs the absolute monomial moment +\begin{equation} + \mu^{\mathrm{abs}}_\lambda + :=\int_\Xi\abs{\alpha^\lambda}\,d\alpha + =\prod_{j=1}^s\frac{2}{\lambda_j+1}. + \label{eq:absolute-box-moment} +\end{equation} +Unlike the signed moment \(\mu_\lambda\), this expression is nonzero for odd +exponents. It bounds a pointwise residual product without pretending that +one fixed approximation-noise value is shared across the domain. + +\subsection{Coefficientwise pointwise-residual functional} + +Consider a canonical term +\[ + c_{\beta,\nu}\alpha^\beta\eta(\alpha)^\nu +\] +that contains at least one pointwise residual factor. Canonicalization means +that every contribution with the same full exponent \((\beta,\nu)\) has +already been summed before an absolute value is taken. The basic sound rule is +\begin{equation} + \left|\int_\Xi + c_{\beta,\nu}\alpha^\beta\eta(\alpha)^\nu\,d\alpha\right| + \leq + \abs{c_{\beta,\nu}}\mu^{\mathrm{abs}}_\beta. + \label{eq:coefficientwise-absolute-rule} +\end{equation} +This improves the cruder replacement of +\(\mu^{\mathrm{abs}}_\beta\) by the full measure \(2^s\). + +There is a further parity refinement. If every non-domain uncertainty power +is even, its product belongs to \([0,1]\). If every domain power is also even, +the integrated term lies between zero and +\(c_{\beta,\nu}\mu^{\mathrm{abs}}_\beta\). It is represented by +\begin{equation} + m_{\beta,\nu}=\frac12c_{\beta,\nu}\mu^{\mathrm{abs}}_\beta, + \qquad + r_{\beta,\nu}=\frac12\abs{c_{\beta,\nu}}\mu^{\mathrm{abs}}_\beta. + \label{eq:even-pointwise-midpoint-radius} +\end{equation} +If all uncertainty powers are even but some domain power is odd, symmetry of +the box gives equal positive and negative absolute moments, and the sound +enclosure has zero midpoint and radius +\begin{equation} + r_{\beta,\nu}=\frac12\abs{c_{\beta,\nu}} + \mu^{\mathrm{abs}}_\beta. + \label{eq:odd-domain-even-noise-radius} +\end{equation} +If any relevant uncertainty power is odd, the full symmetric radius in +\eqref{eq:coefficientwise-absolute-rule} is retained. Interactions with +global symbolic variables are refined only when their parity makes the range +classification unambiguous; otherwise the symmetric rule is used. + +This functional is still coefficientwise and therefore generally not sharp. +For +\[ + Q(\alpha,\eta)=\sum_\nu c_\nu(\alpha)\eta^\nu, +\] +the theoretical pointwise target is +\begin{equation} + \left[ + \int_\Xi\min_{\eta\in[-1,1]^q}Q(\alpha,\eta)\,d\alpha, + \int_\Xi\max_{\eta\in[-1,1]^q}Q(\alpha,\eta)\,d\alpha + \right]. + \label{eq:sharp-pointwise-target} +\end{equation} +Computing this parameterized polynomial optimization problem is outside the +coefficientwise integration method. + +\section{Integration with respect to parameter measure} + +Define the polynomial pullback +\begin{equation} + q(\alpha):=(\widehat f\circ X)(\alpha). + \label{eq:pullback-polynomial} +\end{equation} +Since both \(\widehat f\) and \(X\) are polynomial, \(q\) is polynomial. + +From \eqref{eq:polynomial-remainder-enclosure}, +\[ + f(X(\alpha))=q(\alpha)+r(X(\alpha)), + \qquad + \abs{r(X(\alpha))}\leq \rho. +\] +Therefore, +\begin{align} + \int_\Xi f(X(\alpha))\,d\alpha + &= + \int_\Xi q(\alpha)\,d\alpha + + + \int_\Xi r(X(\alpha))\,d\alpha \\ + &\in + \int_\Xi q(\alpha)\,d\alpha + + + \rho\,\abs{\Xi}[-1,1]. +\end{align} +Since \(\abs{\Xi}=2^s\), +\begin{equation} + \boxed{ + \int_\Xi f(X(\alpha))\,d\alpha + \in + I_{\mathrm{poly}}+2^sE[-1,1], + } + \label{eq:parameter-integral-enclosure} +\end{equation} +where +\[ + I_{\mathrm{poly}}:=\int_\Xi q(\alpha)\,d\alpha. +\] +Equivalently, introducing one fresh approximation noise symbol for the final +scalar integral, +\begin{equation} + \int_\Xi f(X(\alpha))\,d\alpha + \in + I_{\mathrm{poly}}+2^sE\,\eta_{\mathrm{int}}, + \qquad + \eta_{\mathrm{int}}\in[-1,1]. + \label{eq:parameter-integral-noise} +\end{equation} + +\section{Geometric integration over the polynomial-zonotope image} + +Assume now that \(s=n\), and that \(X:\Xi\to\mathcal{X}\subset\R^n\) is injective and continuously differentiable, with +\[ + \det\left.\partial X\right|_\alpha\neq 0 + \qquad + \text{for all }\alpha\in\Xi. +\] +Then the change-of-variables formula gives +\begin{equation} + \int_{\mathcal{X}} f(x)\,dx + = + \int_\Xi f(X(\alpha))J_X(\alpha)\,d\alpha, + \label{eq:change-of-variables} +\end{equation} +where +\begin{equation} + J_X(\alpha):= + \abs{\det\left.\partial X\right|_\alpha}. + \label{eq:jacobian-density} +\end{equation} +Using \eqref{eq:polynomial-remainder-enclosure}, +\begin{align} + \int_{\mathcal{X}} f(x)\,dx + &= + \int_\Xi \widehat f(X(\alpha))J_X(\alpha)\,d\alpha + + + \int_\Xi r(X(\alpha))J_X(\alpha)\,d\alpha. +\end{align} +Since \(J_X(\alpha)\geq 0\), +\[ + \abs{\int_\Xi r(X(\alpha))J_X(\alpha)\,d\alpha} + \leq + \rho\int_\Xi J_X(\alpha)\,d\alpha. +\] +Moreover, +\begin{equation} + \abs{\mathcal{X}}=\int_\Xi J_X(\alpha)\,d\alpha. + \label{eq:image-volume} +\end{equation} +Hence +\begin{equation} + \boxed{ + \int_{\mathcal{X}} f(x)\,dx + \in + I_{\mathrm{poly}}+\rho\abs{\mathcal{X}}[-1,1], + } + \label{eq:geometric-integral-general} +\end{equation} +where +\begin{equation} + I_{\mathrm{poly}} + := + \int_\Xi \widehat f(X(\alpha))J_X(\alpha)\,d\alpha. + \label{eq:geometric-polynomial-part} +\end{equation} + +\section{Fixed-orientation case} + +Suppose that the sign of the Jacobian determinant is known: +\begin{equation} + \sigma\det\left.\partial X\right|_\alpha>0 + \qquad + \text{for all }\alpha\in\Xi, + \qquad + \sigma\in\{-1,1\}. + \label{eq:fixed-orientation} +\end{equation} +Then +\[ + J_X(\alpha)=\sigma\det\left.\partial X\right|_\alpha. +\] +Because \(X\) is polynomial, both \(\det\left.\partial X\right|_\alpha\) and \(\widehat f(X(\alpha))\det\left.\partial X\right|_\alpha\) are polynomials in \(\alpha\). Therefore, +\begin{equation} + I_{\mathrm{poly}} + = + \sigma\int_\Xi \widehat f(X(\alpha))\det\left.\partial X\right|_\alpha\,d\alpha, + \label{eq:fixed-orientation-poly-integral} +\end{equation} +and +\begin{equation} + \abs{\mathcal{X}} + = + \sigma\int_\Xi \det\left.\partial X\right|_\alpha\,d\alpha + \label{eq:fixed-orientation-volume} +\end{equation} +can both be computed by exact polynomial moment integration. + +\section{Affine-zonotope special case} + +Suppose +\begin{equation} + X(\alpha)=c+G\alpha, + \qquad + G\in\R^{n\times n}, + \qquad + \det G\neq 0. + \label{eq:affine-zonotope-map} +\end{equation} +Then \(\left.\partial X\right|_\alpha=G\) and \(J_X(\alpha)=\abs{\det G}\) is constant. Hence +\begin{equation} + \int_{\mathcal{X}} f(x)\,dx + \in + \abs{\det G}\int_\Xi \widehat f(c+G\alpha)\,d\alpha + + + \rho\abs{\det G}\,2^n[-1,1]. + \label{eq:affine-zonotope-integral} +\end{equation} +Since \(\abs{\mathcal{X}}=2^n\abs{\det G}\), this becomes +\begin{equation} + \boxed{ + \int_{\mathcal{X}} f(x)\,dx + \in + \abs{\det G}\int_\Xi \widehat f(c+G\alpha)\,d\alpha + + + \rho\abs{\mathcal{X}}[-1,1]. + } + \label{eq:affine-zonotope-boxed} +\end{equation} + +\section{Jacobian sign changes and injectivity} + +If \(\det\partial X\) changes sign on \(\Xi\), then \(\abs{\det\partial X}\) is generally only piecewise polynomial. A practical certified procedure is to subdivide +\[ + \Xi=\bigcup_{\ell=1}^N \Xi_\ell +\] +into boxes with pairwise disjoint interiors until the sign of \(\det\partial X\) can be certified on every \(\Xi_\ell\). On each subbox, choose \(\sigma_\ell\in\{-1,1\}\) such that +\[ + \sigma_\ell\det\left.\partial X\right|_\alpha>0 + \qquad + \text{for all }\alpha\in\Xi_\ell. +\] +Then each local contribution is again the integral of a polynomial after affine rescaling to \([-1,1]^n\). + +If injectivity of \(X\) is not known, the parameter-space integral +\[ + \int_\Xi f(X(\alpha))\abs{\det\left.\partial X\right|_\alpha}\,d\alpha +\] +may count points of the image with multiplicity. Therefore certified geometric integration over \(\mathcal{X}\) requires either a proof that \(X\) is injective on \(\Xi\), or a subdivision into pieces on which \(X\) is injective and whose images overlap only on sets of measure zero. + +\section{Spatially varying approximation-error radius} + +Assume more generally that +\begin{equation} + f(x)=\widehat f(x)+r(x), + \qquad + \abs{r(x)}\leq \rho(x) + \quad + \text{for all }x\in\mathcal{X}, + \label{eq:variable-remainder} +\end{equation} +where \(\rho:\mathcal{X}\to[0,\infty)\). Then +\begin{equation} + \int_{\mathcal{X}} f(x)\,dx + \in + \int_{\mathcal{X}}\widehat f(x)\,dx + + + \left[ + -\int_{\mathcal{X}}\rho(x)\,dx, + \int_{\mathcal{X}}\rho(x)\,dx + \right]. + \label{eq:variable-remainder-integral} +\end{equation} +If \(\rho\circ X\) is polynomial and the Jacobian sign is fixed, then the +integrated approximation-error radius can itself be computed exactly by +polynomial moment integration. + +\section{Several cells} + +Suppose +\[ + \Omega=\bigcup_{k=1}^N\mathcal{X}_k +\] +with pairwise disjoint interiors, and assume +\[ + f(x)=p_k(x)+r_k(x), + \qquad + \abs{r_k(x)}\leq \rho_k + \quad + \text{for }x\in\mathcal{X}_k. +\] +Let +\[ + I_k^{\mathrm{poly}}:=\int_{\mathcal{X}_k}p_k(x)\,dx, + \qquad + V_k:=\abs{\mathcal{X}_k}. +\] +Then +\begin{equation} + \boxed{ + \int_\Omega f(x)\,dx + \in + \sum_{k=1}^N I_k^{\mathrm{poly}} + + + \left(\sum_{k=1}^N E_kV_k\right)[-1,1]. + } + \label{eq:multi-cell-integral} +\end{equation} +For a scalar final integral, a single fresh final approximation noise symbol +is sufficient: +\begin{equation} + \int_\Omega f(x)\,dx + \in + \sum_{k=1}^N I_k^{\mathrm{poly}} + + + \left(\sum_{k=1}^N E_kV_k\right)\eta_{\mathrm{int}}, + \qquad + \eta_{\mathrm{int}}\in[-1,1]. +\end{equation} + +\section{Coefficient-level implementation} + +Let +\[ + h(\alpha)=\sum_{\lambda\in\Lambda} h_\lambda\alpha^\lambda. +\] +Define the integration operator +\begin{equation} + \mathcal{I}(h) + := + \sum_{\lambda\in\Lambda} h_\lambda\mu_\lambda, + \label{eq:integration-operator} +\end{equation} +where \(\mu_\lambda\) is given by \eqref{eq:multivariate-moment}. The operator \(\mathcal{I}\) is linear, so it should be applied directly to sparse polynomial coefficients without first range-enclosing the polynomial. + +\begin{algorithmblock}[Exact polynomial integration on the reference box] +\label{alg:polynomial-box-integration} +Given \(h(\alpha)=\sum_{\lambda\in\Lambda}h_\lambda\alpha^\lambda\), +initialize \(I\gets0\). For every \(\lambda\in\Lambda\), set +\[ + \mu_\lambda\gets + \begin{cases} + 0,&\text{if some component \(\lambda_j\) is odd},\\[1mm] + \displaystyle\prod_{j=1}^s\frac{2}{\lambda_j+1},&\text{otherwise}, + \end{cases} +\] +and update \(I\gets I+h_\lambda\mu_\lambda\). Return \(I\). +\end{algorithmblock} + +\begin{algorithmblock}[Certified integration over a polynomial-zonotope image] +\label{alg:certified-pz-integration} +Given the parametrization \(X:\Xi=[-1,1]^n\to\R^n\), the polynomial +approximation \(\widehat f\), the bound +\(\abs{f(x)-\widehat f(x)}\leq\rho\), and certificates of injectivity and +fixed orientation, compute +\[ + d(\alpha)=\sigma\det\left.\partial X\right|_\alpha, + \quad q(\alpha)=\widehat f(X(\alpha)), + \quad h(\alpha)=q(\alpha)d(\alpha). +\] +Then set +\[ + I_{\mathrm{poly}}=\int_\Xi h(\alpha)\,d\alpha, + \qquad V=\int_\Xi d(\alpha)\,d\alpha, + \qquad R=\rho V, +\] +and return \([I_{\mathrm{poly}}-R,I_{\mathrm{poly}}+R]\). +\end{algorithmblock} + +\section{Floating-point rigor} + +The mathematical formulas above are exact. A computer implementation must additionally enclose floating-point roundoff. Two common approaches are: +\begin{enumerate}[label=\arabic*.] + \item store the polynomial coefficients as intervals and perform all coefficient operations with outward rounding; + \item store floating-point coefficients and compute an additional certified roundoff-error radius. +\end{enumerate} +If +\[ + I_{\mathrm{poly}} + \in + [\underline{I}_{\mathrm{poly}},\overline{I}_{\mathrm{poly}}] +\] +and +\[ + \abs{\mathcal{X}} + \in + [\underline{V},\overline{V}], + \qquad + 0\leq\underline{V}\leq\overline{V}, +\] +then a rigorous final enclosure is +\begin{equation} + \int_{\mathcal{X}}f(x)\,dx + \in + [\underline{I}_{\mathrm{poly}}-\rho\overline{V}, + \overline{I}_{\mathrm{poly}}+\rho\overline{V}]. + \label{eq:roundoff-safe-final} +\end{equation} + +\section{Recommended computational pipeline} + +The recommended procedure is +\[ +\boxed{ +\begin{aligned} +&\text{polynomial-zonotope parametrization} +\\ +&\quad\longrightarrow\text{polynomial pullback} +\\ +&\quad\longrightarrow\text{Jacobian multiplication} +\\ +&\quad\longrightarrow\text{exact coefficient-level moment integration} +\\ +&\quad\longrightarrow\text{add the integrated approximation-error interval}. +\end{aligned} +} +\] +In particular, one should avoid replacing the polynomial by a range interval and multiplying that interval by the volume. That procedure discards cancellation and the exact moment structure of the polynomial. + +The final enclosure has the form +\[ + \boxed{ + \int_{\mathcal{X}}f(x)\,dx + \in + I_{\mathrm{poly}}+R_{\mathrm{int}}[-1,1], + } +\] +with +\[ + R_{\mathrm{int}}=\rho\abs{\mathcal{X}} +\] +for a constant approximation-error radius, or more generally +\[ + R_{\mathrm{int}}=\int_{\mathcal{X}}\rho(x)\,dx +\] +for a spatially varying certified approximation-error radius. + +The scalable graph/Hilbert alternative is documented in +\texttt{dependency\_preserving\_graph\_hilbert\_integration.tex}. Its +discarded graph component is bounded directly in $L^2$, rather than being +converted into a fresh pointwise approximation symbol. This distinction is +what permits reverse-triangle and dual-witness lower bounds while preserving +the pointwise semantics specified in this note. + +\end{document} diff --git a/docs/dependency_preserving_graph_hilbert_integration.tex b/docs/dependency_preserving_graph_hilbert_integration.tex new file mode 100644 index 0000000..dc526b8 --- /dev/null +++ b/docs/dependency_preserving_graph_hilbert_integration.tex @@ -0,0 +1,170 @@ +\documentclass[11pt]{article} +\usepackage{amsmath,amssymb,booktabs,geometry,hyperref} +\geometry{margin=1in} + +\title{Dependency-Preserving Graph Integration and Hilbert Lower Certificates} +\author{intervalNets implementation note} +\date{} + +\begin{document} +\maketitle + +\section{Scope and semantics} + +This note specifies the graph/Hilbert certification path implemented in +\texttt{graph\_hilbert.py}. It supplements, rather than replaces, the +coefficientwise pointwise-residual functional in +\texttt{certified\_polynomial\_zonotope\_integration.tex}. Approximation +noise remains pointwise: its value may vary with the domain point. The new +method does not integrate such a symbol as one global constant. + +All formulas below use normalized measure +\[ + d\mu(x)=|\Omega|^{-1}\,dx. +\] +Raw squared integrals are recovered by multiplication with \(|\Omega|\). +For the affine box parameterization \(x=c+G\alpha\), the implemented scalable +backend currently requires an axis-aligned \(G\) for the Neumann witness, while +the exact reference moment backend works on the reference box directly. + +\section{Immutable arithmetic graph} + +The exact factored Jacobian is translated to immutable nodes for constants, +noise slices, addition, linear maps, Hadamard products, componentwise squares, +and diagonal scaling. Every node records its domain and approximation-noise +support. Nodes are hash-consed, so repeated occurrences of one approximation +symbol refer to the same graph input. For every fixed noise vector, +\[ + \operatorname{Eval}(\mathcal G;\alpha,\eta) + =\operatorname{Eval}(J_{\rm factored};\alpha,\eta). +\] + +The \texttt{SparseReferenceMomentBackend} expands a small domain-only graph to +canonical exponent maps and computes +\[ + \mathbb E_\mu\!\left[\sum_k P_k(\alpha)^2\right] + =\sum_{k,\beta,\gamma} + c_{k,\beta}c_{k,\gamma}\, + \mathbb E[\alpha^{\beta+\gamma}]. +\] +It is an exact regression oracle, not the scalable 100-dimensional backend. +Trying to expand a degree-six dense graph in 100 variables would merely hide +the original combinatorial problem. + +\section{Certified Hilbert compression} + +At every activation boundary, the scalable backend stores +\[ + y(\alpha)=\widehat y(\alpha)+e(\alpha),\qquad + \widehat y(\alpha)=a_0+A\alpha,qquad + \|e\|_{L^2_\mu(\ell_2)}\leq\varepsilon. +\] +The discarded function is a single certified Hilbert remainder. It is not +replaced by independent pointwise symbols. + +For an affine preactivation \(q=c+a^\top\alpha\), a certified polynomial +enclosure +\[ + \tanh(q)\in p(q)+[-\delta,\delta] +\] +is projected orthogonally onto +\(\operatorname{span}\{1,\alpha_1,\ldots,\alpha_d\}\). The implementation +computes the required moments of \(q^k\) without monomial expansion, by +convolving the one-dimensional uniform moments of the independent summands. +The projection remainder is +\[ + \varepsilon_{\rm proj}^2 + =\mathbb E[p(q)^2]-|\Pi_1p(q)|^2_{L^2_\mu}, +\] +with an explicit floating-point padding. Consequently, +\[ + \|\tanh(q)-\Pi_1p(q)\|_{L^2_\mu} + \leq \varepsilon_{\rm proj}+\delta. +\] +If the incoming preactivation has Hilbert error \(\varepsilon_z\), the +one-Lipschitz property of \(\tanh\) gives the vector bound +\[ + \varepsilon_y\leq \varepsilon_z+ + \left(\sum_i(\varepsilon_{{\rm proj},i}+\delta_i)^2\right)^{1/2}. +\] +Affine layers use a padded spectral-norm upper bound. + +For the reverse derivative graph, \(\tanh'\) is treated by its certified +quadratic enclosure. Products of affine graph nodes are projected without +expansion. If +\(u=c+a^\top\alpha\) and \(v=d+b^\top\alpha\), their affine projection is +\[ + \Pi_1(uv)=cd+\tfrac13a^\top b+(cb+da)^\top\alpha. +\] +Its exact fourth-moment energy is used to certify the discarded quadratic +part. The reverse error recurrence uses +\[ + aD-\widehat a\widehat D=(a-\widehat a)D + +\widehat a(D-\widehat D), +\] +the exact bound \(0\leq\tanh'\leq1\), an affine supremum for +\(\widehat a\), and padded spectral norms at linear maps. + +\section{Reverse-triangle certificates} + +For either the value field or the stacked value/gradient field, let +\(G=P+R\) with exact affine nominal energy \(A=\|P\|_\mu^2\) and +\(\|R\|_\mu\leq E\). Then +\[ + \max\{0,\sqrt A-E\}\leq\|G\|_\mu\leq\sqrt A+E. +\] +The result is intersected with the previous absolute-moment/parity interval. +Thus the new upper endpoint cannot increase and the lower endpoint cannot +decrease. + +\section{Neumann dual lower bound} + +For \(H=W^{1,2}(\Omega)\), any scalar witness \(v\) yields +\[ + \|f\|_H\geq + \frac{|\langle f,v\rangle_H|}{\|v\|_H}. +\] +The implemented basis is +\[ + \phi_0=1,\qquad + \phi_i(\alpha)=\frac{\alpha_i^3}{3}-\alpha_i. +\] +It satisfies \(\partial_n\phi_i=0\) on the boundary of an axis-aligned box. +Integration by parts therefore gives +\[ + \langle f,v\rangle_{W^{1,2}_\mu} + =\int_\Omega f(v-\Delta_xv)\,d\mu. +\] +For \(f=P+R\) and \(\|R\|_{L^2_\mu}\leq E\), +\[ + \|f\|_{W^{1,2}_\mu}\geq + \frac{\max\{0,|\langle P,v-\Delta v\rangle|-E\|v-\Delta v\|\}} + {\|v\|_{W^{1,2}_\mu}}. +\] +All Gram matrices are diagonal and analytic for this basis. A deterministic +one-parameter family of coefficient directions is searched numerically; only +the final candidate is used in the outward-padded certificate. Optimizer +optimality is never assumed. + +\section{Soundness and limitations} + +\begin{itemize} + \item The exact graph backend is used to verify graph evaluation and moment + equivalence on small cases. + \item The scalable backend is a certified compression, not exact integration + of the full 100-dimensional degree-six nominal graph. + \item Polynomial proposals are not proofs. Their uniform residuals are + certified by the repository's outward-rounded subdivision routine. + \item A larger moment budget or subdivision count is accepted only through + explicit recomputation; sampled values are diagnostics, never certificates. + \item The combined result always intersects the pre-existing certificate. +\end{itemize} + +The diagnostic decomposition +\[ + \|P\|_\mu\quad\hbox{versus}\quad E +\] +must be reported for both value and gradient fields. This distinguishes +moment/projection loss from activation-residual and graph-remainder loss. + +\end{document} diff --git a/docs/diagnostics_and_metrics_glossary.tex b/docs/diagnostics_and_metrics_glossary.tex new file mode 100644 index 0000000..806695b --- /dev/null +++ b/docs/diagnostics_and_metrics_glossary.tex @@ -0,0 +1,1193 @@ +\documentclass[11pt]{article} + +\usepackage[a4paper,margin=1in]{geometry} +\usepackage{amsmath,amssymb,mathtools} +\usepackage{booktabs,tabularx,array,longtable} +\usepackage{enumitem} +\usepackage[hidelinks]{hyperref} +\usepackage[T1]{fontenc} + +\newcommand{\R}{\mathbb{R}} +\newcommand{\N}{\mathbb{N}} +\newcommand{\eps}{\varepsilon} +\newcommand{\Sym}{\operatorname{Sym}} +\newcommand{\lb}{\operatorname{lb}} +\newcommand{\ub}{\operatorname{ub}} +\newcommand{\midp}{\operatorname{mid}} +\newcommand{\rad}{\operatorname{rad}} +\newcommand{\wid}{\operatorname{wid}} +\newcommand{\magI}{\operatorname{mag}} +\newcommand{\mig}{\operatorname{mig}} +\newcommand{\Mean}{\operatorname{Mean}} +\newcommand{\Max}{\operatorname{Max}} +\newcommand{\NA}{\texttt{NA}} + +\title{Diagnostics and Metrics for Polynomial-Zonotope\\ +Neural-Network Certification} +\author{} +\date{} + +\begin{document} +\maketitle + +\begin{abstract} +This note fixes a coherent and explicitly computable set of diagnostics for +certified neural networks in PDE applications. It is compatible with the +companion reference \emph{Notation and Terminology for Polynomial-Zonotope +Neural-Network Certification}. In particular, there are exactly two classes +of noise symbols: domain noise symbols \(\alpha\) and approximation noise +symbols \(\eta\), with combined vector \(\eps=(\alpha,\eta)\). The word +\emph{residual} is reserved for an actual function difference, such as an +activation-approximation residual or a PDE residual, and is not used as an +alternative name for \(\eta\). + +The note gives mathematical definitions, aggregation rules, canonical output +schemas, and two required benchmark levels. The \emph{mini benchmark} is the +smallest standardized report suitable for routine method comparisons. The +\emph{medium benchmark} contains the complete mini benchmark and additionally +records per-neuron preactivation intervals, activation approximation-error +radii, and layerwise normalized interval-hull radii. +\end{abstract} + +\section{Compatibility with the notation reference} + +Let +\[ + \Phi:\mathcal X\subseteq\R^{d_0}\longrightarrow\R^{d_{\rm out}} +\] +be the complete neural network and let +\[ + X(\alpha),\qquad \alpha\in[-1,1]^p, +\] +be a polynomial-zonotope parametrization of the physical input set +\(\mathcal X\). The network is written as +\[ + \Phi=A_L\circ\sigma_L\circ\cdots\circ A_1\circ\sigma_1\circ A_0, + \qquad A_\ell(y)=W_\ell y+b_\ell. +\] +At hidden layer \(\ell\), \(z_\ell\) and \(y_\ell\) denote the preactivation +and postactivation, while \(Z_\ell(\eps)\) and \(Y_\ell(\eps)\) denote their +polynomial enclosures. The corresponding polynomial enclosures of the first +and second physical-input derivatives are denoted by \(J_\ell(\eps)\) and +\(H_\ell(\eps)\). + +For the complete network, write +\[ + Y(\eps),\qquad J(\eps),\qquad H(\eps) +\] +for the final polynomial enclosures of +\[ + \Phi(X(\alpha)),\qquad + \left.\partial\Phi\right|_{X(\alpha)},\qquad + \left.\partial^2\Phi\right|_{X(\alpha)}. +\] +The derivatives are always with respect to the physical input \(x\). The +variables \(\alpha\) and \(\eta\) only parametrize the certified enclosure. + +For \(r\in\{0,1,2\}\), with \(\sigma^{(0)}=\sigma\), the canonical scalar +activation enclosure is +\[ + \sigma^{(r)}(t) + \in \widehat\sigma_{\ell i}^{(r)}(t) + +\rho_{\ell i}^{(r)}\eta_{\ell i}^{(r)}, + \qquad \eta_{\ell i}^{(r)}\in[-1,1]. +\] +Here \(\rho_{\ell i}^{(r)}\geq0\) is the +\emph{approximation-error radius}, \(\eta_{\ell i}^{(r)}\) is an +\emph{approximation noise symbol}, and +\[ + r_{\ell i}^{(r)}(t) + =\sigma^{(r)}(t)-\widehat\sigma_{\ell i}^{(r)}(t) +\] +is the actual residual function. + +\section{Cells, indices, and physical-volume weights} + +Let +\[ + \mathcal X=\bigcup_{c=1}^{N_{\rm cell}}\mathcal X_c +\] +be the partition used for certification, with pairwise disjoint interiors. +For cell \(c\), let \(X_c(\alpha)\) denote its domain parametrization. We use +\[ + c=1,\ldots,N_{\rm cell},\qquad + i=1,\ldots,d_{\rm out},\qquad + a,b=1,\ldots,d_0. +\] +Thus \(i\) indexes the output component, \(a,b\) index physical-input +coordinates, and \(c\) indexes the certification cell. + +For unequal physical cell volumes, define +\[ + \omega_c + =\frac{|\mathcal X_c|}{\sum_{k=1}^{N_{\rm cell}}|\mathcal X_k|}, + \qquad \sum_c\omega_c=1. +\] +When all cells have equal volume, \(\omega_c=1/N_{\rm cell}\). If physical +cell volumes are unavailable, the resulting average must be labelled an +\emph{unweighted cell average}; it must not be called a domain average. + +\section{Interval terminology} + +Let \(I=[\underline I,\overline I]\) be a nonempty closed real interval. +Define +\[ + \lb(I)=\underline I,\qquad \ub(I)=\overline I, +\] +\[ + \midp(I)=\frac{\underline I+\overline I}{2},\qquad + \rad(I)=\frac{\overline I-\underline I}{2}, +\] +\[ + \wid(I)=\overline I-\underline I=2\rad(I), +\] +\[ + \magI(I)=\max\{|\underline I|,|\overline I|\}, +\] +and +\[ + \mig(I)= + \begin{cases} + \min\{|\underline I|,|\overline I|\},&0\notin I,\\ + 0,&0\in I. + \end{cases} +\] +The magnitude is a certified upper bound on \(|z|\) for every \(z\in I\), +while the mignitude is the largest certified lower bound on \(|z|\) over the +interval. + +\subsection{Final interval hulls} + +For every cell \(c\), intervalize the final polynomial enclosures over all +domain and approximation noise symbols: +\[ + [Y^i]_c=[\underline Y_c^i,\overline Y_c^i], +\] +\[ + [J_{ia}]_c=[\underline J_{c,ia},\overline J_{c,ia}], +\] +\[ + [H_{iab}]_c=[\underline H_{c,iab},\overline H_{c,iab}]. +\] +These intervals enclose, respectively, +\[ + \Phi^i(X_c(\alpha)),\qquad + \left.\partial_a\Phi^i\right|_{X_c(\alpha)},\qquad + \left.\partial_{ab}\Phi^i\right|_{X_c(\alpha)}. +\] +Unless explicitly stated otherwise, all final width metrics are computed from +these interval hulls. + +\section{Scalar tightness metrics} + +For a scalar interval \(I\), define the absolute width and radius by +\[ + w(I)=\wid(I),\qquad r(I)=\rad(I). +\] +The absolute width is the primary dimensional tightness metric. + +The local relative radius is +\[ + \rho_{\rm loc}(I)= + \begin{cases} + \dfrac{\rad(I)}{\magI(I)},&\magI(I)>0,\\[2mm] + 0,&I=[0,0]. + \end{cases} +\] +No stabilization parameter is used. This metric must always be reported +alongside an absolute metric because an interval can be very small in absolute +terms while still having local relative radius near one when it is centered +near zero. + +The local relative width is +\[ + \rho_{\rm wid}(I)= + \begin{cases} + \dfrac{\wid(I)}{\magI(I)},&\magI(I)>0,\\[2mm] + 0,&I=[0,0]. + \end{cases} +\] +It is dimensionless and explicitly computable, and it satisfies +\[ + \rho_{\rm wid}(I)=2\rho_{\rm loc}(I). +\] +Thus local relative width and local relative radius contain the same +information. The glossary includes both terms for completeness, but the +canonical compact summaries use the radius form to avoid redundant headline +metrics. For intervals crossing zero, the local relative width may be as large +as two. + +The sign-certification indicator is +\[ + s(I)= + \begin{cases} + 1,&0\notin I,\\ + 0,&0\in I. + \end{cases} +\] + +\subsection{Familywise reference scales and normalized radii} + +Define the data-derived scales +\[ + S_Y=\max_{c,i}\magI([Y^i]_c), +\] +\[ + S_J=\max_{c,i,a}\magI([J_{ia}]_c), +\] +\[ + S_H=\max_{c,i,a,b}\magI([H_{iab}]_c). +\] +For a family \(Q\in\{Y,J,H\}\) and an interval \(I\) from that family, +define +\[ + \rho_{\rm glob}(I;S_Q)= + \begin{cases} + \dfrac{\rad(I)}{S_Q},&S_Q>0,\\[2mm] + 0,&S_Q=0. + \end{cases} +\] +This is the canonical \emph{global normalized radius}. Equivalently, +\(\wid(I)/(2S_Q)\) gives the same value. Only the radius form is used in the +canonical schemas. + +\paragraph{Scope of a reference scale.} +A reported value of \(S_Y,S_J,S_H\) is valid only for the exact collection of +cells and components summarized in the same output record. When comparing +methods, the preferred comparison uses a shared reference scale computed from +a designated reference family, or reports absolute widths in addition to each +method's own normalized radius. + +\section{Aggregation rules} + +Let \(q_{c,k}\geq0\) be a cellwise diagnostic, where \(k\) collects all +non-cell indices. Its volume-weighted mean is +\[ + \Mean_\omega(q) + =\sum_{c=1}^{N_{\rm cell}}\omega_c + \left(\frac1{N_k}\sum_{k=1}^{N_k}q_{c,k}\right). +\] +Its maximum is +\[ + \Max(q)=\max_{c,k}q_{c,k}. +\] +The canonical empirical quantiles are +\[ + Q_{0.50}(q),\qquad Q_{0.90}(q),\qquad Q_{0.99}(q), +\] +computed after flattening the finite collection \(\{q_{c,k}\}_{c,k}\). The +implementation must record the interpolation convention used by its numerical +library. The default recommendation is the library's linear interpolation +convention. + +The worst-cell index is +\[ + c_{\max}(q) + =\min\operatorname*{arg\,max}_c\left(\max_k q_{c,k}\right), +\] +where the minimum resolves ties deterministically. + +\section{Function-value, Jacobian, and Hessian diagnostics} + +\subsection{Function values} + +For every \(c,i\), define +\[ + w^Y_{c,i}=\wid([Y^i]_c),\qquad + \rho_{c,i}^{Y,{\rm loc}}=\rho_{\rm loc}([Y^i]_c),\qquad + \rho_{c,i}^{Y,{\rm glob}}=\rho_{\rm glob}([Y^i]_c;S_Y). +\] +The required summaries are the volume-weighted mean, maximum, and +\(0.50,0.90,0.99\) quantiles of \(w^Y_{c,i}\), together with the +volume-weighted mean and maximum of \(\rho_{c,i}^{Y,{\rm glob}}\). + +Define the componentwise width vector and its norms by +\[ + w_c^Y=(w^Y_{c,1},\ldots,w^Y_{c,d_{\rm out}}), +\] +\[ + W_{c,2}^Y=\|w_c^Y\|_2,\qquad + W_{c,\infty}^Y=\|w_c^Y\|_\infty. +\] + +\subsection{Jacobians} + +For every \(c,i,a\), define +\[ + w^J_{c,ia}=\wid([J_{ia}]_c),\qquad + \rho_{c,ia}^{J,{\rm loc}}=\rho_{\rm loc}([J_{ia}]_c),\qquad + \rho_{c,ia}^{J,{\rm glob}}=\rho_{\rm glob}([J_{ia}]_c;S_J). +\] +Let \(W_c^J=(w^J_{c,ia})_{i,a}\). The Jacobian Frobenius width is +\[ + W_{c,F}^J=\|W_c^J\|_F + =\left(\sum_{i=1}^{d_{\rm out}}\sum_{a=1}^{d_0} + (w^J_{c,ia})^2\right)^{1/2}. +\] +For output component \(i\), the gradient width is +\[ + W_{c,i}^{\nabla} + =\left(\sum_{a=1}^{d_0}(w^J_{c,ia})^2\right)^{1/2}. +\] +For physical-input coordinate \(a\), define +\[ + W_{c,a}^{\rm input} + =\left(\sum_{i=1}^{d_{\rm out}}(w^J_{c,ia})^2\right)^{1/2}. +\] +The required summaries are the same entrywise quantities as for function +values. In addition, report +\[\Mean_\omega(W_F^J)\qquad\text{and}\qquad \max_c W_{c,F}^J.\] + +\subsection{Hessians} + +For every \(c,i,a,b\), define +\[ + w^H_{c,iab}=\wid([H_{iab}]_c),\qquad + \rho_{c,iab}^{H,{\rm loc}}=\rho_{\rm loc}([H_{iab}]_c),\qquad + \rho_{c,iab}^{H,{\rm glob}}=\rho_{\rm glob}([H_{iab}]_c;S_H). +\] +The Hessian Frobenius width is +\[ + W_{c,F}^H + =\left(\sum_{i=1}^{d_{\rm out}}\sum_{a=1}^{d_0} + \sum_{b=1}^{d_0}(w^H_{c,iab})^2\right)^{1/2}. +\] +If only entries with \(a\leq b\) are stored, the implementation must either +reconstruct the full symmetric tensor or use weight one for \(a=b\) and weight +two for \(a0,\\[2mm] + 0,&\overline{\mathcal N}=0. + \end{cases} +\] +It lies in \([0,1]\). The equivalent lower-bound informativeness is +\[ + \gamma_{\mathcal N}= + \begin{cases} + \dfrac{\underline{\mathcal N}}{\overline{\mathcal N}}, + &\overline{\mathcal N}>0,\\[2mm] + 1,&\overline{\mathcal N}=0, + \end{cases} + \qquad \gamma_{\mathcal N}=1-\rho_{\mathcal N}. +\] +Only \(\rho_{\mathcal N}\) is required as the canonical tightness +summary. For a nonnegative norm interval, this relative norm width is exactly +the local relative width defined above because +\(\magI([\underline{\mathcal N},\overline{\mathcal N}]) +=\overline{\mathcal N}\). + +\subsection{Domain-volume-normalized norm intervals} + +Let +\[ + V_{\mathcal X}=|\mathcal X|>0 +\] +be the physical integration-domain volume. The +\emph{domain-volume-normalized norm} is +\[ + \mathcal N_{\rm vol}=\frac{\mathcal N}{\sqrt{V_{\mathcal X}}}. +\] +For a certified norm interval, define +\[ + [\mathcal N_{\rm vol}] + =\left[ + \frac{\underline{\mathcal N}}{\sqrt{V_{\mathcal X}}}, + \frac{\overline{\mathcal N}}{\sqrt{V_{\mathcal X}}} + \right]. +\] +For a certified squared-norm interval +\([\mathcal N^2]=[\underline{\mathcal N^2}, +\overline{\mathcal N^2}]\), define +\[ + [\mathcal N_{\rm vol}^2] + =\left[ + \frac{\underline{\mathcal N^2}}{V_{\mathcal X}}, + \frac{\overline{\mathcal N^2}}{V_{\mathcal X}} + \right]. +\] +For the \(L^2\) norm, \(\mathcal N_{\rm vol}\) is the root-mean-square +magnitude of the function over \(\mathcal X\). For \(W^{1,2}\) and +\(W^{2,2}\), it is the square root of the mean value of the corresponding +value-and-derivative energy density. The term \emph{RMS norm} is canonical +only for \(L^2\); for Sobolev norms use \emph{domain-volume-normalized +Sobolev norm}. + +This metric is not a tightness metric. It removes the trivial +\(\sqrt{V_{\mathcal X}}\) scaling of an \(L^2\)-type norm and is useful +when comparing experiments on domains with different volumes. It does not +normalize by the size of the represented function and does not measure +interval overestimation. Positive rescaling leaves the relative norm width +unchanged: +\[ + \frac{\wid([\mathcal N_{\rm vol}])} + {\ub([\mathcal N_{\rm vol}])} + = + \frac{\wid([\mathcal N])}{\ub([\mathcal N])} + =\rho_{\mathcal N}. +\] +Consequently, the domain-volume-normalized interval and the relative norm +width must be reported as separate metrics. The word \emph{normalized} +without a qualifier must not be used for either one. + +For \(L^2\)- and Sobolev norms, retain both the squared-norm interval and the +norm interval. The primary relative-width diagnostic is computed before the +square root. + +Define +\[ + I_0=\int_{\mathcal X}|\Phi(x)|_2^2\,dx, +\] +\[ + I_1=\int_{\mathcal X}\left|\left.\partial\Phi\right|_x\right|_F^2\,dx, +\] +\[ + I_2=\int_{\mathcal X}\left|\left.\partial^2\Phi\right|_x\right|_F^2\,dx. +\] +Then +\[ + \|\Phi\|_{L^2}^2=I_0,\qquad + \|\Phi\|_{W^{1,2}}^2=I_0+I_1, +\] +\[ + \|\Phi\|_{W^{2,2}}^2=I_0+I_1+I_2. +\] +For the terms present in a chosen norm, define +\[ + C_k^{\rm upper} + =\begin{cases} + \dfrac{\overline I_k}{\sum_j\overline I_j}, + &\sum_j\overline I_j>0,\\ + 0,&\sum_j\overline I_j=0, + \end{cases} +\] +and +\[ + C_k^{\rm width} + =\begin{cases} + \dfrac{\overline I_k-\underline I_k} + {\sum_j(\overline I_j-\underline I_j)}, + &\sum_j(\overline I_j-\underline I_j)>0,\\ + 0,&\sum_j(\overline I_j-\underline I_j)=0. + \end{cases} +\] + +\section{PDE residual diagnostics} + +Let +\[ + \mathcal R_\Phi(x)=\mathcal L[\Phi](x)-f(x) +\] +be the PDE residual function. For every cell \(c\), let +\[ + [\mathcal R_\Phi]_c + =[\underline{\mathcal R}_c,\overline{\mathcal R}_c] +\] +be a certified interval enclosure. Define +\[ + M_c^{\mathcal R}=\magI([\mathcal R_\Phi]_c),\qquad + w_c^{\mathcal R}=\wid([\mathcal R_\Phi]_c). +\] +The certified global pointwise residual upper bound is +\[ + \|\mathcal R_\Phi\|_{L^\infty}^{\rm upper} + =\max_c M_c^{\mathcal R}. +\] +If available, also report +\[ + [\|\mathcal R_\Phi\|_{L^2}^2] + \quad\text{and}\quad + [\|\mathcal R_\Phi\|_{L^2}]. +\] +Boundary-condition and initial-condition residuals must be represented by +separate residual functions and reported separately. + +\section{Activation-enclosure diagnostics} + +For cell \(c\), activation layer \(\ell\), neuron \(i\), and derivative order +\(r\in\{0,1,2\}\), let +\[ + [Z_{\ell i}]_c + =[\underline Z_{c,\ell i},\overline Z_{c,\ell i}] +\] +be the certified preactivation interval used to construct the activation +enclosure. Record +\[ + w^Z_{c,\ell i}=\wid([Z_{\ell i}]_c) +\] +and the certified approximation-error radius +\[ + \rho_{c,\ell i}^{(r)}. +\] +The associated approximation-error diameter is +\[ + d_{c,\ell i}^{(r)}=2\rho_{c,\ell i}^{(r)}. +\] + +For comparisons within one layer and derivative order, define +\[ + S_\ell^{(r)} + =\max_{c,i}\magI\left([\sigma^{(r)}(Z_{\ell i})]_c\right) +\] +and +\[ + \widehat\rho_{c,\ell i}^{(r)} + =\begin{cases} + \dfrac{\rho_{c,\ell i}^{(r)}}{S_\ell^{(r)}}, + &S_\ell^{(r)}>0,\\[2mm] + 0,&S_\ell^{(r)}=0. + \end{cases} +\] +This is the canonical normalized activation approximation-error radius. + +\subsection{Layerwise normalized interval-hull radius} + +Let \([Q_{\ell i}]_c\) be the interval hull of a specified scalar neuron +quantity \(Q\) at layer \(\ell\). The quantity identifier must be one of +\[ + Q\in\{Z,Y,\sigma^{(0)}(Z),\sigma^{(1)}(Z),\sigma^{(2)}(Z)\}. +\] +For fixed \(Q\) and \(\ell\), define +\[ + S_{Q,\ell}=\max_{c,i}\magI([Q_{\ell i}]_c) +\] +and +\[ + \nu_{c,\ell i}^{Q} + =\begin{cases} + \dfrac{\rad([Q_{\ell i}]_c)}{S_{Q,\ell}},&S_{Q,\ell}>0,\\[2mm] + 0,&S_{Q,\ell}=0. + \end{cases} +\] +The medium benchmark must include the postactivation-value table +\(Q=Y\). Tables for \(Z\), \(\sigma'(Z)\), and \(\sigma''(Z)\) are optional +extensions but must follow the same schema. + +For a single wide table with layers as columns, aggregate over cells by the +volume-weighted mean +\[ + \bar\nu_{\ell i}^{Q}=\sum_c\omega_c\nu_{c,\ell i}^{Q}. +\] +The wide table entry in row \(i\), column \(\ell\) is +\(\bar\nu_{\ell i}^{Q}\). If layer \(\ell\) has fewer than \(i\) neurons, +the entry is \(\NA\), not zero. + +\section{Representation-complexity and reduction diagnostics} + +For a sparse polynomial +\[ + P(\eps)=\sum_{\lambda\in\Lambda_P}P_\lambda\eps^\lambda, +\] +record +\[ + N_\alpha=p,\qquad N_\eta=q,\qquad + N_{\rm mon}=|\Lambda_P|, +\] +\[ + d_{\max}=\max_{\lambda\in\Lambda_P}|\lambda|_1, +\] +\[ + d_{\alpha,\max}=\max_{\lambda\in\Lambda_P}|\lambda^\alpha|_1, + \qquad + d_{\eta,\max}=\max_{\lambda\in\Lambda_P}|\lambda^\eta|_1, +\] +and +\[ + N_{\alpha\eta} + =\left|\left\{\lambda\in\Lambda_P: + |\lambda^\alpha|_1>0\ \text{and}\ |\lambda^\eta|_1>0 + \right\}\right|. +\] +Duplicate exponent vectors must be merged before counting distinct monomials. +If \(P_\lambda\in U\) and \(\dim U=s\), the stored scalar coefficient count is +\[ + N_{\rm coeff}=s|\Lambda_P|. +\] + +For support reduction with retained support \(\Lambda_{\rm keep}\) and +discarded support \(\Lambda_{\rm drop}\), record +\[ + N_{\rm before}=|\Lambda_P|, +\] +\[ + N_{\rm after}=|\Lambda_{\rm keep}|+N_{\rm new}, +\] +\[ + R_{\rm support} + =\begin{cases} + 1-N_{\rm after}/N_{\rm before},&N_{\rm before}>0,\\ + 0,&N_{\rm before}=0, + \end{cases} +\] +and, for a fixed coefficient norm, +\[ + M_{\rm drop}=\sum_{\lambda\in\Lambda_{\rm drop}}\|P_\lambda\|. +\] +When reduced and unreduced interval hulls are both available, define the +reduction-induced width increase +\[ + \Delta w_{\rm red} + =\wid([P]_{\rm reduced})-\wid([P]_{\rm unreduced}). +\] + +\section{Runtime, memory, and soundness diagnostics} + +Record non-overlapping wall-clock times where applicable: +\[ + T_{\rm value},\ T_{\rm Jacobian},\ T_{\rm Hessian},\ + T_{\rm activation},\ T_{\rm multiplication}, +\] +\[ + T_{\rm reduction},\ T_{\rm integration},\ + T_{\rm intervalization},\ T_{\rm total}. +\] +Also record peak allocated memory \(M_{\rm peak}\). GPU measurements require +device synchronization before and after each timed region. + +For sampled physical points \(x_n\in\mathcal X_c\), test containment of +function values and all derivatives that are part of the benchmark. For a +scalar value \(v\) and interval \(I\), define +\[ + d(v,I)=\max\{\lb(I)-v,\ v-\ub(I),\ 0\}. +\] +Record the number of failures \(N_{\rm fail}\) and the maximum sampled +containment violation \(D_{\rm fail}\). Sampling cannot prove soundness, but +any failure disproves the claimed enclosure. Also record invalid-interval, +NaN-endpoint, and infinite-endpoint counts. + +\section{Canonical data representation} + +All benchmark data must be stored in UTF-8 comma-separated-value files with a +single header row, a period as decimal separator, and no thousands separator. +Missing values are represented by the literal string \(\NA\). Boolean values +are represented by \texttt{0} and \texttt{1}. Infinity is represented by +\texttt{inf} and negative infinity by \texttt{-inf}. Each file must be +accompanied by a metadata file named \texttt{benchmark\_metadata.json}. + +The metadata must contain at least: +\begin{verbatim} +{ + "schema_version": "1.2", + "benchmark_level": "mini" or "medium", + "problem_id": "...", + "model_id": "...", + "method_id": "...", + "git_commit": "..." or null, + "dtype": "float64", + "device": "cpu" or "cuda:...", + "quantile_interpolation": "linear", + "cell_average": "volume_weighted" or "unweighted", + "timestamp_utc": "ISO-8601 timestamp" +} +\end{verbatim} + +\subsection{Canonical tidy metric table} + +Every benchmark must contain \texttt{metrics.csv}. Its exact column order is: +\begin{center} +\small +\begin{tabularx}{\textwidth}{@{}r l X@{}} +\toprule +Position & Column & Meaning \\ +\midrule +1 & \texttt{schema\_version} & Schema version, currently \texttt{1.2}. \\ +2 & \texttt{benchmark\_level} & \texttt{mini} or \texttt{medium}. \\ +3 & \texttt{problem\_id} & PDE or benchmark identifier. \\ +4 & \texttt{model\_id} & Trained-network identifier. \\ +5 & \texttt{method\_id} & Certification-method identifier. \\ +6 & \texttt{run\_id} & Unique identifier for the execution. \\ +7 & \texttt{split\_id} & Domain partition or refinement identifier. \\ +8 & \texttt{quantity} & Canonical quantity identifier. \\ +9 & \texttt{metric} & Canonical metric identifier. \\ +10 & \texttt{aggregation} & \texttt{none}, \texttt{mean\_weighted}, \texttt{max}, \texttt{q50}, \texttt{q90}, or \texttt{q99}. \\ +11 & \texttt{derivative\_order} & \texttt{0}, \texttt{1}, \texttt{2}, or \NA. \\ +12 & \texttt{layer} & Zero-based layer index or \NA. \\ +13 & \texttt{neuron} & Zero-based neuron index or \NA. \\ +14 & \texttt{output\_index} & Zero-based output component or \NA. \\ +15 & \texttt{input\_index\_a} & First physical-input coordinate or \NA. \\ +16 & \texttt{input\_index\_b} & Second coordinate for Hessians or \NA. \\ +17 & \texttt{cell\_id} & Zero-based cell identifier or \NA. \\ +18 & \texttt{value} & Numeric metric value. \\ +19 & \texttt{unit} & Unit string, \texttt{dimensionless}, or \NA. \\ +20 & \texttt{status} & \texttt{ok}, \texttt{missing}, \texttt{invalid}, or \texttt{not\_implemented}. \\ +\bottomrule +\end{tabularx} +\end{center} + +The pair \texttt{quantity}, \texttt{metric} determines the mathematical +meaning. Canonical quantity identifiers include +\begin{center} +\texttt{Y}, \texttt{J}, \texttt{H}, \texttt{L2\_sq}, \texttt{L2}, +\texttt{W12\_sq}, \texttt{W12}, \texttt{W22\_sq}, \texttt{W22}, +\texttt{PDE\_residual}, \texttt{boundary\_residual}, +\texttt{initial\_residual}, \texttt{complexity}, \texttt{runtime}, +\texttt{memory}, and \texttt{soundness}. +\end{center} +Canonical metric identifiers include +\begin{center} +\texttt{lower}, \texttt{upper}, \texttt{width}, +\texttt{local\_relative\_width}, \texttt{local\_relative\_radius}, +\texttt{global\_normalized\_radius}, \texttt{frobenius\_width}, +\texttt{relative\_norm\_width}, +\texttt{domain\_volume\_normalized\_lower}, +\texttt{domain\_volume\_normalized\_upper}, +\texttt{domain\_volume\_normalized\_width}, +\texttt{linf\_upper}, \texttt{n\_alpha}, \texttt{n\_eta}, +\texttt{n\_monomials}, \texttt{n\_mixed\_monomials}, +\texttt{max\_degree}, \texttt{seconds}, \texttt{bytes}, +\texttt{failure\_count}, and \texttt{max\_violation}. +\end{center} + +\subsection{Canonical cellwise interval table} + +The mini and medium benchmarks must contain \texttt{cell\_intervals.csv} with +this exact column order: +\begin{center} +\small +\begin{tabularx}{\textwidth}{@{}r l X@{}} +\toprule +Position & Column & Meaning \\ +\midrule +1 & \texttt{run\_id} & Execution identifier. \\ +2 & \texttt{split\_id} & Partition identifier. \\ +3 & \texttt{cell\_id} & Zero-based cell identifier. \\ +4 & \texttt{cell\_weight} & \(\omega_c\), or \(1/N_{\rm cell}\) for unweighted output. \\ +5 & \texttt{quantity} & \texttt{Y}, \texttt{J}, \texttt{H}, or residual identifier. \\ +6 & \texttt{output\_index} & Output component or \NA. \\ +7 & \texttt{input\_index\_a} & First derivative coordinate or \NA. \\ +8 & \texttt{input\_index\_b} & Second derivative coordinate or \NA. \\ +9 & \texttt{lower} & Interval lower endpoint. \\ +10 & \texttt{upper} & Interval upper endpoint. \\ +11 & \texttt{midpoint} & Interval midpoint. \\ +12 & \texttt{radius} & Interval radius. \\ +13 & \texttt{width} & Interval width. \\ +14 & \texttt{magnitude} & Interval magnitude. \\ +15 & \texttt{mignitude} & Interval mignitude. \\ +16 & \texttt{local\_relative\_radius} & \(\rho_{\rm loc}\). \\ +17 & \texttt{global\_normalized\_radius} & Familywise \(\rho_{\rm glob}\). \\ +18 & \texttt{sign\_certified} & \texttt{1} if zero is excluded, otherwise \texttt{0}. \\ +19 & \texttt{status} & \texttt{ok}, \texttt{invalid}, or \texttt{not\_implemented}. \\ +20 & \texttt{local\_relative\_width} & \(\rho_{\rm wid}=2\rho_{\rm loc}\). \\ +\bottomrule +\end{tabularx} +\end{center} +Rows are sorted lexicographically by +\[ + (\texttt{cell\_id},\texttt{quantity},\texttt{output\_index}, + \texttt{input\_index\_a},\texttt{input\_index\_b}). +\] + +\subsection{Canonical norm table} + +The benchmark must contain \texttt{norms.csv} with exact column order +\begin{center} +\small +\begin{tabularx}{\textwidth}{@{}r l X@{}} +\toprule +Position & Column & Meaning \\ +\midrule +1 & \texttt{run\_id} & Execution identifier. \\ +2 & \texttt{norm} & \texttt{L2}, \texttt{W12}, \texttt{W22}, or residual norm. \\ +3 & \texttt{squared} & \texttt{1} for squared norm, otherwise \texttt{0}. \\ +4 & \texttt{lower} & Certified lower endpoint. \\ +5 & \texttt{upper} & Certified upper endpoint. \\ +6 & \texttt{width} & Upper minus lower. \\ +7 & \texttt{relative\_width} & Width divided by upper endpoint, with zero for \([0,0]\). \\ +8 & \texttt{value\_contribution\_upper} & \(C_0^{\rm upper}\), or \NA. \\ +9 & \texttt{gradient\_contribution\_upper} & \(C_1^{\rm upper}\), or \NA. \\ +10 & \texttt{hessian\_contribution\_upper} & \(C_2^{\rm upper}\), or \NA. \\ +11 & \texttt{value\_contribution\_width} & \(C_0^{\rm width}\), or \NA. \\ +12 & \texttt{gradient\_contribution\_width} & \(C_1^{\rm width}\), or \NA. \\ +13 & \texttt{hessian\_contribution\_width} & \(C_2^{\rm width}\), or \NA. \\ +14 & \texttt{status} & \texttt{ok}, \texttt{invalid}, or \texttt{not\_implemented}. \\ +15 & \texttt{domain\_volume} & Physical integration-domain volume \(V_{\mathcal X}\). \\ +16 & \texttt{domain\_volume\_normalized\_lower} & Lower endpoint divided by \(V_{\mathcal X}\) for squared rows and by \(\sqrt{V_{\mathcal X}}\) otherwise. \\ +17 & \texttt{domain\_volume\_normalized\_upper} & Upper endpoint with the same scaling. \\ +18 & \texttt{domain\_volume\_normalized\_width} & Width with the same scaling. \\ +\bottomrule +\end{tabularx} +\end{center} +Rows are ordered first by \texttt{norm} in the order +\texttt{L2}, \texttt{W12}, \texttt{W22}, followed by other norms, and then by +\texttt{squared} with squared rows first. + +\section{Mini benchmark} + +The mini benchmark is the mandatory smallest benchmark. It is intended for +fast regression tests and standardized comparisons of certification methods. +It must be possible to execute it without storing all intermediate polynomial +objects. + +\subsection{Required outputs} + +The mini benchmark must produce: +\begin{enumerate}[label=\arabic*.] + \item \texttt{benchmark\_metadata.json}; + \item \texttt{metrics.csv}; + \item \texttt{cell\_intervals.csv}; + \item \texttt{norms.csv}; + \item \texttt{complexity.csv}; + \item \texttt{timings.csv}; + \item \texttt{soundness.csv}. +\end{enumerate} + +\subsection{Norm-output program contract} + +The following contract is part of the definition of both benchmark programs. +It is not an optional reporting convention. + +For every implemented norm +\[ + N\in\{\texttt{L2},\texttt{W12},\texttt{W22}\}, +\] +the benchmark program must emit both the squared and unsquared certified norm +rows in \texttt{norms.csv}. Every such row with status \texttt{ok} must +populate all of the following fields: +\[ + \texttt{lower},\quad \texttt{upper},\quad \texttt{width},\quad + \texttt{relative\_width},\quad \texttt{domain\_volume}, +\] +\[ + \texttt{domain\_volume\_normalized\_lower},\quad + \texttt{domain\_volume\_normalized\_upper},\quad + \texttt{domain\_volume\_normalized\_width}. +\] +For a squared norm row, the three domain-volume-normalized endpoints and width +are obtained by division by \(V_{\mathcal X}\). For an unsquared norm row, +they are obtained by division by \(\sqrt{V_{\mathcal X}}\). + +A program that reports only the unnormalized interval, only the relative width, +or only a scalar domain-volume-normalized point estimate does not implement the +mini benchmark. Since the medium benchmark inherits the complete mini +benchmark, such a program also does not implement the medium benchmark. + + +\subsection{Required diagnostics} + +For final function values \(Y\), report: +\begin{itemize} + \item volume-weighted mean width; + \item maximum width; + \item \(0.50,0.90,0.99\) width quantiles; + \item volume-weighted mean global normalized radius; + \item maximum global normalized radius. +\end{itemize} + +For the final Jacobian \(J\), report the same entrywise summaries and also +\[ + \Mean_\omega(W_F^J),\qquad \max_c W_{c,F}^J. +\] + +For the final Hessian \(H\), report the corresponding diagnostics whenever +second derivatives are propagated or required by the PDE. Otherwise rows must +be present in \texttt{metrics.csv} with status \texttt{not\_implemented}; they +must not be silently omitted. + +For each implemented function-space norm, report the squared and unsquared +intervals, absolute widths, relative norm widths, the physical domain volume, +and the corresponding domain-volume-normalized intervals. The minimum +standard set is \(L^2\) and \(W^{1,2}\); \(W^{2,2}\) is required when a +Hessian is propagated. + +\paragraph{Mandatory norm-row contract.} +For every required norm name \(N\in\{\texttt{L2},\texttt{W12}\}\), and +also \(N=\texttt{W22}\) whenever the Hessian is propagated, the mini +benchmark must write exactly two rows to \texttt{norms.csv}: first the squared +row \(\texttt{squared}=1\), then the unsquared row +\(\texttt{squared}=0\). In both rows, columns 15--18 are mandatory and +must contain +\[ + V_{\mathcal X},\qquad + \frac{\underline{\mathcal N^2}}{V_{\mathcal X}},\quad + \frac{\overline{\mathcal N^2}}{V_{\mathcal X}},\quad + \frac{\overline{\mathcal N^2}-\underline{\mathcal N^2}} + {V_{\mathcal X}} +\] +for the squared row, and +\[ + V_{\mathcal X},\qquad + \frac{\underline{\mathcal N}}{\sqrt{V_{\mathcal X}}},\quad + \frac{\overline{\mathcal N}}{\sqrt{V_{\mathcal X}}},\quad + \frac{\overline{\mathcal N}-\underline{\mathcal N}} + {\sqrt{V_{\mathcal X}}} +\] +for the unsquared row. These values must not be omitted, replaced by +\texttt{NA}, or inferred only by downstream plotting code when the underlying +norm interval has status \texttt{ok}. A mini-benchmark implementation is +non-conforming if these columns are absent or unpopulated. + +For PDE certification, report +\[ + \|\mathcal R_\Phi\|_{L^\infty}^{\rm upper} +\] +and, when implemented, certified \(L^2\) residual intervals. Boundary and +initial residuals are separate quantities. + +For representation complexity, record at least +\[ + N_\alpha,\quad N_\eta,\quad N_{\rm mon},\quad + N_{\alpha\eta},\quad d_{\max},\quad N_{\rm coeff} +\] +for \(Y\), \(J\), and \(H\) separately. + +For computational cost, record total runtime, the principal non-overlapping +stage times, and peak memory. For soundness diagnostics, record +\(N_{\rm fail}\), \(D_{\rm fail}\), invalid interval count, NaN endpoint count, +and infinite endpoint count. + +\subsection{Canonical mini-benchmark summary order} + +For human-readable printing, the summary rows must appear in this order: +\begin{enumerate}[label=\arabic*.] + \item run metadata and domain partition; + \item function-value enclosure diagnostics; + \item Jacobian enclosure diagnostics; + \item Hessian enclosure diagnostics; + \item squared and unsquared function-space norm intervals, immediately + followed by their domain-volume-normalized lower endpoint, upper endpoint, + and width; + \item PDE, boundary, and initial residual diagnostics; + \item representation-complexity diagnostics for \(Y,J,H\); + \item runtime and peak-memory diagnostics; + \item soundness and validity diagnostics. +\end{enumerate} +Within each enclosure family, print metrics in the order +\[ + \texttt{mean\_width},\ \texttt{max\_width},\ + \texttt{q50\_width},\ \texttt{q90\_width},\ \texttt{q99\_width}, +\] +\[ + \texttt{mean\_global\_normalized\_radius},\ + \texttt{max\_global\_normalized\_radius}, +\] +followed by family-specific matrix or tensor diagnostics. + +\section{Medium benchmark} + +The medium benchmark contains every mini-benchmark output, file, row, column, +and diagnostic without modification. In particular, it must produce the same +\texttt{norms.csv} required by the mini benchmark, including the mandatory +squared and unsquared domain-volume-normalized norm intervals in columns +15--18. The medium benchmark is therefore not permitted to omit, rename, or +replace these quantities with only relative widths. It additionally diagnoses +where approximation error enters the network and how interval hulls evolve +through the hidden layers. + +\paragraph{Inheritance rule.} +A medium-benchmark program must first satisfy every mini-benchmark conformance +condition. Its additional per-neuron and layerwise files extend the mini +benchmark; they do not form an alternative output schema. Consequently, a +medium-benchmark run is non-conforming whenever the corresponding mini +benchmark would be non-conforming, including whenever any required +\texttt{domain\_volume\_normalized\_lower}, +\texttt{domain\_volume\_normalized\_upper}, or +\texttt{domain\_volume\_normalized\_width} entry is missing from an +implemented norm row. + +\subsection{Per-neuron activation approximation table} + +The medium benchmark must contain +\texttt{activation\_approximation.csv}. Its exact column order is: +\begin{center} +\small +\begin{tabularx}{\textwidth}{@{}r l X@{}} +\toprule +Position & Column & Meaning \\ +\midrule +1 & \texttt{run\_id} & Execution identifier. \\ +2 & \texttt{split\_id} & Partition identifier. \\ +3 & \texttt{cell\_id} & Zero-based cell identifier. \\ +4 & \texttt{cell\_weight} & Physical-volume weight \(\omega_c\). \\ +5 & \texttt{layer} & Zero-based hidden activation-layer index. \\ +6 & \texttt{neuron} & Zero-based neuron index within the layer. \\ +7 & \texttt{derivative\_order} & \texttt{0}, \texttt{1}, or \texttt{2}. \\ +8 & \texttt{preactivation\_lower} & \(\underline Z_{c,\ell i}\). \\ +9 & \texttt{preactivation\_upper} & \(\overline Z_{c,\ell i}\). \\ +10 & \texttt{preactivation\_midpoint} & Midpoint of \([Z_{\ell i}]_c\). \\ +11 & \texttt{preactivation\_radius} & Radius of \([Z_{\ell i}]_c\). \\ +12 & \texttt{preactivation\_width} & Width of \([Z_{\ell i}]_c\). \\ +13 & \texttt{approximation\_kind} & Canonical approximation identifier, e.g. \texttt{affine}, \texttt{quadratic}, or \texttt{interval}. \\ +14 & \texttt{approximation\_error\_radius} & \(\rho_{c,\ell i}^{(r)}\). \\ +15 & \texttt{approximation\_error\_diameter} & \(2\rho_{c,\ell i}^{(r)}\). \\ +16 & \texttt{activation\_scale} & \(S_\ell^{(r)}\). \\ +17 & \texttt{normalized\_approximation\_radius} & \(\widehat\rho_{c,\ell i}^{(r)}\). \\ +18 & \texttt{noise\_symbol\_id} & Stable identifier for \(\eta_{\ell i}^{(r)}\), or \NA if no fresh symbol is introduced. \\ +19 & \texttt{shared\_noise\_group} & Identifier when one approximation noise symbol is reused, otherwise \NA. \\ +20 & \texttt{status} & \texttt{ok}, \texttt{invalid}, or \texttt{not\_implemented}. \\ +\bottomrule +\end{tabularx} +\end{center} +Rows are sorted by +\[ + (\texttt{cell\_id},\texttt{layer},\texttt{neuron}, + \texttt{derivative\_order}). +\] +The derivative-order rows \(0,1,2\) must be present whenever the corresponding +activation enclosure is used by the propagated value, Jacobian, or Hessian. + +\subsection{Layerwise normalized interval-radius table} + +The medium benchmark must contain a separate wide table named +\begin{center}\path{layer_normalized_radius_Y.csv}.\end{center} +It summarizes the postactivation +interval hulls \([Y_{\ell i}]_c\). Its exact column order is +\[ + \texttt{neuron},\ \texttt{layer\_0},\ \texttt{layer\_1},\ \ldots, + \texttt{layer\_{L-1}}. +\] +The row index \texttt{neuron} is zero-based. The entry in row \(i\), column +\texttt{layer\_\(\ell\)} is +\[ + \bar\nu_{\ell i}^{Y} + =\sum_c\omega_c + \begin{cases} + \dfrac{\rad([Y_{\ell i}]_c)} + {\max_{c',j}\magI([Y_{\ell j}]_{c'})}, + &\max_{c',j}\magI([Y_{\ell j}]_{c'})>0,\\[3mm] + 0,&\text{otherwise}. + \end{cases} +\] +If layer \(\ell\) has fewer than \(i+1\) neurons, the entry is \(\NA\). +Columns always follow increasing layer index; rows always follow increasing +neuron index. + +The medium benchmark may additionally provide the same wide schema for other +quantities using the filenames +\begin{center} +\texttt{layer\_normalized\_radius\_Z.csv},\\ +\texttt{layer\_normalized\_radius\_sigma1.csv},\\ +\texttt{layer\_normalized\_radius\_sigma2.csv}. +\end{center} +Here \texttt{sigma1} and \texttt{sigma2} refer to interval hulls of +\(\sigma'(Z_{\ell i})\) and \(\sigma''(Z_{\ell i})\), respectively. These +optional tables must use the identical row and column conventions. + +\subsection{Required medium-benchmark visual summaries} + +A medium-benchmark report should render, from the canonical CSV data: +\begin{enumerate}[label=\arabic*.] + \item one table per derivative order \(r=0,1,2\), with rows corresponding to + neurons and grouped columns containing preactivation interval, approximation + kind, approximation-error radius, and normalized approximation radius; + \item the wide layer table \texttt{layer\_normalized\_radius\_Y.csv}; + \item a ranked list of the largest approximation-error radii; + \item a ranked list of the largest normalized approximation-error radii; + \item the worst cells and neurons according to preactivation width and + approximation-error radius. +\end{enumerate} +These visual summaries are derived views. The CSV files remain the canonical +data representation. + +\section{Implementation invariants and safety checks} + +An implementation conforming to this reference must satisfy the following +invariants. + +\begin{enumerate}[label=\arabic*.] + \item Every reported metric is computable from explicitly stored endpoints, + coefficients, counts, or timings. No hidden stabilization constant is used. + \item The only polynomial variable classes are domain noise symbols \(\alpha\) + and approximation noise symbols \(\eta\). + \item The word \emph{residual} denotes an actual function difference, never a + noise symbol. + \item All derivative indices refer to derivatives with respect to the physical + input \(x\). + \item Missing or unimplemented quantities are represented by \(\NA\) and an + explicit non-\texttt{ok} status; they are never silently replaced by zero. + \item Zero is used only when the mathematical definition prescribes zero, such + as the normalized radius of an identically zero interval family. + \item Cell-weighted means use the stored \texttt{cell\_weight} values, which + must sum to one within numerical tolerance. + \item Distinct monomials are counted only after duplicate exponent vectors are + merged. + \item Squared norm intervals are retained even when unsquared norm intervals + are also reported. + \item Sampling diagnostics are labelled as diagnostics and are not described + as proofs of soundness. +\end{enumerate} + +\section{Canonical vocabulary} + +\renewcommand{\arraystretch}{1.2} +\small +\begin{tabularx}{\textwidth}{@{}l X@{}} +\toprule +Preferred term & Meaning \\ +\midrule +Interval hull & Final interval enclosure obtained from a polynomial enclosure over all domain and approximation noise symbols. \\ +Absolute width & Upper endpoint minus lower endpoint. \\ +Absolute radius & Half the absolute width. \\ +Magnitude & Maximum absolute value of the interval endpoints. \\ +Mignitude & Largest certified lower bound on absolute value over the interval. \\ +Local relative width & Interval width divided by the magnitude of the same interval; equal to twice the local relative radius. \\ +Local relative radius & Interval radius divided by the magnitude of the same interval. \\ +Global normalized radius & Interval radius divided by the maximum magnitude over the same explicitly specified family. \\ +Volume-weighted mean width & Componentwise mean width weighted by physical cell volume. \\ +Relative norm width & Width of a nonnegative norm interval divided by its upper endpoint; this is a tightness metric. \\ +Domain-volume-normalized norm & Norm divided by the square root of the physical integration-domain volume; this is a scale metric, not a tightness metric. \\ +Approximation-error radius & Certified nonnegative scalar \(\rho\) in an activation enclosure. \\ +Approximation noise symbol & A component of \(\eta\); this is the canonical name. \\ +Approximation noise term & A product \(\rho\eta\). \\ +Residual function & An actual difference between a function and its approximation, or between the two sides of a PDE. \\ +Distinct monomial count & Cardinality of the canonical exponent support after duplicate exponents are merged. \\ +Mixed monomial & A monomial depending on at least one domain noise symbol and at least one approximation noise symbol. \\ +Reduction-induced width increase & Difference between reduced and unreduced interval-hull widths. \\ +Mini benchmark & Mandatory compact diagnostic suite for routine regression and method comparison. \\ +Medium benchmark & Complete mini benchmark plus per-neuron approximation diagnostics and layerwise interval-hull-radius tables. \\ +\bottomrule +\end{tabularx} +\normalsize + +\section{Default interpretation} + +No normalized metric replaces the absolute width. The default pair for +pointwise enclosures is +\[ + \boxed{\text{absolute width}} + \qquad\text{and}\qquad + \boxed{\text{global normalized radius}}. +\] +The local relative radius and the equivalent local relative width are secondary +diagnostics. For nonnegative function-space norm intervals, the default +tightness metric is the relative norm width +\[ + \rho_{\mathcal N} + =\frac{\overline{\mathcal N}-\underline{\mathcal N}} + {\overline{\mathcal N}} +\] +when \(\overline{\mathcal N}>0\), and zero for the exact interval \([0,0]\). +The domain-volume-normalized norm interval is reported separately whenever +\(V_{\mathcal X}\) is available. It describes average physical scale over +the domain and must not be interpreted as enclosure tightness. + +The canonical CSV schemas, fixed column orders, benchmark-level requirements, +and implementation invariants are part of the mathematical reference. Agents +must not change them silently. Any extension must preserve the existing +columns and identifiers and must increase the schema version when semantics +change. + +\end{document} diff --git a/docs/direct_integrated_twojet_squares.tex b/docs/direct_integrated_twojet_squares.tex new file mode 100644 index 0000000..d2caa08 --- /dev/null +++ b/docs/direct_integrated_twojet_squares.tex @@ -0,0 +1,1004 @@ +\documentclass[11pt]{article} + +\usepackage[a4paper,margin=1in]{geometry} +\usepackage{amsmath,amssymb,amsthm,mathtools} +\usepackage{booktabs} +\usepackage{enumitem} +\usepackage{microtype} +\usepackage[hidelinks]{hyperref} + +\newtheorem{proposition}{Proposition} +\newtheorem{remark}{Remark} +\newtheorem{warning}{Important implementation warning} +\newtheorem{algorithmblock}{Algorithm} + +\newcommand{\R}{\mathbb{R}} +\newcommand{\N}{\mathbb{N}} +\newcommand{\abs}[1]{\left\lvert #1\right\rvert} +\newcommand{\norm}[1]{\left\lVert #1\right\rVert} +\newcommand{\ip}[2]{\left\langle #1,#2\right\rangle} +\newcommand{\eps}{\varepsilon} +\newcommand{\supp}{\operatorname{supp}} +\newcommand{\diag}{\operatorname{diag}} + +\title{Direct Certified Integration of Squared Polynomial-Zonotope Two-Jets\\ +\large A drop-in optimization without constructing the squared integrand} +\author{} +\date{} + +\begin{document} +\maketitle + +\begin{abstract} +This note specifies a drop-in optimization for the computation of certified +\(L^2\), \(W^{1,2}\), and \(W^{2,2}\) norm enclosures from a +polynomial-zonotope two-jet. The current implementation first constructs the +complete polynomial zonotope representing the squared norm integrand and only +then integrates it. For the Hessian contribution, this entails a costly sparse +polynomial convolution. + +The proposed routine fuses squaring, canonicalization, and integration. It +uses symmetry twice: only the upper-triangular Hessian entries are retained, +with weight two on the off-diagonal entries, and only unordered pairs of input +monomials are processed. More importantly, all selected two-jet coordinates +are contracted simultaneously through a weighted Gram matrix. The complete +squared polynomial zonotope is never constructed. + +The proposed method is algebraically exact: with the same approximation-noise +semantics and the same final interval-enclosure rule, it returns the same +mathematical enclosure as the existing pipeline. It is therefore distinct +from a remainder-norm relaxation such as an \(L^2\) triangle inequality; no +additional overestimation is introduced. +\end{abstract} + +\tableofcontents + +\section{Purpose and scope} + +The target computation is the certified integral of one of the squared +two-jet quantities +\begin{align} + \mathcal{S}_{0}(Y,J,H) + &:= \norm{Y}_2^2, \label{eq:s0}\\ + \mathcal{S}_{1}(Y,J,H) + &:= \norm{Y}_2^2+\norm{J}_F^2, \label{eq:s1}\\ + \mathcal{S}_{2}(Y,J,H) + &:= \norm{Y}_2^2+\norm{J}_F^2+\norm{H}_F^2. \label{eq:s2} +\end{align} +For a vector-valued map \(f:\R^n\to\R^r\), the dense two-jet convention is +\[ + Y\in\R^r,\qquad + J\in\R^{r\times n},\qquad + H\in\R^{r\times n\times n}, +\] +where \(H_{oab}=H_{oba}\). Consequently, +\begin{equation} + \norm{H}_F^2 + = + \sum_{o=1}^{r} + \left( + \sum_{a=1}^{n}H_{oaa}^2 + +2\sum_{1\leq a0}}m_\gamma, + & + R_{\mathrm{app}} + &:=\sum_{\substack{\gamma:\\ + \exists i\in P:\ \gamma_i>0}}r_\gamma. + \label{eq:approximation-radius} +\end{align} +Here \(q_0\) may be viewed as the coefficient with zero exponent, so its +contribution is included in \(b_0\). Under the two-class convention, +\(\gamma_i=0\) for every \(i\in P\) implies \(\gamma_U=0\); hence the +retained object is the scalar \(B=b_0\). The more general coefficient-map +notation is kept below because it matches the implementation. +It is finally interval-enclosed as +\begin{equation} + \boxed{ + \mathcal{I}_{\mathrm{current}}(Q) + = + \left[ + b_0+M_{\mathrm{app}}-\sum_{\nu\neq0}\abs{b_\nu}-R_{\mathrm{app}}, + b_0+M_{\mathrm{app}}+\sum_{\nu\neq0}\abs{b_\nu}+R_{\mathrm{app}} + \right], + } + \label{eq:current-functional} +\end{equation} +with outward rounding in an implementation. + +\begin{warning} +For an approximation-noise term, the rule uses the absolute moment +\(\mu^{\mathrm{abs}}_{\gamma_D}\), not the possibly vanishing signed moment +\(\mu_{\gamma_D}\). Thus an odd domain factor multiplying approximation +noise is retained. The rule remains coefficientwise and is generally not +the sharp pointwise integral enclosure. +\end{warning} + +\section{First symmetry: weighted upper-triangular Hessian coordinates} + +Instead of treating every scalar entry separately, collect precisely the +coordinates required by the Sobolev integrand into one vector. +For \(k\in\{0,1,2\}\), define a coordinate-selection map +\[ + \Phi_k(Y,J,H)\in\R^{N_k} +\] +as follows: +\begin{align*} + \Phi_0(Y,J,H) + &:= + \bigl(Y_o\bigr)_{o},\\ + \Phi_1(Y,J,H) + &:= + \bigl((Y_o)_o,(J_{oa})_{o,a}\bigr),\\ + \Phi_2(Y,J,H) + &:= + \bigl((Y_o)_o,(J_{oa})_{o,a}, + (H_{oaa})_{o,a},(H_{oab})_{o,a0,\\[1mm] + \text{discard}, + &\mu_{\gamma_D}=0,\\[1mm] + \texttt{retained\_coeff[\(\gamma_R\)]} + \mathrel{+}= J_X\mu_{\gamma_D}s, + &\text{otherwise}. +\end{cases} +\label{eq:routing} +\end{equation} +The constant \(c^\top W_kc\) is handled directly as +\[ + \texttt{integrated\_center} + \mathrel{+}= + J_X M\,c^\top W_kc. +\] +Equivalently, it can be passed through +\(\operatorname{route}(0,c^\top W_kc)\). + +After all contributions have been routed, set +\begin{align} + R_{\mathrm{app}} + &:= + M\sum_{\gamma} + \abs{\texttt{approximation\_coeff[\(\gamma\)]}}, + \label{eq:direct-approximation-radius}\\ + b_0 + &:= + \texttt{retained\_coeff[0]},\\ + R_{\mathrm{sym}} + &:= + \sum_{\nu\neq0} + \abs{\texttt{retained\_coeff[\(\nu\)]}}. + \label{eq:direct-symbolic-radius} +\end{align} +The final interval is +\begin{equation} + \boxed{ + [b_0-R_{\mathrm{sym}}-R_{\mathrm{app}}, + b_0+R_{\mathrm{sym}}+R_{\mathrm{app}}]. + } + \label{eq:direct-final} +\end{equation} + +\begin{proposition}[Drop-in enclosure equality] +\label{prop:enclosure-equality} +In exact arithmetic, the direct algorithm +\eqref{eq:unordered-expansion}--\eqref{eq:direct-final} returns precisely +\(\mathcal{I}_{\mathrm{current}}(Q_{\mathrm{old}})\) from +\eqref{eq:current-functional}. +\end{proposition} + +\begin{proof} +Proposition~\ref{prop:integrand-equality} gives equality of the formal squared +integrands. For domain-only terms, moment integration is linear, so applying +\(\mu_{\gamma_D}\) before collecting equal retained exponents produces the same +\(b_\nu\) as collecting the full squared polynomial first and integrating +afterward. For approximation-noise terms, the direct algorithm retains the full exponent +key and collects every contribution before taking its absolute value. +Consequently \eqref{eq:direct-approximation-radius} equals +\eqref{eq:approximation-radius}. The final interval-enclosure step is identical to +\eqref{eq:current-functional}. +\end{proof} + +\begin{warning}[Why some apparently faster variants are not drop-in exact] +The following changes do not necessarily reproduce the current enclosure: +\begin{enumerate}[label=\arabic*.] + \item Taking \(\abs{s}\) for every generated pair before equal exponent + keys have been merged loses cancellations and generally enlarges the + approximation-noise radius. + \item Integrating the \(W^{1,2}\) and Hessian contributions into two + separate intervals and then adding the intervals loses cancellations + between their coefficients. + \item Using the domain moment of an approximation-noise term is inconsistent + with the present approximation-noise semantics and can be unsound. + \item Replacing the approximation part by an \(L^2\) radius and applying a + triangle inequality is a valid alternative enclosure, but it is not the + same enclosure. +\end{enumerate} +\end{warning} + +\section{Reference algorithm} + +\begin{algorithmblock}[Direct integrated squared two-jet enclosure] +\label{alg:direct} +The inputs are two-jet polynomial zonotopes \(Y,J,H\) with aligned noise +metadata, a Sobolev order \(k\in\{0,1,2\}\), an affine cell density +\(J_X\geq0\), domain indices \(D\), and approximation-noise indices \(P\). +Proceed as follows. +\begin{enumerate}[label=\arabic*.,leftmargin=1.8em] + \item Select the coordinates \(v=\Phi_k(Y,J,H)\) and their weights \(w\). + \item Form the union support \(\mathcal{B}\) in a deterministic order, + and extract the center vector \(c\) and coefficient matrix \(A\). + \item Compute + \[ + G\gets(A\diag(w))A^\top,\qquad + g\gets(A\diag(w))c. + \] + \item Initialize + \[ + \texttt{approximation\_coeff}\gets\{\},\qquad + \texttt{retained\_coeff} + \gets\{0:J_X 2^{\abs D}c^\top\diag(w)c\}. + \] + \item For every \(\beta\in\mathcal{B}\), call + \(\operatorname{route}(\beta,2g_\beta)\). + \item For \(i=1,\ldots,\abs{\mathcal{B}}\), set + \(\beta=\mathcal{B}_i\), call + \(\operatorname{route}(2\beta,G_{ii})\), and for every \(j>i\) call + \[ + \operatorname{route}(\beta+\mathcal{B}_j,2G_{ij}). + \] + \item Compute + \[ + R_{\mathrm{app}} + \gets2^{\abs D}\sum_\gamma + \abs{\texttt{approximation\_coeff[\(\gamma\)]}}, + \quad + b_0\gets\texttt{retained\_coeff[0]}, + \] + and + \[ + R_{\mathrm{sym}} + \gets\sum_{\nu\neq0} + \abs{\texttt{retained\_coeff[\(\nu\)]}}. + \] + \item Return the outward-rounded interval + \[ + [b_0-R_{\mathrm{sym}}-R_{\mathrm{app}}, + b_0+R_{\mathrm{sym}}+R_{\mathrm{app}}]. + \] +\end{enumerate} +\end{algorithmblock} + +\begin{algorithmblock}[Routing one implicit squared-polynomial contribution] +\label{alg:route} +For a contribution \(s\eps^\gamma\), define +\(\operatorname{route}(\gamma,s)\) as follows. +\begin{enumerate}[label=\arabic*.,leftmargin=1.8em] + \item If \(s=0\), return. + \item If \(\gamma_i>0\) for some \(i\in P\), perform + \[ + \texttt{approximation\_coeff[\(\gamma\)]} + \mathrel{+}=J_X s + \] + and return. + \item Compute \(\mu=\operatorname{BoxMoment}(\gamma_D)\). If + \(\mu=0\), return. + \item Set \(\nu=\gamma_R\) and perform + \[ + \texttt{retained\_coeff[\(\nu\)]} + \mathrel{+}=J_X\mu s. + \] +\end{enumerate} +\end{algorithmblock} + +\section{Implementation blueprint for \texttt{intervalNets}} +\label{sec:implementation} + +\subsection{Recommended public and private functions} + +A suitable public helper is +\begin{verbatim} +integrate_pz_twojet_squared( + jet: PZTwoJet, + cell: PZIntegrationCell, + integrand_kind: Literal["l2", "w12", "w22"], +) -> Interval +\end{verbatim} +with private helpers of the form +\begin{verbatim} +_twojet_weighted_coordinates(jet, integrand_kind) +_coefficient_matrix(selected_coordinates, union_support) +_route_integrated_quadratic_term(...) +\end{verbatim} + +The adaptive-cell evaluator can then replace +\begin{verbatim} +integrand = _squared_twojet_integrand(jet, integrand_kind) +contribution = integrate_over_cell(integrand, cell, output="interval") +\end{verbatim} +by +\begin{verbatim} +contribution = integrate_pz_twojet_squared( + jet, cell, integrand_kind=integrand_kind +) +\end{verbatim} +The existing integrand constructors should remain available for diagnostics, +rendering, explicit \texttt{output="pz"}, and regression comparison. + +\subsection{Coordinate extraction} + +For PyTorch coefficients, form one flattened coordinate vector: +\begin{enumerate}[label=\arabic*.] + \item append all entries of \(Y\), with weights one; + \item for \(W^{1,2}\) and \(W^{2,2}\), append all entries of \(J\), with + weights one; + \item for \(W^{2,2}\), append \(H_{oaa}\) with weights one and + \(H_{oab}\), \(a{$}l<{$} X@{}} +\toprule +\text{Symbol} & \text{Meaning} \\ +\midrule +x & Physical input point in the domain of the neural network. \\ +\mathcal X\subseteq\R^{d_0} & Physical input set. \\ +X(\alpha) & Polynomial-zonotope parametrization of \(\mathcal X\). \\ +\alpha=(\alpha_1,\ldots,\alpha_p) & Domain noise symbols, with \(\alpha\in[-1,1]^p\). \\ +\eta=(\eta_1,\ldots,\eta_q) & Approximation noise symbols introduced by certified activation enclosures, with \(\eta\in[-1,1]^q\). \\ +\eps=(\alpha,\eta) & Combined vector of all polynomial variables after propagation. \\ +\lambda & Generic multi-index in \(\mathbb N_0^{p+q}\). \\ +\lambda=(\lambda^\alpha,\lambda^\eta) & Split into domain and approximation components. \\ +\eps^\lambda=\alpha^{\lambda^\alpha}\eta^{\lambda^\eta} & Monomial associated with \(\lambda\). \\ +\Lambda_P & Finite exponent support of a sparse polynomial \(P\). \\ +\bottomrule +\end{tabularx} + +The input polynomial zonotope and a general \(U\)-valued propagated +polynomial are written as +\[ + X(\alpha)=x_0+\sum_{\lambda\in\Lambda_X}x_\lambda\alpha^\lambda, + \qquad + P(\eps)=\sum_{\lambda\in\Lambda_P}P_\lambda\eps^\lambda, + \quad P_\lambda\in U. +\] +When several multi-indices occur in one formula, auxiliary letters such as +\(\kappa\) and \(\gamma\) may be used; \(\lambda\) remains the canonical +generic multi-index. + +\section{Network and two-jet notation} + +\noindent\begin{tabularx}{\textwidth}{@{}>{$}l<{$} X@{}} +\toprule +\text{Symbol} & \text{Meaning} \\ +\midrule +\Phi & Complete neural network. \\ +A_\ell(y)=W_\ell y+b_\ell & Affine layer, with weight matrix \(W_\ell\) and bias \(b_\ell\). \\ +\sigma & Scalar activation function. \\ +\sigma_\ell & Componentwise activation layer. \\ +z_\ell,\ y_\ell & Preactivation and postactivation at hidden layer \(\ell\). \\ +Z_\ell(\eps),\ Y_\ell(\eps) & Corresponding polynomial enclosures. \\ +J_\ell(\eps),\ H_\ell(\eps) & Polynomial enclosures of the first and second physical-input derivatives. \\ +\left.\partial F\right|_x & First derivative of \(F\) evaluated at \(x\). \\ +\left.\partial^2F\right|_x & Second derivative of \(F\) evaluated at \(x\). \\ +\left.\partial_aF^i\right|_x,\ \left.\partial_{ab}F^i\right|_x & Coordinate entries of these derivatives. \\ +J_X(\alpha)=\left|\det\left.\partial X\right|_\alpha\right| & Nonnegative geometric Jacobian density of the domain parametrization. \\ +\bottomrule +\end{tabularx} + +We use +\[ + \Phi=A_L\circ\sigma_L\circ\cdots\circ A_1\circ\sigma_1\circ A_0, +\] +and the propagated output coefficients have types +\[ + Y_\lambda\in\R^{d_{\rm out}},\qquad + J_\lambda\in\R^{d_{\rm out}}\otimes\R^{d_0*},\qquad + H_\lambda\in\R^{d_{\rm out}}\otimes\Sym^2(\R^{d_0*}). +\] +The derivatives are always with respect to the physical input \(x\); the +dependence on \(\alpha\) only records evaluation along \(X(\alpha)\). + +\section{Activation enclosures and terminology} + +For \(r\in\{0,1,2\}\), with \(\sigma^{(0)}=\sigma\), the standard enclosure is +\[ + \sigma^{(r)}(t) + \in \widehat\sigma_{\ell i}^{(r)}(t) + +\rho_{\ell i}^{(r)}\eta_{\ell i}^{(r)}, + \qquad \eta_{\ell i}^{(r)}\in[-1,1]. +\] + +\noindent\begin{tabularx}{\textwidth}{@{}>{$}l<{$} X@{}} +\toprule +\text{Symbol or term} & \text{Meaning} \\ +\midrule +\widehat\sigma^{(r)} & Polynomial or affine approximation of \(\sigma^{(r)}\) on the certified preactivation interval. \\ +\rho\geq0 & Certified approximation-error radius. \\ +\eta^{(r)} & Approximation noise symbol. \\ +\rho\eta & Approximation noise term; indices are added when needed. \\ +r(t) & Residual function, defined by +\(r(t)=\sigma^{(r)}(t)-\widehat\sigma^{(r)}(t)\). \\ +\bottomrule +\end{tabularx} +\normalsize + +A fresh approximation noise symbol is introduced for every independently +certified scalar enclosure. In the general two-jet construction this gives +\(\eta_{\ell i}^{(0)},\eta_{\ell i}^{(1)},\eta_{\ell i}^{(2)}\) per neuron. +The same \(\eta_{\ell i}^{(1)}\) is reused wherever the same enclosure of +\(\sigma'\) occurs, including both the Jacobian and Hessian recurrences. +Special constructions may instead derive all activation derivatives from one +shared enclosure; such dependency sharing must be stated explicitly. + +\section{Canonical vocabulary} + +\noindent\begin{tabularx}{\textwidth}{@{}l X@{}} +\toprule +Preferred term & Usage \\ +\midrule +Domain noise symbol & A component of \(\alpha\). \\ +Approximation noise symbol & A component of \(\eta\); this is the canonical name. \\ +Approximation-error radius & The nonnegative coefficient \(\rho\). \\ +Approximation noise term & The product \(\rho\eta\). \\ +Residual function & The actual difference between a function and its approximation. \\ +Polynomial enclosure & A polynomial together with its approximation noise terms. \\ +\bottomrule +\end{tabularx} + +The terms \emph{residual symbol}, \emph{remainder symbol}, and +\emph{pointwise-error variable} are avoided as alternative names for \(\eta\). +The approximation noise symbol is a formal enclosure variable; its coefficient +is the certified uniform bound on the residual over the relevant interval. + +\end{document} diff --git a/docs/polynomial_zonotope_twojet_recurrence.tex b/docs/polynomial_zonotope_twojet_recurrence.tex new file mode 100644 index 0000000..d31c55a --- /dev/null +++ b/docs/polynomial_zonotope_twojet_recurrence.tex @@ -0,0 +1,290 @@ +\documentclass[11pt]{article} + +\usepackage[a4paper,margin=1in]{geometry} +\usepackage{amsmath,amssymb,mathtools} +\usepackage{microtype} +\usepackage[hidelinks]{hyperref} + +\newcommand{\R}{\mathbb{R}} +\newcommand{\N}{\mathbb{N}} +\newcommand{\eps}{\varepsilon} +\newcommand{\etaapp}{\eta} +\newcommand{\Sym}{\operatorname{Sym}} +\newcommand{\diag}{\operatorname{diag}} + +\title{Recurrence Formulas for Polynomial-Zonotope\\ +Two-Jet Enclosures of Neural Networks} +\author{} +\date{} + +\begin{document} +\maketitle + +\section{Network, input, and enclosure convention} + +Consider the neural network +\begin{equation} + \Phi + =A_L\circ\sigma_L\circ\cdots\circ A_1\circ\sigma_1\circ A_0 + :\R^{d_0}\longrightarrow\R^{d_{L+1}}, + \label{eq:network} +\end{equation} +where +\[ + A_\ell(y)=W_\ell y+b_\ell, + \qquad + W_\ell\in\R^{d_{\ell+1}\times d_\ell}, + \qquad \ell=0,\ldots,L, +\] +and each activation is applied componentwise, +\[ + \sigma_\ell(z)_i=\sigma(z_i), + \qquad z\in\R^{d_\ell}. +\] + +The input set is given parametrically by a polynomial zonotope +\begin{equation} + X:[-1,1]^p\longrightarrow\R^{d_0}, + \qquad + X(\alpha)=x_0+\sum_{\lambda\in\Lambda_X}x_\lambda\alpha^\lambda, + \label{eq:input-pz} +\end{equation} +where \(\Lambda_X\subset\N_0^p\setminus\{0\}\) is finite and +\(\alpha^\lambda=\prod_{r=1}^p\alpha_r^{\lambda_r}\). +The variables \(\alpha\) are the \emph{domain noise symbols}. + +For every activation layer \(\ell\), neuron \(i\), and derivative order +\(r\in\{0,1,2\}\), first compute a certified preactivation interval +\(I_{\ell i}\) and choose a polynomial \(\widehat\sigma_{\ell i}^{(r)}\) +and an approximation-error radius \(\rho_{\ell i}^{(r)}\geq0\) such that +\begin{equation} + \sigma^{(r)}(t) + \in + \widehat\sigma_{\ell i}^{(r)}(t) + +\rho_{\ell i}^{(r)}[-1,1] + \qquad\text{for every }t\in I_{\ell i}. + \label{eq:scalar-enclosure} +\end{equation} +Equivalently, at each point one may write +\begin{equation} + \sigma^{(r)}(t) + =\widehat\sigma_{\ell i}^{(r)}(t) + +\rho_{\ell i}^{(r)}\eta_{\ell i}^{(r)}, + \qquad \eta_{\ell i}^{(r)}\in[-1,1]. + \label{eq:scalar-noise} +\end{equation} +These \(\eta_{\ell i}^{(r)}\) are the \emph{approximation noise symbols}. +Their coefficients are certified uniform bounds for the corresponding +residual functions. There are no further kinds of noise symbols. + +If the activation satisfies an autonomous polynomial ODE +\(\sigma'=g(\sigma)\), then +\[ + \sigma''=g'(\sigma)g(\sigma). +\] +These identities may be used to construct or certify the polynomials for the +first and second derivatives. The recurrence below does not depend on how the +three certified scalar enclosures are obtained. + +The goal is to propagate the input parametrization immediately and obtain +polynomial enclosures +\[ + Y(\eps),\qquad + J(\eps),\qquad + H(\eps), + \qquad \eps=(\alpha,\eta), +\] +for +\[ + \Phi(X(\alpha)),\qquad + \left.\partial\Phi\right|_{X(\alpha)},\qquad + \left.\partial^2\Phi\right|_{X(\alpha)}, +\] +respectively. Thus \(J\) and \(H\) are derivatives with respect to the +physical network input \(x\in\R^{d_0}\), not derivatives with respect to +the parameter \(\alpha\). The parameter \(\alpha\) merely records where +the two-jet is evaluated. + +\section{Initialization} + +Initialize the two-jet of the identity map, already evaluated on the input +polynomial zonotope, by +\begin{equation} + Y^{\mathrm{in}}(\alpha)=X(\alpha), + \qquad + J^{\mathrm{in}}=I_{d_0}, + \qquad + H^{\mathrm{in}}=0. + \label{eq:initialization} +\end{equation} +Here +\[ + Y^{\mathrm{in}}\in\R^{d_0},\qquad + J^{\mathrm{in}}\in\R^{d_0\times d_0},\qquad + H^{\mathrm{in}}\in\R^{d_0\times d_0\times d_0}. +\] +All entries are polynomials in \(\alpha\); initially no approximation noise +symbols are present. + +\section{The two recurrence steps} + +The complete algorithm consists only of an affine step and a componentwise +activation step. + +\subsection{Affine step} + +Suppose \((Y,J,H)\) encloses the two-jet of a map with values in \(\R^m\), +and let \(A(y)=Wy+b\) with \(W\in\R^{n\times m}\). Then set +\begin{equation} + \boxed{ + \begin{aligned} + Z_i &= b_i+\sum_{j=1}^m W_{ij}Y_j,\\ + J^Z_{ia} &= \sum_{j=1}^m W_{ij}J_{ja},\\ + H^Z_{iab}&= \sum_{j=1}^m W_{ij}H_{jab}. + \end{aligned}} + \label{eq:affine-recurrence} +\end{equation} +Equivalently, \(Z=WY+b\), \(J^Z=WJ\), and \(H^Z=WH\), where \(W\) +contracts the output index of \(H\). An affine layer introduces no new +noise symbols and no approximation error. + +\subsection{Componentwise activation step} + +Assume that \((Z,J^Z,H^Z)\) is the preactivation two-jet at activation layer +\(\ell\), with \(Z\in\R^{d_\ell}\). Introduce the scalar polynomial +enclosures +\begin{equation} + S_{\ell i}^{(r)} + :=\widehat\sigma_{\ell i}^{(r)}(Z_i) + +\rho_{\ell i}^{(r)}\eta_{\ell i}^{(r)}, + \qquad r=0,1,2. + \label{eq:substituted-enclosure} +\end{equation} +Because \(Z_i\) is already a polynomial in the domain and previously +introduced approximation symbols, the substitution +\(\widehat\sigma_{\ell i}^{(r)}(Z_i)\) is again a polynomial in those symbols. + +The activated two-jet is +\begin{equation} + \boxed{ + \begin{aligned} + Y_i + &=S_{\ell i}^{(0)},\\ + J_{ia} + &=S_{\ell i}^{(1)}J^Z_{ia},\\ + H_{iab} + &=S_{\ell i}^{(2)}J^Z_{ia}J^Z_{ib} + +S_{\ell i}^{(1)}H^Z_{iab}. + \end{aligned}} + \label{eq:activation-recurrence} +\end{equation} +The indices range over +\[ + i=1,\ldots,d_\ell, + \qquad + a,b=1,\ldots,d_0. +\] +Formula \eqref{eq:activation-recurrence} is exactly the componentwise +second-order chain rule with \(\sigma\), \(\sigma'\), and \(\sigma''\) +replaced by their certified polynomial enclosures. + +The same symbol \(\eta_{\ell i}^{(1)}\), and hence the same polynomial +\(S_{\ell i}^{(1)}\), should be reused in the Jacobian formula and in the +second term of the Hessian formula. This preserves the dependency created +by the repeated occurrence of the same quantity \(\sigma'(Z_i)\). +The symbols for derivative orders \(r=0,1,2\) remain distinct because they +certify three different scalar residuals. + +For reference, in compact tensor notation let +\[ + D_\ell=\diag(S_\ell^{(1)}), + \qquad + E_\ell=\diag(S_\ell^{(2)}). +\] +Then +\begin{equation} + J=D_\ell J^Z, + \qquad + H_{iab} + =(E_\ell)_{ii}J^Z_{ia}J^Z_{ib} + +(D_\ell)_{ii}H^Z_{iab}. + \label{eq:compact-activation} +\end{equation} + +\section{Layerwise algorithm} + +Starting from \eqref{eq:initialization}, perform the following operations: +\begin{enumerate} + \item Apply the affine recurrence \eqref{eq:affine-recurrence} with + \(A_0\). + \item For \(\ell=1,\ldots,L\): + \begin{enumerate} + \item apply the activation recurrence + \eqref{eq:activation-recurrence} for \(\sigma_\ell\); + \item apply the affine recurrence + \eqref{eq:affine-recurrence} with \(A_\ell\). + \end{enumerate} +\end{enumerate} +After the final affine map \(A_L\), the resulting arrays have shapes +\begin{equation} + Y\in\R^{d_{L+1}}, + \qquad + J\in\R^{d_{L+1}\times d_0}, + \qquad + H\in\R^{d_{L+1}\times d_0\times d_0}, + \label{eq:output-shapes} +\end{equation} +and every entry is a polynomial in +\begin{equation} + \eps=(\alpha,\eta), + \qquad + \alpha\in[-1,1]^p, + \quad + \eta\in[-1,1]^q, + \label{eq:all-noises} +\end{equation} +for the total number \(q\) of introduced approximation symbols. Consequently, +\((Y,J,H)\) is itself a polynomial-zonotope enclosure of the two-jet of +\(\Phi\) over \(X([-1,1]^p)\). + +\section{Ground-truth identities and implementation invariants} + +The recurrence is justified by the exact identities +\begin{align} + \left.\partial(A\circ f)\right|_x + &=W\left.\partial f\right|_x,\\ + \left.\partial^2(A\circ f)\right|_x + &=W\left.\partial^2 f\right|_x,\\ + \left.\partial_a(\sigma\circ f)_i\right|_x + &=\sigma'(f_i(x))\left.\partial_a f_i\right|_x,\\ + \left.\partial_{ab}(\sigma\circ f)_i\right|_x + &=\sigma''(f_i(x)) + \left.\partial_a f_i\right|_x + \left.\partial_b f_i\right|_x + +\sigma'(f_i(x))\left.\partial_{ab} f_i\right|_x. + \label{eq:exact-identities} +\end{align} +Replacing the three scalar activation quantities in these identities by +\eqref{eq:scalar-noise} gives \eqref{eq:activation-recurrence}. Induction over +the layers therefore yields a pointwise enclosure of the value, Jacobian, and +Hessian, provided every interval \(I_{\ell i}\) contains the corresponding +preactivation range and every scalar bound \eqref{eq:scalar-enclosure} is +certified (with outward rounding in floating-point arithmetic). + +The following are useful implementation invariants: +\begin{itemize} + \item \(H_{iab}=H_{iba}\); only one triangular half need be stored if the + remaining code respects this convention. + \item Domain symbols are created only by the input parametrization. + Activation layers create only approximation symbols. + \item Polynomial coefficients are vector-, matrix-, or tensor-valued, but + the exponent keys belong to the single common variable vector + \(\eps=(\alpha,\eta)\). + \item Canonicalize equal exponent vectors after polynomial products and + substitutions; this does not alter the enclosure. + \item Degree or term truncation is not part of the recurrence above. If + used, every discarded contribution must be transferred to a fresh + certified approximation enclosure. +\end{itemize} + +\end{document} diff --git a/docs/technical-report.md b/docs/technical-report.md index 3735c7e..0f7f0b1 100644 --- a/docs/technical-report.md +++ b/docs/technical-report.md @@ -5,7 +5,7 @@ ## Executive Summary -intervalNets provides interval arithmetic and interval-aware neural-network evaluation with outward rounding. The repository combines a compact mathematical core (`interval.py`) with a PyTorch integration layer (`pytorch.py`) that overloads model evaluation on interval inputs and adds certified bounds for Jacobians, $L^p$ norms, and Sobolev-style norms. This document focuses on those two files. +intervalNets provides interval arithmetic and interval-aware neural-network evaluation with outward rounding. The repository combines a compact mathematical core (`interval.py`) with a PyTorch integration layer (`pytorch.py`) that overloads model evaluation on interval inputs and adds certified bounds for Jacobians/Hessians, $L^p$ norms, and Sobolev-style norms (orders 1 and 2). This document focuses on those two files. The current implementation emphasizes **fast conservative enclosures** for neural-network workloads, especially matrix-by-interval-vector propagation and Jacobian composition. It does not attempt to @@ -16,7 +16,7 @@ compute globally tightest interval enclosures in every step. At a high level, the codebase separates concerns: - `src/intervalnets/interval.py`: scalar and nested-tuple interval representation, shape-aware operations, and outward-rounded arithmetic primitives. -- `src/intervalnets/pytorch.py`: PyTorch-facing interval tensor wrapper, layer-wise interval forward propagation, interval Jacobian propagation, and adaptive box refinement for norm enclosures. +- `src/intervalnets/pytorch.py`: PyTorch-facing interval tensor wrapper, layer-wise interval forward propagation, interval Jacobian/Hessian propagation, and adaptive box refinement for norm enclosures. The architectural pattern is: **core numeric enclosure logic first**, then **framework adaptation**. @@ -141,7 +141,7 @@ $ ### 1) Purpose -`pytorch.py` bridges the pure interval core to PyTorch modules. It introduces `IntervalTensor`, interval forward propagation for supported layers, Jacobian interval bounds, and adaptive interval integration for $L^p$ and Sobolev norms. It monkey-patches `nn.Module` to expose `model.eval(interval)`, `model.lpnorm(...)`, `model.eval_jacobian(...)`, and `model.sobolev_norm(...)`. +`pytorch.py` bridges the pure interval core to PyTorch modules. It introduces `IntervalTensor`, interval forward propagation for supported layers, Jacobian/Hessian interval bounds, and adaptive interval integration for $L^p$ and Sobolev norms. It monkey-patches `nn.Module` to expose `model.eval(interval)`, `model.lpnorm(...)`, `model.eval_jacobian(...)`, `model.eval_hessian(...)`, and `model.sobolev_norm(...)`. ### 2) Structured Code Breakdown @@ -150,11 +150,11 @@ $ Core orchestration and helpers include: - Environment and conversion: `_require_torch`, `IntervalTensor.point`, `IntervalTensor.from_bounds`, `IntervalTensor.to_torch`. -- Layer forward helpers: `_linear_forward`, `_relu_forward`, `_sigmoid_forward`, `_tanh_forward`, `_softplus_forward`, `_leaky_relu_forward`, `_softmax_forward`, plus slope-aware affine-relaxation helpers (`_concretize_affine_bounds`, `_linear_relaxation_step`, `_relu_relaxation_step`, `_sequential_linear_relu_relaxation`). +- Layer forward helpers: `_linear_forward`, `_relu_forward`, `_sigmoid_forward`, `_tanh_forward`, `_softplus_forward`, `_leaky_relu_forward`, `_softmax_forward`, plus slope-aware linear-relaxation helpers (`_concretize_affine_bounds`, `_linear_relaxation_step`, `_relu_relaxation_step`, `_sequential_linear_relu_relaxation`). - Composite helpers: `_interval_add`, `_interval_cat`, `_logsumexp`, `_softmax_component_bounds`. - Norm machinery: `_box_volume`, `_lp_pointwise_power_bounds`, `_split_box`, `_lpnorm_bounds`. -- Jacobian machinery: `_identity_jacobian`, `_matrix_multiply`, `_jacobian_for_layer`, `_eval_jacobian_bounds`. -- Sobolev machinery: `_sobolev_pointwise_power_bounds`, `_sobolev_norm_bounds`. +- Jacobian/Hessian machinery: `_identity_jacobian`, `_matrix_multiply`, `_jacobian_for_layer`, `_hessian_for_layer`, `_hessian_compose`, `_eval_jacobian_bounds`, `_eval_hessian_bounds`. +- Sobolev machinery: `_sobolev_pointwise_power_bounds`, `_sobolev_norm_bounds` with `order ∈ {1,2}`. - Public dispatch/patch: `interval_forward(module, x, enclosure_mode=...)`, `enable_interval_eval(enclosure_mode=...)`. #### Classes and methods @@ -174,7 +174,7 @@ Core orchestration and helpers include: - `_relu_forward`: specialized ReLU propagation that preserves exact `[0, 0]` images on non-positive intervals. - `_interval_abs_bounds` and `_interval_pow_scalar`: scalar interval transformations used in integral bounds. - `_split_box`: adaptive refinement by bisecting widest coordinate. -- Slope-aware helpers keep lower/upper affine forms in the input variables and concretize with outward rounding to preserve certified enclosure guarantees. +- Slope-aware helpers keep lower/upper linear relaxation forms in the input variables and concretize with outward rounding to preserve certified enclosure guarantees. #### Important imports and dependencies @@ -191,10 +191,10 @@ Core orchestration and helpers include: [monkey patch nn.Module] /b |c \d v v v - [eval(interval)] [lpnorm] [eval_jacobian/sobolev_norm] + [eval(interval)] [lpnorm] [eval_jacobian/eval_hessian/sobolev_norm] |e |f |g v v v - [interval_forward] [_lpnorm_bounds] [_eval_jacobian_bounds/_sobolev_norm_bounds] + [interval_forward] [_lpnorm_bounds] [_eval_jacobian_bounds/_eval_hessian_bounds/_sobolev_norm_bounds] /h |i |j |k\ |l |m v v v v v v v [Linear][Acts][Softmax][Add/Cat][Identity] [_split_box + _box_volume] [_jacobian_for_layer] @@ -211,16 +211,16 @@ Core orchestration and helpers include: - (a) `enable_interval_eval` is the single entry for activating interval behavior. - (b) Patched `eval(interval)` routes interval input to interval forward propagation. - (c) Patched `lpnorm` routes to adaptive integral enclosure. -- (d) Patched Jacobian and Sobolev APIs route to derivative-aware enclosure routines. +- (d) Patched Jacobian/Hessian and Sobolev APIs route to derivative-aware enclosure routines. - (e) `eval(interval)` invokes `interval_forward` dispatch by module type. - (f) `lpnorm` invokes `_lpnorm_bounds`. -- (g) Jacobian/Sobolev patched methods invoke `_eval_jacobian_bounds` / `_sobolev_norm_bounds`. +- (g) Jacobian/Hessian/Sobolev patched methods invoke `_eval_jacobian_bounds` / `_eval_hessian_bounds` / `_sobolev_norm_bounds`. - (h) `interval_forward` delegates linear layers to `_linear_forward`. - (i) `interval_forward` delegates monotone activations to dedicated helpers. - (j) `interval_forward` delegates Softmax to specialized bound logic. - (k) branch combinators (`IntervalAdd`, `IntervalCat`) route to structural interval helpers. - (l) `_lpnorm_bounds` repeatedly uses `_split_box` and `_box_volume` for adaptive refinement. -- (m) Jacobian/Sobolev flows rely on `_jacobian_for_layer` (and then aggregation). +- (m) Jacobian/Hessian/Sobolev flows rely on `_jacobian_for_layer` / `_hessian_for_layer` (and then aggregation). - (n) `_linear_forward` uses `_scalar_interval_from_weight` per coefficient/bias term. - (o) activation helpers share `_apply_monotone_bounds` when monotonicity applies. - (p) Softmax helper computes component extrema via `_softmax_component_bounds`. @@ -251,7 +251,9 @@ $ $ \|f\|_{L^p} = \left(\int |f(x)|^p\,dx\right)^{1/p}. $ -- Sobolev-style enclosure similarly accumulates powers of function outputs and Jacobian entries before integration. +- Sobolev-style enclosure accumulates powers of function outputs and derivative entries before integration: + - `order=1`: outputs + Jacobian entries, + - `order=2`: outputs + Jacobian + Hessian entries. ### 6) Implementation Notes @@ -259,10 +261,10 @@ $ - Optional PyTorch dependency is guarded (`try/except ImportError`) and validated via `_require_torch`. - Monkey patching is global (`nn.Module`), one-way for process lifetime, and guarded by `_PATCHED`. - Adaptive integration chooses the box with largest indicator `(integrand width) * (box volume)` for bisection. -- Sobolev refinement avoids over-refining rigorously constant boxes by detecting exact-constant outputs paired with exact-zero Jacobian enclosures and assigning zero refinement indicators to those boxes. +- Sobolev refinement avoids over-refining rigorously constant boxes by detecting exact-constant outputs paired with exact-zero Jacobian enclosures (and exact-zero Hessian enclosures for `order=2`) and assigning zero refinement indicators to those boxes. - Forward enclosure mode is configurable: - `"box"`: baseline midpoint-radius propagation. - - `"slope"`: slope-aware affine relaxation for `nn.Sequential` chains of `nn.Linear` and `nn.ReLU`; when an unsupported layer appears, bounds are first concretized and propagation conservatively continues in `"box"` mode. + - `"slope"`: slope-aware linear relaxation for `nn.Sequential` chains of `nn.Linear` and `nn.ReLU`; when an unsupported layer appears, bounds are first concretized and propagation conservatively continues in `"box"` mode. --- diff --git a/notebooks/affine_pz_twojet_adaquad_benchmarks.ipynb b/notebooks/affine_pz_twojet_adaquad_benchmarks.ipynb new file mode 100644 index 0000000..1e2a409 --- /dev/null +++ b/notebooks/affine_pz_twojet_adaquad_benchmarks.ipynb @@ -0,0 +1,414 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Affine PZ two-jet versus interval AdaQuad benchmarks\n", + "\n", + "Deterministic float64 tanh-network benchmark comparing the affine polynomial-zonotope (PZ) two-jet norm enclosure with the existing interval adaptive quadrature (AdaQuad) norm path. The PZ path uses the closed-form affine enclosures for `tanh`, `tanh'`, and `tanh''`; this notebook intentionally exposes no approximation tuning knobs for the activation enclosure.\n", + "\n", + "The trace helpers run one adaptive refinement pass up to a configured maximum number of refinement steps and record the partial certified result after each step from the same active-cell cache, rather than issuing many independent API calls.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Reproducible setup\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from __future__ import annotations\n", + "\n", + "import math, random, sys, time\n", + "from pathlib import Path\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "import torch.nn as nn\n", + "\n", + "repo_root = Path.cwd().resolve()\n", + "while not (repo_root / \"src\" / \"intervalnets\").exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "if str(repo_root / \"src\") not in sys.path:\n", + " sys.path.insert(0, str(repo_root / \"src\"))\n", + "\n", + "\n", + "from intervalnets import Interval, IntervalTensor, PZIntegrationCell, enable_interval_eval\n", + "from intervalnets.pz_norms import pz_twojet_l2_integrand, pz_twojet_l2_norm, pz_twojet_w12_integrand, pz_twojet_w12_norm, pz_twojet_w22_integrand, pz_twojet_w22_norm\n", + "from intervalnets.pz_integration import integrate_over_cell, integrate_pz_twojet_squared, _dorfler_marking as _pz_dorfler_marking, _evaluate_squared_contribution_cache, _interval_add, _interval_width, _split_box, _sqrt_interval_nonnegative\n", + "from intervalnets.pytorch import _box_volume, _choose_split_dim, _dorfler_marking, _hessian_is_exact_zero, _interval_tensor_is_exact_constant, _jacobian_is_exact_zero, _lp_pointwise_power_bounds_refined, _sobolev_pointwise_power_bounds_refined\n", + "\n", + "enable_interval_eval(\"slope\")\n", + "SEED = 20260722\n", + "random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)\n", + "torch.set_default_dtype(torch.float64)\n", + "print(f\"repo_root={repo_root}\")\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Small tanh network and `IntervalTensor` domain\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def make_tanh_network(input_dim: int = 2, width: int = 5) -> nn.Sequential:\n", + " return nn.Sequential(nn.Linear(input_dim, width), nn.Tanh(), nn.Linear(width, width), nn.Tanh(), nn.Linear(width, 1)).to(dtype=torch.float64)\n", + "\n", + "model = make_tanh_network()\n", + "with torch.no_grad():\n", + " for idx, param in enumerate(model.parameters()):\n", + " gen = torch.Generator().manual_seed(SEED + idx)\n", + " param.copy_(0.25 * torch.randn(param.shape, generator=gen, dtype=torch.float64))\n", + "\n", + "# Interval domain construction with IntervalTensor.\n", + "domain = IntervalTensor.from_bounds(torch.tensor([-0.90, -0.65], dtype=torch.float64), torch.tensor([0.80, 0.70], dtype=torch.float64))\n", + "MAX_REFINEMENT_STEPS = 4\n", + "THETA = 0.5\n", + "FORWARD_REFINE_SPLITS = 1\n", + "FORWARD_REFINE_MAX_CELLS = 256\n", + "model, domain, MAX_REFINEMENT_STEPS\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Direct PZ two-jet norm calls on the affine tanh enclosure path\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Final two-jet monomial diagnostics\n", + "\n", + "Summarize the final propagated polynomial-zonotope two-jet before constructing the norm integrands.\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cell = PZIntegrationCell.from_affine_box(domain)\n", + "t0 = time.perf_counter()\n", + "jet = model.eval_pz_twojet(cell.domain)\n", + "twojet_construction_s = time.perf_counter() - t0\n", + "\n", + "from collections import Counter\n", + "\n", + "\n", + "def summarize_final_twojet_component(component, z):\n", + " kind_counts = Counter(z.noise_kinds)\n", + " approximation_noise = sum(\n", + " count for kind, count in kind_counts.items()\n", + " if \"approximation\" in str(kind)\n", + " )\n", + " return {\n", + " \"component\": component,\n", + " \"shape\": tuple(z.shape),\n", + " \"distinct_monomials\": len(z.terms),\n", + " \"max_degree\": max((sum(exp) for exp in z.terms), default=0),\n", + " \"domain_noise\": kind_counts.get(\"domain\", 0),\n", + " \"approximation_noise\": approximation_noise,\n", + " \"total_noise\": z.num_noise,\n", + " }\n", + "\n", + "\n", + "final_twojet_monomial_diagnostics = pd.DataFrame([\n", + " summarize_final_twojet_component(\"value Y\", jet.Y),\n", + " summarize_final_twojet_component(\"first derivative J\", jet.J),\n", + " summarize_final_twojet_component(\"second derivative H\", jet.H),\n", + "])\n", + "final_twojet_monomial_diagnostics\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "## Optimized PZ two-jet norm bounds and phase timings\n", + "\n", + "# Keep the explicit squared PZ only as an optional regression/benchmark path.\n", + "EXPLICIT_COMPARISON = False\n", + "constructors = {\"L2\": pz_twojet_l2_integrand, \"W12\": pz_twojet_w12_integrand, \"W22\": pz_twojet_w22_integrand}\n", + "norms = {\"L2\": pz_twojet_l2_norm, \"W12\": pz_twojet_w12_norm, \"W22\": pz_twojet_w22_norm}\n", + "kinds = {\"L2\": \"l2\", \"W12\": \"w12\", \"W22\": \"w22\"}\n", + "\n", + "phase_rows = []\n", + "for quantity in [\"L2\", \"W12\", \"W22\"]:\n", + " explicit_integrand = None\n", + " explicit_integrand_s = explicit_integration_s = None\n", + " explicit_bounds = None\n", + " if EXPLICIT_COMPARISON:\n", + " t0 = time.perf_counter(); explicit_integrand = constructors[quantity](jet)\n", + " explicit_integrand_s = time.perf_counter() - t0\n", + " t0 = time.perf_counter(); explicit_integral = integrate_over_cell(explicit_integrand, cell, output=\"interval\")\n", + " explicit_integration_s = time.perf_counter() - t0\n", + " explicit_bounds = _sqrt_interval_nonnegative(explicit_integral)\n", + "\n", + " t0 = time.perf_counter(); direct_integral = integrate_pz_twojet_squared(jet, cell, kinds[quantity])\n", + " direct_integrated_square_s = time.perf_counter() - t0\n", + " # This is the optimized public API used by applications.\n", + " bounds = norms[quantity](jet, cell)\n", + " if explicit_bounds is not None:\n", + " assert math.isclose(float(bounds.lower), float(explicit_bounds.lower), rel_tol=1e-11, abs_tol=1e-11)\n", + " assert math.isclose(float(bounds.upper), float(explicit_bounds.upper), rel_tol=1e-11, abs_tol=1e-11)\n", + " phase_rows.append({\n", + " \"quantity\": quantity, \"bounds\": bounds,\n", + " \"twojet_construction_s\": twojet_construction_s,\n", + " \"explicit_integrand_construction_s\": explicit_integrand_s,\n", + " \"direct_integrated_square_s\": direct_integrated_square_s,\n", + " \"explicit_integrand_integration_s\": explicit_integration_s,\n", + " \"total_cell_s\": twojet_construction_s + direct_integrated_square_s,\n", + " })\n", + "\n", + "direct_pz_norms = pd.DataFrame(phase_rows)\n", + "direct_pz_norms[\"lower\"] = direct_pz_norms[\"bounds\"].map(lambda z: float(z.lower))\n", + "direct_pz_norms[\"upper\"] = direct_pz_norms[\"bounds\"].map(lambda z: float(z.upper))\n", + "direct_pz_norms[\"width\"] = direct_pz_norms[\"upper\"] - direct_pz_norms[\"lower\"]\n", + "direct_pz_norms.drop(columns=\"bounds\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Existing interval AdaQuad norm calls for comparison\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def norm_call(method: str, quantity: str, refinement_steps: int):\n", + " common = dict(iterations=refinement_steps, theta=THETA)\n", + " if method == \"interval\":\n", + " common.update(forward_refine_splits=FORWARD_REFINE_SPLITS, forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS)\n", + " if quantity == \"L2\":\n", + " return model.lpnorm(domain, 2.0, method=\"interval\", **common)\n", + " return model.sobolev_norm(domain, 2.0, order=1 if quantity == \"W12\" else 2, method=\"interval\", **common)\n", + " if quantity == \"L2\":\n", + " return model.pz_l2norm(domain, p=2.0, **common)\n", + " return model.pz_sobolev_norm(domain, p=2.0, order=1 if quantity == \"W12\" else 2, **common)\n", + "\n", + "api_rows = []\n", + "for quantity in [\"L2\", \"W12\", \"W22\"]:\n", + " for method in [\"interval\", \"pz\"]:\n", + " t0 = time.perf_counter(); bounds = norm_call(method, quantity, MAX_REFINEMENT_STEPS)\n", + " api_rows.append({\"quantity\": quantity, \"method\": method, \"refinement_steps\": MAX_REFINEMENT_STEPS, \"lower\": float(bounds.lower), \"upper\": float(bounds.upper), \"width\": float(bounds.upper) - float(bounds.lower), \"seconds\": time.perf_counter() - t0})\n", + "api_comparison = pd.DataFrame(api_rows)\n", + "api_comparison\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5. Trace helpers for tables and convergence diagnostics\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def width(bounds: Interval) -> float:\n", + " return float(bounds.upper) - float(bounds.lower)\n", + "\n", + "def make_row(quantity, method, iteration, bounds, cells, elapsed, extra=None):\n", + " return {\"quantity\": quantity, \"method\": method, \"iteration\": iteration, \"lower\": float(bounds.lower), \"upper\": float(bounds.upper), \"width\": width(bounds), \"runtime_s\": elapsed, \"cells\": cells, **(extra or {})}\n", + "\n", + "def interval_power_bounds(box, quantity):\n", + " if quantity == \"L2\":\n", + " return _lp_pointwise_power_bounds_refined(model, box, 2.0, forward_refine_splits=FORWARD_REFINE_SPLITS, forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS)\n", + " return _sobolev_pointwise_power_bounds_refined(model, box, 2.0, order=1 if quantity == \"W12\" else 2, forward_refine_splits=FORWARD_REFINE_SPLITS, forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS)\n", + "\n", + "def interval_aggregate(boxes, quantity):\n", + " total = Interval.point(0.0)\n", + " for box in boxes:\n", + " total = total + interval_power_bounds(box, quantity) * _box_volume(box)\n", + " return _sqrt_interval_nonnegative(total)\n", + "\n", + "def interval_indicator_split(box, quantity):\n", + " pointwise = interval_power_bounds(box, quantity)\n", + " indicator = _interval_width(pointwise) * _box_volume(box)\n", + " jac = model.eval_jacobian(box)\n", + " if quantity != \"L2\":\n", + " hess = model.eval_hessian(box) if quantity == \"W22\" else None\n", + " if _interval_tensor_is_exact_constant(model.eval(box)) and _jacobian_is_exact_zero(jac) and (hess is None or _hessian_is_exact_zero(hess)):\n", + " indicator = 0.0\n", + " return indicator, _choose_split_dim(box, jac)\n", + "\n", + "def run_interval_trace(quantity, max_refinement_steps):\n", + " boxes, rows, elapsed = [domain], [], 0.0\n", + " for iteration in range(max_refinement_steps + 1):\n", + " t0 = time.perf_counter(); bounds = interval_aggregate(boxes, quantity); elapsed += time.perf_counter() - t0\n", + " rows.append(make_row(quantity, \"interval AdaQuad\", iteration, bounds, len(boxes), elapsed, {\"refined_cells\": None}))\n", + " if iteration == max_refinement_steps: break\n", + " t0 = time.perf_counter(); indicators, split_dims = zip(*(interval_indicator_split(box, quantity) for box in boxes)); marked = set(_dorfler_marking(list(indicators), THETA))\n", + " boxes = [child for idx, box in enumerate(boxes) for child in (_split_box(box, split_dims[idx]) if idx in marked else (box,))]\n", + " rows[-1][\"refined_cells\"] = len(marked); elapsed += time.perf_counter() - t0\n", + " return rows\n", + "\n", + "def pz_integrand_kind(quantity):\n", + " return {\"L2\": \"l2\", \"W12\": \"w12\", \"W22\": \"w22\"}[quantity]\n", + "\n", + "def pz_aggregate(cells):\n", + " total = Interval.point(0.0)\n", + " for cached in cells:\n", + " total = _interval_add(total, cached.contribution)\n", + " return _sqrt_interval_nonnegative(total)\n", + "\n", + "def run_pz_trace(quantity, max_refinement_steps):\n", + " cells = [_evaluate_squared_contribution_cache(model, domain, integrand_kind=pz_integrand_kind(quantity))]\n", + " rows, elapsed = [], 0.0\n", + " for iteration in range(max_refinement_steps + 1):\n", + " t0 = time.perf_counter(); bounds = pz_aggregate(cells); elapsed += time.perf_counter() - t0\n", + " rows.append(make_row(quantity, \"PZ two-jet\", iteration, bounds, len(cells), elapsed, {\"refined_cells\": None}))\n", + " if iteration == max_refinement_steps: break\n", + " t0 = time.perf_counter(); indicators = [_interval_width(c.contribution) for c in cells]; marked = set(_pz_dorfler_marking(indicators, THETA)); new_cells = []\n", + " for idx, cached in enumerate(cells):\n", + " if idx not in marked:\n", + " new_cells.append(cached); continue\n", + " for child_box in _split_box(cached.box, cached.split_dim):\n", + " new_cells.append(_evaluate_squared_contribution_cache(model, child_box, integrand_kind=pz_integrand_kind(quantity)))\n", + " rows[-1][\"refined_cells\"] = len(marked); cells = new_cells; elapsed += time.perf_counter() - t0\n", + " return rows\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 6. Benchmark tables: width, runtime, cells, and refinement steps\n", + "\n", + "The following cell performs one cached adaptive run per `(method, quantity)` pair up to `MAX_REFINEMENT_STEPS`. Each row is the partial certified result after that many refinement steps from the same run.\n", + "\n", + "`runtime_s` below is the separately accumulated adaptive runtime on the same model and domain; the per-cell construction/integrand/integration phases are reported above.\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "all_rows = []\n", + "for quantity in [\"L2\", \"W12\", \"W22\"]:\n", + " all_rows.extend(run_interval_trace(quantity, MAX_REFINEMENT_STEPS))\n", + " all_rows.extend(run_pz_trace(quantity, MAX_REFINEMENT_STEPS))\n", + "results = pd.DataFrame(all_rows)\n", + "results[\"refinement_step\"] = results.pop(\"iteration\")\n", + "results[[\"quantity\", \"method\", \"refinement_step\", \"lower\", \"upper\", \"width\", \"runtime_s\", \"cells\", \"refined_cells\"]]\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "final_summary = results[results[\"refinement_step\"] == MAX_REFINEMENT_STEPS].copy()\n", + "final_summary[[\"quantity\", \"method\", \"lower\", \"upper\", \"width\", \"runtime_s\", \"cells\"]].sort_values([\"quantity\", \"method\"])\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 7. Diagnostic plots: convergence of enclosure width versus refinement step\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "fig, axes = plt.subplots(1, 3, figsize=(14, 4), sharey=False)\n", + "for ax, quantity in zip(axes, [\"L2\", \"W12\", \"W22\"]):\n", + " subset = results[results[\"quantity\"] == quantity]\n", + " for method, group in subset.groupby(\"method\"):\n", + " ax.plot(group[\"refinement_step\"], group[\"width\"], marker=\"o\", label=method)\n", + " ax.set_title(quantity); ax.set_xlabel(\"refinement step\"); ax.set_ylabel(\"certified interval width\"); ax.set_yscale(\"log\"); ax.grid(True, which=\"both\", alpha=0.3); ax.legend()\n", + "fig.suptitle(\"Enclosure-width convergence under adaptive refinement\")\n", + "plt.tight_layout()\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 8. Optional deterministic autograd sanity check\n", + "\n", + "This Monte Carlo/autograd estimate is not a certificate; it only provides a deterministic smoke check that sampled norm estimates are consistent with the certified intervals.\n" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def autograd_pointwise_quantities(samples):\n", + " x = samples.clone().detach().requires_grad_(True)\n", + " y = model(x)[:, 0]\n", + " grad = torch.autograd.grad(y.sum(), x, create_graph=True)[0]\n", + " hess_sq = torch.zeros_like(y)\n", + " for i in range(x.shape[1]):\n", + " for j in range(x.shape[1]):\n", + " hij = torch.autograd.grad(grad[:, i].sum(), x, retain_graph=True)[0][:, j]\n", + " hess_sq = hess_sq + hij.square()\n", + " grad_sq = grad.square().sum(dim=1)\n", + " return {\"L2\": y.square().detach().numpy(), \"W12\": (y.square() + grad_sq).detach().numpy(), \"W22\": (y.square() + grad_sq + hess_sq).detach().numpy()}\n", + "\n", + "generator = torch.Generator().manual_seed(SEED)\n", + "lo = torch.tensor(domain.lower, dtype=torch.float64); hi = torch.tensor(domain.upper, dtype=torch.float64)\n", + "samples = lo + (hi - lo) * torch.rand((2048, len(lo)), generator=generator, dtype=torch.float64)\n", + "volume = float(torch.prod(hi - lo)); pointwise = autograd_pointwise_quantities(samples)\n", + "sanity_rows = []\n", + "for quantity, values in pointwise.items():\n", + " estimate = math.sqrt(max(0.0, volume * float(np.mean(values))))\n", + " for _, certified in final_summary[final_summary[\"quantity\"] == quantity].iterrows():\n", + " sanity_rows.append({\"quantity\": quantity, \"method\": certified[\"method\"], \"mc_autograd_estimate\": estimate, \"certified_lower\": certified[\"lower\"], \"certified_upper\": certified[\"upper\"], \"estimate_inside_certified_interval\": certified[\"lower\"] <= estimate <= certified[\"upper\"]})\n", + "pd.DataFrame(sanity_rows)\n" + ], + "outputs": [], + "execution_count": null + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/benchmark_outputs/hybrid_absolute_moment_norms.json b/notebooks/benchmark_outputs/hybrid_absolute_moment_norms.json new file mode 100644 index 0000000..a8839ed --- /dev/null +++ b/notebooks/benchmark_outputs/hybrid_absolute_moment_norms.json @@ -0,0 +1,121 @@ +[ + { + "checkpoint": "notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt", + "architecture": [ + 100, + 300, + 1 + ], + "physical_domain_volume": 1.267650600228237e-70, + "integration_semantics": "pointwise_residual_absolute_moments_parity", + "l2": { + "squared_lower": 0.0, + "squared_upper": 9.197848680735796e-70, + "squared_absolute_width": 9.197848680735796e-70, + "squared_relative_width": 1.0, + "lower": 0.0, + "upper": 3.0327955224076346e-35, + "absolute_width": 3.0327955224076346e-35, + "relative_width": 1.0, + "domain_volume_normalized_squared_lower": 0.0, + "domain_volume_normalized_squared_upper": 7.255823236371086, + "domain_volume_normalized_squared_width": 7.255823236371086, + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 2.6936635343656206, + "domain_volume_normalized_width": 2.6936635343656206, + "sampled_domain_volume_normalized": 0.37534850060946906 + }, + "w12": { + "squared_lower": 0.0, + "squared_upper": 2.514338731349524e-69, + "squared_absolute_width": 2.514338731349524e-69, + "squared_relative_width": 1.0, + "lower": 0.0, + "upper": 5.014318230177981e-35, + "absolute_width": 5.014318230177981e-35, + "relative_width": 1.0, + "domain_volume_normalized_squared_lower": 0.0, + "domain_volume_normalized_squared_upper": 19.834635276446242, + "domain_volume_normalized_squared_width": 19.834635276446242, + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 4.453609241552995, + "domain_volume_normalized_width": 4.453609241552995, + "sampled_domain_volume_normalized": 2.502463016256655 + }, + "timings_seconds_median": { + "onejet_construction": 0.4144969779999883, + "l2_integration": 0.0007419970002047194, + "w12_integration": 0.014756518000012875, + "construction_plus_l2_plus_w12": 0.43423196599997027 + }, + "sample_soundness_violations": { + "value": 0, + "jacobian_entries": 0 + }, + "quadratic_neurons_per_layer": [ + 4 + ] + }, + { + "checkpoint": "notebooks/checkpoints/pinn_100d_poisson.pt", + "architecture": [ + 100, + 50, + 50, + 50, + 1 + ], + "physical_domain_volume": 1.267650600228237e-70, + "integration_semantics": "pointwise_residual_absolute_moments_parity", + "l2": { + "squared_lower": 0.0, + "squared_upper": 6.94805877613216e-70, + "squared_absolute_width": 6.94805877613216e-70, + "squared_relative_width": 1.0, + "lower": 0.0, + "upper": 2.6359170654882446e-35, + "absolute_width": 2.6359170654882446e-35, + "relative_width": 1.0, + "domain_volume_normalized_squared_lower": 0.0, + "domain_volume_normalized_squared_upper": 5.481051935668378, + "domain_volume_normalized_squared_width": 5.481051935668378, + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 2.341164653685934, + "domain_volume_normalized_width": 2.341164653685934, + "sampled_domain_volume_normalized": 0.3751917686933238 + }, + "w12": { + "squared_lower": 0.0, + "squared_upper": 1.0919842252396966e-68, + "squared_absolute_width": 1.0919842252396966e-68, + "squared_relative_width": 1.0, + "lower": 0.0, + "upper": 1.0449804903631916e-34, + "absolute_width": 1.0449804903631916e-34, + "relative_width": 1.0, + "domain_volume_normalized_squared_lower": 0.0, + "domain_volume_normalized_squared_upper": 86.14236644096472, + "domain_volume_normalized_squared_width": 86.14236644096472, + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 9.281291205482388, + "domain_volume_normalized_width": 9.281291205482388, + "sampled_domain_volume_normalized": 2.5046520721968255 + }, + "timings_seconds_median": { + "onejet_construction": 0.017707030000110535, + "l2_integration": 0.00021356500019464875, + "w12_integration": 0.00019914399990739184, + "construction_plus_l2_plus_w12": 0.018125288000192086 + }, + "sample_soundness_violations": { + "value": 0, + "jacobian_entries": 0 + }, + "quadratic_neurons_per_layer": [ + 1, + 6, + 3 + ], + "deep_w12_integration_note": "absolute moments and parity are applied after the exact factored Jacobian is collapsed to its affine core plus certified pointwise remainder" + } +] diff --git a/notebooks/benchmark_outputs/hybrid_graph_hilbert_norms.json b/notebooks/benchmark_outputs/hybrid_graph_hilbert_norms.json new file mode 100644 index 0000000..c5cd674 --- /dev/null +++ b/notebooks/benchmark_outputs/hybrid_graph_hilbert_norms.json @@ -0,0 +1,484 @@ +[ + { + "checkpoint": "notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt", + "architecture": [ + 100, + 300, + 1 + ], + "physical_domain_volume": 1.267650600228237e-70, + "method": "dependency_preserving_graph_hilbert", + "settings": { + "polynomial_degree": 5, + "residual_subdivisions": 2048, + "derivative_certificate_subdivisions": 64, + "validation_samples": 16384 + }, + "l2": { + "domain_volume_normalized_lower": 0.31760521141821385, + "domain_volume_normalized_upper": 0.4290352754529465, + "domain_volume_normalized_width": 0.11143006403473266, + "relative_norm_width": 0.2597223827739859, + "sampled_domain_volume_normalized": 0.37534850060946906, + "upper_over_sampled": 1.1430318084561522, + "lower_over_sampled": 0.846160863577462, + "raw_squared_lower": 1.2787180813802372e-71, + "raw_squared_upper": 2.333380528363442e-71 + }, + "w12": { + "domain_volume_normalized_lower": 1.3915660133024224, + "domain_volume_normalized_upper": 3.106807462589975, + "domain_volume_normalized_width": 1.7152414492875527, + "relative_norm_width": 0.5520913252402355, + "sampled_domain_volume_normalized": 2.502463016256655, + "upper_over_sampled": 1.2414998513094262, + "lower_over_sampled": 0.5560785531144498, + "raw_squared_lower": 2.454749571898078e-70, + "raw_squared_upper": 1.2235683814120042e-69 + }, + "sample_containment_diagnostic": { + "l2": true, + "w12": true + }, + "previous_absolute_moment_parity": { + "l2": { + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 2.6936635343656206, + "domain_volume_normalized_width": 2.6936635343656206, + "relative_norm_width": 1.0, + "sampled_domain_volume_normalized": 0.37534850060946906, + "upper_over_sampled": 7.176433447827303, + "lower_over_sampled": 0.0, + "raw_squared_lower": -8.985951001340374e-70, + "raw_squared_upper": 9.197848680735797e-70 + }, + "w12": { + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 4.453609241552995, + "domain_volume_normalized_width": 4.453609241552995, + "relative_norm_width": 1.0, + "sampled_domain_volume_normalized": 2.502463016256655, + "upper_over_sampled": 1.779690334131287, + "lower_over_sampled": 0.0, + "raw_squared_lower": -1.9926785949348066e-69, + "raw_squared_upper": 2.514338731349524e-69 + } + }, + "improvement": { + "l2_relative_width_reduction": 0.7402776172260142, + "w12_relative_width_reduction": 0.4479086747597645, + "l2_upper_reduction_fraction": 0.8407242515706448, + "w12_upper_reduction_fraction": 0.30240681342159764 + }, + "decomposition": { + "value_nominal_norm": 0.3733202434355802, + "value_l2_remainder": 0.05571503201736627, + "l2_reverse_triangle_lower": 0.3176052114182139, + "l2_reverse_triangle_upper": 0.42903527545294645, + "w12_dual_lower": 1.3915660133024224, + "gradient_nominal_norm": 2.1622130895896827, + "gradient_l2_remainder": 0.9109006406352889, + "w12_graph_nominal_norm": 2.194204513930222, + "w12_graph_remainder": 0.912602948659753, + "w12_graph_reverse_lower": 1.2816015652704689, + "w12_graph_reverse_upper": 3.106807462589975, + "combined_l2_lower": 0.3176052114182139, + "combined_l2_upper": 0.42903527545294645, + "combined_w12_lower": 1.3915660133024224, + "combined_w12_upper": 3.106807462589975 + }, + "value_layers": [ + { + "layer": 0, + "width": 300, + "preactivation_remainder": 0.0, + "polynomial_projection_remainder": 0.0048918645574333895, + "uniform_approximation_remainder": 0.03517631168523184, + "total_output_remainder": 0.04002667041177232, + "maximum_preactivation_width": 3.342522062749524 + } + ], + "gradient": { + "factor_remainders": [ + 0.9927554786931965 + ], + "product_projection_remainders": [ + 0.0003320101773470841 + ], + "moment_states": 151500 + }, + "w12_witness": { + "lower_bound": 1.3915660133024224, + "nominal_pairing": 3.238851072260739, + "remainder_penalty": 1.2345105784920407, + "witness_norm": 1.4403488405210894, + "transformed_witness_norm": 22.15758537314034, + "coefficients": [ + 0.3449291137147311, + -0.019123769903162127, + -0.01913668631467423, + -0.01907017378841328, + -0.01896004363433614, + -0.01910764646839087, + -0.01911383522903251, + -0.019175411328200442, + -0.0190852512890527, + -0.019128781429098327, + -0.019097827171032544, + -0.019210993813426957, + -0.019109487510147995, + -0.0192410989460646, + -0.01907367040365303, + -0.01909811037799312, + -0.019049058899225424, + -0.01905438225063697, + -0.019113484350665122, + -0.019147238541604494, + -0.019114006845813592, + -0.01914080537744906, + -0.019094714410989345, + -0.01910217448115413, + -0.019102478215971993, + -0.019158387593083692, + -0.0191261469647415, + -0.01908800009476204, + -0.019033867114092804, + -0.019155210913896544, + -0.019038027870063482, + -0.01910059429732797, + -0.01920741618325662, + -0.019098391440346658, + -0.01911961834367098, + -0.0191298984033321, + -0.01905154117119432, + -0.019073379180634824, + -0.0191516152514439, + -0.01896574738156887, + -0.019084674667890763, + -0.019054248214286564, + -0.019073254043786875, + -0.019284045729131026, + -0.01908262513437553, + -0.019164104251444217, + -0.019145924687626045, + -0.018965357503969594, + -0.019025927238303965, + -0.019186110346796608, + -0.0191141469641849, + -0.019150518829590627, + -0.019106815579223065, + -0.019098569658500188, + -0.01906908937685151, + -0.019183364232020432, + -0.01909609370013097, + -0.01908690020599423, + -0.019123215862542193, + -0.019144419891230623, + -0.019121107836758224, + -0.01907418150955922, + -0.019117767723877556, + -0.019152151214964074, + -0.01911309082168923, + -0.019122301001754657, + -0.019102026413075686, + -0.01916568108798173, + -0.01917522099683657, + -0.01906763288387455, + -0.019086923532747987, + -0.019021324966120225, + -0.019185863161153782, + -0.01912706624302287, + -0.01905297694179781, + -0.019149702471861593, + -0.019138878040622698, + -0.019127536749862565, + -0.019089073238310793, + -0.019106071973420798, + -0.01907221782872638, + -0.019067154620903227, + -0.019091056556885484, + -0.01907356703232712, + -0.019216161996275155, + -0.019190779945883463, + -0.019142459643347653, + -0.019014146790192633, + -0.019099190677966137, + -0.01910053379719868, + -0.019060775126817676, + -0.019103048364822014, + -0.019076577493277947, + -0.019222664763663327, + -0.019120443719876517, + -0.01914980834879647, + -0.019121828414643532, + -0.01915304395084987, + -0.019065316669688196, + -0.01907590573802713, + -0.01911695506320105 + ], + "basis": "constant_plus_coordinatewise_cubic_neumann" + }, + "complexity": { + "value_moment_states": 333300, + "gradient_moment_states": 151500 + }, + "timings_seconds": { + "hybrid_onejet_construction": 0.47662268200019753, + "graph_hilbert_certification": 43.14939059400058, + "total_certification": 43.626013276000776 + } + }, + { + "checkpoint": "notebooks/checkpoints/pinn_100d_poisson.pt", + "architecture": [ + 100, + 50, + 50, + 50, + 1 + ], + "physical_domain_volume": 1.267650600228237e-70, + "method": "dependency_preserving_graph_hilbert", + "settings": { + "polynomial_degree": 5, + "residual_subdivisions": 2048, + "derivative_certificate_subdivisions": 64, + "validation_samples": 16384 + }, + "l2": { + "domain_volume_normalized_lower": 0.3008286687076149, + "domain_volume_normalized_upper": 0.4497064043657324, + "domain_volume_normalized_width": 0.14887773565811752, + "relative_norm_width": 0.3310554046213668, + "sampled_domain_volume_normalized": 0.37519176869332377, + "upper_over_sampled": 1.1986041323132433, + "lower_over_sampled": 0.8017997563094402, + "raw_squared_lower": 1.1471970193660698e-71, + "raw_squared_upper": 2.5636439680186364e-71 + }, + "w12": { + "domain_volume_normalized_lower": 1.2109199455968565, + "domain_volume_normalized_upper": 3.397129595777929, + "domain_volume_normalized_width": 2.1862096501810724, + "relative_norm_width": 0.6435461434553954, + "sampled_domain_volume_normalized": 2.5046520721968255, + "upper_over_sampled": 1.3563279441037546, + "lower_over_sampled": 0.4834683264149982, + "raw_squared_lower": 1.8587904470097776e-70, + "raw_squared_upper": 1.4629308429573062e-69 + }, + "sample_containment_diagnostic": { + "l2": true, + "w12": true + }, + "previous_absolute_moment_parity": { + "l2": { + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 2.3411646536859334, + "domain_volume_normalized_width": 2.3411646536859334, + "relative_norm_width": 1.0, + "sampled_domain_volume_normalized": 0.37519176869332377, + "upper_over_sampled": 6.239914755698084, + "lower_over_sampled": 0.0, + "raw_squared_lower": -5e-324, + "raw_squared_upper": 6.948058776132158e-70 + }, + "w12": { + "domain_volume_normalized_lower": 0.0, + "domain_volume_normalized_upper": 9.281291205482317, + "domain_volume_normalized_width": 9.281291205482317, + "relative_norm_width": 1.0, + "sampled_domain_volume_normalized": 2.5046520721968255, + "upper_over_sampled": 3.7056209557049233, + "lower_over_sampled": 0.0, + "raw_squared_lower": -5e-324, + "raw_squared_upper": 1.09198422523968e-68 + } + }, + "improvement": { + "l2_relative_width_reduction": 0.6689445953786333, + "w12_relative_width_reduction": 0.3564538565446046, + "l2_upper_reduction_fraction": 0.8079133803520765, + "w12_upper_reduction_fraction": 0.6339809278076205 + }, + "decomposition": { + "value_nominal_norm": 0.3752675365366736, + "value_l2_remainder": 0.07443886782905872, + "l2_reverse_triangle_lower": 0.3008286687076149, + "l2_reverse_triangle_upper": 0.44970640436573234, + "w12_dual_lower": 1.1320065261732535, + "gradient_nominal_norm": 2.2732585466600117, + "gradient_l2_remainder": 1.0905672898049619, + "w12_graph_nominal_norm": 2.3040247706873926, + "w12_graph_remainder": 1.0931048250905362, + "w12_graph_reverse_lower": 1.2109199455968565, + "w12_graph_reverse_upper": 3.397129595777929, + "combined_l2_lower": 0.3008286687076149, + "combined_l2_upper": 0.44970640436573234, + "combined_w12_lower": 1.2109199455968565, + "combined_w12_upper": 3.397129595777929 + }, + "value_layers": [ + { + "layer": 0, + "width": 50, + "preactivation_remainder": 0.0, + "polynomial_projection_remainder": 0.0015007739701704559, + "uniform_approximation_remainder": 0.008865857777181827, + "total_output_remainder": 0.010364091208998569, + "maximum_preactivation_width": 2.204562489403279 + }, + { + "layer": 1, + "width": 50, + "preactivation_remainder": 0.01848214847279887, + "polynomial_projection_remainder": 0.0013759880041011865, + "uniform_approximation_remainder": 0.00647346269691397, + "total_output_remainder": 0.02631233226105266, + "maximum_preactivation_width": 1.9090414945087009 + }, + { + "layer": 2, + "width": 50, + "preactivation_remainder": 0.04913991218829516, + "polynomial_projection_remainder": 0.0014051299718002828, + "uniform_approximation_remainder": 0.0057572475170688175, + "total_output_remainder": 0.05626361657491562, + "maximum_preactivation_width": 1.9684250726073256 + } + ], + "gradient": { + "factor_remainders": [ + 0.30042105144109754, + 0.2186859485394366, + 0.2160471242588162 + ], + "product_projection_remainders": [ + 0.000135596024964631, + 0.00013571989610426866, + 0.00013554258740462372 + ], + "moment_states": 75750 + }, + "w12_witness": { + "lower_bound": 1.1320065261732535, + "nominal_pairing": 2.594813454282593, + "remainder_penalty": 1.2933491670320258, + "witness_norm": 1.1496968057685726, + "transformed_witness_norm": 17.3746485505673, + "coefficients": [ + 0.34571561485922514, + -0.014975714498835595, + -0.014966610971525891, + -0.014963399322884359, + -0.015008431359193896, + -0.01504791055334925, + -0.015012125398784562, + -0.014999000097018081, + -0.015005664579774525, + -0.014912162634576648, + -0.015023571358447656, + -0.01500960706010439, + -0.01496051939825156, + -0.01495644597040786, + -0.014998644000517935, + -0.015077145119081743, + -0.015041314585228626, + -0.01506751623777035, + -0.014930766747365572, + -0.014938751849202105, + -0.014949582096102513, + -0.014938556593059918, + -0.014923050295100106, + -0.015025678646507412, + -0.0149505442911423, + -0.01504987693302943, + -0.014850178658597977, + -0.015013124978494819, + -0.014971718586213998, + -0.01495964326220702, + -0.014943984823308974, + -0.01494005884003567, + -0.014954635760639524, + -0.015053924252599084, + -0.014923912825799406, + -0.014980419808941561, + -0.015006084090896306, + -0.014987150781620481, + -0.014965165241119877, + -0.01499274143623347, + -0.015040287516621946, + -0.014968684772761835, + -0.014937042657219044, + -0.014952771960155896, + -0.015016848094753404, + -0.014899152022390793, + -0.01503041211059502, + -0.014973765237151864, + -0.014943208688765617, + -0.01499602848970524, + -0.014953420404218003, + -0.015018935307255163, + -0.014993657400509298, + -0.014995935127294087, + -0.014950255686367996, + -0.015015607590803844, + -0.014950979621758982, + -0.014940570839230114, + -0.015065878133938618, + -0.015008481592296143, + -0.015077504660935048, + -0.014962709903143537, + -0.015005250316292275, + -0.014920242884716254, + -0.01490231588937013, + -0.014969332969333718, + -0.014987254118170779, + -0.01493790107688175, + -0.014985829773322221, + -0.014998886645169605, + -0.014991234359134367, + -0.015053140286259864, + -0.014955279068483323, + -0.014905142993793828, + -0.014945847252473625, + -0.014994152350811822, + -0.014943845656807212, + -0.014975347475144106, + -0.01497051961777763, + -0.014984740802838742, + -0.015056094256631607, + -0.014996520231778263, + -0.014963958028641478, + -0.01492130329164413, + -0.015068985688662518, + -0.014938569491200987, + -0.015023668250129787, + -0.014923227013145522, + -0.014995785642561803, + -0.015007685726375777, + -0.015057640702495614, + -0.014965829782483793, + -0.0148888490219843, + -0.01501738083447901, + -0.015021900811003033, + -0.015176856248313192, + -0.01501638447549485, + -0.014955592366612331, + -0.014966683476156482, + -0.015044234336369066, + -0.014988032661049144 + ], + "basis": "constant_plus_coordinatewise_cubic_neumann" + }, + "complexity": { + "value_moment_states": 166650, + "gradient_moment_states": 75750 + }, + "timings_seconds": { + "hybrid_onejet_construction": 0.02323558900025091, + "graph_hilbert_certification": 25.888597799999843, + "total_certification": 25.911833389000094 + } + } +] diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/activation_approximation.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/activation_approximation.csv new file mode 100644 index 0000000..47bf228 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/activation_approximation.csv @@ -0,0 +1,601 @@ +run_id,split_id,cell_id,cell_weight,layer,neuron,derivative_order,preactivation_lower,preactivation_upper,preactivation_midpoint,preactivation_radius,preactivation_width,approximation_kind,approximation_error_radius,approximation_error_diameter,activation_scale,normalized_approximation_radius,noise_symbol_id,shared_noise_group,status +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,0,0,-0.8000887741224202,0.7908767282671686,-0.004606022927625797,0.7954827511947944,1.5909655023895888,affine,0.04776087567042881,0.09552175134085762,0.9319962481713715,0.051245781047014195,eta_l0_n0_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,0,1,-0.8000887741224202,0.7908767282671686,-0.004606022927625797,0.7954827511947944,1.5909655023895888,affine,0.21878416980543125,0.4375683396108625,1.0,0.21878416980543125,eta_l0_n0_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,1,0,-0.6668101129690581,0.6933849725568156,0.013287429793878758,0.6800975427629369,1.3601950855258738,affine,0.032203781618758816,0.06440756323751763,0.9319962481713715,0.034553552851682,eta_l0_n1_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,1,1,-0.6668101129690581,0.6933849725568156,0.013287429793878758,0.6800975427629369,1.3601950855258738,affine,0.17491064771605164,0.3498212954321033,1.0,0.17491064771605164,eta_l0_n1_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,2,0,-0.8409685245821178,0.8705015267430377,0.01476650108045996,0.8557350256625778,1.7114700513251555,affine,0.05704981128360182,0.11409962256720364,0.9319962481713715,0.06121249028151854,eta_l0_n2_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,2,1,-0.8409685245821178,0.8705015267430377,0.01476650108045996,0.8557350256625778,1.7114700513251555,affine,0.2407577465691454,0.4815154931382908,1.0,0.2407577465691454,eta_l0_n2_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,3,0,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.05297381165009602,0.10594762330019204,0.9319962481713715,0.056839082511365886,eta_l0_n3_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,3,1,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.23128307788642288,0.46256615577284577,1.0,0.23128307788642288,eta_l0_n3_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,4,0,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.08534503477820339,0.17069006955640678,0.9319962481713715,0.09157229435811046,eta_l0_n4_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,4,1,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.2959936114346802,0.5919872228693605,1.0,0.2959936114346802,eta_l0_n4_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,5,0,-0.8652278170806447,0.8877201689912209,0.011246175955288096,0.8764739930359328,1.7529479860718655,affine,0.06037470341247909,0.12074940682495817,0.9319962481713715,0.06477998546768575,eta_l0_n5_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,5,1,-0.8652278170806447,0.8877201689912209,0.011246175955288096,0.8764739930359328,1.7529479860718655,affine,0.24820832817273963,0.49641665634547927,1.0,0.24820832817273963,eta_l0_n5_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,6,0,-0.8002253266309622,0.829648498761181,0.014711586065109361,0.8149369126960716,1.6298738253921432,affine,0.05070909200326985,0.1014181840065397,0.9319962481713715,0.05440911602676932,eta_l0_n6_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,6,1,-0.8002253266309622,0.829648498761181,0.014711586065109361,0.8149369126960716,1.6298738253921432,affine,0.22589998281650625,0.4517999656330125,1.0,0.22589998281650625,eta_l0_n6_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,7,0,-0.6724065457781009,0.693292462113097,0.010442958167498073,0.682849503945599,1.365699007891198,affine,0.03252624948541378,0.06505249897082756,0.9319962481713715,0.034899549809596433,eta_l0_n7_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,7,1,-0.6724065457781009,0.693292462113097,0.010442958167498073,0.682849503945599,1.365699007891198,affine,0.17599713778830367,0.35199427557660734,1.0,0.17599713778830367,eta_l0_n7_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,8,0,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.13821186086084938,0.27642372172169877,0.9319962481713715,0.1482965850259899,eta_l0_n8_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,8,1,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.3682178238498487,0.7364356476996974,1.0,0.3682178238498487,eta_l0_n8_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,9,0,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,affine,0.056781076836971985,0.11356215367394397,0.9319962481713715,0.06092414743984176,eta_l0_n9_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,9,1,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,affine,0.24015713034047748,0.48031426068095495,1.0,0.24015713034047748,eta_l0_n9_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,10,0,-0.8080868528523598,0.7839032216460967,-0.01209181560313155,0.7959950372492283,1.5919900744984565,affine,0.04786198015530158,0.09572396031060317,0.9319962481713715,0.05135426269065938,eta_l0_n10_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,10,1,-0.8080868528523598,0.7839032216460967,-0.01209181560313155,0.7959950372492283,1.5919900744984565,affine,0.21891895411782564,0.4378379082356513,1.0,0.21891895411782564,eta_l0_n10_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,11,0,-0.7982094929703909,0.8217535642036393,0.01177203561662421,0.8099815285870151,1.6199630571740302,affine,0.049945300875641425,0.09989060175128285,0.9319962481713715,0.05358959435044603,eta_l0_n11_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,11,1,-0.7982094929703909,0.8217535642036393,0.01177203561662421,0.8099815285870151,1.6199630571740302,affine,0.22410676447470157,0.44821352894940314,1.0,0.22410676447470157,eta_l0_n11_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,12,0,-0.906774438074293,0.9399477281012678,0.01658664501348739,0.9233610830877804,1.8467221661755608,affine,0.06823703003790989,0.13647406007581978,0.9319962481713715,0.07321599220146512,eta_l0_n12_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,12,1,-0.906774438074293,0.9399477281012678,0.01658664501348739,0.9233610830877804,1.8467221661755608,affine,0.2644950169656779,0.5289900339313558,1.0,0.2644950169656779,eta_l0_n12_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,13,0,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.06636055767389427,0.13272111534778855,0.9319962481713715,0.07120260173160287,eta_l0_n13_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,13,1,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.2606246761987212,0.5212493523974424,1.0,0.2606246761987212,eta_l0_n13_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,14,0,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.1281480288819092,0.2562960577638184,0.9319962481713715,0.1374984385756303,eta_l0_n14_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,14,1,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.35680199089624876,0.7136039817924975,1.0,0.35680199089624876,eta_l0_n14_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,15,0,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.055167049976694256,0.11033409995338851,0.9319962481713715,0.059192351991689963,eta_l0_n15_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,15,1,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.2364612101423731,0.4729224202847462,1.0,0.2364612101423731,eta_l0_n15_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,16,0,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.18859813104338555,0.3771962620867711,0.9319962481713715,0.20235932431426154,eta_l0_n16_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,16,1,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.41417314403421934,0.8283462880684387,1.0,0.41417314403421934,eta_l0_n16_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,17,0,-0.9249747098227338,0.8972162631509849,-0.013879223335874435,0.9110954864868593,1.8221909729737187,affine,0.06613657047259132,0.13227314094518264,0.9319962481713715,0.07096227114900404,eta_l0_n17_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,17,1,-0.9249747098227338,0.8972162631509849,-0.013879223335874435,0.9110954864868593,1.8221909729737187,affine,0.2603096831220119,0.5206193662440238,1.0,0.2603096831220119,eta_l0_n17_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,18,0,-1.6733730976128967,1.6691489651366265,-0.0021120662381350908,1.6712610313747616,3.342522062749523,affine,0.2180433724141755,0.436086744828351,0.9319962481713715,0.23395305811797928,eta_l0_n18_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,18,1,-1.6733730976128967,1.6691489651366265,-0.0021120662381350908,1.6712610313747616,3.342522062749523,affine,0.4340486250718861,0.8680972501437721,1.0,0.4340486250718861,eta_l0_n18_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,19,0,-0.9806865471881266,0.9462170674146505,-0.01723473988673807,0.9634518073013886,1.9269036146027771,affine,0.07522189449895324,0.15044378899790647,0.9319962481713715,0.08071051213622671,eta_l0_n19_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,19,1,-0.9806865471881266,0.9462170674146505,-0.01723473988673807,0.9634518073013886,1.9269036146027771,affine,0.277989649600464,0.555979299200928,1.0,0.277989649600464,eta_l0_n19_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,20,0,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.05759508702140571,0.11519017404281141,0.9319962481713715,0.061797552441236185,eta_l0_n20_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,20,1,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.2418920890071064,0.4837841780142128,1.0,0.2418920890071064,eta_l0_n20_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,21,0,-0.9711358295108821,0.9996589259075808,0.01426154819834935,0.9853973777092314,1.9707947554184628,affine,0.07912843252686426,0.15825686505372852,0.9319962481713715,0.08490209341734867,eta_l0_n21_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,21,1,-0.9711358295108821,0.9996589259075808,0.01426154819834935,0.9853973777092314,1.9707947554184628,affine,0.2852222128166809,0.5704444256333618,1.0,0.2852222128166809,eta_l0_n21_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,22,0,-1.0915414624371893,1.1306988296348472,0.019578683598828972,1.1111201460360183,2.2222402920720365,affine,0.10280415066198174,0.2056083013239635,0.9319962481713715,0.1103053267260348,eta_l0_n22_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,22,1,-1.0915414624371893,1.1306988296348472,0.019578683598828972,1.1111201460360183,2.2222402920720365,affine,0.32342725517235416,0.6468545103447083,1.0,0.32342725517235416,eta_l0_n22_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,23,0,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.09752239534692254,0.1950447906938451,0.9319962481713715,0.10463818447581405,eta_l0_n23_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,23,1,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.3156441037740703,0.6312882075481406,1.0,0.3156441037740703,eta_l0_n23_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,24,0,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.13521015581349047,0.27042031162698094,0.9319962481713715,0.14507585849061122,eta_l0_n24_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,24,1,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.3648891411432009,0.7297782822864018,1.0,0.3648891411432009,eta_l0_n24_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,25,0,-1.2223185723825791,1.1844726444174873,-0.018922963982545893,1.2033956084000332,2.4067912168000665,affine,0.12114127036711338,0.24228254073422675,0.9319962481713715,0.12998042707231847,eta_l0_n25_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,25,1,-1.2223185723825791,1.1844726444174873,-0.018922963982545893,1.2033956084000332,2.4067912168000665,affine,0.3482249401892906,0.6964498803785812,1.0,0.3482249401892906,eta_l0_n25_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,26,0,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.09212749605465377,0.18425499210930754,0.9319962481713715,0.09884964261971338,eta_l0_n26_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,26,1,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.30710974875608243,0.6142194975121649,1.0,0.30710974875608243,eta_l0_n26_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,27,0,-1.0765750272011247,1.0932061682436134,0.00831557052124432,1.084890597722369,2.169781195444738,affine,0.09767678010090204,0.19535356020180408,0.9319962481713715,0.10480383402030782,eta_l0_n27_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,27,1,-1.0765750272011247,1.0932061682436134,0.00831557052124432,1.084890597722369,2.169781195444738,affine,0.315989484240928,0.631978968481856,1.0,0.315989484240928,eta_l0_n27_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,28,0,-0.8349571475513465,0.8610004970616331,0.013021674755143264,0.8479788223064898,1.6959576446129796,affine,0.055810473951034785,0.11162094790206957,0.9319962481713715,0.059882723842009065,eta_l0_n28_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,28,1,-0.8349571475513465,0.8610004970616331,0.013021674755143264,0.8479788223064898,1.6959576446129796,affine,0.23798244361387216,0.4759648872277443,1.0,0.23798244361387216,eta_l0_n28_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,29,0,-1.191499282547206,1.2302241779957939,0.019362447724293963,1.2108617302715,2.421723460543,affine,0.1226544191509072,0.2453088383018144,0.9319962481713715,0.13160398380525887,eta_l0_n29_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,29,1,-1.191499282547206,1.2302241779957939,0.019362447724293963,1.2108617302715,2.421723460543,affine,0.3501010164013204,0.7002020328026408,1.0,0.3501010164013204,eta_l0_n29_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,30,0,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.03916738782673008,0.07833477565346016,0.9319962481713715,0.042025263410210796,eta_l0_n30_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,30,1,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.19569049953193574,0.39138099906387147,1.0,0.19569049953193574,eta_l0_n30_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,31,0,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,affine,0.07920689094127416,0.15841378188254832,0.9319962481713715,0.08498627660431304,eta_l0_n31_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,31,1,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,affine,0.2852835771335912,0.5705671542671824,1.0,0.2852835771335912,eta_l0_n31_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,32,0,-1.2946227610182182,1.2514994629324607,-0.021561649042878717,1.2730611119753394,2.546122223950679,affine,0.1353723538326648,0.2707447076653296,0.9319962481713715,0.1452498914006069,eta_l0_n32_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,32,1,-1.2946227610182182,1.2514994629324607,-0.021561649042878717,1.2730611119753394,2.546122223950679,affine,0.3650408857692888,0.7300817715385776,1.0,0.3650408857692888,eta_l0_n32_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,33,0,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,affine,0.05575080421097712,0.11150160842195424,0.9319962481713715,0.05981870025803569,eta_l0_n33_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,33,1,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,affine,0.23778197001319706,0.4755639400263941,1.0,0.23778197001319706,eta_l0_n33_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,34,0,-0.9062201538536022,0.9391835025457022,0.016481674346049968,0.9227018281996522,1.8454036563993044,affine,0.06812369722603763,0.13624739445207526,0.9319962481713715,0.07309438998247055,eta_l0_n34_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,34,1,-0.9062201538536022,0.9391835025457022,0.016481674346049968,0.9227018281996522,1.8454036563993044,affine,0.26427067826313716,0.5285413565262743,1.0,0.26427067826313716,eta_l0_n34_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,35,0,-0.9404150670737785,0.9548630677126784,0.007224000319449919,0.9476390673932285,1.895278134786457,affine,0.07239481230028884,0.14478962460057768,0.9319962481713715,0.07767714992665635,eta_l0_n35_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,35,1,-0.9404150670737785,0.9548630677126784,0.007224000319449919,0.9476390673932285,1.895278134786457,affine,0.27282519963288465,0.5456503992657693,1.0,0.27282519963288465,eta_l0_n35_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,36,0,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.1329675800582422,0.2659351601164844,0.9319962481713715,0.1426696516419803,eta_l0_n36_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,36,1,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.36239116506943936,0.7247823301388787,1.0,0.36239116506943936,eta_l0_n36_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,37,0,-1.1019913101592176,1.1337401489755885,0.01587441940818546,1.117865729567403,2.235731459134806,affine,0.10410160451057877,0.20820320902115755,0.9319962481713715,0.11169745019342289,eta_l0_n37_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,37,1,-1.1019913101592176,1.1337401489755885,0.01587441940818546,1.117865729567403,2.235731459134806,affine,0.3253861193108082,0.6507722386216164,1.0,0.3253861193108082,eta_l0_n37_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,38,0,-0.7748642347846233,0.7429797929158597,-0.0159422209343818,0.7589220138502415,1.517844027700483,affine,0.04255388603436006,0.08510777206872013,0.9319962481713715,0.04565885980534058,eta_l0_n38_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,38,1,-0.7748642347846233,0.7429797929158597,-0.0159422209343818,0.7589220138502415,1.517844027700483,affine,0.20497042177762226,0.4099408435552445,1.0,0.20497042177762226,eta_l0_n38_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,39,0,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,affine,0.08683626610744645,0.1736725322148929,0.9319962481713715,0.09317233441424688,eta_l0_n39_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,39,1,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,affine,0.29854643485100885,0.5970928697020177,1.0,0.29854643485100885,eta_l0_n39_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,40,0,-1.0016438696095071,0.9635624254813989,-0.019040722064054105,0.9826031475454531,1.9652062950909062,affine,0.07865281463649128,0.15730562927298256,0.9319962481713715,0.08439177173815074,eta_l0_n40_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,40,1,-1.0016438696095071,0.9635624254813989,-0.019040722064054105,0.9826031475454531,1.9652062950909062,affine,0.2842463860855686,0.5684927721711373,1.0,0.2842463860855686,eta_l0_n40_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,41,0,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,affine,0.052595132871832324,0.10519026574366465,0.9319962481713715,0.056432773173740666,eta_l0_n41_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,41,1,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,affine,0.2304090999181408,0.4608181998362816,1.0,0.2304090999181408,eta_l0_n41_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,42,0,-0.8258438072260423,0.7979791433617529,-0.013932331932144715,0.8119114752938976,1.6238229505877952,affine,0.05024729084278002,0.10049458168556004,0.9319962481713715,0.05391361922472113,eta_l0_n42_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,42,1,-0.8258438072260423,0.7979791433617529,-0.013932331932144715,0.8119114752938976,1.6238229505877952,affine,0.2247944505364547,0.4495889010729094,1.0,0.2247944505364547,eta_l0_n42_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,43,0,-1.506554210451975,1.539800993136927,0.016623391342476035,1.523177601794451,3.046355203588902,affine,0.18746340190277452,0.37492680380554905,0.9319962481713715,0.20114179887589478,eta_l0_n43_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,43,1,-1.506554210451975,1.539800993136927,0.016623391342476035,1.523177601794451,3.046355203588902,affine,0.41330507513892245,0.8266101502778449,1.0,0.41330507513892245,eta_l0_n43_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,44,0,-1.298895240010546,1.254490838972456,-0.02220220051904498,1.276693039491501,2.553386078983002,affine,0.13612331689707471,0.27224663379414943,0.9319962481713715,0.14605564900519313,eta_l0_n44_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,44,1,-1.298895240010546,1.254490838972456,-0.02220220051904498,1.276693039491501,2.553386078983002,affine,0.36586726001736714,0.7317345200347343,1.0,0.36586726001736714,eta_l0_n44_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,45,0,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,affine,0.031247814786823872,0.062495629573647744,0.9319962481713715,0.03352783323767003,eta_l0_n45_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,45,1,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,affine,0.17204478770874515,0.3440895754174903,1.0,0.17204478770874515,eta_l0_n45_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,46,0,-0.8663927459559603,0.9013360930495738,0.017471673546806787,0.8838644195027671,1.7677288390055341,affine,0.06161881063103398,0.12323762126206796,0.9319962481713715,0.06611486983121823,eta_l0_n46_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,46,1,-0.8663927459559603,0.9013360930495738,0.017471673546806787,0.8838644195027671,1.7677288390055341,affine,0.25074390701983446,0.5014878140396689,1.0,0.25074390701983446,eta_l0_n46_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,47,0,-0.8527698834317742,0.8310757190829284,-0.010847082174422873,0.8419228012573513,1.6838456025147026,affine,0.054847789658709296,0.10969557931741859,0.9319962481713715,0.05884979662345606,eta_l0_n47_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,47,1,-0.8527698834317742,0.8310757190829284,-0.010847082174422873,0.8419228012573513,1.6838456025147026,affine,0.23581240905165818,0.47162481810331636,1.0,0.23581240905165818,eta_l0_n47_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,48,0,-1.0115982611523093,1.0428605240202082,0.015631131433949452,1.0272293925862588,2.0544587851725176,affine,0.0867955966979921,0.1735911933959842,0.9319962481713715,0.09312869753316055,eta_l0_n48_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,48,1,-1.0115982611523093,1.0428605240202082,0.015631131433949452,1.0272293925862588,2.0544587851725176,affine,0.2985041035683372,0.5970082071366744,1.0,0.2985041035683372,eta_l0_n48_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,49,0,-0.8165600505003638,0.847871599185807,0.015655774342721585,0.8322158248430854,1.6644316496861709,affine,0.053361052968857854,0.10672210593771571,0.9319962481713715,0.057254579161187834,eta_l0_n49_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,49,1,-0.8165600505003638,0.847871599185807,0.015655774342721585,0.8322158248430854,1.6644316496861709,affine,0.23222316361302536,0.4644463272260507,1.0,0.23222316361302536,eta_l0_n49_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,50,0,-1.1685921066522982,1.2049616256427143,0.018184759495208036,1.1867768661475062,2.3735537322950124,affine,0.11778720959870133,0.23557441919740266,0.9319962481713715,0.1263816349366281,eta_l0_n50_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,50,1,-1.1685921066522982,1.2049616256427143,0.018184759495208036,1.1867768661475062,2.3735537322950124,affine,0.34397954463048375,0.6879590892609675,1.0,0.34397954463048375,eta_l0_n50_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,51,0,-0.8006207796149774,0.8344702881968007,0.01692475429091167,0.817545533905889,1.635091067811778,affine,0.05111893061710033,0.10223786123420066,0.9319962481713715,0.054848858798947436,eta_l0_n51_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,51,1,-0.8006207796149774,0.8344702881968007,0.01692475429091167,0.817545533905889,1.635091067811778,affine,0.2268290575123888,0.4536581150247776,1.0,0.2268290575123888,eta_l0_n51_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,52,0,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.04215920397663842,0.08431840795327684,0.9319962481713715,0.04523537949788653,eta_l0_n52_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,52,1,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.20395651747432286,0.4079130349486457,1.0,0.20395651747432286,eta_l0_n52_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,53,0,-1.0909113768846286,1.05038646716867,-0.020262454857979284,1.0706489220266493,2.1412978440532986,affine,0.09500119573358663,0.19000239146717326,0.9319962481713715,0.10193302378629127,eta_l0_n53_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,53,1,-1.0909113768846286,1.05038646716867,-0.020262454857979284,1.0706489220266493,2.1412978440532986,affine,0.31165202620511157,0.6233040524102231,1.0,0.31165202620511157,eta_l0_n53_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,54,0,-1.2066771566490586,1.173895383021899,-0.016390886813579808,1.1902862698354788,2.3805725396709576,affine,0.11848491224053499,0.23696982448106998,0.9319962481713715,0.12713024593501207,eta_l0_n54_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,54,1,-1.2066771566490586,1.173895383021899,-0.016390886813579808,1.1902862698354788,2.3805725396709576,affine,0.34490796924504935,0.6898159384900987,1.0,0.34490796924504935,eta_l0_n54_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,55,0,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.03217402595959646,0.06434805191919292,0.9319962481713715,0.03452162605023753,eta_l0_n55_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,55,1,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.17478486104675106,0.3495697220935021,1.0,0.17478486104675106,eta_l0_n55_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,56,0,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,affine,0.03540540227173134,0.07081080454346268,0.9319962481713715,0.03798878197331664,eta_l0_n56_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,56,1,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,affine,0.18492049713101028,0.36984099426202055,1.0,0.18492049713101028,eta_l0_n56_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,57,0,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.1231062852532452,0.2462125705064904,0.9319962481713715,0.1320888206307553,eta_l0_n57_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,57,1,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.3506497194153906,0.7012994388307812,1.0,0.3506497194153906,eta_l0_n57_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,58,0,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.037684701643931236,0.07536940328786247,0.9319962481713715,0.04043439200304799,eta_l0_n58_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,58,1,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.191521183406278,0.383042366812556,1.0,0.191521183406278,eta_l0_n58_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,59,0,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.07040104306089869,0.14080208612179737,0.9319962481713715,0.07553790393365795,eta_l0_n59_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,59,1,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.2689123493482129,0.5378246986964258,1.0,0.2689123493482129,eta_l0_n59_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,60,0,-1.5860529702984403,1.5733837593857998,-0.0063346054563202525,1.57971836484212,3.15943672968424,affine,0.1992008157072346,0.3984016314144692,0.9319962481713715,0.21373564120894015,eta_l0_n60_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,60,1,-1.5860529702984403,1.5733837593857998,-0.0063346054563202525,1.57971836484212,3.15943672968424,affine,0.4218660257056953,0.8437320514113906,1.0,0.4218660257056953,eta_l0_n60_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,61,0,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,affine,0.10458222490461823,0.20916444980923646,0.9319962481713715,0.11221313938743249,eta_l0_n61_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,61,1,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,affine,0.32606958196707225,0.6521391639341445,1.0,0.32606958196707225,eta_l0_n61_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,62,0,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,affine,0.09705393191182805,0.1941078638236561,0.9319962481713715,0.1041355392816798,eta_l0_n62_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,62,1,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,affine,0.3148977187880977,0.6297954375761954,1.0,0.3148977187880977,eta_l0_n62_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,63,0,-1.4015171493211274,1.3643005656294995,-0.018608291845813918,1.3829088574753134,2.765817714950627,affine,0.15815871965502917,0.31631743931005835,0.9319962481713715,0.16969888018899795,eta_l0_n63_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,63,1,-1.4015171493211274,1.3643005656294995,-0.018608291845813918,1.3829088574753134,2.765817714950627,affine,0.3885130831309809,0.7770261662619617,1.0,0.3885130831309809,eta_l0_n63_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,64,0,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,affine,0.05902823505223593,0.11805647010447186,0.9319962481713715,0.0633352711108576,eta_l0_n64_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,64,1,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,affine,0.245112538951466,0.490225077902932,1.0,0.245112538951466,eta_l0_n64_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,65,0,-0.8429087446687172,0.8658273177411767,0.01145928653622974,0.854368031204947,1.708736062409894,affine,0.05681525626691875,0.1136305125338375,0.9319962481713715,0.060960820795570206,eta_l0_n65_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,65,1,-0.8429087446687172,0.8658273177411767,0.01145928653622974,0.854368031204947,1.708736062409894,affine,0.24030458215024153,0.48060916430048306,1.0,0.24030458215024153,eta_l0_n65_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,66,0,-0.9905468239121493,0.9617857365141982,-0.014380543698975568,0.9761662802131738,1.9523325604263475,affine,0.0774704270215625,0.154940854043125,0.9319962481713715,0.08312311039187527,eta_l0_n66_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,66,1,-0.9905468239121493,0.9617857365141982,-0.014380543698975568,0.9761662802131738,1.9523325604263475,affine,0.28221320191889177,0.5644264038377835,1.0,0.28221320191889177,eta_l0_n66_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,67,0,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.06370430392497341,0.12740860784994681,0.9319962481713715,0.06835253258794206,eta_l0_n67_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,67,1,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.25522800306256815,0.5104560061251363,1.0,0.25522800306256815,eta_l0_n67_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,68,0,-1.0577074974769154,1.0951935167927762,0.01874300965793041,1.0764505071348458,2.1529010142696916,affine,0.0961007655593182,0.1922015311186364,0.9319962481713715,0.10311282448601401,eta_l0_n68_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,68,1,-1.0577074974769154,1.0951935167927762,0.01874300965793041,1.0764505071348458,2.1529010142696916,affine,0.313395395818585,0.62679079163717,1.0,0.313395395818585,eta_l0_n68_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,69,0,-1.3791784207269757,1.42228292189483,0.02155225058392718,1.400730671310903,2.801461342621806,affine,0.16189286350766371,0.32378572701532743,0.9319962481713715,0.17370548843442934,eta_l0_n69_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,69,1,-1.3791784207269757,1.42228292189483,0.02155225058392718,1.400730671310903,2.801461342621806,affine,0.39193665167048064,0.7838733033409613,1.0,0.39193665167048064,eta_l0_n69_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,70,0,-0.9308513790946814,0.9636725950147781,0.01641060796004834,0.9472619870547297,1.8945239741094595,affine,0.07236808784580497,0.14473617569160993,0.9319962481713715,0.0776484755038394,eta_l0_n70_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,70,1,-0.9308513790946814,0.9636725950147781,0.01641060796004834,0.9472619870547297,1.8945239741094595,affine,0.2726039266590261,0.5452078533180522,1.0,0.2726039266590261,eta_l0_n70_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,71,0,-1.438885897394498,1.3949902367415656,-0.02194783032646619,1.4169380670680318,2.8338761341360637,affine,0.16528217318108282,0.33056434636216564,0.9319962481713715,0.17734210143589701,eta_l0_n71_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,71,1,-1.438885897394498,1.3949902367415656,-0.02194783032646619,1.4169380670680318,2.8338761341360637,affine,0.39499506530744755,0.7899901306148951,1.0,0.39499506530744755,eta_l0_n71_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,72,0,-1.1711538278994709,1.2088264922154857,0.018836332158007396,1.1899901600574783,2.3799803201149565,affine,0.11843682940143908,0.23687365880287817,0.9319962481713715,0.12707865469825522,eta_l0_n72_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,72,1,-1.1711538278994709,1.2088264922154857,0.018836332158007396,1.1899901600574783,2.3799803201149565,affine,0.3448007599664116,0.6896015199328231,1.0,0.3448007599664116,eta_l0_n72_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,73,0,-1.1745387071873483,1.2100713466785744,0.017766319745613046,1.1923050269329614,2.3846100538659227,affine,0.11889769438589372,0.23779538877178744,0.9319962481713715,0.12757314701553532,eta_l0_n73_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,73,1,-1.1745387071873483,1.2100713466785744,0.017766319745613046,1.1923050269329614,2.3846100538659227,affine,0.3454105213869702,0.6908210427739404,1.0,0.3454105213869702,eta_l0_n73_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,74,0,-0.9300731152852425,0.8929332780733105,-0.018569918605966018,0.9115031966792765,1.823006393358553,affine,0.06623391994248702,0.13246783988497404,0.9319962481713715,0.0710667237904033,eta_l0_n74_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,74,1,-0.9300731152852425,0.8929332780733105,-0.018569918605966018,0.9115031966792765,1.823006393358553,affine,0.26038337545599277,0.5207667509119855,1.0,0.26038337545599277,eta_l0_n74_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,75,0,-0.8829617050674686,0.9081151180277419,0.012576706480136646,0.8955384115476053,1.7910768230952105,affine,0.06352122059880379,0.12704244119760758,0.9319962481713715,0.0681560904600592,eta_l0_n75_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,75,1,-0.8829617050674686,0.9081151180277419,0.012576706480136646,0.8955384115476053,1.7910768230952105,affine,0.25491234963215775,0.5098246992643155,1.0,0.25491234963215775,eta_l0_n75_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,76,0,-1.1805875736327942,1.148801902205817,-0.01589283571348865,1.1646947379193056,2.329389475838611,affine,0.1133540174298754,0.2267080348597508,0.9319962481713715,0.1216249718303934,eta_l0_n76_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,76,1,-1.1805875736327942,1.148801902205817,-0.01589283571348865,1.1646947379193056,2.329389475838611,affine,0.3382125951197492,0.6764251902394984,1.0,0.3382125951197492,eta_l0_n76_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,77,0,-1.3659675145450216,1.3314526620255038,-0.0172574262597589,1.3487100882852627,2.6974201765705255,affine,0.1510226319798477,0.3020452639596954,0.9319962481713715,0.16204210293352844,eta_l0_n77_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,77,1,-1.3659675145450216,1.3314526620255038,-0.0172574262597589,1.3487100882852627,2.6974201765705255,affine,0.38163061019264893,0.7632612203852979,1.0,0.38163061019264893,eta_l0_n77_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,78,0,-0.6125577319415736,0.6569759697827164,0.022209118920571425,0.634766850862145,1.26953370172429,affine,0.026981033085969944,0.05396206617193989,0.9319962481713715,0.028949722854473108,eta_l0_n78_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,78,1,-0.6125577319415736,0.6569759697827164,0.022209118920571425,0.634766850862145,1.26953370172429,affine,0.1573444023687779,0.3146888047375558,1.0,0.1573444023687779,eta_l0_n78_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,79,0,-0.8548400889214125,0.8380958072828193,-0.008372140819296603,0.8464679481021159,1.6929358962042318,affine,0.05555260530301887,0.11110521060603774,0.9319962481713715,0.05960603962946865,eta_l0_n79_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,79,1,-0.8548400889214125,0.8380958072828193,-0.008372140819296603,0.8464679481021159,1.6929358962042318,affine,0.23748054605143304,0.4749610921028661,1.0,0.23748054605143304,eta_l0_n79_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,80,0,-1.2719651427464111,1.229509990212115,-0.021227576267148107,1.250737566479263,2.501475132958526,affine,0.13078781192491887,0.26157562384983774,0.9319962481713715,0.14033083521691406,eta_l0_n80_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,80,1,-1.2719651427464111,1.229509990212115,-0.021227576267148107,1.250737566479263,2.501475132958526,affine,0.3598172673204265,0.719634534640853,1.0,0.3598172673204265,eta_l0_n80_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,81,0,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,affine,0.023591200034515373,0.047182400069030746,0.9319962481713715,0.025312548286329072,eta_l0_n81_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,81,1,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,affine,0.14477030561706322,0.28954061123412644,1.0,0.14477030561706322,eta_l0_n81_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,82,0,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.046288370646411676,0.09257674129282335,0.9319962481713715,0.049665833673935955,eta_l0_n82_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,82,1,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.21485561047209328,0.42971122094418657,1.0,0.21485561047209328,eta_l0_n82_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,83,0,-0.8932013736751792,0.8717751463069086,-0.0107131136841353,0.8824882599910439,1.7649765199820877,affine,0.06135632904886263,0.12271265809772526,0.9319962481713715,0.0658332360985864,eta_l0_n83_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,83,1,-0.8932013736751792,0.8717751463069086,-0.0107131136841353,0.8824882599910439,1.7649765199820877,affine,0.250342804135403,0.500685608270806,1.0,0.250342804135403,eta_l0_n83_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,84,0,-0.7976625548491152,0.8270409447667988,0.0146891949588418,0.812351749807957,1.624703499615914,affine,0.05031809937737942,0.10063619875475883,0.9319962481713715,0.05398959435309566,eta_l0_n84_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,84,1,-0.7976625548491152,0.8270409447667988,0.0146891949588418,0.812351749807957,1.624703499615914,affine,0.22494720723711653,0.44989441447423306,1.0,0.22494720723711653,eta_l0_n84_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,85,0,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.06751399250682856,0.13502798501365712,0.9319962481713715,0.07244019773609044,eta_l0_n85_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,85,1,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.26306409874430814,0.5261281974886163,1.0,0.26306409874430814,eta_l0_n85_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,86,0,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,affine,0.1063931461831313,0.2127862923662626,0.9319962481713715,0.11415619579143217,eta_l0_n86_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,86,1,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,affine,0.3285811233329348,0.6571622466658696,1.0,0.3285811233329348,eta_l0_n86_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,87,0,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,affine,0.13197455335417327,0.26394910670834654,0.9319962481713715,0.14160416805659323,eta_l0_n87_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,87,1,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,affine,0.3611638067539207,0.7223276135078414,1.0,0.3611638067539207,eta_l0_n87_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,88,0,-0.8196835402297238,0.8511525501491992,0.015734504959737716,0.8354180451894615,1.670836090378923,affine,0.053858323789688196,0.10771664757937639,0.9319962481713715,0.05778813369191263,eta_l0_n88_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,88,1,-0.8196835402297238,0.8511525501491992,0.015734504959737716,0.8354180451894615,1.670836090378923,affine,0.2333895181269373,0.4667790362538746,1.0,0.2333895181269373,eta_l0_n88_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,89,0,-0.8579518670180813,0.8906473389988186,0.01634773599036865,0.87429960300845,1.7485992060169,affine,0.06004773422309601,0.12009546844619202,0.9319962481713715,0.06442915874491234,eta_l0_n89_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,89,1,-0.8579518670180813,0.8906473389988186,0.01634773599036865,0.87429960300845,1.7485992060169,affine,0.2473735088482814,0.4947470176965628,1.0,0.2473735088482814,eta_l0_n89_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,90,0,-0.8207744817317322,0.7924840586718498,-0.014145211529941193,0.806629270201791,1.613258540403582,affine,0.04945446424308742,0.09890892848617484,0.9319962481713715,0.05306294348300203,eta_l0_n90_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,90,1,-0.8207744817317322,0.7924840586718498,-0.014145211529941193,0.806629270201791,1.613258540403582,affine,0.22283997146248546,0.44567994292497093,1.0,0.22283997146248546,eta_l0_n90_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,91,0,-1.2894025896139478,1.2579584264319816,-0.015722081590983095,1.2736805080229647,2.5473610160459295,affine,0.13547413367370534,0.2709482673474107,0.9319962481713715,0.14535909767825045,eta_l0_n91_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,91,1,-1.2894025896139478,1.2579584264319816,-0.015722081590983095,1.2736805080229647,2.5473610160459295,affine,0.3652544063984313,0.7305088127968626,1.0,0.3652544063984313,eta_l0_n91_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,92,0,-1.1249469056327446,1.0858598844704395,-0.019543510581152557,1.105403395051592,2.210806790103184,affine,0.10169126819839032,0.20338253639678064,0.9319962481713715,0.10911124202260926,eta_l0_n92_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,92,1,-1.1249469056327446,1.0858598844704395,-0.019543510581152557,1.105403395051592,2.210806790103184,affine,0.32179887662215556,0.6435977532443111,1.0,0.32179887662215556,eta_l0_n92_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,93,0,-0.6468941826251269,0.6720388239177191,0.012572320646296098,0.659466503271423,1.318933006542846,affine,0.02973109792160489,0.05946219584320978,0.9319962481713715,0.03190044807577172,eta_l0_n93_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,93,1,-0.6468941826251269,0.6720388239177191,0.012572320646296098,0.659466503271423,1.318933006542846,affine,0.16698133324584757,0.33396266649169515,1.0,0.16698133324584757,eta_l0_n93_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,94,0,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,affine,0.12258952797033353,0.24517905594066705,0.9319962481713715,0.13153435779474543,eta_l0_n94_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,94,1,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,affine,0.3499788038850754,0.6999576077701508,1.0,0.3499788038850754,eta_l0_n94_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,95,0,-0.8425271444430554,0.8784439097769791,0.017958382666961814,0.8604855271100172,1.7209710542200345,affine,0.05782891292611981,0.11565782585223962,0.9319962481713715,0.0620484396150557,eta_l0_n95_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,95,1,-0.8425271444430554,0.8784439097769791,0.017958382666961814,0.8604855271100172,1.7209710542200345,affine,0.2424173386872116,0.4848346773744232,1.0,0.2424173386872116,eta_l0_n95_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,96,0,-0.6659994683686798,0.635867396179219,-0.01506603609473045,0.6509334322739494,1.3018668645478988,affine,0.0287543739277934,0.0575087478555868,0.9319962481713715,0.030852456739188686,eta_l0_n96_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,96,1,-0.6659994683686798,0.635867396179219,-0.01506603609473045,0.6509334322739494,1.3018668645478988,affine,0.16366932524311711,0.32733865048623423,1.0,0.16366932524311711,eta_l0_n96_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,97,0,-1.2151238777580005,1.1786727174263023,-0.018225580165849076,1.1968982975921514,2.3937965951843028,affine,0.11982580429260405,0.2396516085852081,0.9319962481713715,0.12856897710447757,eta_l0_n97_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,97,1,-1.2151238777580005,1.1786727174263023,-0.018225580165849076,1.1968982975921514,2.3937965951843028,affine,0.34658140653981456,0.6931628130796291,1.0,0.34658140653981456,eta_l0_n97_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,98,0,-1.2733732597983927,1.309028506901763,0.017827623551685212,1.2912008833500779,2.5824017667001558,affine,0.13909500430324453,0.27819000860648907,0.9319962481713715,0.14924416764140055,eta_l0_n98_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,98,1,-1.2733732597983927,1.309028506901763,0.017827623551685212,1.2912008833500779,2.5824017667001558,affine,0.36921820119777526,0.7384364023955505,1.0,0.36921820119777526,eta_l0_n98_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,99,0,-0.9135757537667057,0.8771689762111492,-0.018203388777778273,0.8953723649889275,1.790744729977855,affine,0.06352641759977425,0.1270528351995485,0.9319962481713715,0.06816166666380537,eta_l0_n99_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,99,1,-0.9135757537667057,0.8771689762111492,-0.018203388777778273,0.8953723649889275,1.790744729977855,affine,0.25477731278545024,0.5095546255709005,1.0,0.25477731278545024,eta_l0_n99_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,100,0,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,affine,0.17294317701844078,0.34588635403688156,0.9319962481713715,0.18556209572491833,eta_l0_n100_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,100,1,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,affine,0.40173890570989007,0.8034778114197801,1.0,0.40173890570989007,eta_l0_n100_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,101,0,-1.583787966015745,1.5801226097140433,-0.0018326781508508638,1.5819552878648941,3.1639105757297883,affine,0.1996618192557774,0.3993236385115548,0.9319962481713715,0.2142302822007331,eta_l0_n101_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,101,1,-1.583787966015745,1.5801226097140433,-0.0018326781508508638,1.5819552878648941,3.1639105757297883,affine,0.42219407424579525,0.8443881484915905,1.0,0.42219407424579525,eta_l0_n101_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,102,0,-1.0583150318257148,1.026256340812422,-0.016029345506646475,1.0422856863190684,2.0845713726381367,affine,0.08960906604109521,0.17921813208219042,0.9319962481713715,0.09614745361573417,eta_l0_n102_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,102,1,-1.0583150318257148,1.026256340812422,-0.016029345506646475,1.0422856863190684,2.0845713726381367,affine,0.30315034046603295,0.6063006809320659,1.0,0.30315034046603295,eta_l0_n102_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,103,0,-1.1476452573399842,1.1090705510943302,-0.019287353122827033,1.1283579042171572,2.2567158084343144,affine,0.10617588471459573,0.21235176942919146,0.9319962481713715,0.11392308168935092,eta_l0_n103_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,103,1,-1.1476452573399842,1.1090705510943302,-0.019287353122827033,1.1283579042171572,2.2567158084343144,affine,0.32827755135072456,0.6565551027014491,1.0,0.32827755135072456,eta_l0_n103_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,104,0,-0.8577016693145987,0.8877385522479757,0.015018441466688515,0.8727201107812872,1.7454402215625744,affine,0.05978300541739091,0.11956601083478181,0.9319962481713715,0.06414511381852502,eta_l0_n104_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,104,1,-0.8577016693145987,0.8877385522479757,0.015018441466688515,0.8727201107812872,1.7454402215625744,affine,0.24683057948606552,0.49366115897213103,1.0,0.24683057948606552,eta_l0_n104_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,105,0,-1.1779127957269429,1.2185421121727307,0.02031465822289391,1.1982274539498368,2.3964549078996735,affine,0.12010463419158925,0.2402092683831785,0.9319962481713715,0.12886815202018378,eta_l0_n105_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,105,1,-1.1779127957269429,1.2185421121727307,0.02031465822289391,1.1982274539498368,2.3964549078996735,affine,0.34689200531234776,0.6937840106246955,1.0,0.34689200531234776,eta_l0_n105_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,106,0,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,affine,0.03822436614129144,0.07644873228258288,0.9319962481713715,0.04101343349427616,eta_l0_n106_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,106,1,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,affine,0.19290116812859953,0.38580233625719906,1.0,0.19290116812859953,eta_l0_n106_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,107,0,-0.6364463917753737,0.563347610914743,-0.036549390430315354,0.5998970013450583,1.1997940026901166,affine,0.023429717443423096,0.04685943488684619,0.9319962481713715,0.02513928300612101,eta_l0_n107_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,107,1,-0.6364463917753737,0.563347610914743,-0.036549390430315354,0.5998970013450583,1.1997940026901166,affine,0.14365662849300698,0.28731325698601395,1.0,0.14365662849300698,eta_l0_n107_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,108,0,-1.0426471018400016,1.0803325368651484,0.01884271751257338,1.061489819352575,2.12297963870515,affine,0.09324930213724093,0.18649860427448187,0.9319962481713715,0.10005330206017594,eta_l0_n108_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,108,1,-1.0426471018400016,1.0803325368651484,0.01884271751257338,1.061489819352575,2.12297963870515,affine,0.3089373526281335,0.617874705256267,1.0,0.3089373526281335,eta_l0_n108_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,109,0,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,affine,0.06945072467805004,0.13890144935610008,0.9319962481713715,0.07451824491173245,eta_l0_n109_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,109,1,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,affine,0.26690106136144665,0.5338021227228933,1.0,0.26690106136144665,eta_l0_n109_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,110,0,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.1296818009212755,0.259363601842551,0.9319962481713715,0.13914412335427143,eta_l0_n110_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,110,1,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.3585026397894039,0.7170052795788078,1.0,0.3585026397894039,eta_l0_n110_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,111,0,-0.958157890561465,0.9243831398125949,-0.016887375374435076,0.94127051518703,1.88254103037406,affine,0.07132673835576632,0.14265347671153264,0.9319962481713715,0.07653114322693182,eta_l0_n111_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,111,1,-0.958157890561465,0.9243831398125949,-0.016887375374435076,0.94127051518703,1.88254103037406,affine,0.2705801126588766,0.5411602253177532,1.0,0.2705801126588766,eta_l0_n111_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,112,0,-0.7914234807966222,0.818891668119116,0.013734093661246893,0.8051575744578691,1.6103151489157381,affine,0.04923190847633619,0.09846381695267238,0.9319962481713715,0.05282414878056852,eta_l0_n112_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,112,1,-0.7914234807966222,0.818891668119116,0.013734093661246893,0.8051575744578691,1.6103151489157381,affine,0.22230035899282263,0.44460071798564527,1.0,0.22230035899282263,eta_l0_n112_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,113,0,-0.8496861508209814,0.8720170059064725,0.01116542754274552,0.8608515783637269,1.7217031567274539,affine,0.05784896434081916,0.11569792868163832,0.9319962481713715,0.062069954095117924,eta_l0_n113_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,113,1,-0.8496861508209814,0.8720170059064725,0.01116542754274552,0.8608515783637269,1.7217031567274539,affine,0.24263703680786103,0.48527407361572206,1.0,0.24263703680786103,eta_l0_n113_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,114,0,-0.6630886315286884,0.6437069553096086,-0.009690838109539857,0.6533977934191485,1.306795586838297,affine,0.029010392935394484,0.05802078587078897,0.9319962481713715,0.031127156351020178,eta_l0_n114_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,114,1,-0.6630886315286884,0.6437069553096086,-0.009690838109539857,0.6533977934191485,1.306795586838297,affine,0.164672309515897,0.329344619031794,1.0,0.164672309515897,eta_l0_n114_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,115,0,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.039720764451005815,0.07944152890201163,0.9319962481713715,0.04261901754319308,eta_l0_n115_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,115,1,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.19732520226170253,0.39465040452340505,1.0,0.19732520226170253,eta_l0_n115_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,116,0,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.0629088693592051,0.1258177387184102,0.9319962481713715,0.06749905858809604,eta_l0_n116_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,116,1,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.253502859142495,0.50700571828499,1.0,0.253502859142495,eta_l0_n116_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,117,0,-1.3740507569108842,1.4125160594440573,0.019232651266586576,1.3932834081774708,2.7865668163549415,affine,0.1603277382012225,0.320655476402445,0.9319962481713715,0.17202616267586318,eta_l0_n117_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,117,1,-1.3740507569108842,1.4125160594440573,0.019232651266586576,1.3932834081774708,2.7865668163549415,affine,0.39052967606442457,0.7810593521288491,1.0,0.39052967606442457,eta_l0_n117_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,118,0,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.03443316994739421,0.06886633989478842,0.9319962481713715,0.03694561004398248,eta_l0_n118_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,118,1,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.1818036059699658,0.3636072119399316,1.0,0.1818036059699658,eta_l0_n118_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,119,0,-1.1687352625088665,1.1298298900732497,-0.01945268621780838,1.1492825762910581,2.2985651525821162,affine,0.11030672337739635,0.2206134467547927,0.9319962481713715,0.11835532985655713,eta_l0_n119_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,119,1,-1.1687352625088665,1.1298298900732497,-0.01945268621780838,1.1492825762910581,2.2985651525821162,affine,0.3340246272819401,0.6680492545638802,1.0,0.3340246272819401,eta_l0_n119_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,120,0,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.041338081872310646,0.08267616374462129,0.9319962481713715,0.044354343650436646,eta_l0_n120_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,120,1,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.20172634433294967,0.40345268866589934,1.0,0.20172634433294967,eta_l0_n120_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,121,0,-0.666212706257072,0.6906969277491457,0.012242110746036872,0.6784548170031088,1.3569096340062177,affine,0.03199782957754133,0.06399565915508267,0.9319962481713715,0.034332573377116975,eta_l0_n121_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,121,1,-0.666212706257072,0.6906969277491457,0.012242110746036872,0.6784548170031088,1.3569096340062177,affine,0.17429024028187629,0.34858048056375257,1.0,0.17429024028187629,eta_l0_n121_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,122,0,-0.8964792612600482,0.9290067815766619,0.016263760158306884,0.9127430214183551,1.8254860428367101,affine,0.06642863621157087,0.13285727242314174,0.9319962481713715,0.0712756476669381,eta_l0_n122_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,122,1,-0.8964792612600482,0.9290067815766619,0.016263760158306884,0.9127430214183551,1.8254860428367101,affine,0.26084735534552284,0.5216947106910457,1.0,0.26084735534552284,eta_l0_n122_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,123,0,-1.1041675633687147,1.1420328248526195,0.018932630741952394,1.123100194110667,2.246200388221334,affine,0.10514210954629312,0.21028421909258624,0.9319962481713715,0.11281387639981147,eta_l0_n123_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,123,1,-1.1041675633687147,1.1420328248526195,0.018932630741952394,1.123100194110667,2.246200388221334,affine,0.3268151844988936,0.6536303689977871,1.0,0.3268151844988936,eta_l0_n123_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,124,0,-0.7854236701342572,0.7505279392833624,-0.017447865425447406,0.7679758047088098,1.5359516094176195,affine,0.04383925134039441,0.08767850268078882,0.9319962481713715,0.04703801268128435,eta_l0_n124_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,124,1,-0.7854236701342572,0.7505279392833624,-0.017447865425447406,0.7679758047088098,1.5359516094176195,affine,0.2083621946695587,0.4167243893391174,1.0,0.2083621946695587,eta_l0_n124_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,125,0,-1.3977310453244942,1.362160377008622,-0.01778533415793615,1.379945711166558,2.759891422333116,affine,0.1575371349875396,0.3150742699750792,0.9319962481713715,0.16903194116568196,eta_l0_n125_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,125,1,-1.3977310453244942,1.362160377008622,-0.01778533415793615,1.379945711166558,2.759891422333116,affine,0.38793782616216815,0.7758756523243363,1.0,0.38793782616216815,eta_l0_n125_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,126,0,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.11361256119173553,0.22722512238347106,0.9319962481713715,0.12190238041692732,eta_l0_n126_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,126,1,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.33846737843434105,0.6769347568686821,1.0,0.33846737843434105,eta_l0_n126_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,127,0,-1.1653861727789858,1.1295881895992121,-0.017898991589886837,1.147487181189099,2.294974362378198,affine,0.10994269110721135,0.2198853822144227,0.9319962481713715,0.11796473571961799,eta_l0_n127_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,127,1,-1.1653861727789858,1.1295881895992121,-0.017898991589886837,1.147487181189099,2.294974362378198,affine,0.33355880708885166,0.6671176141777033,1.0,0.33355880708885166,eta_l0_n127_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,128,0,-0.9183696950325081,0.9538988826345168,0.01776459380100437,0.9361342888335125,1.872268577667025,affine,0.07044155189568786,0.14088310379137572,0.9319962481713715,0.07558136852363742,eta_l0_n128_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,128,1,-0.9183696950325081,0.9538988826345168,0.01776459380100437,0.9361342888335125,1.872268577667025,affine,0.2688296286984237,0.5376592573968474,1.0,0.2688296286984237,eta_l0_n128_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,129,0,-0.8382147723907126,0.8049901321285021,-0.01661232013110525,0.8216024522596074,1.6432049045192147,affine,0.0517349305265229,0.1034698610530458,0.9319962481713715,0.05550980556845558,eta_l0_n129_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,129,1,-0.8382147723907126,0.8049901321285021,-0.01661232013110525,0.8216024522596074,1.6432049045192147,affine,0.2283246773501364,0.4566493547002728,1.0,0.2283246773501364,eta_l0_n129_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,130,0,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,affine,0.08784979328981053,0.17569958657962106,0.9319962481713715,0.0942598143095283,eta_l0_n130_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,130,1,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,affine,0.30022639347126356,0.6004527869425271,1.0,0.30022639347126356,eta_l0_n130_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,131,0,-1.047450072089849,1.0179226028488386,-0.014763734620505176,1.0326863374693438,2.0653726749386876,affine,0.08780704324716622,0.17561408649433244,0.9319962481713715,0.09421394498041008,eta_l0_n131_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,131,1,-1.047450072089849,1.0179226028488386,-0.014763734620505176,1.0326863374693438,2.0653726749386876,affine,0.3002092776642886,0.6004185553285772,1.0,0.3002092776642886,eta_l0_n131_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,132,0,-1.060935041080445,1.0970640029002237,0.018064480909889324,1.0789995219903343,2.1579990439806687,affine,0.09658521996959714,0.19317043993919428,0.9319962481713715,0.10363262744791378,eta_l0_n132_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,132,1,-1.060935041080445,1.0970640029002237,0.018064480909889324,1.0789995219903343,2.1579990439806687,affine,0.3141573803296547,0.6283147606593094,1.0,0.3141573803296547,eta_l0_n132_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,133,0,-1.253402395011484,1.2165648038644405,-0.018418795573521773,1.2349835994379623,2.4699671988759246,affine,0.12755412399079488,0.25510824798158976,0.9319962481713715,0.13686119900274618,eta_l0_n133_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,133,1,-1.253402395011484,1.2165648038644405,-0.018418795573521773,1.2349835994379623,2.4699671988759246,affine,0.356068269207211,0.712136538414422,1.0,0.356068269207211,eta_l0_n133_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,134,0,-1.0114330530784947,0.9766652867437464,-0.01738388316737416,0.9940491699111206,1.9880983398222412,affine,0.08071068741298588,0.16142137482597177,0.9319962481713715,0.08659979862724206,eta_l0_n134_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,134,1,-1.0114330530784947,0.9766652867437464,-0.01738388316737416,0.9940491699111206,1.9880983398222412,affine,0.2879753490369872,0.5759506980739744,1.0,0.2879753490369872,eta_l0_n134_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,135,0,-1.283272451573751,1.3273446149675863,0.02203608169691762,1.3053085332706686,2.6106170665413373,affine,0.142031026455352,0.284062052910704,0.9319962481713715,0.15239441868357817,eta_l0_n135_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,135,1,-1.283272451573751,1.3273446149675863,0.02203608169691762,1.3053085332706686,2.6106170665413373,affine,0.3723051142259514,0.7446102284519028,1.0,0.3723051142259514,eta_l0_n135_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,136,0,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.055506724224711264,0.11101344844942253,0.9319962481713715,0.059556810806501145,eta_l0_n136_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,136,1,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.23735487630608,0.47470975261216,1.0,0.23735487630608,eta_l0_n136_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,137,0,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.1106683443332991,0.2213366886665982,0.9319962481713715,0.11874333673599712,eta_l0_n137_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,137,1,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.33448127202207456,0.6689625440441491,1.0,0.33448127202207456,eta_l0_n137_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,138,0,-1.4795147536405517,1.4449804711041994,-0.01726714126817619,1.4622476123723755,2.924495224744751,affine,0.1747409545584442,0.3494819091168884,0.9319962481713715,0.18749104934842353,eta_l0_n138_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,138,1,-1.4795147536405517,1.4449804711041994,-0.01726714126817619,1.4622476123723755,2.924495224744751,affine,0.4032069921724032,0.8064139843448064,1.0,0.4032069921724032,eta_l0_n138_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,139,0,-1.1846112842973124,1.1451154059873383,-0.01974793915498707,1.1648633451423254,2.3297266902846507,affine,0.11340669701661678,0.22681339403323356,0.9319962481713715,0.12168149521967178,eta_l0_n139_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,139,1,-1.1846112842973124,1.1451154059873383,-0.01974793915498707,1.1648633451423254,2.3297266902846507,affine,0.3382067958346997,0.6764135916693994,1.0,0.3382067958346997,eta_l0_n139_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,140,0,-1.0016612677883296,1.0308531994790653,0.01459596584536782,1.0162572336336975,2.032514467267395,affine,0.08475915274863734,0.16951830549727467,0.9319962481713715,0.0909436630404248,eta_l0_n140_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,140,1,-1.0016612677883296,1.0308531994790653,0.01459596584536782,1.0162572336336975,2.032514467267395,affine,0.2950819976681076,0.5901639953362152,1.0,0.2950819976681076,eta_l0_n140_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,141,0,-0.8542154631766357,0.8897074866463954,0.017746011734879885,0.8719614749115155,1.743922949823031,affine,0.05967715153016984,0.11935430306033969,0.9319962481713715,0.06403153622909935,eta_l0_n141_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,141,1,-0.8542154631766357,0.8897074866463954,0.017746011734879885,0.8719614749115155,1.743922949823031,affine,0.24652077232990388,0.49304154465980776,1.0,0.24652077232990388,eta_l0_n141_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,142,0,-1.0907597002338707,1.1219900064229051,0.01561515309451722,1.106374853328388,2.212749706656776,affine,0.10185940078171347,0.20371880156342695,0.9319962481713715,0.10929164251633769,eta_l0_n142_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,142,1,-1.0907597002338707,1.1219900064229051,0.01561515309451722,1.106374853328388,2.212749706656776,affine,0.3221303897798582,0.6442607795597164,1.0,0.3221303897798582,eta_l0_n142_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,143,0,-0.8162896758725653,0.8281241957461958,0.0059172599368152445,0.8222069358093805,1.644413871618761,affine,0.05177898004970977,0.10355796009941955,0.9319962481713715,0.05555706919561427,eta_l0_n143_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,143,1,-0.8162896758725653,0.8281241957461958,0.0059172599368152445,0.8222069358093805,1.644413871618761,affine,0.22865439410150246,0.4573087882030049,1.0,0.22865439410150246,eta_l0_n143_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,144,0,-1.0737179807403916,1.0426127742833173,-0.015552603228537132,1.0581653775118545,2.116330755023709,affine,0.09260083445632895,0.1852016689126579,0.9319962481713715,0.0993575184857417,eta_l0_n144_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,144,1,-1.0737179807403916,1.0426127742833173,-0.015552603228537132,1.0581653775118545,2.116330755023709,affine,0.30798318336415104,0.6159663667283021,1.0,0.30798318336415104,eta_l0_n144_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,145,0,-0.908246631810817,0.8772921563875723,-0.01547723771162235,0.8927693940991946,1.7855387881983893,affine,0.06307662244905878,0.12615324489811755,0.9319962481713715,0.06767905189835112,eta_l0_n145_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,145,1,-0.908246631810817,0.8772921563875723,-0.01547723771162235,0.8927693940991946,1.7855387881983893,affine,0.253906143084322,0.507812286168644,1.0,0.253906143084322,eta_l0_n145_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,146,0,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.145374579693725,0.29074915938745,0.9319962481713715,0.15598193660002174,eta_l0_n146_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,146,1,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.37586776251971343,0.7517355250394269,1.0,0.37586776251971343,eta_l0_n146_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,147,0,-0.9610675137578785,0.9233653689554445,-0.018851072401216973,0.9422164413566615,1.884432882713323,affine,0.071503963689264,0.143007927378528,0.9319962481713715,0.07672129992964967,eta_l0_n147_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,147,1,-0.9610675137578785,0.9233653689554445,-0.018851072401216973,0.9422164413566615,1.884432882713323,affine,0.27086854893595547,0.5417370978719109,1.0,0.27086854893595547,eta_l0_n147_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,148,0,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.16491939624199328,0.32983879248398656,0.9319962481713715,0.17695285422615628,eta_l0_n148_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,148,1,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.3947057909211337,0.7894115818422675,1.0,0.3947057909211337,eta_l0_n148_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,149,0,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,affine,0.05466287032502593,0.10932574065005186,0.9319962481713715,0.05865138452249945,eta_l0_n149_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,149,1,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,affine,0.23527771722316493,0.47055543444632986,1.0,0.23527771722316493,eta_l0_n149_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,150,0,-0.81442882440053,0.8392548862181488,0.012413030908809408,0.8268418553093394,1.6536837106186788,affine,0.0525136087308466,0.1050272174616932,0.9319962481713715,0.05634530056733729,eta_l0_n150_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,150,1,-0.81442882440053,0.8392548862181488,0.012413030908809408,0.8268418553093394,1.6536837106186788,affine,0.23029984075508209,0.46059968151016417,1.0,0.23029984075508209,eta_l0_n150_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,151,0,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,affine,0.04347210480581693,0.08694420961163386,0.9319962481713715,0.04664407704549414,eta_l0_n151_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,151,1,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,affine,0.20749637537672616,0.4149927507534523,1.0,0.20749637537672616,eta_l0_n151_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,152,0,-1.2728778080271927,1.3140653792633163,0.0205937856180618,1.2934715936452545,2.586943187290509,affine,0.13957622498198222,0.27915244996396443,0.9319962481713715,0.1497605009202972,eta_l0_n152_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,152,1,-1.2728778080271927,1.3140653792633163,0.0205937856180618,1.2934715936452545,2.586943187290509,affine,0.36969406958175527,0.7393881391635105,1.0,0.36969406958175527,eta_l0_n152_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,153,0,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.03558178463885487,0.07116356927770974,0.9319962481713715,0.038178034202034944,eta_l0_n153_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,153,1,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.18526418828007996,0.3705283765601599,1.0,0.18526418828007996,eta_l0_n153_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,154,0,-0.79668091660671,0.7634063707671254,-0.016637272919792334,0.7800436436869177,1.5600872873738354,affine,0.04555995415804837,0.09111990831609675,0.9319962481713715,0.04888426777193528,eta_l0_n154_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,154,1,-0.79668091660671,0.7634063707671254,-0.016637272919792334,0.7800436436869177,1.5600872873738354,affine,0.21290628402333134,0.4258125680466627,1.0,0.21290628402333134,eta_l0_n154_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,155,0,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.060409021979322915,0.12081804395864583,0.9319962481713715,0.06481680811253134,eta_l0_n155_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,155,1,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.24812396510144952,0.49624793020289903,1.0,0.24812396510144952,eta_l0_n155_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,156,0,-0.9292268209994728,0.9689797402104269,0.019876459605477015,0.9491032806049499,1.8982065612098997,affine,0.07271276999907768,0.14542553999815536,0.9319962481713715,0.07801830762918217,eta_l0_n156_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,156,1,-0.9292268209994728,0.9689797402104269,0.019876459605477015,0.9491032806049499,1.8982065612098997,affine,0.27316687115563565,0.5463337423112713,1.0,0.27316687115563565,eta_l0_n156_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,157,0,-1.1731237389534528,1.2050166691047852,0.015946465075666216,1.189070204029119,2.378140408058238,affine,0.11823823949624425,0.2364764789924885,0.9319962481713715,0.12686557454306738,eta_l0_n157_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,157,1,-1.1731237389534528,1.2050166691047852,0.015946465075666216,1.189070204029119,2.378140408058238,affine,0.34459962052402604,0.6891992410480521,1.0,0.34459962052402604,eta_l0_n157_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,158,0,-0.8913860173428696,0.9226653347131664,0.015639658685148383,0.907025676028018,1.814051352056036,affine,0.06545990610854242,0.13091981221708485,0.9319962481713715,0.07023623350091635,eta_l0_n158_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,158,1,-0.8913860173428696,0.9226653347131664,0.015639658685148383,0.907025676028018,1.814051352056036,affine,0.25887689896386723,0.5177537979277345,1.0,0.25887689896386723,eta_l0_n158_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,159,0,-1.1273921839493979,1.0959081779819333,-0.015742002983732295,1.1116501809656656,2.223300361931331,affine,0.10288727370103208,0.20577454740206416,0.9319962481713715,0.11039451489520762,eta_l0_n159_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,159,1,-1.1273921839493979,1.0959081779819333,-0.015742002983732295,1.1116501809656656,2.223300361931331,affine,0.32363040412414623,0.6472608082482925,1.0,0.32363040412414623,eta_l0_n159_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,160,0,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.1775418997412546,0.3550837994825092,0.9319962481713715,0.19049636743667336,eta_l0_n160_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,160,1,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.4055171391129622,0.8110342782259244,1.0,0.4055171391129622,eta_l0_n160_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,161,0,-0.8118887083898723,0.7855637226605154,-0.01316249286467841,0.7987262155251939,1.5974524310503877,affine,0.04827148630569767,0.09654297261139534,0.9319962481713715,0.05179364874097832,eta_l0_n161_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,161,1,-0.8118887083898723,0.7855637226605154,-0.01316249286467841,0.7987262155251939,1.5974524310503877,affine,0.21992204690021305,0.4398440938004261,1.0,0.21992204690021305,eta_l0_n161_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,162,0,-0.9685612086655193,0.9950593658781908,0.013249078606335729,0.981810287271855,1.96362057454371,affine,0.07847764604365189,0.15695529208730377,0.9319962481713715,0.08420382184759799,eta_l0_n162_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,162,1,-0.9685612086655193,0.9950593658781908,0.013249078606335729,0.981810287271855,1.96362057454371,affine,0.28406853154096634,0.5681370630819327,1.0,0.28406853154096634,eta_l0_n162_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,163,0,-0.8271071303752251,0.8610459715485377,0.016969420586656292,0.8440765509618814,1.6881531019237628,affine,0.055219362719068694,0.11043872543813739,0.9319962481713715,0.05924848176954806,eta_l0_n163_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,163,1,-0.8271071303752251,0.8610459715485377,0.016969420586656292,0.8440765509618814,1.6881531019237628,affine,0.2365171171742459,0.4730342343484918,1.0,0.2365171171742459,eta_l0_n163_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,164,0,-1.1176997476072552,1.1538089483476912,0.018054600370218,1.1357543479774732,2.2715086959549464,affine,0.10762484486853492,0.21524968973706984,0.9319962481713715,0.11547776622459678,eta_l0_n164_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,164,1,-1.1176997476072552,1.1538089483476912,0.018054600370218,1.1357543479774732,2.2715086959549464,affine,0.33034402088890125,0.6606880417778025,1.0,0.33034402088890125,eta_l0_n164_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,165,0,-0.9571068847382636,0.9915651490227835,0.017229132142259962,0.9743360168805235,1.948672033761047,affine,0.07715881571356727,0.15431763142713453,0.9319962481713715,0.08278876214894336,eta_l0_n165_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,165,1,-0.9571068847382636,0.9915651490227835,0.017229132142259962,0.9743360168805235,1.948672033761047,affine,0.28157507368831103,0.5631501473766221,1.0,0.28157507368831103,eta_l0_n165_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,166,0,-1.1951923778641027,1.1529662485775045,-0.021113064643299095,1.1740793132208036,2.348158626441607,affine,0.11525571941871461,0.23051143883742922,0.9319962481713715,0.12366543282213072,eta_l0_n166_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,166,1,-1.1951923778641027,1.1529662485775045,-0.021113064643299095,1.1740793132208036,2.348158626441607,affine,0.3406246922472449,0.6812493844944898,1.0,0.3406246922472449,eta_l0_n166_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,167,0,-1.1001154688531452,1.072608887706285,-0.01375329057343011,1.086362178279715,2.17272435655943,affine,0.0979784771275863,0.1959569542551726,0.9319962481713715,0.10512754457952542,eta_l0_n167_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,167,1,-1.1001154688531452,1.072608887706285,-0.01375329057343011,1.086362178279715,2.17272435655943,affine,0.31637193416115644,0.6327438683223129,1.0,0.31637193416115644,eta_l0_n167_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,168,0,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.09087186490139544,0.18174372980279088,0.9319962481713715,0.09750239346959937,eta_l0_n168_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,168,1,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.3051084494179191,0.6102168988358382,1.0,0.3051084494179191,eta_l0_n168_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,169,0,-0.8810525006112326,0.8477341734459158,-0.0166591635826584,0.8643933370285742,1.7287866740571485,affine,0.05844734457479079,0.11689468914958158,0.9319962481713715,0.06271199555735094,eta_l0_n169_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,169,1,-0.8810525006112326,0.8477341734459158,-0.0166591635826584,0.8643933370285742,1.7287866740571485,affine,0.2438371059123533,0.4876742118247066,1.0,0.2438371059123533,eta_l0_n169_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,170,0,-0.8468298613304297,0.8765102975610901,0.014840218115330206,0.8616700794457599,1.7233401588915198,affine,0.05799879967277095,0.1159975993455419,0.9319962481713715,0.06223072226585442,eta_l0_n170_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,170,1,-0.8468298613304297,0.8765102975610901,0.014840218115330206,0.8616700794457599,1.7233401588915198,affine,0.24288770303173635,0.4857754060634727,1.0,0.24288770303173635,eta_l0_n170_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,171,0,-1.1567718049908575,1.1214757897709777,-0.017648007609939897,1.1391237973809176,2.2782475947618352,affine,0.10828744905510314,0.21657489811020628,0.9319962481713715,0.11618871778461462,eta_l0_n171_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,171,1,-1.1567718049908575,1.1214757897709777,-0.017648007609939897,1.1391237973809176,2.2782475947618352,affine,0.3312768331711643,0.6625536663423286,1.0,0.3312768331711643,eta_l0_n171_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,172,0,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,affine,0.028361496955946756,0.05672299391189351,0.9319962481713715,0.03043091322695085,eta_l0_n172_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,172,1,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,affine,0.1624303925044039,0.3248607850088078,1.0,0.1624303925044039,eta_l0_n172_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,173,0,-0.6736564656359886,0.6502014587413382,-0.011727503447325205,0.6619289621886634,1.3238579243773267,affine,0.030016059137825183,0.060032118275650366,0.9319962481713715,0.032206201684522186,eta_l0_n173_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,173,1,-0.6736564656359886,0.6502014587413382,-0.011727503447325205,0.6619289621886634,1.3238579243773267,affine,0.16793748849175272,0.33587497698350544,1.0,0.16793748849175272,eta_l0_n173_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,174,0,-1.267195182787436,1.2257910549274453,-0.020702063929995296,1.2464931188574406,2.4929862377148813,affine,0.12991639486248988,0.25983278972497975,0.9319962481713715,0.13939583460490648,eta_l0_n174_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,174,1,-1.267195182787436,1.2257910549274453,-0.020702063929995296,1.2464931188574406,2.4929862377148813,affine,0.3588123469275569,0.7176246938551138,1.0,0.3588123469275569,eta_l0_n174_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,175,0,-1.3831413857359587,1.428157169742103,0.022507892003072127,1.4056492777390308,2.8112985554780616,affine,0.16292477100837002,0.32584954201674005,0.9319962481713715,0.17481268978071263,eta_l0_n175_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,175,1,-1.3831413857359587,1.428157169742103,0.022507892003072127,1.4056492777390308,2.8112985554780616,affine,0.39286284593744103,0.7857256918748821,1.0,0.39286284593744103,eta_l0_n175_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,176,0,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,affine,0.08877578314687136,0.17755156629374272,0.9319962481713715,0.09525336965792983,eta_l0_n176_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,176,1,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,affine,0.30169992359450337,0.6033998471890067,1.0,0.30169992359450337,eta_l0_n176_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,177,0,-0.8553465281258207,0.839658428320634,-0.007844049902593353,0.8475024782232273,1.6950049564464547,affine,0.05571402887752242,0.11142805775504484,0.9319962481713715,0.05977924158690171,eta_l0_n177_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,177,1,-0.8553465281258207,0.839658428320634,-0.007844049902593353,0.8475024782232273,1.6950049564464547,affine,0.23785857921394304,0.4757171584278861,1.0,0.23785857921394304,eta_l0_n177_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,178,0,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.09034133178274757,0.18068266356549514,0.9319962481713715,0.09693314963445646,eta_l0_n178_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,178,1,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.304261390372095,0.60852278074419,1.0,0.304261390372095,eta_l0_n178_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,179,0,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.12411459017998151,0.24822918035996303,0.9319962481713715,0.1331706972249097,eta_l0_n179_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,179,1,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.35189623949265825,0.7037924789853165,1.0,0.35189623949265825,eta_l0_n179_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,180,0,-1.0633221779271937,1.0983992430851335,0.017538532578969868,1.0808607105061636,2.1617214210123272,affine,0.09693932115397964,0.1938786423079593,0.9319962481713715,0.1040125658704957,eta_l0_n180_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,180,1,-1.0633221779271937,1.0983992430851335,0.017538532578969868,1.0808607105061636,2.1617214210123272,affine,0.3147125704027749,0.6294251408055498,1.0,0.3147125704027749,eta_l0_n180_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,181,0,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.10855051376169962,0.21710102752339924,0.9319962481713715,0.11647097718974918,eta_l0_n181_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,181,1,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.3315986341833693,0.6631972683667386,1.0,0.3315986341833693,eta_l0_n181_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,182,0,-0.9468539302832147,0.9082390537521353,-0.01930743826553971,0.927546492017675,1.85509298403535,affine,0.06897207704411946,0.13794415408823893,0.9319962481713715,0.07400467242163959,eta_l0_n182_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,182,1,-0.9468539302832147,0.9082390537521353,-0.01930743826553971,0.927546492017675,1.85509298403535,affine,0.2658832386164963,0.5317664772329926,1.0,0.2658832386164963,eta_l0_n182_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,183,0,-0.9012005330706286,0.9364696817911623,0.01763457436026683,0.9188351074308955,1.837670214861791,affine,0.06747144526222183,0.13494289052444366,0.9319962481713715,0.07239454600230909,eta_l0_n183_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,183,1,-0.9012005330706286,0.9364696817911623,0.01763457436026683,0.9188351074308955,1.837670214861791,affine,0.26292614612453097,0.5258522922490619,1.0,0.26292614612453097,eta_l0_n183_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,184,0,-1.4793999358375651,1.4405298344240638,-0.019435050706750667,1.4599648851308145,2.919929770261629,affine,0.17427052042194865,0.3485410408438973,0.9319962481713715,0.18698628965929542,eta_l0_n184_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,184,1,-1.4793999358375651,1.4405298344240638,-0.019435050706750667,1.4599648851308145,2.919929770261629,affine,0.4027895634419543,0.8055791268839086,1.0,0.4027895634419543,eta_l0_n184_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,185,0,-0.7321511044331092,0.7389444504690766,0.0033966730179837423,0.7355477774510929,1.4710955549021858,affine,0.03929340910885843,0.07858681821771686,0.9319962481713715,0.04216047992248283,eta_l0_n185_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,185,1,-0.7321511044331092,0.7389444504690766,0.0033966730179837423,0.7355477774510929,1.4710955549021858,affine,0.1962133109401037,0.3924266218802074,1.0,0.1962133109401037,eta_l0_n185_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,186,0,-0.8942204521232298,0.8708685501338451,-0.011675950994692319,0.8825445011285374,1.7650890022570749,affine,0.06136968976427765,0.1227393795285553,0.9319962481713715,0.06584757168785647,eta_l0_n186_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,186,1,-0.8942204521232298,0.8708685501338451,-0.011675950994692319,0.8825445011285374,1.7650890022570749,affine,0.2503530688112541,0.5007061376225082,1.0,0.2503530688112541,eta_l0_n186_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,187,0,-1.0714555174736886,1.0473124800913576,-0.012071518691165517,1.059383998782523,2.118767997565046,affine,0.09281648755288631,0.18563297510577262,0.9319962481713715,0.09958890685986925,eta_l0_n187_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,187,1,-1.0714555174736886,1.0473124800913576,-0.012071518691165517,1.059383998782523,2.118767997565046,affine,0.3083893023272184,0.6167786046544368,1.0,0.3083893023272184,eta_l0_n187_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,188,0,-0.8561385666094258,0.8238401922534623,-0.01614918717798175,0.839989379431444,1.679978758862888,affine,0.05457352556779011,0.10914705113558022,0.9319962481713715,0.058555520663164044,eta_l0_n188_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,188,1,-0.8561385666094258,0.8238401922534623,-0.01614918717798175,0.839989379431444,1.679978758862888,affine,0.23504641413088656,0.4700928282617731,1.0,0.23504641413088656,eta_l0_n188_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,189,0,-1.055476274410492,1.0911960330035593,0.017859879296533654,1.0733361537070256,2.1466723074140512,affine,0.09549999712492331,0.19099999424984662,0.9319962481713715,0.10246822056665959,eta_l0_n189_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,189,1,-1.055476274410492,1.0911960330035593,0.017859879296533654,1.0733361537070256,2.1466723074140512,affine,0.31248672028935975,0.6249734405787195,1.0,0.31248672028935975,eta_l0_n189_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,190,0,-0.8409602339792388,0.8183358148780697,-0.01131220955058454,0.8296480244286543,1.6592960488573085,affine,0.052940748618230955,0.10588149723646191,0.9319962481713715,0.05680360701247848,eta_l0_n190_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,190,1,-0.8409602339792388,0.8183358148780697,-0.01131220955058454,0.8296480244286543,1.6592960488573085,affine,0.23133788022140234,0.4626757604428047,1.0,0.23133788022140234,eta_l0_n190_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,191,0,-0.9923663640591593,0.9567994816892675,-0.017783441184945903,0.9745829228742134,1.9491658457484269,affine,0.07720636019263961,0.15441272038527923,0.9319962481713715,0.08283977574386463,eta_l0_n191_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,191,1,-0.9923663640591593,0.9567994816892675,-0.017783441184945903,0.9745829228742134,1.9491658457484269,affine,0.2816476439026049,0.5632952878052098,1.0,0.2816476439026049,eta_l0_n191_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,192,0,-0.622907066400142,0.5904531146635353,-0.01622697586830335,0.6066800905318387,1.2133601810636774,affine,0.02390234411526149,0.04780468823052298,0.9319962481713715,0.025646395210451995,eta_l0_n192_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,192,1,-0.622907066400142,0.5904531146635353,-0.01622697586830335,0.6066800905318387,1.2133601810636774,affine,0.14666339508907572,0.29332679017815144,1.0,0.14666339508907572,eta_l0_n192_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,193,0,-0.7374416418193389,0.7107433024551729,-0.013349169682082995,0.7240924721372559,1.4481849442745118,affine,0.037799429557977056,0.07559885911595411,0.9319962481713715,0.04055749111881259,eta_l0_n193_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,193,1,-0.7374416418193389,0.7107433024551729,-0.013349169682082995,0.7240924721372559,1.4481849442745118,affine,0.19177416905298977,0.38354833810597955,1.0,0.19177416905298977,eta_l0_n193_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,194,0,-0.9262917881438898,0.9624928742496369,0.018100543052873563,0.9443923311967634,1.8887846623935267,affine,0.07187781562146196,0.14375563124292393,0.9319962481713715,0.07712243022704247,eta_l0_n194_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,194,1,-0.9262917881438898,0.9624928742496369,0.018100543052873563,0.9443923311967634,1.8887846623935267,affine,0.2716137525954708,0.5432275051909417,1.0,0.2716137525954708,eta_l0_n194_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,195,0,-1.4438322648629722,1.4816268957431415,0.01889731544008466,1.4627295803030569,2.9254591606061138,affine,0.17484682179969427,0.34969364359938854,0.9319962481713715,0.18760464126626417,eta_l0_n195_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,195,1,-1.4438322648629722,1.4816268957431415,0.01889731544008466,1.4627295803030569,2.9254591606061138,affine,0.4032760717116363,0.8065521434232726,1.0,0.4032760717116363,eta_l0_n195_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,196,0,-1.1524474377340879,1.1941743187475986,0.02086344050675537,1.1733108782408432,2.3466217564816865,affine,0.11510051619594604,0.23020103239189207,0.9319962481713715,0.12349890508870573,eta_l0_n196_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,196,1,-1.1524474377340879,1.1941743187475986,0.02086344050675537,1.1733108782408432,2.3466217564816865,affine,0.34042627982208706,0.6808525596441741,1.0,0.34042627982208706,eta_l0_n196_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,197,0,-0.6221827602303874,0.6073776418168665,-0.007402559206760473,0.614780201023627,1.229560402047254,affine,0.024711480698816016,0.04942296139763203,0.9319962481713715,0.026514571005303203,eta_l0_n197_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,197,1,-0.6221827602303874,0.6073776418168665,-0.007402559206760473,0.614780201023627,1.229560402047254,affine,0.14984737210450283,0.29969474420900566,1.0,0.14984737210450283,eta_l0_n197_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,198,0,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,affine,0.06546697285142163,0.13093394570284325,0.9319962481713715,0.07024381587358476,eta_l0_n198_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,198,1,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,affine,0.2588549749285447,0.5177099498570894,1.0,0.2588549749285447,eta_l0_n198_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,199,0,-1.0952666713142436,1.1346180087386013,0.01967566871217885,1.1149423400264225,2.229884680052845,affine,0.10355036232782984,0.20710072465565968,0.9319962481713715,0.11110598624297191,eta_l0_n199_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,199,1,-1.0952666713142436,1.1346180087386013,0.01967566871217885,1.1149423400264225,2.229884680052845,affine,0.3245088156471295,0.649017631294259,1.0,0.3245088156471295,eta_l0_n199_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,200,0,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.08060765055937986,0.16121530111875973,0.9319962481713715,0.08648924361824048,eta_l0_n200_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,200,1,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.287807187210534,0.575614374421068,1.0,0.287807187210534,eta_l0_n200_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,201,0,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.10398467023247389,0.20796934046494778,0.9319962481713715,0.11157198372471734,eta_l0_n201_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,201,1,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.32518038985547915,0.6503607797109583,1.0,0.32518038985547915,eta_l0_n201_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,202,0,-0.7951331280804155,0.8148313974508754,0.009849134685229965,0.8049822627656454,1.6099645255312909,affine,0.049187121407205665,0.09837424281441133,0.9319962481713715,0.05277609379191551,eta_l0_n202_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,202,1,-0.7951331280804155,0.8148313974508754,0.009849134685229965,0.8049822627656454,1.6099645255312909,affine,0.22227638354605228,0.44455276709210456,1.0,0.22227638354605228,eta_l0_n202_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,203,0,-0.9291054457961189,0.8942568543418766,-0.01742429572712112,0.9116811500689977,1.8233623001379955,affine,0.0662563014773734,0.1325126029547468,0.9319962481713715,0.07109073840948603,eta_l0_n203_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,203,1,-0.9291054457961189,0.8942568543418766,-0.01742429572712112,0.9116811500689977,1.8233623001379955,affine,0.26046312975463887,0.5209262595092777,1.0,0.26046312975463887,eta_l0_n203_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,204,0,-0.7641244024694367,0.7349866898999595,-0.014568856284738585,0.7495555461846981,1.4991110923693962,affine,0.041244716478901274,0.08248943295780255,0.9319962481713715,0.04425416578642425,eta_l0_n204_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,204,1,-0.7641244024694367,0.7349866898999595,-0.014568856284738585,0.7495555461846981,1.4991110923693962,affine,0.20144524470009967,0.40289048940019934,1.0,0.20144524470009967,eta_l0_n204_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,205,0,-1.0117373774449647,0.9756317416300787,-0.01805281790744301,0.9936845595375217,1.9873691190750433,affine,0.08064859882773769,0.16129719765547537,0.9319962481713715,0.08653317970536333,eta_l0_n205_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,205,1,-1.0117373774449647,0.9756317416300787,-0.01805281790744301,0.9936845595375217,1.9873691190750433,affine,0.2878479178458295,0.575695835691659,1.0,0.2878479178458295,eta_l0_n205_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,206,0,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.13911287947176756,0.27822575894353513,0.9319962481713715,0.14926334708397676,eta_l0_n206_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,206,1,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.3691976371738509,0.7383952743477018,1.0,0.3691976371738509,eta_l0_n206_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,207,0,-1.3112708446293027,1.2772121752583023,-0.017029334685500164,1.2942415099438025,2.588483019887605,affine,0.13971997178468812,0.27943994356937624,0.9319962481713715,0.14991473630803395,eta_l0_n207_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,207,1,-1.3112708446293027,1.2772121752583023,-0.017029334685500164,1.2942415099438025,2.588483019887605,affine,0.3699089369805241,0.7398178739610481,1.0,0.3699089369805241,eta_l0_n207_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,208,0,-1.1651989100051086,1.200260436473233,0.01753076323406222,1.1827296732391708,2.3654593464783416,affine,0.11697091939156241,0.23394183878312483,0.9319962481713715,0.12550578354909248,eta_l0_n208_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,208,1,-1.1651989100051086,1.200260436473233,0.01753076323406222,1.1827296732391708,2.3654593464783416,affine,0.3429378820617447,0.6858757641234894,1.0,0.3429378820617447,eta_l0_n208_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,209,0,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,affine,0.12336957207392958,0.24673914414785916,0.9319962481713715,0.13237131835668603,eta_l0_n209_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,209,1,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,affine,0.35104070176541363,0.7020814035308273,1.0,0.35104070176541363,eta_l0_n209_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,210,0,-0.9228447532260015,0.9507075443892258,0.01393139558161216,0.9367761488076136,1.8735522976152272,affine,0.07053047421855775,0.1410609484371155,0.9319962481713715,0.07567677912539075,eta_l0_n210_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,210,1,-0.9228447532260015,0.9507075443892258,0.01393139558161216,0.9367761488076136,1.8735522976152272,affine,0.2691003564686259,0.5382007129372518,1.0,0.2691003564686259,eta_l0_n210_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,211,0,-1.3219320199504636,1.2845693941220586,-0.018681312914202497,1.3032507070362611,2.6065014140725222,affine,0.14158975427736184,0.2831795085547237,0.9319962481713715,0.15192094877545786,eta_l0_n211_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,211,1,-1.3219320199504636,1.2845693941220586,-0.018681312914202497,1.3032507070362611,2.6065014140725222,affine,0.3718937045390255,0.743787409078051,1.0,0.3718937045390255,eta_l0_n211_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,212,0,-1.1136630602458557,1.0736732395863258,-0.01999491032976497,1.0936681499160907,2.1873362998321815,affine,0.09941980308099137,0.19883960616198273,0.9319962481713715,0.10667403787950708,eta_l0_n212_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,212,1,-1.1136630602458557,1.0736732395863258,-0.01999491032976497,1.0936681499160907,2.1873362998321815,affine,0.3184140898770352,0.6368281797540704,1.0,0.3184140898770352,eta_l0_n212_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,213,0,-1.397693525091097,1.3709670065547253,-0.013363259268185823,1.384330265822911,2.768660531645822,affine,0.1584389554911889,0.3168779109823778,0.9319962481713715,0.1699995636270585,eta_l0_n213_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,213,1,-1.397693525091097,1.3709670065547253,-0.013363259268185823,1.384330265822911,2.768660531645822,affine,0.38883905368764476,0.7776781073752895,1.0,0.38883905368764476,eta_l0_n213_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,214,0,-0.6308511366333199,0.5817134666571077,-0.024568834988106136,0.6062823016452138,1.2125646032904276,affine,0.02393379841437915,0.0478675968287583,0.9319962481713715,0.02568014459429273,eta_l0_n214_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,214,1,-0.6308511366333199,0.5817134666571077,-0.024568834988106136,0.6062823016452138,1.2125646032904276,affine,0.14637875612598206,0.2927575122519641,1.0,0.14637875612598206,eta_l0_n214_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,215,0,-0.6750710083304649,0.648698160588605,-0.013186423870929942,0.6618845844595349,1.3237691689190698,affine,0.030018619591721033,0.060037239183442066,0.9319962481713715,0.03220894896371014,eta_l0_n215_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,215,1,-0.6750710083304649,0.648698160588605,-0.013186423870929942,0.6618845844595349,1.3237691689190698,affine,0.16790532846119766,0.3358106569223953,1.0,0.16790532846119766,eta_l0_n215_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,216,0,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.13353496019544117,0.26706992039088234,0.9319962481713715,0.1432784310639063,eta_l0_n216_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,216,1,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.3629650350912027,0.7259300701824054,1.0,0.3629650350912027,eta_l0_n216_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,217,0,-1.0176420196916924,0.9812025096472035,-0.018219755022244488,0.999422264669448,1.998844529338896,affine,0.08169289794408681,0.16338579588817362,0.9319962481713715,0.0876536768301083,eta_l0_n217_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,217,1,-1.0176420196916924,0.9812025096472035,-0.018219755022244488,0.999422264669448,1.998844529338896,affine,0.28968717014881595,0.5793743402976319,1.0,0.28968717014881595,eta_l0_n217_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,218,0,-0.9798393779987576,1.0123201516079219,0.01624038680458212,0.9960797648033397,1.9921595296066794,affine,0.08107290966168675,0.1621458193233735,0.9319962481713715,0.08698845067322568,eta_l0_n218_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,218,1,-0.9798393779987576,1.0123201516079219,0.01624038680458212,0.9960797648033397,1.9921595296066794,affine,0.28864444116563753,0.5772888823312751,1.0,0.28864444116563753,eta_l0_n218_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,219,0,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.08893857072936846,0.17787714145873693,0.9319962481713715,0.09542803514914452,eta_l0_n219_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,219,1,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.30212709905295965,0.6042541981059193,1.0,0.30212709905295965,eta_l0_n219_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,220,0,-1.0276272819085601,0.9881549383021562,-0.019736171803201974,1.007891110105358,2.015782220210716,affine,0.08325051921200739,0.16650103842401479,0.9319962481713715,0.08932495101278524,eta_l0_n220_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,220,1,-1.0276272819085601,0.9881549383021562,-0.019736171803201974,1.007891110105358,2.015782220210716,affine,0.2923627454325827,0.5847254908651655,1.0,0.2923627454325827,eta_l0_n220_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,221,0,-0.918271405219569,0.9526902849630147,0.017209439871722854,0.9354808450912918,1.8709616901825836,affine,0.07032498333747385,0.1406499666749477,0.9319962481713715,0.0754562944598279,eta_l0_n221_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,221,1,-0.918271405219569,0.9526902849630147,0.017209439871722854,0.9354808450912918,1.8709616901825836,affine,0.2686165914228864,0.5372331828457728,1.0,0.2686165914228864,eta_l0_n221_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,222,0,-1.093166004600465,1.1314416045523363,0.019137799975935676,1.1123038045764007,2.2246076091528013,affine,0.1030323751704313,0.2060647503408626,0.9319962481713715,0.11055020379383132,eta_l0_n222_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,222,1,-1.093166004600465,1.1314416045523363,0.019137799975935676,1.1123038045764007,2.2246076091528013,affine,0.32376979893757774,0.6475395978751555,1.0,0.32376979893757774,eta_l0_n222_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,223,0,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,affine,0.0371415219972684,0.0742830439945368,0.9319962481713715,0.03985157887721343,eta_l0_n223_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,223,1,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,affine,0.18978987632131178,0.37957975264262356,1.0,0.18978987632131178,eta_l0_n223_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,224,0,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.13683906908689789,0.27367813817379577,0.9319962481713715,0.1468236265493383,eta_l0_n224_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,224,1,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.36670914743226213,0.7334182948645243,1.0,0.36670914743226213,eta_l0_n224_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,225,0,-0.8019294265090884,0.8378016275339952,0.017936100512453423,0.8198655270215418,1.6397310540430836,affine,0.051479085767128514,0.10295817153425703,0.9319962481713715,0.055235292918971876,eta_l0_n225_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,225,1,-0.8019294265090884,0.8378016275339952,0.017936100512453423,0.8198655270215418,1.6397310540430836,affine,0.2276662882782467,0.4553325765564934,1.0,0.2276662882782467,eta_l0_n225_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,226,0,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.0847732214456068,0.1695464428912136,0.9319962481713715,0.09095875826961383,eta_l0_n226_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,226,1,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.2949929960678926,0.5899859921357852,1.0,0.2949929960678926,eta_l0_n226_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,227,0,-1.222732562277716,1.2664926147953328,0.02188002625880836,1.2446125885365245,2.489225177073049,affine,0.12953797657304725,0.2590759531460945,0.9319962481713715,0.1389898047628496,eta_l0_n227_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,227,1,-1.222732562277716,1.2664926147953328,0.02188002625880836,1.2446125885365245,2.489225177073049,affine,0.35834507775656466,0.7166901555131293,1.0,0.35834507775656466,eta_l0_n227_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,228,0,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,affine,0.08376097929774685,0.1675219585954937,0.9319962481713715,0.08987265717226926,eta_l0_n228_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,228,1,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,affine,0.2932994718153777,0.5865989436307554,1.0,0.2932994718153777,eta_l0_n228_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,229,0,-0.7782243522405144,0.7634707519083993,-0.007376800166057573,0.7708475520744569,1.5416951041489138,affine,0.04419558317224093,0.08839116634448187,0.9319962481713715,0.04742034451206765,eta_l0_n229_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,229,1,-0.7782243522405144,0.7634707519083993,-0.007376800166057573,0.7708475520744569,1.5416951041489138,affine,0.20955349419685776,0.4191069883937155,1.0,0.20955349419685776,eta_l0_n229_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,230,0,-1.4736909184242275,1.4349512883956,-0.019369815014313785,1.4543211034099137,2.9086422068198274,affine,0.1730901226391359,0.3461802452782718,0.9319962481713715,0.18571976333461465,eta_l0_n230_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,230,1,-1.4736909184242275,1.4349512883956,-0.019369815014313785,1.4543211034099137,2.9086422068198274,affine,0.4018009870697764,0.8036019741395528,1.0,0.4018009870697764,eta_l0_n230_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,231,0,-0.9741772249753825,1.0156090479613666,0.020715911492992067,0.9948931364683746,1.9897862729367493,affine,0.08088571108096311,0.16177142216192622,0.9319962481713715,0.08678759301838969,eta_l0_n231_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,231,1,-0.9741772249753825,1.0156090479613666,0.020715911492992067,0.9948931364683746,1.9897862729367493,affine,0.28819280957062676,0.5763856191412535,1.0,0.28819280957062676,eta_l0_n231_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,232,0,-0.9915097879699294,1.031502035920235,0.01999612397515277,1.0115059119450822,2.0230118238901644,affine,0.08391577855064876,0.16783155710129752,0.9319962481713715,0.09003875145988643,eta_l0_n232_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,232,1,-0.9915097879699294,1.031502035920235,0.01999612397515277,1.0115059119450822,2.0230118238901644,affine,0.29350402543245935,0.5870080508649187,1.0,0.29350402543245935,eta_l0_n232_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,233,0,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.09369831588937973,0.18739663177875945,0.9319962481713715,0.10053507841176511,eta_l0_n233_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,233,1,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.30969189457663343,0.6193837891532669,1.0,0.30969189457663343,eta_l0_n233_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,234,0,-0.9553499690208584,0.9911598413697276,0.01790493617443456,0.973254905195293,1.946509810390586,affine,0.0769698294054751,0.1539396588109502,0.9319962481713715,0.08258598632397307,eta_l0_n234_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,234,1,-0.9553499690208584,0.9911598413697276,0.01790493617443456,0.973254905195293,1.946509810390586,affine,0.28121030953724074,0.5624206190744815,1.0,0.28121030953724074,eta_l0_n234_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,235,0,-0.9690273603532127,1.0071673197752016,0.019069979710994445,0.9880973400642071,1.9761946801284143,affine,0.07964352957996403,0.15928705915992805,0.9319962481713715,0.08545477488372841,eta_l0_n235_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,235,1,-0.9690273603532127,1.0071673197752016,0.019069979710994445,0.9880973400642071,1.9761946801284143,affine,0.28602852557731817,0.5720570511546363,1.0,0.28602852557731817,eta_l0_n235_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,236,0,-1.1558838037706318,1.191758770838782,0.017937483534075094,1.173821287304707,2.347642574609414,affine,0.11518710792645188,0.23037421585290377,0.9319962481713715,0.12359181504480882,eta_l0_n236_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,236,1,-1.1558838037706318,1.191758770838782,0.017937483534075094,1.173821287304707,2.347642574609414,affine,0.34060209629286436,0.6812041925857287,1.0,0.34060209629286436,eta_l0_n236_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,237,0,-0.9032024251467424,0.9272904688323386,0.01204402184279807,0.9152464469895405,1.830492893979081,affine,0.06683064114202937,0.13366128228405874,0.9319962481713715,0.0717069851656107,eta_l0_n237_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,237,1,-0.9032024251467424,0.9272904688323386,0.01204402184279807,0.9152464469895405,1.830492893979081,affine,0.2617639868026934,0.5235279736053868,1.0,0.2617639868026934,eta_l0_n237_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,238,0,-1.2475163628525583,1.2970872348018667,0.024785435974654213,1.2723017988272125,2.544603597654425,affine,0.1352337806803164,0.2704675613606328,0.9319962481713715,0.14510120716220973,eta_l0_n238_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,238,1,-1.2475163628525583,1.2970872348018667,0.024785435974654213,1.2723017988272125,2.544603597654425,affine,0.3648170558558874,0.7296341117117748,1.0,0.3648170558558874,eta_l0_n238_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,239,0,-0.9472028721092912,0.9617653086795485,0.007281218285128621,0.9544840903944198,1.9089681807888397,affine,0.07359532437748174,0.1471906487549635,0.9319962481713715,0.0789652581991396,eta_l0_n239_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,239,1,-0.9472028721092912,0.9617653086795485,0.007281218285128621,0.9544840903944198,1.9089681807888397,affine,0.27511529379093747,0.5502305875818749,1.0,0.27511529379093747,eta_l0_n239_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,240,0,-1.139020179466013,1.1757004326923162,0.01834012661315154,1.1573603060791646,2.3147206121583292,affine,0.11190478616491507,0.22380957232983015,0.9319962481713715,0.12006999640232296,eta_l0_n240_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,240,1,-1.139020179466013,1.1757004326923162,0.01834012661315154,1.1573603060791646,2.3147206121583292,affine,0.3362207708362268,0.6724415416724536,1.0,0.3362207708362268,eta_l0_n240_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,241,0,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.05101514148501563,0.10203028297003126,0.9319962481713715,0.054737496620946895,eta_l0_n241_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,241,1,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.2266731457928317,0.4533462915856634,1.0,0.2266731457928317,eta_l0_n241_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,242,0,-0.8216464738019207,0.8370838065220392,0.007718666360059245,0.8293651401619799,1.6587302803239599,affine,0.05288344570803522,0.10576689141607044,0.9319962481713715,0.05674212295574739,eta_l0_n242_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,242,1,-0.8216464738019207,0.8370838065220392,0.007718666360059245,0.8293651401619799,1.6587302803239599,affine,0.23126511064546856,0.4625302212909371,1.0,0.23126511064546856,eta_l0_n242_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,243,0,-1.094745104288666,1.126225789433019,0.01574034257217649,1.1104854468608425,2.220970893721685,affine,0.10266022844570051,0.20532045689140102,0.9319962481713715,0.11015090312554969,eta_l0_n243_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,243,1,-1.094745104288666,1.126225789433019,0.01574034257217649,1.1104854468608425,2.220970893721685,affine,0.32329968942006604,0.6465993788401321,1.0,0.32329968942006604,eta_l0_n243_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,244,0,-1.3579344396070396,1.4019798358364988,0.022022698114729566,1.3799571377217692,2.7599142754435384,affine,0.15755628942850283,0.31511257885700567,0.9319962481713715,0.16905249322369809,eta_l0_n244_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,244,1,-1.3579344396070396,1.4019798358364988,0.022022698114729566,1.3799571377217692,2.7599142754435384,affine,0.38789260455289937,0.7757852091057987,1.0,0.38789260455289937,eta_l0_n244_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,245,0,-1.0757844619013353,1.058689201892231,-0.008547630004552165,1.067236831896783,2.134473663793566,affine,0.09429752179835456,0.18859504359670912,0.9319962481713715,0.10117800579494987,eta_l0_n245_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,245,1,-1.0757844619013353,1.058689201892231,-0.008547630004552165,1.067236831896783,2.134473663793566,affine,0.31077188650760995,0.6215437730152199,1.0,0.31077188650760995,eta_l0_n245_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,246,0,-0.8996611244567155,0.9223465961082321,0.011342735825758288,0.9110038602824738,1.8220077205649476,affine,0.06610913113535623,0.13221826227071246,0.9319962481713715,0.07093282968152073,eta_l0_n246_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,246,1,-0.8996611244567155,0.9223465961082321,0.011342735825758288,0.9110038602824738,1.8220077205649476,affine,0.26030628422287794,0.5206125684457559,1.0,0.26030628422287794,eta_l0_n246_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,247,0,-0.8285968972008307,0.8196796009213531,-0.004458648139738841,0.8241382490610919,1.6482764981221838,affine,0.05207163037664759,0.10414326075329518,0.9319962481713715,0.0558710729563719,eta_l0_n247_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,247,1,-0.8285968972008307,0.8196796009213531,-0.004458648139738841,0.8241382490610919,1.6482764981221838,affine,0.22936955208021276,0.4587391041604255,1.0,0.22936955208021276,eta_l0_n247_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,248,0,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,affine,0.07260971125925632,0.14521942251851264,0.9319962481713715,0.0779077291370224,eta_l0_n248_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,248,1,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,affine,0.27311766338374865,0.5462353267674973,1.0,0.27311766338374865,eta_l0_n248_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,249,0,-0.8971474300455685,0.9299772734367954,0.016414921695613427,0.9135623517411819,1.8271247034823639,affine,0.06656829091883472,0.13313658183766944,0.9319962481713715,0.07142549237665431,eta_l0_n249_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,249,1,-0.8971474300455685,0.9299772734367954,0.016414921695613427,0.9135623517411819,1.8271247034823639,affine,0.2611280939599575,0.522256187919915,1.0,0.2611280939599575,eta_l0_n249_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,250,0,-0.7261210591057342,0.7589229120372533,0.016400926465759524,0.7425219855714937,1.4850439711429875,affine,0.04029261137626906,0.08058522275253811,0.9319962481713715,0.043232589675468544,eta_l0_n250_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,250,1,-0.7261210591057342,0.7589229120372533,0.016400926465759524,0.7425219855714937,1.4850439711429875,affine,0.19875206432670667,0.39750412865341334,1.0,0.19875206432670667,eta_l0_n250_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,251,0,-0.6364866647888496,0.6638130918043734,0.013663213507761895,0.6501498782966115,1.300299756593223,affine,0.028655556908585626,0.05731111381717125,0.9319962481713715,0.030746429467725243,eta_l0_n251_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,251,1,-0.6364866647888496,0.6638130918043734,0.013663213507761895,0.6501498782966115,1.300299756593223,affine,0.1633843769776558,0.3267687539553116,1.0,0.1633843769776558,eta_l0_n251_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,252,0,-1.0954772645448594,1.1341057511714407,0.01931424331329068,1.11479150785815,2.2295830157163,affine,0.10351882023537858,0.20703764047075715,0.9319962481713715,0.11107214265989618,eta_l0_n252_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,252,1,-1.0954772645448594,1.1341057511714407,0.01931424331329068,1.11479150785815,2.2295830157163,affine,0.32447162950315533,0.6489432590063107,1.0,0.32447162950315533,eta_l0_n252_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,253,0,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.04924698734954847,0.09849397469909695,0.9319962481713715,0.052840327894209664,eta_l0_n253_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,253,1,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.22222072025332534,0.4444414405066507,1.0,0.22222072025332534,eta_l0_n253_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,254,0,-0.9847557236564193,1.020181147346501,0.017712711845040863,1.0024684355014601,2.0049368710029203,affine,0.08224546002753337,0.16449092005506674,0.9319962481713715,0.08824655698872559,eta_l0_n254_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,254,1,-0.9847557236564193,1.020181147346501,0.017712711845040863,1.0024684355014601,2.0049368710029203,affine,0.29066857170882565,0.5813371434176513,1.0,0.29066857170882565,eta_l0_n254_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,255,0,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.13466273029786982,0.26932546059573964,0.9319962481713715,0.14448848969304928,eta_l0_n255_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,255,1,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.3642931767979276,0.7285863535958552,1.0,0.3642931767979276,eta_l0_n255_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,256,0,-0.8345849622272768,0.862889794074723,0.014152415923723072,0.8487373781509999,1.6974747563019998,affine,0.055936304533085686,0.11187260906617137,0.9319962481713715,0.0600177357396404,eta_l0_n256_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,256,1,-0.8345849622272768,0.862889794074723,0.014152415923723072,0.8487373781509999,1.6974747563019998,affine,0.2382428389544612,0.4764856779089224,1.0,0.2382428389544612,eta_l0_n256_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,257,0,-0.6351214718940504,0.5726601313824508,-0.03123067025579984,0.6038908016382506,1.2077816032765012,affine,0.023764750123252524,0.04752950024650505,0.9319962481713715,0.025498761577506655,eta_l0_n257_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,257,1,-0.6351214718940504,0.5726601313824508,-0.03123067025579984,0.6038908016382506,1.2077816032765012,affine,0.14532038190436053,0.29064076380872106,1.0,0.14532038190436053,eta_l0_n257_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,258,0,-0.9998125161825591,0.967562441340464,-0.01612503742104754,0.9836874787615115,1.967374957523023,affine,0.07883014658578365,0.1576602931715673,0.9319962481713715,0.0845820428359586,eta_l0_n258_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,258,1,-0.9998125161825591,0.967562441340464,-0.01612503742104754,0.9836874787615115,1.967374957523023,affine,0.28464282835773846,0.5692856567154769,1.0,0.28464282835773846,eta_l0_n258_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,259,0,-0.8697080952856967,0.8389415568536377,-0.015383269216029505,0.8543248260696672,1.7086496521393344,affine,0.05682902813904301,0.11365805627808602,0.9319962481713715,0.06097559754188359,eta_l0_n259_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,259,1,-0.8697080952856967,0.8389415568536377,-0.015383269216029505,0.8543248260696672,1.7086496521393344,affine,0.24024190927679612,0.48048381855359223,1.0,0.24024190927679612,eta_l0_n259_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,260,0,-0.8381656949954069,0.8252083496681949,-0.006478672663606022,0.8316870223318009,1.6633740446636018,affine,0.0532387319093588,0.1064774638187176,0.9319962481713715,0.05712333286085234,eta_l0_n260_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,260,1,-0.8381656949954069,0.8252083496681949,-0.006478672663606022,0.8316870223318009,1.6633740446636018,affine,0.2321211134765584,0.4642422269531168,1.0,0.2321211134765584,eta_l0_n260_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,261,0,-0.833623752115959,0.8706092838419001,0.01849276586297055,0.8521165179789295,1.704233035957859,affine,0.056499013590415766,0.11299802718083153,0.9319962481713715,0.06062150325311928,eta_l0_n261_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,261,1,-0.833623752115959,0.8706092838419001,0.01849276586297055,0.8521165179789295,1.704233035957859,affine,0.2393991173845697,0.4787982347691394,1.0,0.2393991173845697,eta_l0_n261_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,262,0,-0.9956755857415504,0.9684535524140813,-0.013611016663734543,0.9820645690778158,1.9641291381556316,affine,0.07852507002697835,0.1570501400539567,0.9319962481713715,0.0842547061547178,eta_l0_n262_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,262,1,-0.9956755857415504,0.9684535524140813,-0.013611016663734543,0.9820645690778158,1.9641291381556316,affine,0.28414712102329764,0.5682942420465953,1.0,0.28414712102329764,eta_l0_n262_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,263,0,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.08863715985000005,0.1772743197000001,0.9319962481713715,0.09510463161617988,eta_l0_n263_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,263,1,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.3015525088126787,0.6031050176253574,1.0,0.3015525088126787,eta_l0_n263_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,264,0,-0.7417572625689896,0.7667071698008693,0.012474953615939866,0.7542322161849294,1.5084644323698588,affine,0.04187984998861028,0.08375969997722056,0.9319962481713715,0.0449356422526173,eta_l0_n264_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,264,1,-0.7417572625689896,0.7667071698008693,0.012474953615939866,0.7542322161849294,1.5084644323698588,affine,0.20324099935783696,0.4064819987156739,1.0,0.20324099935783696,eta_l0_n264_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,265,0,-0.8885673887593571,0.8568782922249832,-0.015844548267186953,0.8727228404921702,1.7454456809843404,affine,0.05978837364507889,0.11957674729015778,0.9319962481713715,0.06415087374266475,eta_l0_n265_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,265,1,-0.8885673887593571,0.8568782922249832,-0.015844548267186953,0.8727228404921702,1.7454456809843404,affine,0.24682017229808212,0.49364034459616424,1.0,0.24682017229808212,eta_l0_n265_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,266,0,-1.097939440105263,1.0589211186241407,-0.019509160740561082,1.0784302793647018,2.1568605587294036,affine,0.09648454749320724,0.19296909498641449,0.9319962481713715,0.10352460933454968,eta_l0_n266_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,266,1,-1.097939440105263,1.0589211186241407,-0.019509160740561082,1.0784302793647018,2.1568605587294036,affine,0.31396790847240585,0.6279358169448117,1.0,0.31396790847240585,eta_l0_n266_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,267,0,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.07059108594245113,0.14118217188490226,0.9319962481713715,0.07574181342570291,eta_l0_n267_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,267,1,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.2691747136306971,0.5383494272613942,1.0,0.2691747136306971,eta_l0_n267_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,268,0,-0.8940710602099515,0.8588320503931359,-0.0176195049084078,0.8764515553015437,1.7529031106030875,affine,0.06040647288144008,0.12081294576288017,0.9319962481713715,0.06481407301795575,eta_l0_n268_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,268,1,-0.8940710602099515,0.8588320503931359,-0.0176195049084078,0.8764515553015437,1.7529031106030875,affine,0.24811831778222848,0.49623663556445696,1.0,0.24811831778222848,eta_l0_n268_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,269,0,-0.8629278062752647,0.8343272647993644,-0.01430027073795015,0.8486275355373145,1.697255071074629,affine,0.05591978122054115,0.1118395624410823,0.9319962481713715,0.06000000679215058,eta_l0_n269_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,269,1,-0.8629278062752647,0.8343272647993644,-0.01430027073795015,0.8486275355373145,1.697255071074629,affine,0.23820126653711426,0.47640253307422853,1.0,0.23820126653711426,eta_l0_n269_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,270,0,-0.970677453881365,1.0031156697628496,0.016219107940742328,0.9868965618221073,1.9737931236442146,affine,0.07940926081926343,0.15881852163852686,0.9319962481713715,0.08520341254062859,eta_l0_n270_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,270,1,-0.970677453881365,1.0031156697628496,0.016219107940742328,0.9868965618221073,1.9737931236442146,affine,0.2856827070911946,0.5713654141823892,1.0,0.2856827070911946,eta_l0_n270_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,271,0,-0.9732209605163527,0.9367859559488056,-0.0182175022837735,0.9550034582325791,1.9100069164651583,affine,0.07373664133513287,0.14747328267026574,0.9319962481713715,0.07911688644649403,eta_l0_n271_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,271,1,-0.9732209605163527,0.9367859559488056,-0.0182175022837735,0.9550034582325791,1.9100069164651583,affine,0.27516733078326927,0.5503346615665385,1.0,0.27516733078326927,eta_l0_n271_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,272,0,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.0785284283481333,0.1570568566962666,0.9319962481713715,0.08425830951810208,eta_l0_n272_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,272,1,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.28407907522631537,0.5681581504526307,1.0,0.28407907522631537,eta_l0_n272_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,273,0,-0.8018194571083685,0.7795301587031829,-0.011144649202592838,0.7906748079057757,1.5813496158115514,affine,0.047074883359198356,0.09414976671839671,0.9319962481713715,0.050509734831617505,eta_l0_n273_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,273,1,-0.8018194571083685,0.7795301587031829,-0.011144649202592838,0.7906748079057757,1.5813496158115514,affine,0.2169475124104976,0.4338950248209952,1.0,0.2169475124104976,eta_l0_n273_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,274,0,-1.3704733539073402,1.4030091777593323,0.01626791192599608,1.3867412658333362,2.7734825316666725,affine,0.15895094587993705,0.3179018917598741,0.9319962481713715,0.17054891174917028,eta_l0_n274_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,274,1,-1.3704733539073402,1.4030091777593323,0.01626791192599608,1.3867412658333362,2.7734825316666725,affine,0.38928688741470735,0.7785737748294147,1.0,0.38928688741470735,eta_l0_n274_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,275,0,-1.054322247914134,1.0201728471958655,-0.017074700359134276,1.0372475475549998,2.0744950951099996,affine,0.08867093565940438,0.17734187131880877,0.9319962481713715,0.09514087190091344,eta_l0_n275_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,275,1,-1.054322247914134,1.0201728471958655,-0.017074700359134276,1.0372475475549998,2.0744950951099996,affine,0.3015876428654989,0.6031752857309978,1.0,0.3015876428654989,eta_l0_n275_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,276,0,-1.1254872684487631,1.1656844838811946,0.020098607716215744,1.1455858761649789,2.2911717523299577,affine,0.10957807722918857,0.21915615445837713,0.9319962481713715,0.11757351753742233,eta_l0_n276_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,276,1,-1.1254872684487631,1.1656844838811946,0.020098607716215744,1.1455858761649789,2.2911717523299577,affine,0.333009874225618,0.666019748451236,1.0,0.333009874225618,eta_l0_n276_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,277,0,-1.1043859523878041,1.1426379378077016,0.019125992709948747,1.1235119450977529,2.2470238901955057,affine,0.10522390427298968,0.21044780854597936,0.9319962481713715,0.11290163933539951,eta_l0_n277_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,277,1,-1.1043859523878041,1.1426379378077016,0.019125992709948747,1.1235119450977529,2.2470238901955057,affine,0.32692761757395183,0.6538552351479037,1.0,0.32692761757395183,eta_l0_n277_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,278,0,-0.7530154612851288,0.7784502098291963,0.012717374272033788,0.7657328355571625,1.531465671114325,affine,0.04349218747112233,0.08698437494224466,0.9319962481713715,0.04666562505638453,eta_l0_n278_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,278,1,-0.7530154612851288,0.7784502098291963,0.012717374272033788,0.7657328355571625,1.531465671114325,affine,0.20758068201875263,0.41516136403750525,1.0,0.20758068201875263,eta_l0_n278_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,279,0,-0.9664557004775315,0.9399092435870476,-0.013273228445241925,0.9531824720322896,1.9063649440645791,affine,0.07338862410217874,0.14677724820435747,0.9319962481713715,0.07874347589507072,eta_l0_n279_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,279,1,-0.9664557004775315,0.9399092435870476,-0.013273228445241925,0.9531824720322896,1.9063649440645791,affine,0.2746272259394916,0.5492544518789833,1.0,0.2746272259394916,eta_l0_n279_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,280,0,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.10125806852097143,0.20251613704194285,0.9319962481713715,0.10864643363065613,eta_l0_n280_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,280,1,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.3211816722610421,0.6423633445220842,1.0,0.3211816722610421,eta_l0_n280_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,281,0,-0.7251348527214971,0.758782585137245,0.016823866207873905,0.741958718929371,1.483917437858742,affine,0.040218829577515165,0.08043765915503033,0.9319962481713715,0.04315342433666095,eta_l0_n281_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,281,1,-0.7251348527214971,0.758782585137245,0.016823866207873905,0.741958718929371,1.483917437858742,affine,0.19853197030781827,0.39706394061563655,1.0,0.19853197030781827,eta_l0_n281_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,282,0,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.1874287668164584,0.3748575336329168,0.9319962481713715,0.20110463661651434,eta_l0_n282_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,282,1,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.41327305208332726,0.8265461041666545,1.0,0.41327305208332726,eta_l0_n282_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,283,0,-0.8267574423081061,0.7934742831840171,-0.016641579562044484,0.8101158627460616,1.6202317254921232,affine,0.04999348869782195,0.0999869773956439,0.9319962481713715,0.05364129823045099,eta_l0_n283_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,283,1,-0.8267574423081061,0.7934742831840171,-0.016641579562044484,0.8101158627460616,1.6202317254921232,affine,0.2240945199925709,0.4481890399851418,1.0,0.2240945199925709,eta_l0_n283_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,284,0,-0.8807211491905227,0.9069751074714274,0.013126979140452355,0.893848128330975,1.78769625666195,affine,0.06324291874219658,0.12648583748439315,0.9319962481713715,0.06785748211570884,eta_l0_n284_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,284,1,-0.8807211491905227,0.9069751074714274,0.013126979140452355,0.893848128330975,1.78769625666195,affine,0.25431416160486126,0.5086283232097225,1.0,0.25431416160486126,eta_l0_n284_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,285,0,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.06959998889061066,0.1391999777812213,0.9319962481713715,0.0746784002909558,eta_l0_n285_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,285,1,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.26717870611859196,0.5343574122371839,1.0,0.26717870611859196,eta_l0_n285_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,286,0,-1.3222298401611514,1.2810144283174631,-0.020607705921844133,1.3016221342393073,2.6032442684786146,affine,0.14126131622843827,0.28252263245687653,0.9319962481713715,0.15156854601679012,eta_l0_n286_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,286,1,-1.3222298401611514,1.2810144283174631,-0.020607705921844133,1.3016221342393073,2.6032442684786146,affine,0.3715097252062655,0.743019450412531,1.0,0.3715097252062655,eta_l0_n286_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,287,0,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,affine,0.08700633057614827,0.17401266115229655,0.9319962481713715,0.09335480775470882,eta_l0_n287_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,287,1,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,affine,0.29874920593166787,0.5974984118633357,1.0,0.29874920593166787,eta_l0_n287_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,288,0,-0.9609248528732853,0.9933100955407531,0.016192621333733892,0.9771174742070192,1.9542349484140384,affine,0.07765047558068246,0.15530095116136491,0.9319962481713715,0.08331629631882854,eta_l0_n288_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,288,1,-0.9609248528732853,0.9933100955407531,0.016192621333733892,0.9771174742070192,1.9542349484140384,affine,0.28250049405507804,0.5650009881101561,1.0,0.28250049405507804,eta_l0_n288_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,289,0,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.12823730959170523,0.25647461918341047,0.9319962481713715,0.13759423371426008,eta_l0_n289_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,289,1,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.3568584940492191,0.7137169880984382,1.0,0.3568584940492191,eta_l0_n289_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,290,0,-1.2209164590474624,1.265004290450195,0.022043915701366323,1.2429603747488287,2.4859207494976574,affine,0.12920111825149752,0.25840223650299504,0.9319962481713715,0.13862836734053094,eta_l0_n290_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,290,1,-1.2209164590474624,1.265004290450195,0.022043915701366323,1.2429603747488287,2.4859207494976574,affine,0.35794597035653486,0.7158919407130697,1.0,0.35794597035653486,eta_l0_n290_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,291,0,-0.8933386502203347,0.9310906530126255,0.018876001396145425,0.9122146516164801,1.8244293032329602,affine,0.0663563654821971,0.1327127309643942,0.9319962481713715,0.07119810365373463,eta_l0_n291_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,291,1,-0.8933386502203347,0.9310906530126255,0.018876001396145425,0.9122146516164801,1.8244293032329602,affine,0.260624237096595,0.52124847419319,1.0,0.260624237096595,eta_l0_n291_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,292,0,-0.8645810082391754,0.838881124514349,-0.012849941862413172,0.8517310663767622,1.7034621327535244,affine,0.05640316746034286,0.11280633492068572,0.9319962481713715,0.06051866364377433,eta_l0_n292_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,292,1,-0.8645810082391754,0.838881124514349,-0.012849941862413172,0.8517310663767622,1.7034621327535244,affine,0.23933924698846587,0.47867849397693174,1.0,0.23933924698846587,eta_l0_n292_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,293,0,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.042222681286823445,0.08444536257364689,0.9319962481713715,0.045303488473979046,eta_l0_n293_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,293,1,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.2041600550732502,0.4083201101465004,1.0,0.2041600550732502,eta_l0_n293_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,294,0,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.09129259489700804,0.18258518979401608,0.9319962481713715,0.09795382232077564,eta_l0_n294_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,294,1,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.3058624608389326,0.6117249216778652,1.0,0.3058624608389326,eta_l0_n294_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,295,0,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.16854344612361224,0.3370868922472245,0.9319962481713715,0.1808413354177164,eta_l0_n295_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,295,1,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.3979471867839069,0.7958943735678138,1.0,0.3979471867839069,eta_l0_n295_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,296,0,-0.7369816136118256,0.7101516754046736,-0.01341496910357598,0.7235666445082496,1.4471332890164992,affine,0.03773032047057046,0.07546064094114092,0.9319962481713715,0.04048333943897248,eta_l0_n296_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,296,1,-0.7369816136118256,0.7101516754046736,-0.01341496910357598,0.7235666445082496,1.4471332890164992,affine,0.19157267510947118,0.38314535021894236,1.0,0.19157267510947118,eta_l0_n296_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,297,0,-0.9825135379376,1.01382408988808,0.01565527597523997,0.99816881391284,1.99633762782568,affine,0.08144974624643898,0.16289949249287797,0.9319962481713715,0.08739278340041381,eta_l0_n297_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,297,1,-0.9825135379376,1.01382408988808,0.01565527597523997,0.99816881391284,1.99633762782568,affine,0.2893225528047418,0.5786451056094836,1.0,0.2893225528047418,eta_l0_n297_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,298,0,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,affine,0.055178454356791036,0.11035690871358207,0.9319962481713715,0.05920458850028016,eta_l0_n298_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,298,1,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,affine,0.23657700326353948,0.47315400652707895,1.0,0.23657700326353948,eta_l0_n298_r1,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,299,0,-0.7428390480563718,0.7269071605470809,-0.007965943754645433,0.7348731043017264,1.4697462086034527,affine,0.03921345041839409,0.07842690083678817,0.9319962481713715,0.042074686990782484,eta_l0_n299_r0,NA,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,0,299,1,-0.7428390480563718,0.7269071605470809,-0.007965943754645433,0.7348731043017264,1.4697462086034527,affine,0.19593380084189632,0.39186760168379264,1.0,0.19593380084189632,eta_l0_n299_r1,NA,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/benchmark_metadata.json b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/benchmark_metadata.json new file mode 100644 index 0000000..118ac37 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/benchmark_metadata.json @@ -0,0 +1,13 @@ +{ + "schema_version": "1.2", + "benchmark_level": "medium", + "problem_id": "poisson_100d_ridge", + "model_id": "pinn_100d_poisson_shallow_300_seed_20260804", + "method_id": "affine_pz_topk96_symbolic", + "git_commit": "fb576a55aed822c24764f8d4db93e38495445e37", + "dtype": "float64", + "device": "cpu", + "quantile_interpolation": "linear", + "cell_average": "volume_weighted", + "timestamp_utc": "2026-08-04T11:32:52.998799+00:00" +} diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/cell_intervals.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/cell_intervals.csv new file mode 100644 index 0000000..eec9a39 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/cell_intervals.csv @@ -0,0 +1,102 @@ +run_id,split_id,cell_id,cell_weight,quantity,output_index,input_index_a,input_index_b,lower,upper,midpoint,radius,width,magnitude,mignitude,local_relative_radius,global_normalized_radius,sign_certified,status,local_relative_width +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,0,NA,-0.5291032689303383,0.7917320969045791,0.1313144139871204,0.6604176829174587,1.3208353658349175,0.7917320969045791,0.0,0.8341428691592547,0.6843924821460999,0,ok,1.6682857383185095 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,1,NA,-0.5592419162371889,0.817648798798601,0.12920344128070604,0.6884453575178949,1.3768907150357899,0.817648798798601,0.0,0.841981739017352,0.7134376308220677,0,ok,1.683963478034704 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,2,NA,-0.46167813695587107,0.7101561503491247,0.12423900669662682,0.5859171436524979,1.1718342873049958,0.7101561503491247,0.0,0.8250539594206868,0.6071873886005624,0,ok,1.6501079188413736 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,3,NA,-0.5426051194812953,0.8023746875764397,0.1298847840475722,0.6724899035288675,1.344979807057735,0.8023746875764397,0.0,0.8381245245411627,0.6969029542957228,0,ok,1.6762490490823254 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,4,NA,-0.4844542861354129,0.7377393715118,0.12664254268819355,0.6110968288236065,1.222193657647213,0.7377393715118,0.0,0.8283370149695638,0.633281158770048,0,ok,1.6566740299391276 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,5,NA,-0.5598917567230854,0.8284683071573953,0.13428827521715492,0.6941800319402404,1.3883600638804807,0.8284683071573953,0.0,0.8379077702104031,0.7193804881436202,0,ok,1.6758155404208062 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,6,NA,-0.5367277984723896,0.7933301233850784,0.12830116245634438,0.665028960928734,1.330057921857468,0.7933301233850784,0.0,0.8382751912799009,0.6891711609816829,0,ok,1.6765503825598018 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,7,NA,-0.6725105258712704,0.9259232858184864,0.126706379973608,0.7992169058448784,1.5984338116897567,0.9259232858184864,0.0,0.8631567194450632,0.8282304609833794,0,ok,1.7263134388901265 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,8,NA,-0.5465434092621674,0.8039457921499856,0.12870119144390912,0.6752446007060765,1.350489201412153,0.8039457921499856,0.0,0.8399130977478909,0.69975765381004,0,ok,1.6798261954957818 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,9,NA,-0.5073436576696282,0.7679342727221585,0.13029530752626517,0.6376389651958934,1.2752779303917867,0.7679342727221585,0.0,0.8303301309050879,0.660786840497168,0,ok,1.6606602618101758 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,10,NA,-0.5752451106752176,0.8421676081227122,0.13346124872374732,0.7087063593989649,1.4174127187979297,0.8421676081227122,0.0,0.8415264996700031,0.7344341573034541,0,ok,1.6830529993400063 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,11,NA,-0.5652918203948106,0.8271014954530932,0.13090483752914128,0.6961966579239519,1.3923933158479038,0.8271014954530932,0.0,0.8417306240542698,0.7214703226502549,0,ok,1.6834612481085396 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,12,NA,-0.5289614692046908,0.7891029097204242,0.1300707202578667,0.6590321894625575,1.318064378925115,0.7891029097204242,0.0,0.835166340593079,0.6829566918438049,0,ok,1.670332681186158 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,13,NA,-0.48222842328459575,0.7415022097687382,0.1296368932420712,0.611865316526667,1.223730633053334,0.7415022097687382,0.0,0.8251699165097529,0.6340775444820006,0,ok,1.6503398330195058 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,14,NA,-0.5991818133439206,0.8549375111417448,0.12787784889891207,0.7270596622428327,1.4541193244856654,0.8549375111417448,0.0,0.8504243325010563,0.7534537305429303,0,ok,1.7008486650021126 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,15,NA,-0.5377245530013925,0.793613413906181,0.12794443045239423,0.6656689834537868,1.3313379669075736,0.793613413906181,0.0,0.8387824245275175,0.6898344179111692,0,ok,1.677564849055035 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,16,NA,-0.5870500093141308,0.8445351721038633,0.12874258139486627,0.715792590708997,1.431585181417994,0.8445351721038633,0.0,0.8475580583883212,0.7417776363785601,0,ok,1.6951161167766424 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,17,NA,-0.4992684856719644,0.7599895426427712,0.1303605284854034,0.6296290141573678,1.2592580283147357,0.7599895426427712,0.0,0.8284706286456384,0.6524861083772947,0,ok,1.6569412572912767 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,18,NA,-0.5660375751412741,0.8117575456448274,0.12285998525177666,0.6888975603930507,1.3777951207861014,0.8117575456448274,0.0,0.8486494078054037,0.7139062497827144,0,ok,1.6972988156108073 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,19,NA,-0.460561271847678,0.7169362739544868,0.1281875010534044,0.5887487729010824,1.1774975458021648,0.7169362739544868,0.0,0.8212009829739175,0.6101218130111796,0,ok,1.642401965947835 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,20,NA,-0.5008108055447058,0.7503431977820736,0.12476619611868389,0.6255770016633897,1.2511540033267794,0.7503431977820736,0.0,0.8337211605469629,0.6482869977838444,0,ok,1.6674423210939258 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,21,NA,-0.6301561373389277,0.8902124499074757,0.130028156284274,0.7601842936232017,1.5203685872464034,0.8902124499074757,0.0,0.8539358146498754,0.7877808681665586,0,ok,1.707871629299751 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,22,NA,-0.5563864777940233,0.8031580286207072,0.12338577541334195,0.6797722532073652,1.3595445064147305,0.8031580286207072,0.0,0.8463742239802584,0.7044496713222956,0,ok,1.6927484479605168 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,23,NA,-0.48407143797880997,0.7429435736758709,0.12943606784853048,0.6135075058273405,1.227015011654681,0.7429435736758709,0.0,0.8257794098573085,0.6357793493256819,0,ok,1.651558819714617 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,24,NA,-0.5329262711247771,0.7901657552463681,0.12861974206079552,0.6615460131855726,1.3230920263711452,0.7901657552463681,0.0,0.8372243529831374,0.6855617736003561,0,ok,1.6744487059662747 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,25,NA,-0.5798147111679391,0.8381645189729167,0.12917490390248876,0.7089896150704279,1.4179792301408558,0.8381645189729167,0.0,0.8458835932821648,0.7347276958580521,0,ok,1.6917671865643296 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,26,NA,-0.5978342699142182,0.8549400530653876,0.1285528915755847,0.7263871614898029,1.4527743229796057,0.8549400530653876,0.0,0.8496351982637165,0.7527568163452698,0,ok,1.699270396527433 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,27,NA,-0.5238616559386213,0.7883154530919647,0.1322268985766717,0.656088554515293,1.312177109030586,0.7883154530919647,0.0,0.8322665145557585,0.6799061956499558,0,ok,1.664533029111517 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,28,NA,-0.4662703436819486,0.7243848813588267,0.12905726883843907,0.5953276125203877,1.1906552250407754,0.7243848813588267,0.0,0.8218388150283468,0.6169394808192359,0,ok,1.6436776300566935 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,29,NA,-0.567985299187635,0.8210279829137351,0.12652134186305009,0.694506641050685,1.38901328210137,0.8210279829137351,0.0,0.8458988676439015,0.7197189539745188,0,ok,1.691797735287803 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,30,NA,-0.6978840532373684,0.96496922474446,0.13354258575354583,0.8314266389909142,1.6628532779818284,0.96496922474446,0.0,0.8616094872985093,0.8616094872985093,0,ok,1.7232189745970186 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,31,NA,-0.5287241720506252,0.7851224534573921,0.12819914070338345,0.6569233127540086,1.3138466255080172,0.7851224534573921,0.0,0.8367144639172637,0.6807712576823088,0,ok,1.6734289278345273 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,32,NA,-0.5512151468762239,0.8021661550400181,0.12547550408189712,0.676690650958121,1.353381301916242,0.8021661550400181,0.0,0.843579159637273,0.7012561992713497,0,ok,1.687158319274546 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,33,NA,-0.5801238098577022,0.8390142170678001,0.12944520360504896,0.7095690134627511,1.4191380269255023,0.8390142170678001,0.0,0.8457175087480209,0.7353281278484886,0,ok,1.6914350174960417 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,34,NA,-0.49947984021358827,0.7561638140286967,0.12834198690755422,0.6278218271211424,1.2556436542422849,0.7561638140286967,0.0,0.830272244550063,0.6506133159711909,0,ok,1.660544489100126 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,35,NA,-0.5870587424614309,0.837744494348168,0.12534287594336857,0.7124016184047994,1.4248032368095989,0.837744494348168,0.0,0.8503805434843289,0.7382635633726612,0,ok,1.7007610869686578 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,36,NA,-0.5195971004524618,0.7766262174626231,0.12851455850508064,0.6481116589575424,1.2962233179150848,0.7766262174626231,0.0,0.8345219931861678,0.6716397190067623,0,ok,1.6690439863723356 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,37,NA,-0.599468511176408,0.861517649837326,0.13102456933045903,0.730493080506867,1.460986161013734,0.861517649837326,0.0,0.8479142367481393,0.7570117904022419,0,ok,1.6958284734962785 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,38,NA,-0.5126660464293469,0.7623252083693222,0.12482958096998764,0.6374956273993345,1.274991254798669,0.7623252083693222,0.0,0.8362515372710702,0.6606382991832243,0,ok,1.6725030745421403 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,39,NA,-0.4472838980117689,0.7065142254328275,0.1296151637105293,0.5768990617222982,1.1537981234445964,0.7065142254328275,0.0,0.8165427403374307,0.5978419279382415,0,ok,1.6330854806748614 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,40,NA,-0.5386275190384694,0.7882227390923923,0.12479761002696144,0.6634251290654308,1.3268502581308617,0.7882227390923923,0.0,0.8416721519977196,0.687509106045446,0,ok,1.6833443039954392 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,41,NA,-0.5257970551739006,0.7888983221542523,0.13155063349017582,0.6573476886640764,1.314695377328153,0.7888983221542523,0.0,0.8332476698252453,0.681211039490045,0,ok,1.6664953396504907 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,42,NA,-0.6047201869933004,0.8643838184822338,0.12983181574446667,0.7345520027377671,1.4691040054755342,0.8643838184822338,0.0,0.8497984194423751,0.7612180615731955,0,ok,1.6995968388847502 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,43,NA,-0.596309051567432,0.8598306115635793,0.13176077999807367,0.7280698315655056,1.4561396631310113,0.8598306115635793,0.0,0.8467596079668877,0.7545005715164758,0,ok,1.6935192159337753 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,44,NA,-0.47292519221055884,0.7395398025332199,0.13330730516133055,0.6062324973718893,1.2124649947437787,0.7395398025332199,0.0,0.8197428932091285,0.628240240026753,0,ok,1.639485786418257 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,45,NA,-0.5247425082957363,0.7889649197790652,0.13211120574166446,0.6568537140374008,1.3137074280748016,0.7889649197790652,0.0,0.8325512295544653,0.6806991323597358,0,ok,1.6651024591089305 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,46,NA,-0.6274520476324948,0.8943431014245257,0.1334455268960154,0.7608975745285103,1.5217951490570205,0.8943431014245257,0.0,0.8507893372426522,0.7885200429371296,0,ok,1.7015786744853043 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,47,NA,-0.5053508889655194,0.7652106863870478,0.1299298987107642,0.6352807876762836,1.2705615753525672,0.7652106863870478,0.0,0.8302037582299981,0.6583430552870912,0,ok,1.6604075164599963 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,48,NA,-0.5273994027025097,0.7871721684980735,0.12988638289778187,0.6572857856002916,1.3145715712005832,0.7871721684980735,0.0,0.834996220527454,0.6811468891915717,0,ok,1.669992441054908 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,49,NA,-0.5560230580672452,0.8175374804487954,0.1307572111907751,0.6867802692580203,1.3735605385160405,0.8175374804487954,0.0,0.840059673938136,0.7117120957301941,0,ok,1.680119347876272 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,50,NA,-0.4979154984025504,0.7608140364908171,0.13144926904413337,0.6293647674466838,1.2587295348933676,0.7608140364908171,0.0,0.8272254943528241,0.6522122688558799,0,ok,1.6544509887056482 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,51,NA,-0.48631342324535354,0.7478171096099594,0.13075184318230293,0.6170652664276565,1.234130532855313,0.7478171096099594,0.0,0.8251553200614794,0.6394662654563577,0,ok,1.6503106401229588 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,52,NA,-0.5750550897397898,0.8365300887361348,0.13073749949817248,0.7057925892379623,1.4115851784759246,0.8365300887361348,0.0,0.8437145283133853,0.7314146100616504,0,ok,1.6874290566267707 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,53,NA,-0.4898880127304422,0.749103920079287,0.12960795367442238,0.6194959664048646,1.2389919328097292,0.749103920079287,0.0,0.8269826786372919,0.6419852058690447,0,ok,1.6539653572745838 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,54,NA,-0.5372198995725416,0.7979251968543705,0.13035264864091445,0.667572548213456,1.335145096426912,0.7979251968543705,0.0,0.8366355027328394,0.6918070867910222,0,ok,1.6732710054656788 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,55,NA,-0.5156583697890303,0.7705896893930276,0.12746565980199864,0.643124029591029,1.286248059182058,0.7705896893930276,0.0,0.8345868604829116,0.6664710263286779,0,ok,1.6691737209658233 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,56,NA,-0.5305407605927447,0.789966921805321,0.12971308060628817,0.6602538411990329,1.3205076823980657,0.789966921805321,0.0,0.8357993518135503,0.684222692567091,0,ok,1.6715987036271005 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,57,NA,-0.473563405800619,0.7434172326100232,0.13492691340470211,0.6084903192053212,1.2169806384106423,0.7434172326100232,0.0,0.8185044582152144,0.630580026390437,0,ok,1.637008916430429 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,58,NA,-0.5157932293584538,0.7733778149494767,0.1287922927955114,0.6445855221539653,1.2891710443079305,0.7733778149494767,0.0,0.8334678208943385,0.6679855746950503,0,ok,1.666935641788677 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,59,NA,-0.47219793841609076,0.7281234794272438,0.1279627705055765,0.6001607089216673,1.2003214178433346,0.7281234794272438,0.0,0.8242567722081501,0.621948030602323,0,ok,1.6485135444163002 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,60,NA,-0.6300481533778419,0.8867900066101089,0.12837092661613347,0.7584190799939754,1.5168381599879508,0.8867900066101089,0.0,0.8552408961994835,0.7859515729061903,0,ok,1.710481792398967 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,61,NA,-0.6326350641610969,0.8922679446254453,0.1298164402321742,0.7624515043932711,1.5249030087865423,0.8922679446254453,0.0,0.8545095775163498,0.7901303843085576,0,ok,1.7090191550326996 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,62,NA,-0.49751271383741796,0.7523065185415283,0.12739690235205517,0.6249096161894732,1.2498192323789463,0.7523065185415283,0.0,0.8306582500454266,0.647595384562611,0,ok,1.6613165000908532 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,63,NA,-0.4415598889658365,0.6925728471715935,0.12550647910287852,0.567066368068715,1.13413273613743,0.6925728471715935,0.0,0.8187822701749918,0.5876522831273543,0,ok,1.6375645403499837 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,64,NA,-0.5236578063702678,0.7769445227597979,0.12664335819476502,0.6503011645650328,1.3006023291300657,0.7769445227597979,0.0,0.8369981968018605,0.6739087090961304,0,ok,1.673996393603721 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,65,NA,-0.5663176277864376,0.8252193291679827,0.12945085069077256,0.6957684784772101,1.3915369569544203,0.8252193291679827,0.0,0.8431315819743463,0.7210265992280337,0,ok,1.6862631639486927 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,66,NA,-0.5324945594403825,0.7887510871976045,0.12812826387861098,0.6606228233189935,1.321245646637987,0.7887510871976045,0.0,0.8375555153478841,0.6846050696528041,0,ok,1.6751110306957682 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,67,NA,-0.5577937283593639,0.8205875552934097,0.13139691346702287,0.6891906418263868,1.3783812836527736,0.8205875552934097,0.0,0.8398745964164172,0.7142099707987019,0,ok,1.6797491928328343 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,68,NA,-0.49010984009802583,0.7517518123241559,0.13082098611306503,0.6209308262110909,1.2418616524221817,0.7517518123241559,0.0,0.8259784892189194,0.6434721546436092,0,ok,1.6519569784378387 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,69,NA,-0.5431884091134801,0.8045636819369907,0.1306876364117553,0.6738760455252354,1.3477520910504708,0.8045636819369907,0.0,0.8375670697723713,0.6983394166831476,0,ok,1.6751341395447426 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,70,NA,-0.5614205388513879,0.8143987265944249,0.12648909387151852,0.6879096327229064,1.3758192654458128,0.8143987265944249,0.0,0.8446840721369266,0.7128824578888269,0,ok,1.6893681442738533 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,71,NA,-0.5141593613977303,0.764364385058986,0.12510251183062782,0.6392618732283581,1.2785237464567163,0.764364385058986,0.0,0.8363313175286501,0.6624686641148015,0,ok,1.6726626350573002 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,72,NA,-0.5084449689809197,0.761967947035631,0.12676148902735562,0.6352064580082754,1.2704129160165507,0.761967947035631,0.0,0.8336393420215247,0.6582660272678527,0,ok,1.6672786840430494 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,73,NA,-0.48488155283207696,0.7446114141298289,0.12986493064887597,0.614746483480953,1.229492966961906,0.7446114141298289,0.0,0.8255936879497888,0.6370633049398524,0,ok,1.6511873758995776 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,74,NA,-0.6098631436602433,0.8777533990109976,0.13394512767537714,0.7438082713356204,1.4876165426712409,0.8777533990109976,0.0,0.8474000467257673,0.7708103556697296,0,ok,1.6948000934515346 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,75,NA,-0.5163755657306874,0.7817251914638796,0.1326748128665961,0.6490503785972835,1.298100757194567,0.7817251914638796,0.0,0.8302794712063127,0.6726125164967441,0,ok,1.6605589424126255 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,76,NA,-0.4773968052194186,0.7330026343902925,0.12780291458543694,0.6051997198048555,1.210399439609711,0.7330026343902925,0.0,0.8256446722162974,0.6271699700735249,0,ok,1.651289344432595 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,77,NA,-0.5422972892502792,0.7990273225392516,0.12836501664448619,0.6706623058947654,1.3413246117895308,0.7990273225392516,0.0,0.8393484014582238,0.6950090103364364,0,ok,1.6786968029164475 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,78,NA,-0.6227793822835531,0.876182764193313,0.12670169095487993,0.749481073238433,1.498962146476866,0.876182764193313,0.0,0.8553935364483778,0.776689094345893,0,ok,1.7107870728967556 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,79,NA,-0.5773517857982878,0.8463486772641676,0.1344984457329399,0.7118502315312277,1.4237004630624555,0.8463486772641676,0.0,0.8410838826289565,0.7376921597885545,0,ok,1.682167765257913 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,80,NA,-0.598966830976286,0.84373705849528,0.12238511375949701,0.721351944735783,1.442703889471566,0.84373705849528,0.0,0.8549487514774348,0.7475388087395316,0,ok,1.7098975029548695 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,81,NA,-0.5615014478387987,0.8212824455316399,0.12989049884642057,0.6913919466852193,1.3827838933704386,0.8212824455316399,0.0,0.8418443014907756,0.7164911884814891,0,ok,1.6836886029815512 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,82,NA,-0.5208532114936125,0.7828429250388763,0.1309948567726319,0.6518480682662444,1.3036961365324888,0.7828429250388763,0.0,0.832667764397147,0.6755117692368529,0,ok,1.665335528794294 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,83,NA,-0.5512602667169995,0.8032111060304405,0.12597541965672054,0.67723568637372,1.35447137274744,0.8032111060304405,0.0,0.8431602617158704,0.7018210208238126,0,ok,1.6863205234317409 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,84,NA,-0.5286648802633269,0.7867699999389217,0.1290525598377974,0.6577174401011243,1.3154348802022486,0.7867699999389217,0.0,0.8359716818793091,0.681594213820963,0,ok,1.6719433637586183 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,85,NA,-0.6256845061619133,0.894729605301375,0.13452254956973086,0.7602070557316442,1.5204141114632883,0.894729605301375,0.0,0.8496500520686144,0.7878044565959704,0,ok,1.6993001041372289 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,86,NA,-0.5022585056787926,0.760413784040895,0.1290776391810512,0.6313361448598438,1.2626722897196876,0.760413784040895,0.0,0.8302534200588486,0.6542552121566697,0,ok,1.6605068401176972 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,87,NA,-0.5208330455321617,0.7867331830226707,0.1329500687452545,0.6537831142774162,1.3075662285548324,0.7867331830226707,0.0,0.8310099642238894,0.6775170622156876,0,ok,1.6620199284477788 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,88,NA,-0.5434362563941455,0.8038989829099736,0.13023136325791407,0.6736676196520596,1.3473352393041191,0.8038989829099736,0.0,0.8380003383180069,0.6981234244340363,0,ok,1.6760006766360138 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,89,NA,-0.5188867406915404,0.782382967629547,0.13174811346900328,0.6506348541605437,1.3012697083210873,0.782382967629547,0.0,0.8316066185998758,0.6742545124512573,0,ok,1.6632132371997517 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,90,NA,-0.4324989985483607,0.6861696440035064,0.12683532272757286,0.5593343212759336,1.1186686425518673,0.6861696440035064,0.0,0.8151545702495043,0.5796395438663391,0,ok,1.6303091404990087 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,91,NA,-0.6072320581704852,0.8586202999784408,0.1256941209039778,0.732926179074463,1.465852358148926,0.8586202999784408,0.0,0.8536091903404405,0.7595332164800946,0,ok,1.707218380680881 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,92,NA,-0.49398127298458294,0.7504761761653279,0.12824745159037249,0.6222287245749554,1.2444574491499107,0.7504761761653279,0.0,0.8291118950028868,0.644817169935893,0,ok,1.6582237900057737 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,93,NA,-0.509515538287324,0.7664943085854607,0.12848938514906838,0.6380049234363924,1.2760098468727847,0.7664943085854607,0.0,0.8323674635155593,0.6611660839290981,0,ok,1.6647349270311187 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,94,NA,-0.514925926396652,0.7682987664196065,0.12668642001147723,0.6416123464081293,1.2832246928162585,0.7682987664196065,0.0,0.8351078700778658,0.6649044652984026,0,ok,1.6702157401557316 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,95,NA,-0.5059170898651054,0.75079408128606,0.12243849571047727,0.6283555855755827,1.2567111711511654,0.75079408128606,0.0,0.836921336006341,0.6511664511808465,0,ok,1.673842672012682 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,96,NA,-0.48611503898809877,0.7458186559759985,0.12985180849394987,0.6159668474820487,1.2319336949640973,0.7458186559759985,0.0,0.8258935902803045,0.638327971179772,0,ok,1.651787180560609 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,97,NA,-0.5869216121849404,0.8483358214130042,0.1307071046140319,0.7176287167989723,1.4352574335979447,0.8483358214130042,0.0,0.8459252794532197,0.7436804183978121,0,ok,1.6918505589064394 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,98,NA,-0.6209679195368148,0.874833204257987,0.1269326423605861,0.747900561897401,1.495801123794802,0.874833204257987,0.0,0.8549064647491891,0.7750512065247029,0,ok,1.7098129294983782 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,J,0,99,NA,-0.5813009047242994,0.8343643974380488,0.1265317463568747,0.7078326510811741,1.4156653021623482,0.8343643974380488,0.0,0.8483495379891618,0.7335287312075885,0,ok,1.6966990759783236 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,0,1.0,Y,0,NA,NA,-2.9913949982448904,3.506439915785475,0.25752245877029223,3.2489174570151826,6.497834914030365,3.506439915785475,0.0,0.9265572874610045,0.9265572874610045,0,ok,1.853114574922009 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/complexity.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/complexity.csv new file mode 100644 index 0000000..d69f530 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/complexity.csv @@ -0,0 +1,4 @@ +run_id,quantity,n_alpha,n_eta,n_monomials,n_mixed_monomials,max_degree,n_coefficients,status +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,Y,100.0,400.0,400.0,0.0,1.0,400.0,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,J,100.0,400.0,196.0,0.0,1.0,19600.0,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,H,NA,NA,NA,NA,NA,NA,not_implemented diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/layer_normalized_radius_Y.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/layer_normalized_radius_Y.csv new file mode 100644 index 0000000..d064387 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/layer_normalized_radius_Y.csv @@ -0,0 +1,301 @@ +neuron,layer_0 +0,0.6162421199489911 +1,0.5419218285316451 +2,0.6525327778653724 +3,0.637024294710363 +4,0.7427237147703024 +5,0.6646561232806381 +6,0.6281257110189709 +7,0.543776295629057 +8,0.8657529873940282 +9,0.651544860521241 +10,0.6165398200770281 +11,0.6251195724642008 +12,0.6912848856671185 +13,0.685035917252616 +14,0.8453483334854439 +15,0.6454932120893029 +16,0.9546592428300092 +17,0.6844219675544602 +18,0.9989769533006161 +19,0.7132754622721597 +20,0.6544510118719792 +21,0.7250216825362106 +22,0.7881857569830729 +23,0.7751149638665825 +24,0.859768585369206 +25,0.830398867531548 +26,0.7610666779300438 +27,0.7755950750012318 +28,0.6479592660850625 +29,0.8336659398768762 +30,0.577638923433512 +31,0.7251830806294721 +32,0.8600723383926094 +33,0.6476726254761521 +34,0.6909176216103874 +35,0.7047202246124595 +36,0.8552648324434805 +37,0.7914027620828362 +38,0.5933413914154003 +39,0.7468845735772127 +40,0.7235186720351032 +41,0.6355710553440279 +42,0.626288058482591 +43,0.9528482053020632 +44,0.8615678731924009 +45,0.5368203060683006 +46,0.6688985438254108 +47,0.6443708874499796 +48,0.7467942927395395 +49,0.6385533374042454 +50,0.8230485319677043 +51,0.6296994068635823 +52,0.5916061892419623 +53,0.7685769031707625 +54,0.8246169214452626 +55,0.54171952478903 +56,0.5591547684566563 +57,0.834630628282293 +58,0.5705099526169742 +59,0.6983825094353423 +60,0.9711665349434576 +61,0.7925580496557257 +62,0.7739003345523898 +63,0.9033425122231032 +64,0.6596971791975124 +65,0.6517398863272531 +66,0.7201117129977588 +67,0.6761713582560301 +68,0.7714366430971834 +69,0.9099648233044394 +70,0.704484787216771 +71,0.9159053838275886 +72,0.8244760671147741 +73,0.8255080588076058 +74,0.6846305582944902 +75,0.675608962006266 +76,0.8131197354843077 +77,0.8903252064464775 +78,0.5109893040753523 +79,0.6470779993909104 +80,0.8507588073142239 +81,0.48828299860482227 +82,0.6098286677909557 +83,0.6681325395936913 +84,0.6265530750327327 +85,0.6889388762532255 +86,0.7968470016124662 +87,0.8531743537692537 +88,0.6404709513492913 +89,0.6633758900956943 +90,0.6230628600687211 +91,0.8603494246601253 +92,0.7854578489955437 +93,0.5279852281313057 +94,0.8334946520781183 +95,0.6553095660427952 +96,0.5221521700057307 +97,0.8275395025216328 +98,0.8675218865905903 +99,0.6754901047331782 +100,0.9290957181679524 +101,0.9718747288736591 +102,0.7544446448407093 +103,0.7963318065887969 +104,0.6624648458375416 +105,0.8281177477259958 +106,0.5729591361537771 +107,0.48646887613813905 +108,0.764051344262394 +109,0.6952094840415755 +110,0.8484577743164452 +111,0.701198157325423 +112,0.6221638583532983 +113,0.6555523479516163 +114,0.5238569605481382 +115,0.5803652319175282 +116,0.6733924570906795 +117,0.9072138935425135 +118,0.5538870821400664 +119,0.8060555734457937 +120,0.5878475917970294 +121,0.5408231103857551 +122,0.6853393974816521 +123,0.7938620192397763 +124,0.5990550539427488 +125,0.9022322562957132 +126,0.8136390896551343 +127,0.8052346460926463 +128,0.6983654751243557 +129,0.6321577929598001 +130,0.7496539427504885 +131,0.7495822560561621 +132,0.7726884769714508 +133,0.8440855668704697 +134,0.7295759818673087 +135,0.8732070662815837 +136,0.6468853179208286 +137,0.806865572195593 +138,0.9320892519043357 +139,0.8131811376887522 +140,0.7411601804149776 +141,0.6620114115895328 +142,0.785938628164867 +143,0.6325580465007923 +144,0.7624114006192217 +145,0.6740164849680114 +146,0.8796752345182725 +147,0.7017082917790937 +148,0.9152976338348483 +149,0.6435628091587109 +150,0.6353369849744391 +151,0.5975395068039396 +152,0.8684341609763699 +153,0.5598472710329968 +154,0.6066249584541327 +155,0.6646263568977663 +156,0.7054739897318261 +157,0.8240778609863428 +158,0.6821186903787475 +159,0.78845386314625 +160,0.9367525217150431 +161,0.6182182494715668 +162,0.7231220561650594 +163,0.6456272556641447 +164,0.799794685950358 +165,0.7191218970415708 +166,0.8173451869250589 +167,0.7762967129222806 +168,0.7577494022439597 +169,0.6576045490629244 +170,0.6560185667358831 +171,0.801364092260314 +172,0.5199058524673331 +173,0.5296623781354116 +174,0.8489691556477169 +175,0.9117735772737228 +176,0.7521245822606443 +177,0.6476925002821613 +178,0.7563424348594338 +179,0.8367961820471885 +180,0.7736008406491968 +181,0.8019484928702707 +182,0.693601603310003 +183,0.6887513360557742 +184,0.931283851277937 +185,0.5784294415472628 +186,0.6681619265262403 +187,0.7630300157898852 +188,0.6431987744455856 +189,0.7699110668821418 +190,0.6370287818625128 +191,0.719251344513818 +192,0.4914141614607233 +193,0.5709855737214663 +194,0.7029058056270725 +195,0.932253345899195 +196,0.816999899919923 +197,0.49713666100544623 +198,0.6821086174382271 +199,0.7900017819271218 +200,0.7292898027159896 +201,0.7910902417132392 +202,0.6220698339112714 +203,0.6847364428358164 +204,0.5873884707760092 +205,0.7293810867623591 +206,0.86752801895761 +207,0.8687592485001452 +208,0.8212440643846498 +209,0.8352457761128697 +210,0.6987361037936566 +211,0.8723927041740711 +212,0.7798132270081817 +213,0.9038894681178925 +214,0.49108414803207956 +215,0.5296268453931839 +216,0.8563636346631119 +217,0.7323919388719169 +218,0.7306481333777458 +219,0.7526980757677582 +220,0.7368032693234934 +221,0.6980074007702309 +222,0.7887509191444781 +223,0.5676430654113679 +224,0.8630306925047158 +225,0.6311002366227472 +226,0.741103947737733 +227,0.8481680525951041 +228,0.7382943637762912 +229,0.6008984291499322 +230,0.9293008484031721 +231,0.7300032302497331 +232,0.738678469054186 +233,0.7652617361979039 +234,0.7185404049956695 +235,0.7264295487074044 +236,0.8172424721389996 +237,0.6867625929112331 +238,0.8597439029896542 +239,0.7084514124737799 +240,0.8097671992413316 +241,0.6293817962278792 +242,0.6368686413963487 +243,0.7878994162668858 +244,0.902222207242424 +245,0.7669338079892857 +246,0.6843791934140323 +247,0.6337265624309671 +248,0.7052846827158022 +249,0.6857994352130584 +250,0.5828781625114009 +251,0.5216217649225898 +252,0.7899318569568655 +253,0.6221052967589766 +254,0.7339873002011027 +255,0.8586812771175738 +256,0.6484040726146777 +257,0.48934342656612595 +258,0.7241077021581477 +259,0.6516994095776407 +260,0.6382654051471263 +261,0.65038165873561 +262,0.7232558861886125 +263,0.7518161929425176 +264,0.590377345136714 +265,0.6624628318580533 +266,0.7724032715998305 +267,0.6988892946404283 +268,0.66461720593681 +269,0.648338441739094 +270,0.7258076409702059 +271,0.7086962762919159 +272,0.723201435588892 +273,0.6132558048139611 +274,0.9047843465339248 +275,0.7518906902116689 +276,0.8043475276533333 +277,0.7940551159124855 +278,0.5976623127230066 +279,0.7077269709075823 +280,0.784406875698381 +281,0.5825145547288328 +282,0.9527891403677229 +283,0.6251814278730241 +284,0.674642251412045 +285,0.6956741249057677 +286,0.8717298877588612 +287,0.7472835883496265 +288,0.7206120437267526 +289,0.8454980596041871 +290,0.847466837773604 +291,0.685029461329672 +292,0.6501792735220139 +293,0.5919296981013319 +294,0.7589313102422693 +295,0.9215966179594822 +296,0.5706432778521059 +297,0.7317465260702745 +298,0.6456245621083724 +299,0.5779864956945218 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/metrics.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/metrics.csv new file mode 100644 index 0000000..6ecb9a2 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/metrics.csv @@ -0,0 +1,79 @@ +schema_version,benchmark_level,problem_id,model_id,method_id,run_id,split_id,quantity,metric,aggregation,derivative_order,layer,neuron,output_index,input_index_a,input_index_b,cell_id,value,unit,status +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,mean_width,mean_weighted,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,max_width,max,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,q50_width,q50,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,q90_width,q90,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,q99_width,q99,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,mean_global_normalized_radius,mean_weighted,0,NA,NA,NA,NA,NA,NA,0.9265572874610045,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,Y,max_global_normalized_radius,max,0,NA,NA,NA,NA,NA,NA,0.9265572874610045,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,mean_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,1.3377060235764833,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,max_width,max,1,NA,NA,NA,NA,NA,NA,1.6628532779818284,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,q50_width,q50,1,NA,NA,NA,NA,NA,NA,1.3210405062364523,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,q90_width,q90,1,NA,NA,NA,NA,NA,NA,1.470955259195105,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,q99_width,q99,1,NA,NA,NA,NA,NA,NA,1.5990780063526777,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,mean_global_normalized_radius,mean_weighted,1,NA,NA,NA,NA,NA,NA,0.6931340343681586,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,max_global_normalized_radius,max,1,NA,NA,NA,NA,NA,NA,0.8616094872985093,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,mean_frobenius_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,13.416510613277604,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,J,max_frobenius_width,max,1,NA,NA,NA,NA,NA,NA,13.416510613277604,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,mean_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,max_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,q50_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,q90_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,q99_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,mean_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,H,max_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,width,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,upper,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,width,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,L2,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,9.340887300803004e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,width,none,NA,NA,NA,NA,NA,NA,NA,9.340887300803004e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,73.6866081167887,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,73.6866081167887,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,upper,none,NA,NA,NA,NA,NA,NA,NA,9.664826589651262e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,width,none,NA,NA,NA,NA,NA,NA,NA,9.664826589651262e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,8.584090407072186,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,W12,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,8.584090407072186,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,PDE_residual,linf_upper,max,2,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,boundary_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,initial_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,196.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,runtime,seconds,onejet_construction,NA,NA,NA,NA,NA,NA,NA,0.20227178599998297,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,runtime,seconds,L2_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,0.028450207000787486,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,runtime,seconds,W12_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,1.1954289400000562,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,runtime,seconds,W12_total,NA,NA,NA,NA,NA,NA,NA,1.3977007260000391,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,memory,bytes,peak,NA,NA,NA,NA,NA,NA,NA,NA,bytes,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,soundness,failure_count,Y,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,soundness,max_violation,Y,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,soundness,failure_count,J,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,affine_pz_topk96_symbolic,pinn100d_shallow300_medium_affine_pz_topk96_symbolic,single_cell,soundness,max_violation,J,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/norms.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/norms.csv new file mode 100644 index 0000000..31635c1 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/norms.csv @@ -0,0 +1,5 @@ +run_id,norm,squared,lower,upper,width,relative_width,value_contribution_upper,gradient_contribution_upper,hessian_contribution_upper,value_contribution_width,gradient_contribution_width,hessian_contribution_width,status,domain_volume,domain_volume_normalized_lower,domain_volume_normalized_upper,domain_volume_normalized_width +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,L2,1,0.0,1.244073696539206e-69,1.244073696539206e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,9.81401102413563,9.81401102413563 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,L2,0,0.0,3.527142889846123e-35,3.527142889846123e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,3.1327321979600535,3.1327321979600535 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,W12,1,0.0,9.340887300803004e-69,9.340887300803004e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,73.6866081167887,73.6866081167887 +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,W12,0,0.0,9.664826589651262e-35,9.664826589651262e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,8.584090407072186,8.584090407072186 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/soundness.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/soundness.csv new file mode 100644 index 0000000..561b882 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/soundness.csv @@ -0,0 +1,3 @@ +run_id,quantity,sample_count,failure_count,max_violation,invalid_interval_count,nan_endpoint_count,infinite_endpoint_count,status +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,Y,16384,0,0.0,0,0,0,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,J,16384,0,0.0,0,0,0,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/timings.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/timings.csv new file mode 100644 index 0000000..1f74f59 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/affine_pz_topk96_symbolic/timings.csv @@ -0,0 +1,5 @@ +run_id,stage,seconds,status +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,onejet_construction,0.20227178599998297,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,L2_symbolic_integration,0.028450207000787486,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,W12_symbolic_integration,1.1954289400000562,ok +pinn100d_shallow300_medium_affine_pz_topk96_symbolic,W12_total,1.3977007260000391,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/activation_approximation.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/activation_approximation.csv new file mode 100644 index 0000000..4fc8fef --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/activation_approximation.csv @@ -0,0 +1,601 @@ +run_id,split_id,cell_id,cell_weight,layer,neuron,derivative_order,preactivation_lower,preactivation_upper,preactivation_midpoint,preactivation_radius,preactivation_width,approximation_kind,approximation_error_radius,approximation_error_diameter,activation_scale,normalized_approximation_radius,noise_symbol_id,shared_noise_group,status +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,0,0,-0.8000887741224202,0.7908767282671686,-0.004606022927625797,0.7954827511947944,1.5909655023895888,affine,0.04776087567042881,0.09552175134085762,0.9319962481713715,0.051245781047014195,eta_l0_n0_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,0,1,-0.8000887741224202,0.7908767282671686,-0.004606022927625797,0.7954827511947944,1.5909655023895888,affine,0.21878416980543125,0.4375683396108625,1.0,0.21878416980543125,eta_l0_n0_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,1,0,-0.6668101129690581,0.6933849725568156,0.013287429793878758,0.6800975427629369,1.3601950855258738,affine,0.032203781618758816,0.06440756323751763,0.9319962481713715,0.034553552851682,eta_l0_n1_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,1,1,-0.6668101129690581,0.6933849725568156,0.013287429793878758,0.6800975427629369,1.3601950855258738,affine,0.17491064771605164,0.3498212954321033,1.0,0.17491064771605164,eta_l0_n1_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,2,0,-0.8409685245821178,0.8705015267430377,0.01476650108045996,0.8557350256625778,1.7114700513251555,affine,0.05704981128360182,0.11409962256720364,0.9319962481713715,0.06121249028151854,eta_l0_n2_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,2,1,-0.8409685245821178,0.8705015267430377,0.01476650108045996,0.8557350256625778,1.7114700513251555,affine,0.2407577465691454,0.4815154931382908,1.0,0.2407577465691454,eta_l0_n2_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,3,0,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.05297381165009602,0.10594762330019204,0.9319962481713715,0.056839082511365886,eta_l0_n3_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,3,1,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.23128307788642288,0.46256615577284577,1.0,0.23128307788642288,eta_l0_n3_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,4,0,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.08534503477820339,0.17069006955640678,0.9319962481713715,0.09157229435811046,eta_l0_n4_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,4,1,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.2959936114346802,0.5919872228693605,1.0,0.2959936114346802,eta_l0_n4_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,5,0,-0.8652278170806447,0.8877201689912209,0.011246175955288096,0.8764739930359328,1.7529479860718655,affine,0.06037470341247909,0.12074940682495817,0.9319962481713715,0.06477998546768575,eta_l0_n5_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,5,1,-0.8652278170806447,0.8877201689912209,0.011246175955288096,0.8764739930359328,1.7529479860718655,affine,0.24820832817273963,0.49641665634547927,1.0,0.24820832817273963,eta_l0_n5_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,6,0,-0.8002253266309622,0.829648498761181,0.014711586065109361,0.8149369126960716,1.6298738253921432,affine,0.05070909200326985,0.1014181840065397,0.9319962481713715,0.05440911602676932,eta_l0_n6_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,6,1,-0.8002253266309622,0.829648498761181,0.014711586065109361,0.8149369126960716,1.6298738253921432,affine,0.22589998281650625,0.4517999656330125,1.0,0.22589998281650625,eta_l0_n6_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,7,0,-0.6724065457781009,0.693292462113097,0.010442958167498073,0.682849503945599,1.365699007891198,affine,0.03252624948541378,0.06505249897082756,0.9319962481713715,0.034899549809596433,eta_l0_n7_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,7,1,-0.6724065457781009,0.693292462113097,0.010442958167498073,0.682849503945599,1.365699007891198,affine,0.17599713778830367,0.35199427557660734,1.0,0.17599713778830367,eta_l0_n7_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,8,0,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.13821186086084938,0.27642372172169877,0.9319962481713715,0.1482965850259899,eta_l0_n8_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,8,1,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.3682178238498487,0.7364356476996974,1.0,0.3682178238498487,eta_l0_n8_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,9,0,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,affine,0.056781076836971985,0.11356215367394397,0.9319962481713715,0.06092414743984176,eta_l0_n9_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,9,1,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,affine,0.24015713034047748,0.48031426068095495,1.0,0.24015713034047748,eta_l0_n9_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,10,0,-0.8080868528523598,0.7839032216460967,-0.01209181560313155,0.7959950372492283,1.5919900744984565,affine,0.04786198015530158,0.09572396031060317,0.9319962481713715,0.05135426269065938,eta_l0_n10_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,10,1,-0.8080868528523598,0.7839032216460967,-0.01209181560313155,0.7959950372492283,1.5919900744984565,affine,0.21891895411782564,0.4378379082356513,1.0,0.21891895411782564,eta_l0_n10_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,11,0,-0.7982094929703909,0.8217535642036393,0.01177203561662421,0.8099815285870151,1.6199630571740302,affine,0.049945300875641425,0.09989060175128285,0.9319962481713715,0.05358959435044603,eta_l0_n11_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,11,1,-0.7982094929703909,0.8217535642036393,0.01177203561662421,0.8099815285870151,1.6199630571740302,affine,0.22410676447470157,0.44821352894940314,1.0,0.22410676447470157,eta_l0_n11_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,12,0,-0.906774438074293,0.9399477281012678,0.01658664501348739,0.9233610830877804,1.8467221661755608,affine,0.06823703003790989,0.13647406007581978,0.9319962481713715,0.07321599220146512,eta_l0_n12_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,12,1,-0.906774438074293,0.9399477281012678,0.01658664501348739,0.9233610830877804,1.8467221661755608,affine,0.2644950169656779,0.5289900339313558,1.0,0.2644950169656779,eta_l0_n12_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,13,0,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.06636055767389427,0.13272111534778855,0.9319962481713715,0.07120260173160287,eta_l0_n13_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,13,1,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.2606246761987212,0.5212493523974424,1.0,0.2606246761987212,eta_l0_n13_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,14,0,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.1281480288819092,0.2562960577638184,0.9319962481713715,0.1374984385756303,eta_l0_n14_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,14,1,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.35680199089624876,0.7136039817924975,1.0,0.35680199089624876,eta_l0_n14_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,15,0,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.055167049976694256,0.11033409995338851,0.9319962481713715,0.059192351991689963,eta_l0_n15_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,15,1,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.2364612101423731,0.4729224202847462,1.0,0.2364612101423731,eta_l0_n15_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,16,0,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.18859813104338555,0.3771962620867711,0.9319962481713715,0.20235932431426154,eta_l0_n16_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,16,1,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.41417314403421934,0.8283462880684387,1.0,0.41417314403421934,eta_l0_n16_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,17,0,-0.9249747098227338,0.8972162631509849,-0.013879223335874435,0.9110954864868593,1.8221909729737187,affine,0.06613657047259132,0.13227314094518264,0.9319962481713715,0.07096227114900404,eta_l0_n17_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,17,1,-0.9249747098227338,0.8972162631509849,-0.013879223335874435,0.9110954864868593,1.8221909729737187,affine,0.2603096831220119,0.5206193662440238,1.0,0.2603096831220119,eta_l0_n17_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,18,0,-1.6733730976128967,1.6691489651366265,-0.0021120662381350908,1.6712610313747616,3.342522062749523,affine,0.2180433724141755,0.436086744828351,0.9319962481713715,0.23395305811797928,eta_l0_n18_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,18,1,-1.6733730976128967,1.6691489651366265,-0.0021120662381350908,1.6712610313747616,3.342522062749523,quadratic,0.13549735134988083,0.27099470269976167,1.0,0.13549735134988083,eta_l0_n18_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,19,0,-0.9806865471881266,0.9462170674146505,-0.01723473988673807,0.9634518073013886,1.9269036146027771,affine,0.07522189449895324,0.15044378899790647,0.9319962481713715,0.08071051213622671,eta_l0_n19_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,19,1,-0.9806865471881266,0.9462170674146505,-0.01723473988673807,0.9634518073013886,1.9269036146027771,affine,0.277989649600464,0.555979299200928,1.0,0.277989649600464,eta_l0_n19_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,20,0,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.05759508702140571,0.11519017404281141,0.9319962481713715,0.061797552441236185,eta_l0_n20_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,20,1,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.2418920890071064,0.4837841780142128,1.0,0.2418920890071064,eta_l0_n20_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,21,0,-0.9711358295108821,0.9996589259075808,0.01426154819834935,0.9853973777092314,1.9707947554184628,affine,0.07912843252686426,0.15825686505372852,0.9319962481713715,0.08490209341734867,eta_l0_n21_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,21,1,-0.9711358295108821,0.9996589259075808,0.01426154819834935,0.9853973777092314,1.9707947554184628,affine,0.2852222128166809,0.5704444256333618,1.0,0.2852222128166809,eta_l0_n21_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,22,0,-1.0915414624371893,1.1306988296348472,0.019578683598828972,1.1111201460360183,2.2222402920720365,affine,0.10280415066198174,0.2056083013239635,0.9319962481713715,0.1103053267260348,eta_l0_n22_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,22,1,-1.0915414624371893,1.1306988296348472,0.019578683598828972,1.1111201460360183,2.2222402920720365,affine,0.32342725517235416,0.6468545103447083,1.0,0.32342725517235416,eta_l0_n22_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,23,0,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.09752239534692254,0.1950447906938451,0.9319962481713715,0.10463818447581405,eta_l0_n23_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,23,1,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.3156441037740703,0.6312882075481406,1.0,0.3156441037740703,eta_l0_n23_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,24,0,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.13521015581349047,0.27042031162698094,0.9319962481713715,0.14507585849061122,eta_l0_n24_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,24,1,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.3648891411432009,0.7297782822864018,1.0,0.3648891411432009,eta_l0_n24_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,25,0,-1.2223185723825791,1.1844726444174873,-0.018922963982545893,1.2033956084000332,2.4067912168000665,affine,0.12114127036711338,0.24228254073422675,0.9319962481713715,0.12998042707231847,eta_l0_n25_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,25,1,-1.2223185723825791,1.1844726444174873,-0.018922963982545893,1.2033956084000332,2.4067912168000665,affine,0.3482249401892906,0.6964498803785812,1.0,0.3482249401892906,eta_l0_n25_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,26,0,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.09212749605465377,0.18425499210930754,0.9319962481713715,0.09884964261971338,eta_l0_n26_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,26,1,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.30710974875608243,0.6142194975121649,1.0,0.30710974875608243,eta_l0_n26_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,27,0,-1.0765750272011247,1.0932061682436134,0.00831557052124432,1.084890597722369,2.169781195444738,affine,0.09767678010090204,0.19535356020180408,0.9319962481713715,0.10480383402030782,eta_l0_n27_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,27,1,-1.0765750272011247,1.0932061682436134,0.00831557052124432,1.084890597722369,2.169781195444738,affine,0.315989484240928,0.631978968481856,1.0,0.315989484240928,eta_l0_n27_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,28,0,-0.8349571475513465,0.8610004970616331,0.013021674755143264,0.8479788223064898,1.6959576446129796,affine,0.055810473951034785,0.11162094790206957,0.9319962481713715,0.059882723842009065,eta_l0_n28_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,28,1,-0.8349571475513465,0.8610004970616331,0.013021674755143264,0.8479788223064898,1.6959576446129796,affine,0.23798244361387216,0.4759648872277443,1.0,0.23798244361387216,eta_l0_n28_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,29,0,-1.191499282547206,1.2302241779957939,0.019362447724293963,1.2108617302715,2.421723460543,affine,0.1226544191509072,0.2453088383018144,0.9319962481713715,0.13160398380525887,eta_l0_n29_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,29,1,-1.191499282547206,1.2302241779957939,0.019362447724293963,1.2108617302715,2.421723460543,affine,0.3501010164013204,0.7002020328026408,1.0,0.3501010164013204,eta_l0_n29_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,30,0,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.03916738782673008,0.07833477565346016,0.9319962481713715,0.042025263410210796,eta_l0_n30_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,30,1,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.19569049953193574,0.39138099906387147,1.0,0.19569049953193574,eta_l0_n30_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,31,0,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,affine,0.07920689094127416,0.15841378188254832,0.9319962481713715,0.08498627660431304,eta_l0_n31_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,31,1,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,affine,0.2852835771335912,0.5705671542671824,1.0,0.2852835771335912,eta_l0_n31_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,32,0,-1.2946227610182182,1.2514994629324607,-0.021561649042878717,1.2730611119753394,2.546122223950679,affine,0.1353723538326648,0.2707447076653296,0.9319962481713715,0.1452498914006069,eta_l0_n32_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,32,1,-1.2946227610182182,1.2514994629324607,-0.021561649042878717,1.2730611119753394,2.546122223950679,affine,0.3650408857692888,0.7300817715385776,1.0,0.3650408857692888,eta_l0_n32_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,33,0,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,affine,0.05575080421097712,0.11150160842195424,0.9319962481713715,0.05981870025803569,eta_l0_n33_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,33,1,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,affine,0.23778197001319706,0.4755639400263941,1.0,0.23778197001319706,eta_l0_n33_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,34,0,-0.9062201538536022,0.9391835025457022,0.016481674346049968,0.9227018281996522,1.8454036563993044,affine,0.06812369722603763,0.13624739445207526,0.9319962481713715,0.07309438998247055,eta_l0_n34_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,34,1,-0.9062201538536022,0.9391835025457022,0.016481674346049968,0.9227018281996522,1.8454036563993044,affine,0.26427067826313716,0.5285413565262743,1.0,0.26427067826313716,eta_l0_n34_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,35,0,-0.9404150670737785,0.9548630677126784,0.007224000319449919,0.9476390673932285,1.895278134786457,affine,0.07239481230028884,0.14478962460057768,0.9319962481713715,0.07767714992665635,eta_l0_n35_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,35,1,-0.9404150670737785,0.9548630677126784,0.007224000319449919,0.9476390673932285,1.895278134786457,affine,0.27282519963288465,0.5456503992657693,1.0,0.27282519963288465,eta_l0_n35_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,36,0,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.1329675800582422,0.2659351601164844,0.9319962481713715,0.1426696516419803,eta_l0_n36_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,36,1,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.36239116506943936,0.7247823301388787,1.0,0.36239116506943936,eta_l0_n36_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,37,0,-1.1019913101592176,1.1337401489755885,0.01587441940818546,1.117865729567403,2.235731459134806,affine,0.10410160451057877,0.20820320902115755,0.9319962481713715,0.11169745019342289,eta_l0_n37_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,37,1,-1.1019913101592176,1.1337401489755885,0.01587441940818546,1.117865729567403,2.235731459134806,affine,0.3253861193108082,0.6507722386216164,1.0,0.3253861193108082,eta_l0_n37_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,38,0,-0.7748642347846233,0.7429797929158597,-0.0159422209343818,0.7589220138502415,1.517844027700483,affine,0.04255388603436006,0.08510777206872013,0.9319962481713715,0.04565885980534058,eta_l0_n38_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,38,1,-0.7748642347846233,0.7429797929158597,-0.0159422209343818,0.7589220138502415,1.517844027700483,affine,0.20497042177762226,0.4099408435552445,1.0,0.20497042177762226,eta_l0_n38_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,39,0,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,affine,0.08683626610744645,0.1736725322148929,0.9319962481713715,0.09317233441424688,eta_l0_n39_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,39,1,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,affine,0.29854643485100885,0.5970928697020177,1.0,0.29854643485100885,eta_l0_n39_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,40,0,-1.0016438696095071,0.9635624254813989,-0.019040722064054105,0.9826031475454531,1.9652062950909062,affine,0.07865281463649128,0.15730562927298256,0.9319962481713715,0.08439177173815074,eta_l0_n40_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,40,1,-1.0016438696095071,0.9635624254813989,-0.019040722064054105,0.9826031475454531,1.9652062950909062,affine,0.2842463860855686,0.5684927721711373,1.0,0.2842463860855686,eta_l0_n40_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,41,0,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,affine,0.052595132871832324,0.10519026574366465,0.9319962481713715,0.056432773173740666,eta_l0_n41_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,41,1,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,affine,0.2304090999181408,0.4608181998362816,1.0,0.2304090999181408,eta_l0_n41_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,42,0,-0.8258438072260423,0.7979791433617529,-0.013932331932144715,0.8119114752938976,1.6238229505877952,affine,0.05024729084278002,0.10049458168556004,0.9319962481713715,0.05391361922472113,eta_l0_n42_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,42,1,-0.8258438072260423,0.7979791433617529,-0.013932331932144715,0.8119114752938976,1.6238229505877952,affine,0.2247944505364547,0.4495889010729094,1.0,0.2247944505364547,eta_l0_n42_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,43,0,-1.506554210451975,1.539800993136927,0.016623391342476035,1.523177601794451,3.046355203588902,affine,0.18746340190277452,0.37492680380554905,0.9319962481713715,0.20114179887589478,eta_l0_n43_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,43,1,-1.506554210451975,1.539800993136927,0.016623391342476035,1.523177601794451,3.046355203588902,affine,0.41330507513892245,0.8266101502778449,1.0,0.41330507513892245,eta_l0_n43_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,44,0,-1.298895240010546,1.254490838972456,-0.02220220051904498,1.276693039491501,2.553386078983002,affine,0.13612331689707471,0.27224663379414943,0.9319962481713715,0.14605564900519313,eta_l0_n44_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,44,1,-1.298895240010546,1.254490838972456,-0.02220220051904498,1.276693039491501,2.553386078983002,affine,0.36586726001736714,0.7317345200347343,1.0,0.36586726001736714,eta_l0_n44_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,45,0,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,affine,0.031247814786823872,0.062495629573647744,0.9319962481713715,0.03352783323767003,eta_l0_n45_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,45,1,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,affine,0.17204478770874515,0.3440895754174903,1.0,0.17204478770874515,eta_l0_n45_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,46,0,-0.8663927459559603,0.9013360930495738,0.017471673546806787,0.8838644195027671,1.7677288390055341,affine,0.06161881063103398,0.12323762126206796,0.9319962481713715,0.06611486983121823,eta_l0_n46_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,46,1,-0.8663927459559603,0.9013360930495738,0.017471673546806787,0.8838644195027671,1.7677288390055341,affine,0.25074390701983446,0.5014878140396689,1.0,0.25074390701983446,eta_l0_n46_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,47,0,-0.8527698834317742,0.8310757190829284,-0.010847082174422873,0.8419228012573513,1.6838456025147026,affine,0.054847789658709296,0.10969557931741859,0.9319962481713715,0.05884979662345606,eta_l0_n47_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,47,1,-0.8527698834317742,0.8310757190829284,-0.010847082174422873,0.8419228012573513,1.6838456025147026,affine,0.23581240905165818,0.47162481810331636,1.0,0.23581240905165818,eta_l0_n47_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,48,0,-1.0115982611523093,1.0428605240202082,0.015631131433949452,1.0272293925862588,2.0544587851725176,affine,0.0867955966979921,0.1735911933959842,0.9319962481713715,0.09312869753316055,eta_l0_n48_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,48,1,-1.0115982611523093,1.0428605240202082,0.015631131433949452,1.0272293925862588,2.0544587851725176,affine,0.2985041035683372,0.5970082071366744,1.0,0.2985041035683372,eta_l0_n48_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,49,0,-0.8165600505003638,0.847871599185807,0.015655774342721585,0.8322158248430854,1.6644316496861709,affine,0.053361052968857854,0.10672210593771571,0.9319962481713715,0.057254579161187834,eta_l0_n49_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,49,1,-0.8165600505003638,0.847871599185807,0.015655774342721585,0.8322158248430854,1.6644316496861709,affine,0.23222316361302536,0.4644463272260507,1.0,0.23222316361302536,eta_l0_n49_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,50,0,-1.1685921066522982,1.2049616256427143,0.018184759495208036,1.1867768661475062,2.3735537322950124,affine,0.11778720959870133,0.23557441919740266,0.9319962481713715,0.1263816349366281,eta_l0_n50_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,50,1,-1.1685921066522982,1.2049616256427143,0.018184759495208036,1.1867768661475062,2.3735537322950124,affine,0.34397954463048375,0.6879590892609675,1.0,0.34397954463048375,eta_l0_n50_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,51,0,-0.8006207796149774,0.8344702881968007,0.01692475429091167,0.817545533905889,1.635091067811778,affine,0.05111893061710033,0.10223786123420066,0.9319962481713715,0.054848858798947436,eta_l0_n51_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,51,1,-0.8006207796149774,0.8344702881968007,0.01692475429091167,0.817545533905889,1.635091067811778,affine,0.2268290575123888,0.4536581150247776,1.0,0.2268290575123888,eta_l0_n51_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,52,0,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.04215920397663842,0.08431840795327684,0.9319962481713715,0.04523537949788653,eta_l0_n52_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,52,1,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.20395651747432286,0.4079130349486457,1.0,0.20395651747432286,eta_l0_n52_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,53,0,-1.0909113768846286,1.05038646716867,-0.020262454857979284,1.0706489220266493,2.1412978440532986,affine,0.09500119573358663,0.19000239146717326,0.9319962481713715,0.10193302378629127,eta_l0_n53_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,53,1,-1.0909113768846286,1.05038646716867,-0.020262454857979284,1.0706489220266493,2.1412978440532986,affine,0.31165202620511157,0.6233040524102231,1.0,0.31165202620511157,eta_l0_n53_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,54,0,-1.2066771566490586,1.173895383021899,-0.016390886813579808,1.1902862698354788,2.3805725396709576,affine,0.11848491224053499,0.23696982448106998,0.9319962481713715,0.12713024593501207,eta_l0_n54_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,54,1,-1.2066771566490586,1.173895383021899,-0.016390886813579808,1.1902862698354788,2.3805725396709576,affine,0.34490796924504935,0.6898159384900987,1.0,0.34490796924504935,eta_l0_n54_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,55,0,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.03217402595959646,0.06434805191919292,0.9319962481713715,0.03452162605023753,eta_l0_n55_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,55,1,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.17478486104675106,0.3495697220935021,1.0,0.17478486104675106,eta_l0_n55_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,56,0,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,affine,0.03540540227173134,0.07081080454346268,0.9319962481713715,0.03798878197331664,eta_l0_n56_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,56,1,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,quadratic,0.014350815397436175,0.02870163079487235,1.0,0.014350815397436175,eta_l0_n56_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,57,0,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.1231062852532452,0.2462125705064904,0.9319962481713715,0.1320888206307553,eta_l0_n57_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,57,1,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.3506497194153906,0.7012994388307812,1.0,0.3506497194153906,eta_l0_n57_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,58,0,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.037684701643931236,0.07536940328786247,0.9319962481713715,0.04043439200304799,eta_l0_n58_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,58,1,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.191521183406278,0.383042366812556,1.0,0.191521183406278,eta_l0_n58_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,59,0,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.07040104306089869,0.14080208612179737,0.9319962481713715,0.07553790393365795,eta_l0_n59_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,59,1,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.2689123493482129,0.5378246986964258,1.0,0.2689123493482129,eta_l0_n59_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,60,0,-1.5860529702984403,1.5733837593857998,-0.0063346054563202525,1.57971836484212,3.15943672968424,affine,0.1992008157072346,0.3984016314144692,0.9319962481713715,0.21373564120894015,eta_l0_n60_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,60,1,-1.5860529702984403,1.5733837593857998,-0.0063346054563202525,1.57971836484212,3.15943672968424,quadratic,0.12300515623212498,0.24601031246424995,1.0,0.12300515623212498,eta_l0_n60_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,61,0,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,affine,0.10458222490461823,0.20916444980923646,0.9319962481713715,0.11221313938743249,eta_l0_n61_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,61,1,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,affine,0.32606958196707225,0.6521391639341445,1.0,0.32606958196707225,eta_l0_n61_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,62,0,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,affine,0.09705393191182805,0.1941078638236561,0.9319962481713715,0.1041355392816798,eta_l0_n62_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,62,1,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,affine,0.3148977187880977,0.6297954375761954,1.0,0.3148977187880977,eta_l0_n62_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,63,0,-1.4015171493211274,1.3643005656294995,-0.018608291845813918,1.3829088574753134,2.765817714950627,affine,0.15815871965502917,0.31631743931005835,0.9319962481713715,0.16969888018899795,eta_l0_n63_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,63,1,-1.4015171493211274,1.3643005656294995,-0.018608291845813918,1.3829088574753134,2.765817714950627,affine,0.3885130831309809,0.7770261662619617,1.0,0.3885130831309809,eta_l0_n63_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,64,0,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,affine,0.05902823505223593,0.11805647010447186,0.9319962481713715,0.0633352711108576,eta_l0_n64_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,64,1,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,affine,0.245112538951466,0.490225077902932,1.0,0.245112538951466,eta_l0_n64_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,65,0,-0.8429087446687172,0.8658273177411767,0.01145928653622974,0.854368031204947,1.708736062409894,affine,0.05681525626691875,0.1136305125338375,0.9319962481713715,0.060960820795570206,eta_l0_n65_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,65,1,-0.8429087446687172,0.8658273177411767,0.01145928653622974,0.854368031204947,1.708736062409894,affine,0.24030458215024153,0.48060916430048306,1.0,0.24030458215024153,eta_l0_n65_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,66,0,-0.9905468239121493,0.9617857365141982,-0.014380543698975568,0.9761662802131738,1.9523325604263475,affine,0.0774704270215625,0.154940854043125,0.9319962481713715,0.08312311039187527,eta_l0_n66_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,66,1,-0.9905468239121493,0.9617857365141982,-0.014380543698975568,0.9761662802131738,1.9523325604263475,affine,0.28221320191889177,0.5644264038377835,1.0,0.28221320191889177,eta_l0_n66_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,67,0,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.06370430392497341,0.12740860784994681,0.9319962481713715,0.06835253258794206,eta_l0_n67_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,67,1,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.25522800306256815,0.5104560061251363,1.0,0.25522800306256815,eta_l0_n67_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,68,0,-1.0577074974769154,1.0951935167927762,0.01874300965793041,1.0764505071348458,2.1529010142696916,affine,0.0961007655593182,0.1922015311186364,0.9319962481713715,0.10311282448601401,eta_l0_n68_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,68,1,-1.0577074974769154,1.0951935167927762,0.01874300965793041,1.0764505071348458,2.1529010142696916,affine,0.313395395818585,0.62679079163717,1.0,0.313395395818585,eta_l0_n68_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,69,0,-1.3791784207269757,1.42228292189483,0.02155225058392718,1.400730671310903,2.801461342621806,affine,0.16189286350766371,0.32378572701532743,0.9319962481713715,0.17370548843442934,eta_l0_n69_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,69,1,-1.3791784207269757,1.42228292189483,0.02155225058392718,1.400730671310903,2.801461342621806,affine,0.39193665167048064,0.7838733033409613,1.0,0.39193665167048064,eta_l0_n69_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,70,0,-0.9308513790946814,0.9636725950147781,0.01641060796004834,0.9472619870547297,1.8945239741094595,affine,0.07236808784580497,0.14473617569160993,0.9319962481713715,0.0776484755038394,eta_l0_n70_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,70,1,-0.9308513790946814,0.9636725950147781,0.01641060796004834,0.9472619870547297,1.8945239741094595,affine,0.2726039266590261,0.5452078533180522,1.0,0.2726039266590261,eta_l0_n70_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,71,0,-1.438885897394498,1.3949902367415656,-0.02194783032646619,1.4169380670680318,2.8338761341360637,affine,0.16528217318108282,0.33056434636216564,0.9319962481713715,0.17734210143589701,eta_l0_n71_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,71,1,-1.438885897394498,1.3949902367415656,-0.02194783032646619,1.4169380670680318,2.8338761341360637,affine,0.39499506530744755,0.7899901306148951,1.0,0.39499506530744755,eta_l0_n71_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,72,0,-1.1711538278994709,1.2088264922154857,0.018836332158007396,1.1899901600574783,2.3799803201149565,affine,0.11843682940143908,0.23687365880287817,0.9319962481713715,0.12707865469825522,eta_l0_n72_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,72,1,-1.1711538278994709,1.2088264922154857,0.018836332158007396,1.1899901600574783,2.3799803201149565,affine,0.3448007599664116,0.6896015199328231,1.0,0.3448007599664116,eta_l0_n72_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,73,0,-1.1745387071873483,1.2100713466785744,0.017766319745613046,1.1923050269329614,2.3846100538659227,affine,0.11889769438589372,0.23779538877178744,0.9319962481713715,0.12757314701553532,eta_l0_n73_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,73,1,-1.1745387071873483,1.2100713466785744,0.017766319745613046,1.1923050269329614,2.3846100538659227,affine,0.3454105213869702,0.6908210427739404,1.0,0.3454105213869702,eta_l0_n73_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,74,0,-0.9300731152852425,0.8929332780733105,-0.018569918605966018,0.9115031966792765,1.823006393358553,affine,0.06623391994248702,0.13246783988497404,0.9319962481713715,0.0710667237904033,eta_l0_n74_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,74,1,-0.9300731152852425,0.8929332780733105,-0.018569918605966018,0.9115031966792765,1.823006393358553,affine,0.26038337545599277,0.5207667509119855,1.0,0.26038337545599277,eta_l0_n74_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,75,0,-0.8829617050674686,0.9081151180277419,0.012576706480136646,0.8955384115476053,1.7910768230952105,affine,0.06352122059880379,0.12704244119760758,0.9319962481713715,0.0681560904600592,eta_l0_n75_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,75,1,-0.8829617050674686,0.9081151180277419,0.012576706480136646,0.8955384115476053,1.7910768230952105,affine,0.25491234963215775,0.5098246992643155,1.0,0.25491234963215775,eta_l0_n75_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,76,0,-1.1805875736327942,1.148801902205817,-0.01589283571348865,1.1646947379193056,2.329389475838611,affine,0.1133540174298754,0.2267080348597508,0.9319962481713715,0.1216249718303934,eta_l0_n76_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,76,1,-1.1805875736327942,1.148801902205817,-0.01589283571348865,1.1646947379193056,2.329389475838611,affine,0.3382125951197492,0.6764251902394984,1.0,0.3382125951197492,eta_l0_n76_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,77,0,-1.3659675145450216,1.3314526620255038,-0.0172574262597589,1.3487100882852627,2.6974201765705255,affine,0.1510226319798477,0.3020452639596954,0.9319962481713715,0.16204210293352844,eta_l0_n77_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,77,1,-1.3659675145450216,1.3314526620255038,-0.0172574262597589,1.3487100882852627,2.6974201765705255,affine,0.38163061019264893,0.7632612203852979,1.0,0.38163061019264893,eta_l0_n77_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,78,0,-0.6125577319415736,0.6569759697827164,0.022209118920571425,0.634766850862145,1.26953370172429,affine,0.026981033085969944,0.05396206617193989,0.9319962481713715,0.028949722854473108,eta_l0_n78_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,78,1,-0.6125577319415736,0.6569759697827164,0.022209118920571425,0.634766850862145,1.26953370172429,affine,0.1573444023687779,0.3146888047375558,1.0,0.1573444023687779,eta_l0_n78_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,79,0,-0.8548400889214125,0.8380958072828193,-0.008372140819296603,0.8464679481021159,1.6929358962042318,affine,0.05555260530301887,0.11110521060603774,0.9319962481713715,0.05960603962946865,eta_l0_n79_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,79,1,-0.8548400889214125,0.8380958072828193,-0.008372140819296603,0.8464679481021159,1.6929358962042318,affine,0.23748054605143304,0.4749610921028661,1.0,0.23748054605143304,eta_l0_n79_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,80,0,-1.2719651427464111,1.229509990212115,-0.021227576267148107,1.250737566479263,2.501475132958526,affine,0.13078781192491887,0.26157562384983774,0.9319962481713715,0.14033083521691406,eta_l0_n80_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,80,1,-1.2719651427464111,1.229509990212115,-0.021227576267148107,1.250737566479263,2.501475132958526,affine,0.3598172673204265,0.719634534640853,1.0,0.3598172673204265,eta_l0_n80_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,81,0,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,affine,0.023591200034515373,0.047182400069030746,0.9319962481713715,0.025312548286329072,eta_l0_n81_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,81,1,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,affine,0.14477030561706322,0.28954061123412644,1.0,0.14477030561706322,eta_l0_n81_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,82,0,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.046288370646411676,0.09257674129282335,0.9319962481713715,0.049665833673935955,eta_l0_n82_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,82,1,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.21485561047209328,0.42971122094418657,1.0,0.21485561047209328,eta_l0_n82_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,83,0,-0.8932013736751792,0.8717751463069086,-0.0107131136841353,0.8824882599910439,1.7649765199820877,affine,0.06135632904886263,0.12271265809772526,0.9319962481713715,0.0658332360985864,eta_l0_n83_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,83,1,-0.8932013736751792,0.8717751463069086,-0.0107131136841353,0.8824882599910439,1.7649765199820877,affine,0.250342804135403,0.500685608270806,1.0,0.250342804135403,eta_l0_n83_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,84,0,-0.7976625548491152,0.8270409447667988,0.0146891949588418,0.812351749807957,1.624703499615914,affine,0.05031809937737942,0.10063619875475883,0.9319962481713715,0.05398959435309566,eta_l0_n84_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,84,1,-0.7976625548491152,0.8270409447667988,0.0146891949588418,0.812351749807957,1.624703499615914,affine,0.22494720723711653,0.44989441447423306,1.0,0.22494720723711653,eta_l0_n84_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,85,0,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.06751399250682856,0.13502798501365712,0.9319962481713715,0.07244019773609044,eta_l0_n85_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,85,1,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.26306409874430814,0.5261281974886163,1.0,0.26306409874430814,eta_l0_n85_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,86,0,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,affine,0.1063931461831313,0.2127862923662626,0.9319962481713715,0.11415619579143217,eta_l0_n86_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,86,1,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,affine,0.3285811233329348,0.6571622466658696,1.0,0.3285811233329348,eta_l0_n86_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,87,0,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,affine,0.13197455335417327,0.26394910670834654,0.9319962481713715,0.14160416805659323,eta_l0_n87_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,87,1,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,affine,0.3611638067539207,0.7223276135078414,1.0,0.3611638067539207,eta_l0_n87_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,88,0,-0.8196835402297238,0.8511525501491992,0.015734504959737716,0.8354180451894615,1.670836090378923,affine,0.053858323789688196,0.10771664757937639,0.9319962481713715,0.05778813369191263,eta_l0_n88_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,88,1,-0.8196835402297238,0.8511525501491992,0.015734504959737716,0.8354180451894615,1.670836090378923,affine,0.2333895181269373,0.4667790362538746,1.0,0.2333895181269373,eta_l0_n88_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,89,0,-0.8579518670180813,0.8906473389988186,0.01634773599036865,0.87429960300845,1.7485992060169,affine,0.06004773422309601,0.12009546844619202,0.9319962481713715,0.06442915874491234,eta_l0_n89_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,89,1,-0.8579518670180813,0.8906473389988186,0.01634773599036865,0.87429960300845,1.7485992060169,affine,0.2473735088482814,0.4947470176965628,1.0,0.2473735088482814,eta_l0_n89_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,90,0,-0.8207744817317322,0.7924840586718498,-0.014145211529941193,0.806629270201791,1.613258540403582,affine,0.04945446424308742,0.09890892848617484,0.9319962481713715,0.05306294348300203,eta_l0_n90_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,90,1,-0.8207744817317322,0.7924840586718498,-0.014145211529941193,0.806629270201791,1.613258540403582,affine,0.22283997146248546,0.44567994292497093,1.0,0.22283997146248546,eta_l0_n90_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,91,0,-1.2894025896139478,1.2579584264319816,-0.015722081590983095,1.2736805080229647,2.5473610160459295,affine,0.13547413367370534,0.2709482673474107,0.9319962481713715,0.14535909767825045,eta_l0_n91_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,91,1,-1.2894025896139478,1.2579584264319816,-0.015722081590983095,1.2736805080229647,2.5473610160459295,affine,0.3652544063984313,0.7305088127968626,1.0,0.3652544063984313,eta_l0_n91_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,92,0,-1.1249469056327446,1.0858598844704395,-0.019543510581152557,1.105403395051592,2.210806790103184,affine,0.10169126819839032,0.20338253639678064,0.9319962481713715,0.10911124202260926,eta_l0_n92_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,92,1,-1.1249469056327446,1.0858598844704395,-0.019543510581152557,1.105403395051592,2.210806790103184,affine,0.32179887662215556,0.6435977532443111,1.0,0.32179887662215556,eta_l0_n92_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,93,0,-0.6468941826251269,0.6720388239177191,0.012572320646296098,0.659466503271423,1.318933006542846,affine,0.02973109792160489,0.05946219584320978,0.9319962481713715,0.03190044807577172,eta_l0_n93_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,93,1,-0.6468941826251269,0.6720388239177191,0.012572320646296098,0.659466503271423,1.318933006542846,affine,0.16698133324584757,0.33396266649169515,1.0,0.16698133324584757,eta_l0_n93_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,94,0,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,affine,0.12258952797033353,0.24517905594066705,0.9319962481713715,0.13153435779474543,eta_l0_n94_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,94,1,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,affine,0.3499788038850754,0.6999576077701508,1.0,0.3499788038850754,eta_l0_n94_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,95,0,-0.8425271444430554,0.8784439097769791,0.017958382666961814,0.8604855271100172,1.7209710542200345,affine,0.05782891292611981,0.11565782585223962,0.9319962481713715,0.0620484396150557,eta_l0_n95_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,95,1,-0.8425271444430554,0.8784439097769791,0.017958382666961814,0.8604855271100172,1.7209710542200345,affine,0.2424173386872116,0.4848346773744232,1.0,0.2424173386872116,eta_l0_n95_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,96,0,-0.6659994683686798,0.635867396179219,-0.01506603609473045,0.6509334322739494,1.3018668645478988,affine,0.0287543739277934,0.0575087478555868,0.9319962481713715,0.030852456739188686,eta_l0_n96_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,96,1,-0.6659994683686798,0.635867396179219,-0.01506603609473045,0.6509334322739494,1.3018668645478988,affine,0.16366932524311711,0.32733865048623423,1.0,0.16366932524311711,eta_l0_n96_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,97,0,-1.2151238777580005,1.1786727174263023,-0.018225580165849076,1.1968982975921514,2.3937965951843028,affine,0.11982580429260405,0.2396516085852081,0.9319962481713715,0.12856897710447757,eta_l0_n97_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,97,1,-1.2151238777580005,1.1786727174263023,-0.018225580165849076,1.1968982975921514,2.3937965951843028,affine,0.34658140653981456,0.6931628130796291,1.0,0.34658140653981456,eta_l0_n97_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,98,0,-1.2733732597983927,1.309028506901763,0.017827623551685212,1.2912008833500779,2.5824017667001558,affine,0.13909500430324453,0.27819000860648907,0.9319962481713715,0.14924416764140055,eta_l0_n98_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,98,1,-1.2733732597983927,1.309028506901763,0.017827623551685212,1.2912008833500779,2.5824017667001558,affine,0.36921820119777526,0.7384364023955505,1.0,0.36921820119777526,eta_l0_n98_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,99,0,-0.9135757537667057,0.8771689762111492,-0.018203388777778273,0.8953723649889275,1.790744729977855,affine,0.06352641759977425,0.1270528351995485,0.9319962481713715,0.06816166666380537,eta_l0_n99_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,99,1,-0.9135757537667057,0.8771689762111492,-0.018203388777778273,0.8953723649889275,1.790744729977855,affine,0.25477731278545024,0.5095546255709005,1.0,0.25477731278545024,eta_l0_n99_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,100,0,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,affine,0.17294317701844078,0.34588635403688156,0.9319962481713715,0.18556209572491833,eta_l0_n100_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,100,1,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,affine,0.40173890570989007,0.8034778114197801,1.0,0.40173890570989007,eta_l0_n100_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,101,0,-1.583787966015745,1.5801226097140433,-0.0018326781508508638,1.5819552878648941,3.1639105757297883,affine,0.1996618192557774,0.3993236385115548,0.9319962481713715,0.2142302822007331,eta_l0_n101_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,101,1,-1.583787966015745,1.5801226097140433,-0.0018326781508508638,1.5819552878648941,3.1639105757297883,quadratic,0.12229427866343796,0.24458855732687593,1.0,0.12229427866343796,eta_l0_n101_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,102,0,-1.0583150318257148,1.026256340812422,-0.016029345506646475,1.0422856863190684,2.0845713726381367,affine,0.08960906604109521,0.17921813208219042,0.9319962481713715,0.09614745361573417,eta_l0_n102_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,102,1,-1.0583150318257148,1.026256340812422,-0.016029345506646475,1.0422856863190684,2.0845713726381367,affine,0.30315034046603295,0.6063006809320659,1.0,0.30315034046603295,eta_l0_n102_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,103,0,-1.1476452573399842,1.1090705510943302,-0.019287353122827033,1.1283579042171572,2.2567158084343144,affine,0.10617588471459573,0.21235176942919146,0.9319962481713715,0.11392308168935092,eta_l0_n103_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,103,1,-1.1476452573399842,1.1090705510943302,-0.019287353122827033,1.1283579042171572,2.2567158084343144,affine,0.32827755135072456,0.6565551027014491,1.0,0.32827755135072456,eta_l0_n103_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,104,0,-0.8577016693145987,0.8877385522479757,0.015018441466688515,0.8727201107812872,1.7454402215625744,affine,0.05978300541739091,0.11956601083478181,0.9319962481713715,0.06414511381852502,eta_l0_n104_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,104,1,-0.8577016693145987,0.8877385522479757,0.015018441466688515,0.8727201107812872,1.7454402215625744,affine,0.24683057948606552,0.49366115897213103,1.0,0.24683057948606552,eta_l0_n104_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,105,0,-1.1779127957269429,1.2185421121727307,0.02031465822289391,1.1982274539498368,2.3964549078996735,affine,0.12010463419158925,0.2402092683831785,0.9319962481713715,0.12886815202018378,eta_l0_n105_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,105,1,-1.1779127957269429,1.2185421121727307,0.02031465822289391,1.1982274539498368,2.3964549078996735,affine,0.34689200531234776,0.6937840106246955,1.0,0.34689200531234776,eta_l0_n105_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,106,0,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,affine,0.03822436614129144,0.07644873228258288,0.9319962481713715,0.04101343349427616,eta_l0_n106_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,106,1,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,affine,0.19290116812859953,0.38580233625719906,1.0,0.19290116812859953,eta_l0_n106_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,107,0,-0.6364463917753737,0.563347610914743,-0.036549390430315354,0.5998970013450583,1.1997940026901166,affine,0.023429717443423096,0.04685943488684619,0.9319962481713715,0.02513928300612101,eta_l0_n107_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,107,1,-0.6364463917753737,0.563347610914743,-0.036549390430315354,0.5998970013450583,1.1997940026901166,affine,0.14365662849300698,0.28731325698601395,1.0,0.14365662849300698,eta_l0_n107_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,108,0,-1.0426471018400016,1.0803325368651484,0.01884271751257338,1.061489819352575,2.12297963870515,affine,0.09324930213724093,0.18649860427448187,0.9319962481713715,0.10005330206017594,eta_l0_n108_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,108,1,-1.0426471018400016,1.0803325368651484,0.01884271751257338,1.061489819352575,2.12297963870515,affine,0.3089373526281335,0.617874705256267,1.0,0.3089373526281335,eta_l0_n108_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,109,0,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,affine,0.06945072467805004,0.13890144935610008,0.9319962481713715,0.07451824491173245,eta_l0_n109_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,109,1,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,affine,0.26690106136144665,0.5338021227228933,1.0,0.26690106136144665,eta_l0_n109_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,110,0,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.1296818009212755,0.259363601842551,0.9319962481713715,0.13914412335427143,eta_l0_n110_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,110,1,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.3585026397894039,0.7170052795788078,1.0,0.3585026397894039,eta_l0_n110_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,111,0,-0.958157890561465,0.9243831398125949,-0.016887375374435076,0.94127051518703,1.88254103037406,affine,0.07132673835576632,0.14265347671153264,0.9319962481713715,0.07653114322693182,eta_l0_n111_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,111,1,-0.958157890561465,0.9243831398125949,-0.016887375374435076,0.94127051518703,1.88254103037406,affine,0.2705801126588766,0.5411602253177532,1.0,0.2705801126588766,eta_l0_n111_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,112,0,-0.7914234807966222,0.818891668119116,0.013734093661246893,0.8051575744578691,1.6103151489157381,affine,0.04923190847633619,0.09846381695267238,0.9319962481713715,0.05282414878056852,eta_l0_n112_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,112,1,-0.7914234807966222,0.818891668119116,0.013734093661246893,0.8051575744578691,1.6103151489157381,affine,0.22230035899282263,0.44460071798564527,1.0,0.22230035899282263,eta_l0_n112_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,113,0,-0.8496861508209814,0.8720170059064725,0.01116542754274552,0.8608515783637269,1.7217031567274539,affine,0.05784896434081916,0.11569792868163832,0.9319962481713715,0.062069954095117924,eta_l0_n113_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,113,1,-0.8496861508209814,0.8720170059064725,0.01116542754274552,0.8608515783637269,1.7217031567274539,affine,0.24263703680786103,0.48527407361572206,1.0,0.24263703680786103,eta_l0_n113_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,114,0,-0.6630886315286884,0.6437069553096086,-0.009690838109539857,0.6533977934191485,1.306795586838297,affine,0.029010392935394484,0.05802078587078897,0.9319962481713715,0.031127156351020178,eta_l0_n114_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,114,1,-0.6630886315286884,0.6437069553096086,-0.009690838109539857,0.6533977934191485,1.306795586838297,affine,0.164672309515897,0.329344619031794,1.0,0.164672309515897,eta_l0_n114_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,115,0,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.039720764451005815,0.07944152890201163,0.9319962481713715,0.04261901754319308,eta_l0_n115_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,115,1,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.19732520226170253,0.39465040452340505,1.0,0.19732520226170253,eta_l0_n115_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,116,0,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.0629088693592051,0.1258177387184102,0.9319962481713715,0.06749905858809604,eta_l0_n116_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,116,1,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.253502859142495,0.50700571828499,1.0,0.253502859142495,eta_l0_n116_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,117,0,-1.3740507569108842,1.4125160594440573,0.019232651266586576,1.3932834081774708,2.7865668163549415,affine,0.1603277382012225,0.320655476402445,0.9319962481713715,0.17202616267586318,eta_l0_n117_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,117,1,-1.3740507569108842,1.4125160594440573,0.019232651266586576,1.3932834081774708,2.7865668163549415,affine,0.39052967606442457,0.7810593521288491,1.0,0.39052967606442457,eta_l0_n117_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,118,0,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.03443316994739421,0.06886633989478842,0.9319962481713715,0.03694561004398248,eta_l0_n118_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,118,1,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.1818036059699658,0.3636072119399316,1.0,0.1818036059699658,eta_l0_n118_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,119,0,-1.1687352625088665,1.1298298900732497,-0.01945268621780838,1.1492825762910581,2.2985651525821162,affine,0.11030672337739635,0.2206134467547927,0.9319962481713715,0.11835532985655713,eta_l0_n119_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,119,1,-1.1687352625088665,1.1298298900732497,-0.01945268621780838,1.1492825762910581,2.2985651525821162,affine,0.3340246272819401,0.6680492545638802,1.0,0.3340246272819401,eta_l0_n119_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,120,0,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.041338081872310646,0.08267616374462129,0.9319962481713715,0.044354343650436646,eta_l0_n120_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,120,1,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.20172634433294967,0.40345268866589934,1.0,0.20172634433294967,eta_l0_n120_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,121,0,-0.666212706257072,0.6906969277491457,0.012242110746036872,0.6784548170031088,1.3569096340062177,affine,0.03199782957754133,0.06399565915508267,0.9319962481713715,0.034332573377116975,eta_l0_n121_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,121,1,-0.666212706257072,0.6906969277491457,0.012242110746036872,0.6784548170031088,1.3569096340062177,affine,0.17429024028187629,0.34858048056375257,1.0,0.17429024028187629,eta_l0_n121_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,122,0,-0.8964792612600482,0.9290067815766619,0.016263760158306884,0.9127430214183551,1.8254860428367101,affine,0.06642863621157087,0.13285727242314174,0.9319962481713715,0.0712756476669381,eta_l0_n122_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,122,1,-0.8964792612600482,0.9290067815766619,0.016263760158306884,0.9127430214183551,1.8254860428367101,affine,0.26084735534552284,0.5216947106910457,1.0,0.26084735534552284,eta_l0_n122_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,123,0,-1.1041675633687147,1.1420328248526195,0.018932630741952394,1.123100194110667,2.246200388221334,affine,0.10514210954629312,0.21028421909258624,0.9319962481713715,0.11281387639981147,eta_l0_n123_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,123,1,-1.1041675633687147,1.1420328248526195,0.018932630741952394,1.123100194110667,2.246200388221334,affine,0.3268151844988936,0.6536303689977871,1.0,0.3268151844988936,eta_l0_n123_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,124,0,-0.7854236701342572,0.7505279392833624,-0.017447865425447406,0.7679758047088098,1.5359516094176195,affine,0.04383925134039441,0.08767850268078882,0.9319962481713715,0.04703801268128435,eta_l0_n124_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,124,1,-0.7854236701342572,0.7505279392833624,-0.017447865425447406,0.7679758047088098,1.5359516094176195,affine,0.2083621946695587,0.4167243893391174,1.0,0.2083621946695587,eta_l0_n124_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,125,0,-1.3977310453244942,1.362160377008622,-0.01778533415793615,1.379945711166558,2.759891422333116,affine,0.1575371349875396,0.3150742699750792,0.9319962481713715,0.16903194116568196,eta_l0_n125_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,125,1,-1.3977310453244942,1.362160377008622,-0.01778533415793615,1.379945711166558,2.759891422333116,affine,0.38793782616216815,0.7758756523243363,1.0,0.38793782616216815,eta_l0_n125_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,126,0,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.11361256119173553,0.22722512238347106,0.9319962481713715,0.12190238041692732,eta_l0_n126_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,126,1,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.33846737843434105,0.6769347568686821,1.0,0.33846737843434105,eta_l0_n126_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,127,0,-1.1653861727789858,1.1295881895992121,-0.017898991589886837,1.147487181189099,2.294974362378198,affine,0.10994269110721135,0.2198853822144227,0.9319962481713715,0.11796473571961799,eta_l0_n127_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,127,1,-1.1653861727789858,1.1295881895992121,-0.017898991589886837,1.147487181189099,2.294974362378198,affine,0.33355880708885166,0.6671176141777033,1.0,0.33355880708885166,eta_l0_n127_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,128,0,-0.9183696950325081,0.9538988826345168,0.01776459380100437,0.9361342888335125,1.872268577667025,affine,0.07044155189568786,0.14088310379137572,0.9319962481713715,0.07558136852363742,eta_l0_n128_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,128,1,-0.9183696950325081,0.9538988826345168,0.01776459380100437,0.9361342888335125,1.872268577667025,affine,0.2688296286984237,0.5376592573968474,1.0,0.2688296286984237,eta_l0_n128_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,129,0,-0.8382147723907126,0.8049901321285021,-0.01661232013110525,0.8216024522596074,1.6432049045192147,affine,0.0517349305265229,0.1034698610530458,0.9319962481713715,0.05550980556845558,eta_l0_n129_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,129,1,-0.8382147723907126,0.8049901321285021,-0.01661232013110525,0.8216024522596074,1.6432049045192147,affine,0.2283246773501364,0.4566493547002728,1.0,0.2283246773501364,eta_l0_n129_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,130,0,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,affine,0.08784979328981053,0.17569958657962106,0.9319962481713715,0.0942598143095283,eta_l0_n130_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,130,1,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,affine,0.30022639347126356,0.6004527869425271,1.0,0.30022639347126356,eta_l0_n130_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,131,0,-1.047450072089849,1.0179226028488386,-0.014763734620505176,1.0326863374693438,2.0653726749386876,affine,0.08780704324716622,0.17561408649433244,0.9319962481713715,0.09421394498041008,eta_l0_n131_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,131,1,-1.047450072089849,1.0179226028488386,-0.014763734620505176,1.0326863374693438,2.0653726749386876,affine,0.3002092776642886,0.6004185553285772,1.0,0.3002092776642886,eta_l0_n131_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,132,0,-1.060935041080445,1.0970640029002237,0.018064480909889324,1.0789995219903343,2.1579990439806687,affine,0.09658521996959714,0.19317043993919428,0.9319962481713715,0.10363262744791378,eta_l0_n132_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,132,1,-1.060935041080445,1.0970640029002237,0.018064480909889324,1.0789995219903343,2.1579990439806687,affine,0.3141573803296547,0.6283147606593094,1.0,0.3141573803296547,eta_l0_n132_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,133,0,-1.253402395011484,1.2165648038644405,-0.018418795573521773,1.2349835994379623,2.4699671988759246,affine,0.12755412399079488,0.25510824798158976,0.9319962481713715,0.13686119900274618,eta_l0_n133_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,133,1,-1.253402395011484,1.2165648038644405,-0.018418795573521773,1.2349835994379623,2.4699671988759246,affine,0.356068269207211,0.712136538414422,1.0,0.356068269207211,eta_l0_n133_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,134,0,-1.0114330530784947,0.9766652867437464,-0.01738388316737416,0.9940491699111206,1.9880983398222412,affine,0.08071068741298588,0.16142137482597177,0.9319962481713715,0.08659979862724206,eta_l0_n134_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,134,1,-1.0114330530784947,0.9766652867437464,-0.01738388316737416,0.9940491699111206,1.9880983398222412,affine,0.2879753490369872,0.5759506980739744,1.0,0.2879753490369872,eta_l0_n134_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,135,0,-1.283272451573751,1.3273446149675863,0.02203608169691762,1.3053085332706686,2.6106170665413373,affine,0.142031026455352,0.284062052910704,0.9319962481713715,0.15239441868357817,eta_l0_n135_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,135,1,-1.283272451573751,1.3273446149675863,0.02203608169691762,1.3053085332706686,2.6106170665413373,affine,0.3723051142259514,0.7446102284519028,1.0,0.3723051142259514,eta_l0_n135_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,136,0,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.055506724224711264,0.11101344844942253,0.9319962481713715,0.059556810806501145,eta_l0_n136_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,136,1,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.23735487630608,0.47470975261216,1.0,0.23735487630608,eta_l0_n136_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,137,0,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.1106683443332991,0.2213366886665982,0.9319962481713715,0.11874333673599712,eta_l0_n137_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,137,1,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.33448127202207456,0.6689625440441491,1.0,0.33448127202207456,eta_l0_n137_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,138,0,-1.4795147536405517,1.4449804711041994,-0.01726714126817619,1.4622476123723755,2.924495224744751,affine,0.1747409545584442,0.3494819091168884,0.9319962481713715,0.18749104934842353,eta_l0_n138_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,138,1,-1.4795147536405517,1.4449804711041994,-0.01726714126817619,1.4622476123723755,2.924495224744751,affine,0.4032069921724032,0.8064139843448064,1.0,0.4032069921724032,eta_l0_n138_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,139,0,-1.1846112842973124,1.1451154059873383,-0.01974793915498707,1.1648633451423254,2.3297266902846507,affine,0.11340669701661678,0.22681339403323356,0.9319962481713715,0.12168149521967178,eta_l0_n139_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,139,1,-1.1846112842973124,1.1451154059873383,-0.01974793915498707,1.1648633451423254,2.3297266902846507,affine,0.3382067958346997,0.6764135916693994,1.0,0.3382067958346997,eta_l0_n139_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,140,0,-1.0016612677883296,1.0308531994790653,0.01459596584536782,1.0162572336336975,2.032514467267395,affine,0.08475915274863734,0.16951830549727467,0.9319962481713715,0.0909436630404248,eta_l0_n140_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,140,1,-1.0016612677883296,1.0308531994790653,0.01459596584536782,1.0162572336336975,2.032514467267395,affine,0.2950819976681076,0.5901639953362152,1.0,0.2950819976681076,eta_l0_n140_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,141,0,-0.8542154631766357,0.8897074866463954,0.017746011734879885,0.8719614749115155,1.743922949823031,affine,0.05967715153016984,0.11935430306033969,0.9319962481713715,0.06403153622909935,eta_l0_n141_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,141,1,-0.8542154631766357,0.8897074866463954,0.017746011734879885,0.8719614749115155,1.743922949823031,affine,0.24652077232990388,0.49304154465980776,1.0,0.24652077232990388,eta_l0_n141_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,142,0,-1.0907597002338707,1.1219900064229051,0.01561515309451722,1.106374853328388,2.212749706656776,affine,0.10185940078171347,0.20371880156342695,0.9319962481713715,0.10929164251633769,eta_l0_n142_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,142,1,-1.0907597002338707,1.1219900064229051,0.01561515309451722,1.106374853328388,2.212749706656776,affine,0.3221303897798582,0.6442607795597164,1.0,0.3221303897798582,eta_l0_n142_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,143,0,-0.8162896758725653,0.8281241957461958,0.0059172599368152445,0.8222069358093805,1.644413871618761,affine,0.05177898004970977,0.10355796009941955,0.9319962481713715,0.05555706919561427,eta_l0_n143_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,143,1,-0.8162896758725653,0.8281241957461958,0.0059172599368152445,0.8222069358093805,1.644413871618761,affine,0.22865439410150246,0.4573087882030049,1.0,0.22865439410150246,eta_l0_n143_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,144,0,-1.0737179807403916,1.0426127742833173,-0.015552603228537132,1.0581653775118545,2.116330755023709,affine,0.09260083445632895,0.1852016689126579,0.9319962481713715,0.0993575184857417,eta_l0_n144_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,144,1,-1.0737179807403916,1.0426127742833173,-0.015552603228537132,1.0581653775118545,2.116330755023709,affine,0.30798318336415104,0.6159663667283021,1.0,0.30798318336415104,eta_l0_n144_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,145,0,-0.908246631810817,0.8772921563875723,-0.01547723771162235,0.8927693940991946,1.7855387881983893,affine,0.06307662244905878,0.12615324489811755,0.9319962481713715,0.06767905189835112,eta_l0_n145_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,145,1,-0.908246631810817,0.8772921563875723,-0.01547723771162235,0.8927693940991946,1.7855387881983893,affine,0.253906143084322,0.507812286168644,1.0,0.253906143084322,eta_l0_n145_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,146,0,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.145374579693725,0.29074915938745,0.9319962481713715,0.15598193660002174,eta_l0_n146_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,146,1,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.37586776251971343,0.7517355250394269,1.0,0.37586776251971343,eta_l0_n146_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,147,0,-0.9610675137578785,0.9233653689554445,-0.018851072401216973,0.9422164413566615,1.884432882713323,affine,0.071503963689264,0.143007927378528,0.9319962481713715,0.07672129992964967,eta_l0_n147_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,147,1,-0.9610675137578785,0.9233653689554445,-0.018851072401216973,0.9422164413566615,1.884432882713323,affine,0.27086854893595547,0.5417370978719109,1.0,0.27086854893595547,eta_l0_n147_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,148,0,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.16491939624199328,0.32983879248398656,0.9319962481713715,0.17695285422615628,eta_l0_n148_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,148,1,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.3947057909211337,0.7894115818422675,1.0,0.3947057909211337,eta_l0_n148_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,149,0,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,affine,0.05466287032502593,0.10932574065005186,0.9319962481713715,0.05865138452249945,eta_l0_n149_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,149,1,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,affine,0.23527771722316493,0.47055543444632986,1.0,0.23527771722316493,eta_l0_n149_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,150,0,-0.81442882440053,0.8392548862181488,0.012413030908809408,0.8268418553093394,1.6536837106186788,affine,0.0525136087308466,0.1050272174616932,0.9319962481713715,0.05634530056733729,eta_l0_n150_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,150,1,-0.81442882440053,0.8392548862181488,0.012413030908809408,0.8268418553093394,1.6536837106186788,affine,0.23029984075508209,0.46059968151016417,1.0,0.23029984075508209,eta_l0_n150_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,151,0,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,affine,0.04347210480581693,0.08694420961163386,0.9319962481713715,0.04664407704549414,eta_l0_n151_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,151,1,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,affine,0.20749637537672616,0.4149927507534523,1.0,0.20749637537672616,eta_l0_n151_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,152,0,-1.2728778080271927,1.3140653792633163,0.0205937856180618,1.2934715936452545,2.586943187290509,affine,0.13957622498198222,0.27915244996396443,0.9319962481713715,0.1497605009202972,eta_l0_n152_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,152,1,-1.2728778080271927,1.3140653792633163,0.0205937856180618,1.2934715936452545,2.586943187290509,affine,0.36969406958175527,0.7393881391635105,1.0,0.36969406958175527,eta_l0_n152_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,153,0,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.03558178463885487,0.07116356927770974,0.9319962481713715,0.038178034202034944,eta_l0_n153_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,153,1,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.18526418828007996,0.3705283765601599,1.0,0.18526418828007996,eta_l0_n153_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,154,0,-0.79668091660671,0.7634063707671254,-0.016637272919792334,0.7800436436869177,1.5600872873738354,affine,0.04555995415804837,0.09111990831609675,0.9319962481713715,0.04888426777193528,eta_l0_n154_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,154,1,-0.79668091660671,0.7634063707671254,-0.016637272919792334,0.7800436436869177,1.5600872873738354,affine,0.21290628402333134,0.4258125680466627,1.0,0.21290628402333134,eta_l0_n154_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,155,0,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.060409021979322915,0.12081804395864583,0.9319962481713715,0.06481680811253134,eta_l0_n155_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,155,1,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.24812396510144952,0.49624793020289903,1.0,0.24812396510144952,eta_l0_n155_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,156,0,-0.9292268209994728,0.9689797402104269,0.019876459605477015,0.9491032806049499,1.8982065612098997,affine,0.07271276999907768,0.14542553999815536,0.9319962481713715,0.07801830762918217,eta_l0_n156_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,156,1,-0.9292268209994728,0.9689797402104269,0.019876459605477015,0.9491032806049499,1.8982065612098997,affine,0.27316687115563565,0.5463337423112713,1.0,0.27316687115563565,eta_l0_n156_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,157,0,-1.1731237389534528,1.2050166691047852,0.015946465075666216,1.189070204029119,2.378140408058238,affine,0.11823823949624425,0.2364764789924885,0.9319962481713715,0.12686557454306738,eta_l0_n157_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,157,1,-1.1731237389534528,1.2050166691047852,0.015946465075666216,1.189070204029119,2.378140408058238,affine,0.34459962052402604,0.6891992410480521,1.0,0.34459962052402604,eta_l0_n157_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,158,0,-0.8913860173428696,0.9226653347131664,0.015639658685148383,0.907025676028018,1.814051352056036,affine,0.06545990610854242,0.13091981221708485,0.9319962481713715,0.07023623350091635,eta_l0_n158_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,158,1,-0.8913860173428696,0.9226653347131664,0.015639658685148383,0.907025676028018,1.814051352056036,affine,0.25887689896386723,0.5177537979277345,1.0,0.25887689896386723,eta_l0_n158_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,159,0,-1.1273921839493979,1.0959081779819333,-0.015742002983732295,1.1116501809656656,2.223300361931331,affine,0.10288727370103208,0.20577454740206416,0.9319962481713715,0.11039451489520762,eta_l0_n159_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,159,1,-1.1273921839493979,1.0959081779819333,-0.015742002983732295,1.1116501809656656,2.223300361931331,affine,0.32363040412414623,0.6472608082482925,1.0,0.32363040412414623,eta_l0_n159_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,160,0,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.1775418997412546,0.3550837994825092,0.9319962481713715,0.19049636743667336,eta_l0_n160_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,160,1,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.4055171391129622,0.8110342782259244,1.0,0.4055171391129622,eta_l0_n160_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,161,0,-0.8118887083898723,0.7855637226605154,-0.01316249286467841,0.7987262155251939,1.5974524310503877,affine,0.04827148630569767,0.09654297261139534,0.9319962481713715,0.05179364874097832,eta_l0_n161_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,161,1,-0.8118887083898723,0.7855637226605154,-0.01316249286467841,0.7987262155251939,1.5974524310503877,affine,0.21992204690021305,0.4398440938004261,1.0,0.21992204690021305,eta_l0_n161_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,162,0,-0.9685612086655193,0.9950593658781908,0.013249078606335729,0.981810287271855,1.96362057454371,affine,0.07847764604365189,0.15695529208730377,0.9319962481713715,0.08420382184759799,eta_l0_n162_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,162,1,-0.9685612086655193,0.9950593658781908,0.013249078606335729,0.981810287271855,1.96362057454371,affine,0.28406853154096634,0.5681370630819327,1.0,0.28406853154096634,eta_l0_n162_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,163,0,-0.8271071303752251,0.8610459715485377,0.016969420586656292,0.8440765509618814,1.6881531019237628,affine,0.055219362719068694,0.11043872543813739,0.9319962481713715,0.05924848176954806,eta_l0_n163_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,163,1,-0.8271071303752251,0.8610459715485377,0.016969420586656292,0.8440765509618814,1.6881531019237628,affine,0.2365171171742459,0.4730342343484918,1.0,0.2365171171742459,eta_l0_n163_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,164,0,-1.1176997476072552,1.1538089483476912,0.018054600370218,1.1357543479774732,2.2715086959549464,affine,0.10762484486853492,0.21524968973706984,0.9319962481713715,0.11547776622459678,eta_l0_n164_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,164,1,-1.1176997476072552,1.1538089483476912,0.018054600370218,1.1357543479774732,2.2715086959549464,affine,0.33034402088890125,0.6606880417778025,1.0,0.33034402088890125,eta_l0_n164_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,165,0,-0.9571068847382636,0.9915651490227835,0.017229132142259962,0.9743360168805235,1.948672033761047,affine,0.07715881571356727,0.15431763142713453,0.9319962481713715,0.08278876214894336,eta_l0_n165_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,165,1,-0.9571068847382636,0.9915651490227835,0.017229132142259962,0.9743360168805235,1.948672033761047,affine,0.28157507368831103,0.5631501473766221,1.0,0.28157507368831103,eta_l0_n165_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,166,0,-1.1951923778641027,1.1529662485775045,-0.021113064643299095,1.1740793132208036,2.348158626441607,affine,0.11525571941871461,0.23051143883742922,0.9319962481713715,0.12366543282213072,eta_l0_n166_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,166,1,-1.1951923778641027,1.1529662485775045,-0.021113064643299095,1.1740793132208036,2.348158626441607,affine,0.3406246922472449,0.6812493844944898,1.0,0.3406246922472449,eta_l0_n166_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,167,0,-1.1001154688531452,1.072608887706285,-0.01375329057343011,1.086362178279715,2.17272435655943,affine,0.0979784771275863,0.1959569542551726,0.9319962481713715,0.10512754457952542,eta_l0_n167_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,167,1,-1.1001154688531452,1.072608887706285,-0.01375329057343011,1.086362178279715,2.17272435655943,affine,0.31637193416115644,0.6327438683223129,1.0,0.31637193416115644,eta_l0_n167_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,168,0,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.09087186490139544,0.18174372980279088,0.9319962481713715,0.09750239346959937,eta_l0_n168_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,168,1,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.3051084494179191,0.6102168988358382,1.0,0.3051084494179191,eta_l0_n168_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,169,0,-0.8810525006112326,0.8477341734459158,-0.0166591635826584,0.8643933370285742,1.7287866740571485,affine,0.05844734457479079,0.11689468914958158,0.9319962481713715,0.06271199555735094,eta_l0_n169_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,169,1,-0.8810525006112326,0.8477341734459158,-0.0166591635826584,0.8643933370285742,1.7287866740571485,affine,0.2438371059123533,0.4876742118247066,1.0,0.2438371059123533,eta_l0_n169_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,170,0,-0.8468298613304297,0.8765102975610901,0.014840218115330206,0.8616700794457599,1.7233401588915198,affine,0.05799879967277095,0.1159975993455419,0.9319962481713715,0.06223072226585442,eta_l0_n170_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,170,1,-0.8468298613304297,0.8765102975610901,0.014840218115330206,0.8616700794457599,1.7233401588915198,affine,0.24288770303173635,0.4857754060634727,1.0,0.24288770303173635,eta_l0_n170_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,171,0,-1.1567718049908575,1.1214757897709777,-0.017648007609939897,1.1391237973809176,2.2782475947618352,affine,0.10828744905510314,0.21657489811020628,0.9319962481713715,0.11618871778461462,eta_l0_n171_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,171,1,-1.1567718049908575,1.1214757897709777,-0.017648007609939897,1.1391237973809176,2.2782475947618352,affine,0.3312768331711643,0.6625536663423286,1.0,0.3312768331711643,eta_l0_n171_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,172,0,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,affine,0.028361496955946756,0.05672299391189351,0.9319962481713715,0.03043091322695085,eta_l0_n172_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,172,1,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,affine,0.1624303925044039,0.3248607850088078,1.0,0.1624303925044039,eta_l0_n172_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,173,0,-0.6736564656359886,0.6502014587413382,-0.011727503447325205,0.6619289621886634,1.3238579243773267,affine,0.030016059137825183,0.060032118275650366,0.9319962481713715,0.032206201684522186,eta_l0_n173_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,173,1,-0.6736564656359886,0.6502014587413382,-0.011727503447325205,0.6619289621886634,1.3238579243773267,affine,0.16793748849175272,0.33587497698350544,1.0,0.16793748849175272,eta_l0_n173_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,174,0,-1.267195182787436,1.2257910549274453,-0.020702063929995296,1.2464931188574406,2.4929862377148813,affine,0.12991639486248988,0.25983278972497975,0.9319962481713715,0.13939583460490648,eta_l0_n174_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,174,1,-1.267195182787436,1.2257910549274453,-0.020702063929995296,1.2464931188574406,2.4929862377148813,affine,0.3588123469275569,0.7176246938551138,1.0,0.3588123469275569,eta_l0_n174_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,175,0,-1.3831413857359587,1.428157169742103,0.022507892003072127,1.4056492777390308,2.8112985554780616,affine,0.16292477100837002,0.32584954201674005,0.9319962481713715,0.17481268978071263,eta_l0_n175_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,175,1,-1.3831413857359587,1.428157169742103,0.022507892003072127,1.4056492777390308,2.8112985554780616,affine,0.39286284593744103,0.7857256918748821,1.0,0.39286284593744103,eta_l0_n175_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,176,0,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,affine,0.08877578314687136,0.17755156629374272,0.9319962481713715,0.09525336965792983,eta_l0_n176_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,176,1,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,affine,0.30169992359450337,0.6033998471890067,1.0,0.30169992359450337,eta_l0_n176_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,177,0,-0.8553465281258207,0.839658428320634,-0.007844049902593353,0.8475024782232273,1.6950049564464547,affine,0.05571402887752242,0.11142805775504484,0.9319962481713715,0.05977924158690171,eta_l0_n177_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,177,1,-0.8553465281258207,0.839658428320634,-0.007844049902593353,0.8475024782232273,1.6950049564464547,affine,0.23785857921394304,0.4757171584278861,1.0,0.23785857921394304,eta_l0_n177_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,178,0,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.09034133178274757,0.18068266356549514,0.9319962481713715,0.09693314963445646,eta_l0_n178_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,178,1,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.304261390372095,0.60852278074419,1.0,0.304261390372095,eta_l0_n178_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,179,0,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.12411459017998151,0.24822918035996303,0.9319962481713715,0.1331706972249097,eta_l0_n179_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,179,1,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.35189623949265825,0.7037924789853165,1.0,0.35189623949265825,eta_l0_n179_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,180,0,-1.0633221779271937,1.0983992430851335,0.017538532578969868,1.0808607105061636,2.1617214210123272,affine,0.09693932115397964,0.1938786423079593,0.9319962481713715,0.1040125658704957,eta_l0_n180_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,180,1,-1.0633221779271937,1.0983992430851335,0.017538532578969868,1.0808607105061636,2.1617214210123272,affine,0.3147125704027749,0.6294251408055498,1.0,0.3147125704027749,eta_l0_n180_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,181,0,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.10855051376169962,0.21710102752339924,0.9319962481713715,0.11647097718974918,eta_l0_n181_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,181,1,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.3315986341833693,0.6631972683667386,1.0,0.3315986341833693,eta_l0_n181_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,182,0,-0.9468539302832147,0.9082390537521353,-0.01930743826553971,0.927546492017675,1.85509298403535,affine,0.06897207704411946,0.13794415408823893,0.9319962481713715,0.07400467242163959,eta_l0_n182_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,182,1,-0.9468539302832147,0.9082390537521353,-0.01930743826553971,0.927546492017675,1.85509298403535,affine,0.2658832386164963,0.5317664772329926,1.0,0.2658832386164963,eta_l0_n182_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,183,0,-0.9012005330706286,0.9364696817911623,0.01763457436026683,0.9188351074308955,1.837670214861791,affine,0.06747144526222183,0.13494289052444366,0.9319962481713715,0.07239454600230909,eta_l0_n183_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,183,1,-0.9012005330706286,0.9364696817911623,0.01763457436026683,0.9188351074308955,1.837670214861791,affine,0.26292614612453097,0.5258522922490619,1.0,0.26292614612453097,eta_l0_n183_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,184,0,-1.4793999358375651,1.4405298344240638,-0.019435050706750667,1.4599648851308145,2.919929770261629,affine,0.17427052042194865,0.3485410408438973,0.9319962481713715,0.18698628965929542,eta_l0_n184_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,184,1,-1.4793999358375651,1.4405298344240638,-0.019435050706750667,1.4599648851308145,2.919929770261629,affine,0.4027895634419543,0.8055791268839086,1.0,0.4027895634419543,eta_l0_n184_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,185,0,-0.7321511044331092,0.7389444504690766,0.0033966730179837423,0.7355477774510929,1.4710955549021858,affine,0.03929340910885843,0.07858681821771686,0.9319962481713715,0.04216047992248283,eta_l0_n185_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,185,1,-0.7321511044331092,0.7389444504690766,0.0033966730179837423,0.7355477774510929,1.4710955549021858,affine,0.1962133109401037,0.3924266218802074,1.0,0.1962133109401037,eta_l0_n185_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,186,0,-0.8942204521232298,0.8708685501338451,-0.011675950994692319,0.8825445011285374,1.7650890022570749,affine,0.06136968976427765,0.1227393795285553,0.9319962481713715,0.06584757168785647,eta_l0_n186_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,186,1,-0.8942204521232298,0.8708685501338451,-0.011675950994692319,0.8825445011285374,1.7650890022570749,affine,0.2503530688112541,0.5007061376225082,1.0,0.2503530688112541,eta_l0_n186_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,187,0,-1.0714555174736886,1.0473124800913576,-0.012071518691165517,1.059383998782523,2.118767997565046,affine,0.09281648755288631,0.18563297510577262,0.9319962481713715,0.09958890685986925,eta_l0_n187_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,187,1,-1.0714555174736886,1.0473124800913576,-0.012071518691165517,1.059383998782523,2.118767997565046,affine,0.3083893023272184,0.6167786046544368,1.0,0.3083893023272184,eta_l0_n187_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,188,0,-0.8561385666094258,0.8238401922534623,-0.01614918717798175,0.839989379431444,1.679978758862888,affine,0.05457352556779011,0.10914705113558022,0.9319962481713715,0.058555520663164044,eta_l0_n188_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,188,1,-0.8561385666094258,0.8238401922534623,-0.01614918717798175,0.839989379431444,1.679978758862888,affine,0.23504641413088656,0.4700928282617731,1.0,0.23504641413088656,eta_l0_n188_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,189,0,-1.055476274410492,1.0911960330035593,0.017859879296533654,1.0733361537070256,2.1466723074140512,affine,0.09549999712492331,0.19099999424984662,0.9319962481713715,0.10246822056665959,eta_l0_n189_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,189,1,-1.055476274410492,1.0911960330035593,0.017859879296533654,1.0733361537070256,2.1466723074140512,affine,0.31248672028935975,0.6249734405787195,1.0,0.31248672028935975,eta_l0_n189_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,190,0,-0.8409602339792388,0.8183358148780697,-0.01131220955058454,0.8296480244286543,1.6592960488573085,affine,0.052940748618230955,0.10588149723646191,0.9319962481713715,0.05680360701247848,eta_l0_n190_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,190,1,-0.8409602339792388,0.8183358148780697,-0.01131220955058454,0.8296480244286543,1.6592960488573085,affine,0.23133788022140234,0.4626757604428047,1.0,0.23133788022140234,eta_l0_n190_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,191,0,-0.9923663640591593,0.9567994816892675,-0.017783441184945903,0.9745829228742134,1.9491658457484269,affine,0.07720636019263961,0.15441272038527923,0.9319962481713715,0.08283977574386463,eta_l0_n191_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,191,1,-0.9923663640591593,0.9567994816892675,-0.017783441184945903,0.9745829228742134,1.9491658457484269,affine,0.2816476439026049,0.5632952878052098,1.0,0.2816476439026049,eta_l0_n191_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,192,0,-0.622907066400142,0.5904531146635353,-0.01622697586830335,0.6066800905318387,1.2133601810636774,affine,0.02390234411526149,0.04780468823052298,0.9319962481713715,0.025646395210451995,eta_l0_n192_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,192,1,-0.622907066400142,0.5904531146635353,-0.01622697586830335,0.6066800905318387,1.2133601810636774,affine,0.14666339508907572,0.29332679017815144,1.0,0.14666339508907572,eta_l0_n192_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,193,0,-0.7374416418193389,0.7107433024551729,-0.013349169682082995,0.7240924721372559,1.4481849442745118,affine,0.037799429557977056,0.07559885911595411,0.9319962481713715,0.04055749111881259,eta_l0_n193_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,193,1,-0.7374416418193389,0.7107433024551729,-0.013349169682082995,0.7240924721372559,1.4481849442745118,affine,0.19177416905298977,0.38354833810597955,1.0,0.19177416905298977,eta_l0_n193_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,194,0,-0.9262917881438898,0.9624928742496369,0.018100543052873563,0.9443923311967634,1.8887846623935267,affine,0.07187781562146196,0.14375563124292393,0.9319962481713715,0.07712243022704247,eta_l0_n194_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,194,1,-0.9262917881438898,0.9624928742496369,0.018100543052873563,0.9443923311967634,1.8887846623935267,affine,0.2716137525954708,0.5432275051909417,1.0,0.2716137525954708,eta_l0_n194_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,195,0,-1.4438322648629722,1.4816268957431415,0.01889731544008466,1.4627295803030569,2.9254591606061138,affine,0.17484682179969427,0.34969364359938854,0.9319962481713715,0.18760464126626417,eta_l0_n195_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,195,1,-1.4438322648629722,1.4816268957431415,0.01889731544008466,1.4627295803030569,2.9254591606061138,affine,0.4032760717116363,0.8065521434232726,1.0,0.4032760717116363,eta_l0_n195_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,196,0,-1.1524474377340879,1.1941743187475986,0.02086344050675537,1.1733108782408432,2.3466217564816865,affine,0.11510051619594604,0.23020103239189207,0.9319962481713715,0.12349890508870573,eta_l0_n196_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,196,1,-1.1524474377340879,1.1941743187475986,0.02086344050675537,1.1733108782408432,2.3466217564816865,affine,0.34042627982208706,0.6808525596441741,1.0,0.34042627982208706,eta_l0_n196_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,197,0,-0.6221827602303874,0.6073776418168665,-0.007402559206760473,0.614780201023627,1.229560402047254,affine,0.024711480698816016,0.04942296139763203,0.9319962481713715,0.026514571005303203,eta_l0_n197_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,197,1,-0.6221827602303874,0.6073776418168665,-0.007402559206760473,0.614780201023627,1.229560402047254,affine,0.14984737210450283,0.29969474420900566,1.0,0.14984737210450283,eta_l0_n197_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,198,0,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,affine,0.06546697285142163,0.13093394570284325,0.9319962481713715,0.07024381587358476,eta_l0_n198_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,198,1,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,affine,0.2588549749285447,0.5177099498570894,1.0,0.2588549749285447,eta_l0_n198_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,199,0,-1.0952666713142436,1.1346180087386013,0.01967566871217885,1.1149423400264225,2.229884680052845,affine,0.10355036232782984,0.20710072465565968,0.9319962481713715,0.11110598624297191,eta_l0_n199_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,199,1,-1.0952666713142436,1.1346180087386013,0.01967566871217885,1.1149423400264225,2.229884680052845,affine,0.3245088156471295,0.649017631294259,1.0,0.3245088156471295,eta_l0_n199_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,200,0,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.08060765055937986,0.16121530111875973,0.9319962481713715,0.08648924361824048,eta_l0_n200_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,200,1,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.287807187210534,0.575614374421068,1.0,0.287807187210534,eta_l0_n200_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,201,0,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.10398467023247389,0.20796934046494778,0.9319962481713715,0.11157198372471734,eta_l0_n201_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,201,1,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.32518038985547915,0.6503607797109583,1.0,0.32518038985547915,eta_l0_n201_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,202,0,-0.7951331280804155,0.8148313974508754,0.009849134685229965,0.8049822627656454,1.6099645255312909,affine,0.049187121407205665,0.09837424281441133,0.9319962481713715,0.05277609379191551,eta_l0_n202_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,202,1,-0.7951331280804155,0.8148313974508754,0.009849134685229965,0.8049822627656454,1.6099645255312909,affine,0.22227638354605228,0.44455276709210456,1.0,0.22227638354605228,eta_l0_n202_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,203,0,-0.9291054457961189,0.8942568543418766,-0.01742429572712112,0.9116811500689977,1.8233623001379955,affine,0.0662563014773734,0.1325126029547468,0.9319962481713715,0.07109073840948603,eta_l0_n203_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,203,1,-0.9291054457961189,0.8942568543418766,-0.01742429572712112,0.9116811500689977,1.8233623001379955,affine,0.26046312975463887,0.5209262595092777,1.0,0.26046312975463887,eta_l0_n203_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,204,0,-0.7641244024694367,0.7349866898999595,-0.014568856284738585,0.7495555461846981,1.4991110923693962,affine,0.041244716478901274,0.08248943295780255,0.9319962481713715,0.04425416578642425,eta_l0_n204_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,204,1,-0.7641244024694367,0.7349866898999595,-0.014568856284738585,0.7495555461846981,1.4991110923693962,affine,0.20144524470009967,0.40289048940019934,1.0,0.20144524470009967,eta_l0_n204_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,205,0,-1.0117373774449647,0.9756317416300787,-0.01805281790744301,0.9936845595375217,1.9873691190750433,affine,0.08064859882773769,0.16129719765547537,0.9319962481713715,0.08653317970536333,eta_l0_n205_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,205,1,-1.0117373774449647,0.9756317416300787,-0.01805281790744301,0.9936845595375217,1.9873691190750433,affine,0.2878479178458295,0.575695835691659,1.0,0.2878479178458295,eta_l0_n205_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,206,0,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.13911287947176756,0.27822575894353513,0.9319962481713715,0.14926334708397676,eta_l0_n206_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,206,1,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.3691976371738509,0.7383952743477018,1.0,0.3691976371738509,eta_l0_n206_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,207,0,-1.3112708446293027,1.2772121752583023,-0.017029334685500164,1.2942415099438025,2.588483019887605,affine,0.13971997178468812,0.27943994356937624,0.9319962481713715,0.14991473630803395,eta_l0_n207_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,207,1,-1.3112708446293027,1.2772121752583023,-0.017029334685500164,1.2942415099438025,2.588483019887605,affine,0.3699089369805241,0.7398178739610481,1.0,0.3699089369805241,eta_l0_n207_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,208,0,-1.1651989100051086,1.200260436473233,0.01753076323406222,1.1827296732391708,2.3654593464783416,affine,0.11697091939156241,0.23394183878312483,0.9319962481713715,0.12550578354909248,eta_l0_n208_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,208,1,-1.1651989100051086,1.200260436473233,0.01753076323406222,1.1827296732391708,2.3654593464783416,affine,0.3429378820617447,0.6858757641234894,1.0,0.3429378820617447,eta_l0_n208_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,209,0,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,affine,0.12336957207392958,0.24673914414785916,0.9319962481713715,0.13237131835668603,eta_l0_n209_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,209,1,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,affine,0.35104070176541363,0.7020814035308273,1.0,0.35104070176541363,eta_l0_n209_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,210,0,-0.9228447532260015,0.9507075443892258,0.01393139558161216,0.9367761488076136,1.8735522976152272,affine,0.07053047421855775,0.1410609484371155,0.9319962481713715,0.07567677912539075,eta_l0_n210_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,210,1,-0.9228447532260015,0.9507075443892258,0.01393139558161216,0.9367761488076136,1.8735522976152272,affine,0.2691003564686259,0.5382007129372518,1.0,0.2691003564686259,eta_l0_n210_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,211,0,-1.3219320199504636,1.2845693941220586,-0.018681312914202497,1.3032507070362611,2.6065014140725222,affine,0.14158975427736184,0.2831795085547237,0.9319962481713715,0.15192094877545786,eta_l0_n211_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,211,1,-1.3219320199504636,1.2845693941220586,-0.018681312914202497,1.3032507070362611,2.6065014140725222,affine,0.3718937045390255,0.743787409078051,1.0,0.3718937045390255,eta_l0_n211_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,212,0,-1.1136630602458557,1.0736732395863258,-0.01999491032976497,1.0936681499160907,2.1873362998321815,affine,0.09941980308099137,0.19883960616198273,0.9319962481713715,0.10667403787950708,eta_l0_n212_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,212,1,-1.1136630602458557,1.0736732395863258,-0.01999491032976497,1.0936681499160907,2.1873362998321815,affine,0.3184140898770352,0.6368281797540704,1.0,0.3184140898770352,eta_l0_n212_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,213,0,-1.397693525091097,1.3709670065547253,-0.013363259268185823,1.384330265822911,2.768660531645822,affine,0.1584389554911889,0.3168779109823778,0.9319962481713715,0.1699995636270585,eta_l0_n213_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,213,1,-1.397693525091097,1.3709670065547253,-0.013363259268185823,1.384330265822911,2.768660531645822,affine,0.38883905368764476,0.7776781073752895,1.0,0.38883905368764476,eta_l0_n213_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,214,0,-0.6308511366333199,0.5817134666571077,-0.024568834988106136,0.6062823016452138,1.2125646032904276,affine,0.02393379841437915,0.0478675968287583,0.9319962481713715,0.02568014459429273,eta_l0_n214_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,214,1,-0.6308511366333199,0.5817134666571077,-0.024568834988106136,0.6062823016452138,1.2125646032904276,affine,0.14637875612598206,0.2927575122519641,1.0,0.14637875612598206,eta_l0_n214_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,215,0,-0.6750710083304649,0.648698160588605,-0.013186423870929942,0.6618845844595349,1.3237691689190698,affine,0.030018619591721033,0.060037239183442066,0.9319962481713715,0.03220894896371014,eta_l0_n215_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,215,1,-0.6750710083304649,0.648698160588605,-0.013186423870929942,0.6618845844595349,1.3237691689190698,affine,0.16790532846119766,0.3358106569223953,1.0,0.16790532846119766,eta_l0_n215_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,216,0,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.13353496019544117,0.26706992039088234,0.9319962481713715,0.1432784310639063,eta_l0_n216_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,216,1,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.3629650350912027,0.7259300701824054,1.0,0.3629650350912027,eta_l0_n216_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,217,0,-1.0176420196916924,0.9812025096472035,-0.018219755022244488,0.999422264669448,1.998844529338896,affine,0.08169289794408681,0.16338579588817362,0.9319962481713715,0.0876536768301083,eta_l0_n217_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,217,1,-1.0176420196916924,0.9812025096472035,-0.018219755022244488,0.999422264669448,1.998844529338896,affine,0.28968717014881595,0.5793743402976319,1.0,0.28968717014881595,eta_l0_n217_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,218,0,-0.9798393779987576,1.0123201516079219,0.01624038680458212,0.9960797648033397,1.9921595296066794,affine,0.08107290966168675,0.1621458193233735,0.9319962481713715,0.08698845067322568,eta_l0_n218_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,218,1,-0.9798393779987576,1.0123201516079219,0.01624038680458212,0.9960797648033397,1.9921595296066794,affine,0.28864444116563753,0.5772888823312751,1.0,0.28864444116563753,eta_l0_n218_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,219,0,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.08893857072936846,0.17787714145873693,0.9319962481713715,0.09542803514914452,eta_l0_n219_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,219,1,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.30212709905295965,0.6042541981059193,1.0,0.30212709905295965,eta_l0_n219_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,220,0,-1.0276272819085601,0.9881549383021562,-0.019736171803201974,1.007891110105358,2.015782220210716,affine,0.08325051921200739,0.16650103842401479,0.9319962481713715,0.08932495101278524,eta_l0_n220_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,220,1,-1.0276272819085601,0.9881549383021562,-0.019736171803201974,1.007891110105358,2.015782220210716,affine,0.2923627454325827,0.5847254908651655,1.0,0.2923627454325827,eta_l0_n220_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,221,0,-0.918271405219569,0.9526902849630147,0.017209439871722854,0.9354808450912918,1.8709616901825836,affine,0.07032498333747385,0.1406499666749477,0.9319962481713715,0.0754562944598279,eta_l0_n221_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,221,1,-0.918271405219569,0.9526902849630147,0.017209439871722854,0.9354808450912918,1.8709616901825836,affine,0.2686165914228864,0.5372331828457728,1.0,0.2686165914228864,eta_l0_n221_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,222,0,-1.093166004600465,1.1314416045523363,0.019137799975935676,1.1123038045764007,2.2246076091528013,affine,0.1030323751704313,0.2060647503408626,0.9319962481713715,0.11055020379383132,eta_l0_n222_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,222,1,-1.093166004600465,1.1314416045523363,0.019137799975935676,1.1123038045764007,2.2246076091528013,affine,0.32376979893757774,0.6475395978751555,1.0,0.32376979893757774,eta_l0_n222_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,223,0,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,affine,0.0371415219972684,0.0742830439945368,0.9319962481713715,0.03985157887721343,eta_l0_n223_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,223,1,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,affine,0.18978987632131178,0.37957975264262356,1.0,0.18978987632131178,eta_l0_n223_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,224,0,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.13683906908689789,0.27367813817379577,0.9319962481713715,0.1468236265493383,eta_l0_n224_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,224,1,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.36670914743226213,0.7334182948645243,1.0,0.36670914743226213,eta_l0_n224_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,225,0,-0.8019294265090884,0.8378016275339952,0.017936100512453423,0.8198655270215418,1.6397310540430836,affine,0.051479085767128514,0.10295817153425703,0.9319962481713715,0.055235292918971876,eta_l0_n225_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,225,1,-0.8019294265090884,0.8378016275339952,0.017936100512453423,0.8198655270215418,1.6397310540430836,affine,0.2276662882782467,0.4553325765564934,1.0,0.2276662882782467,eta_l0_n225_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,226,0,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.0847732214456068,0.1695464428912136,0.9319962481713715,0.09095875826961383,eta_l0_n226_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,226,1,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.2949929960678926,0.5899859921357852,1.0,0.2949929960678926,eta_l0_n226_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,227,0,-1.222732562277716,1.2664926147953328,0.02188002625880836,1.2446125885365245,2.489225177073049,affine,0.12953797657304725,0.2590759531460945,0.9319962481713715,0.1389898047628496,eta_l0_n227_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,227,1,-1.222732562277716,1.2664926147953328,0.02188002625880836,1.2446125885365245,2.489225177073049,affine,0.35834507775656466,0.7166901555131293,1.0,0.35834507775656466,eta_l0_n227_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,228,0,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,affine,0.08376097929774685,0.1675219585954937,0.9319962481713715,0.08987265717226926,eta_l0_n228_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,228,1,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,affine,0.2932994718153777,0.5865989436307554,1.0,0.2932994718153777,eta_l0_n228_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,229,0,-0.7782243522405144,0.7634707519083993,-0.007376800166057573,0.7708475520744569,1.5416951041489138,affine,0.04419558317224093,0.08839116634448187,0.9319962481713715,0.04742034451206765,eta_l0_n229_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,229,1,-0.7782243522405144,0.7634707519083993,-0.007376800166057573,0.7708475520744569,1.5416951041489138,affine,0.20955349419685776,0.4191069883937155,1.0,0.20955349419685776,eta_l0_n229_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,230,0,-1.4736909184242275,1.4349512883956,-0.019369815014313785,1.4543211034099137,2.9086422068198274,affine,0.1730901226391359,0.3461802452782718,0.9319962481713715,0.18571976333461465,eta_l0_n230_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,230,1,-1.4736909184242275,1.4349512883956,-0.019369815014313785,1.4543211034099137,2.9086422068198274,affine,0.4018009870697764,0.8036019741395528,1.0,0.4018009870697764,eta_l0_n230_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,231,0,-0.9741772249753825,1.0156090479613666,0.020715911492992067,0.9948931364683746,1.9897862729367493,affine,0.08088571108096311,0.16177142216192622,0.9319962481713715,0.08678759301838969,eta_l0_n231_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,231,1,-0.9741772249753825,1.0156090479613666,0.020715911492992067,0.9948931364683746,1.9897862729367493,affine,0.28819280957062676,0.5763856191412535,1.0,0.28819280957062676,eta_l0_n231_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,232,0,-0.9915097879699294,1.031502035920235,0.01999612397515277,1.0115059119450822,2.0230118238901644,affine,0.08391577855064876,0.16783155710129752,0.9319962481713715,0.09003875145988643,eta_l0_n232_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,232,1,-0.9915097879699294,1.031502035920235,0.01999612397515277,1.0115059119450822,2.0230118238901644,affine,0.29350402543245935,0.5870080508649187,1.0,0.29350402543245935,eta_l0_n232_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,233,0,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.09369831588937973,0.18739663177875945,0.9319962481713715,0.10053507841176511,eta_l0_n233_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,233,1,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.30969189457663343,0.6193837891532669,1.0,0.30969189457663343,eta_l0_n233_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,234,0,-0.9553499690208584,0.9911598413697276,0.01790493617443456,0.973254905195293,1.946509810390586,affine,0.0769698294054751,0.1539396588109502,0.9319962481713715,0.08258598632397307,eta_l0_n234_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,234,1,-0.9553499690208584,0.9911598413697276,0.01790493617443456,0.973254905195293,1.946509810390586,affine,0.28121030953724074,0.5624206190744815,1.0,0.28121030953724074,eta_l0_n234_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,235,0,-0.9690273603532127,1.0071673197752016,0.019069979710994445,0.9880973400642071,1.9761946801284143,affine,0.07964352957996403,0.15928705915992805,0.9319962481713715,0.08545477488372841,eta_l0_n235_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,235,1,-0.9690273603532127,1.0071673197752016,0.019069979710994445,0.9880973400642071,1.9761946801284143,affine,0.28602852557731817,0.5720570511546363,1.0,0.28602852557731817,eta_l0_n235_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,236,0,-1.1558838037706318,1.191758770838782,0.017937483534075094,1.173821287304707,2.347642574609414,affine,0.11518710792645188,0.23037421585290377,0.9319962481713715,0.12359181504480882,eta_l0_n236_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,236,1,-1.1558838037706318,1.191758770838782,0.017937483534075094,1.173821287304707,2.347642574609414,affine,0.34060209629286436,0.6812041925857287,1.0,0.34060209629286436,eta_l0_n236_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,237,0,-0.9032024251467424,0.9272904688323386,0.01204402184279807,0.9152464469895405,1.830492893979081,affine,0.06683064114202937,0.13366128228405874,0.9319962481713715,0.0717069851656107,eta_l0_n237_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,237,1,-0.9032024251467424,0.9272904688323386,0.01204402184279807,0.9152464469895405,1.830492893979081,affine,0.2617639868026934,0.5235279736053868,1.0,0.2617639868026934,eta_l0_n237_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,238,0,-1.2475163628525583,1.2970872348018667,0.024785435974654213,1.2723017988272125,2.544603597654425,affine,0.1352337806803164,0.2704675613606328,0.9319962481713715,0.14510120716220973,eta_l0_n238_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,238,1,-1.2475163628525583,1.2970872348018667,0.024785435974654213,1.2723017988272125,2.544603597654425,affine,0.3648170558558874,0.7296341117117748,1.0,0.3648170558558874,eta_l0_n238_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,239,0,-0.9472028721092912,0.9617653086795485,0.007281218285128621,0.9544840903944198,1.9089681807888397,affine,0.07359532437748174,0.1471906487549635,0.9319962481713715,0.0789652581991396,eta_l0_n239_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,239,1,-0.9472028721092912,0.9617653086795485,0.007281218285128621,0.9544840903944198,1.9089681807888397,affine,0.27511529379093747,0.5502305875818749,1.0,0.27511529379093747,eta_l0_n239_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,240,0,-1.139020179466013,1.1757004326923162,0.01834012661315154,1.1573603060791646,2.3147206121583292,affine,0.11190478616491507,0.22380957232983015,0.9319962481713715,0.12006999640232296,eta_l0_n240_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,240,1,-1.139020179466013,1.1757004326923162,0.01834012661315154,1.1573603060791646,2.3147206121583292,affine,0.3362207708362268,0.6724415416724536,1.0,0.3362207708362268,eta_l0_n240_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,241,0,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.05101514148501563,0.10203028297003126,0.9319962481713715,0.054737496620946895,eta_l0_n241_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,241,1,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.2266731457928317,0.4533462915856634,1.0,0.2266731457928317,eta_l0_n241_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,242,0,-0.8216464738019207,0.8370838065220392,0.007718666360059245,0.8293651401619799,1.6587302803239599,affine,0.05288344570803522,0.10576689141607044,0.9319962481713715,0.05674212295574739,eta_l0_n242_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,242,1,-0.8216464738019207,0.8370838065220392,0.007718666360059245,0.8293651401619799,1.6587302803239599,affine,0.23126511064546856,0.4625302212909371,1.0,0.23126511064546856,eta_l0_n242_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,243,0,-1.094745104288666,1.126225789433019,0.01574034257217649,1.1104854468608425,2.220970893721685,affine,0.10266022844570051,0.20532045689140102,0.9319962481713715,0.11015090312554969,eta_l0_n243_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,243,1,-1.094745104288666,1.126225789433019,0.01574034257217649,1.1104854468608425,2.220970893721685,affine,0.32329968942006604,0.6465993788401321,1.0,0.32329968942006604,eta_l0_n243_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,244,0,-1.3579344396070396,1.4019798358364988,0.022022698114729566,1.3799571377217692,2.7599142754435384,affine,0.15755628942850283,0.31511257885700567,0.9319962481713715,0.16905249322369809,eta_l0_n244_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,244,1,-1.3579344396070396,1.4019798358364988,0.022022698114729566,1.3799571377217692,2.7599142754435384,affine,0.38789260455289937,0.7757852091057987,1.0,0.38789260455289937,eta_l0_n244_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,245,0,-1.0757844619013353,1.058689201892231,-0.008547630004552165,1.067236831896783,2.134473663793566,affine,0.09429752179835456,0.18859504359670912,0.9319962481713715,0.10117800579494987,eta_l0_n245_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,245,1,-1.0757844619013353,1.058689201892231,-0.008547630004552165,1.067236831896783,2.134473663793566,affine,0.31077188650760995,0.6215437730152199,1.0,0.31077188650760995,eta_l0_n245_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,246,0,-0.8996611244567155,0.9223465961082321,0.011342735825758288,0.9110038602824738,1.8220077205649476,affine,0.06610913113535623,0.13221826227071246,0.9319962481713715,0.07093282968152073,eta_l0_n246_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,246,1,-0.8996611244567155,0.9223465961082321,0.011342735825758288,0.9110038602824738,1.8220077205649476,affine,0.26030628422287794,0.5206125684457559,1.0,0.26030628422287794,eta_l0_n246_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,247,0,-0.8285968972008307,0.8196796009213531,-0.004458648139738841,0.8241382490610919,1.6482764981221838,affine,0.05207163037664759,0.10414326075329518,0.9319962481713715,0.0558710729563719,eta_l0_n247_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,247,1,-0.8285968972008307,0.8196796009213531,-0.004458648139738841,0.8241382490610919,1.6482764981221838,affine,0.22936955208021276,0.4587391041604255,1.0,0.22936955208021276,eta_l0_n247_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,248,0,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,affine,0.07260971125925632,0.14521942251851264,0.9319962481713715,0.0779077291370224,eta_l0_n248_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,248,1,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,affine,0.27311766338374865,0.5462353267674973,1.0,0.27311766338374865,eta_l0_n248_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,249,0,-0.8971474300455685,0.9299772734367954,0.016414921695613427,0.9135623517411819,1.8271247034823639,affine,0.06656829091883472,0.13313658183766944,0.9319962481713715,0.07142549237665431,eta_l0_n249_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,249,1,-0.8971474300455685,0.9299772734367954,0.016414921695613427,0.9135623517411819,1.8271247034823639,affine,0.2611280939599575,0.522256187919915,1.0,0.2611280939599575,eta_l0_n249_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,250,0,-0.7261210591057342,0.7589229120372533,0.016400926465759524,0.7425219855714937,1.4850439711429875,affine,0.04029261137626906,0.08058522275253811,0.9319962481713715,0.043232589675468544,eta_l0_n250_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,250,1,-0.7261210591057342,0.7589229120372533,0.016400926465759524,0.7425219855714937,1.4850439711429875,affine,0.19875206432670667,0.39750412865341334,1.0,0.19875206432670667,eta_l0_n250_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,251,0,-0.6364866647888496,0.6638130918043734,0.013663213507761895,0.6501498782966115,1.300299756593223,affine,0.028655556908585626,0.05731111381717125,0.9319962481713715,0.030746429467725243,eta_l0_n251_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,251,1,-0.6364866647888496,0.6638130918043734,0.013663213507761895,0.6501498782966115,1.300299756593223,affine,0.1633843769776558,0.3267687539553116,1.0,0.1633843769776558,eta_l0_n251_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,252,0,-1.0954772645448594,1.1341057511714407,0.01931424331329068,1.11479150785815,2.2295830157163,affine,0.10351882023537858,0.20703764047075715,0.9319962481713715,0.11107214265989618,eta_l0_n252_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,252,1,-1.0954772645448594,1.1341057511714407,0.01931424331329068,1.11479150785815,2.2295830157163,affine,0.32447162950315533,0.6489432590063107,1.0,0.32447162950315533,eta_l0_n252_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,253,0,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.04924698734954847,0.09849397469909695,0.9319962481713715,0.052840327894209664,eta_l0_n253_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,253,1,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.22222072025332534,0.4444414405066507,1.0,0.22222072025332534,eta_l0_n253_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,254,0,-0.9847557236564193,1.020181147346501,0.017712711845040863,1.0024684355014601,2.0049368710029203,affine,0.08224546002753337,0.16449092005506674,0.9319962481713715,0.08824655698872559,eta_l0_n254_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,254,1,-0.9847557236564193,1.020181147346501,0.017712711845040863,1.0024684355014601,2.0049368710029203,affine,0.29066857170882565,0.5813371434176513,1.0,0.29066857170882565,eta_l0_n254_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,255,0,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.13466273029786982,0.26932546059573964,0.9319962481713715,0.14448848969304928,eta_l0_n255_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,255,1,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.3642931767979276,0.7285863535958552,1.0,0.3642931767979276,eta_l0_n255_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,256,0,-0.8345849622272768,0.862889794074723,0.014152415923723072,0.8487373781509999,1.6974747563019998,affine,0.055936304533085686,0.11187260906617137,0.9319962481713715,0.0600177357396404,eta_l0_n256_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,256,1,-0.8345849622272768,0.862889794074723,0.014152415923723072,0.8487373781509999,1.6974747563019998,affine,0.2382428389544612,0.4764856779089224,1.0,0.2382428389544612,eta_l0_n256_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,257,0,-0.6351214718940504,0.5726601313824508,-0.03123067025579984,0.6038908016382506,1.2077816032765012,affine,0.023764750123252524,0.04752950024650505,0.9319962481713715,0.025498761577506655,eta_l0_n257_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,257,1,-0.6351214718940504,0.5726601313824508,-0.03123067025579984,0.6038908016382506,1.2077816032765012,affine,0.14532038190436053,0.29064076380872106,1.0,0.14532038190436053,eta_l0_n257_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,258,0,-0.9998125161825591,0.967562441340464,-0.01612503742104754,0.9836874787615115,1.967374957523023,affine,0.07883014658578365,0.1576602931715673,0.9319962481713715,0.0845820428359586,eta_l0_n258_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,258,1,-0.9998125161825591,0.967562441340464,-0.01612503742104754,0.9836874787615115,1.967374957523023,affine,0.28464282835773846,0.5692856567154769,1.0,0.28464282835773846,eta_l0_n258_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,259,0,-0.8697080952856967,0.8389415568536377,-0.015383269216029505,0.8543248260696672,1.7086496521393344,affine,0.05682902813904301,0.11365805627808602,0.9319962481713715,0.06097559754188359,eta_l0_n259_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,259,1,-0.8697080952856967,0.8389415568536377,-0.015383269216029505,0.8543248260696672,1.7086496521393344,affine,0.24024190927679612,0.48048381855359223,1.0,0.24024190927679612,eta_l0_n259_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,260,0,-0.8381656949954069,0.8252083496681949,-0.006478672663606022,0.8316870223318009,1.6633740446636018,affine,0.0532387319093588,0.1064774638187176,0.9319962481713715,0.05712333286085234,eta_l0_n260_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,260,1,-0.8381656949954069,0.8252083496681949,-0.006478672663606022,0.8316870223318009,1.6633740446636018,affine,0.2321211134765584,0.4642422269531168,1.0,0.2321211134765584,eta_l0_n260_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,261,0,-0.833623752115959,0.8706092838419001,0.01849276586297055,0.8521165179789295,1.704233035957859,affine,0.056499013590415766,0.11299802718083153,0.9319962481713715,0.06062150325311928,eta_l0_n261_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,261,1,-0.833623752115959,0.8706092838419001,0.01849276586297055,0.8521165179789295,1.704233035957859,affine,0.2393991173845697,0.4787982347691394,1.0,0.2393991173845697,eta_l0_n261_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,262,0,-0.9956755857415504,0.9684535524140813,-0.013611016663734543,0.9820645690778158,1.9641291381556316,affine,0.07852507002697835,0.1570501400539567,0.9319962481713715,0.0842547061547178,eta_l0_n262_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,262,1,-0.9956755857415504,0.9684535524140813,-0.013611016663734543,0.9820645690778158,1.9641291381556316,affine,0.28414712102329764,0.5682942420465953,1.0,0.28414712102329764,eta_l0_n262_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,263,0,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.08863715985000005,0.1772743197000001,0.9319962481713715,0.09510463161617988,eta_l0_n263_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,263,1,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.3015525088126787,0.6031050176253574,1.0,0.3015525088126787,eta_l0_n263_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,264,0,-0.7417572625689896,0.7667071698008693,0.012474953615939866,0.7542322161849294,1.5084644323698588,affine,0.04187984998861028,0.08375969997722056,0.9319962481713715,0.0449356422526173,eta_l0_n264_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,264,1,-0.7417572625689896,0.7667071698008693,0.012474953615939866,0.7542322161849294,1.5084644323698588,affine,0.20324099935783696,0.4064819987156739,1.0,0.20324099935783696,eta_l0_n264_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,265,0,-0.8885673887593571,0.8568782922249832,-0.015844548267186953,0.8727228404921702,1.7454456809843404,affine,0.05978837364507889,0.11957674729015778,0.9319962481713715,0.06415087374266475,eta_l0_n265_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,265,1,-0.8885673887593571,0.8568782922249832,-0.015844548267186953,0.8727228404921702,1.7454456809843404,affine,0.24682017229808212,0.49364034459616424,1.0,0.24682017229808212,eta_l0_n265_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,266,0,-1.097939440105263,1.0589211186241407,-0.019509160740561082,1.0784302793647018,2.1568605587294036,affine,0.09648454749320724,0.19296909498641449,0.9319962481713715,0.10352460933454968,eta_l0_n266_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,266,1,-1.097939440105263,1.0589211186241407,-0.019509160740561082,1.0784302793647018,2.1568605587294036,affine,0.31396790847240585,0.6279358169448117,1.0,0.31396790847240585,eta_l0_n266_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,267,0,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.07059108594245113,0.14118217188490226,0.9319962481713715,0.07574181342570291,eta_l0_n267_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,267,1,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.2691747136306971,0.5383494272613942,1.0,0.2691747136306971,eta_l0_n267_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,268,0,-0.8940710602099515,0.8588320503931359,-0.0176195049084078,0.8764515553015437,1.7529031106030875,affine,0.06040647288144008,0.12081294576288017,0.9319962481713715,0.06481407301795575,eta_l0_n268_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,268,1,-0.8940710602099515,0.8588320503931359,-0.0176195049084078,0.8764515553015437,1.7529031106030875,affine,0.24811831778222848,0.49623663556445696,1.0,0.24811831778222848,eta_l0_n268_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,269,0,-0.8629278062752647,0.8343272647993644,-0.01430027073795015,0.8486275355373145,1.697255071074629,affine,0.05591978122054115,0.1118395624410823,0.9319962481713715,0.06000000679215058,eta_l0_n269_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,269,1,-0.8629278062752647,0.8343272647993644,-0.01430027073795015,0.8486275355373145,1.697255071074629,affine,0.23820126653711426,0.47640253307422853,1.0,0.23820126653711426,eta_l0_n269_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,270,0,-0.970677453881365,1.0031156697628496,0.016219107940742328,0.9868965618221073,1.9737931236442146,affine,0.07940926081926343,0.15881852163852686,0.9319962481713715,0.08520341254062859,eta_l0_n270_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,270,1,-0.970677453881365,1.0031156697628496,0.016219107940742328,0.9868965618221073,1.9737931236442146,affine,0.2856827070911946,0.5713654141823892,1.0,0.2856827070911946,eta_l0_n270_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,271,0,-0.9732209605163527,0.9367859559488056,-0.0182175022837735,0.9550034582325791,1.9100069164651583,affine,0.07373664133513287,0.14747328267026574,0.9319962481713715,0.07911688644649403,eta_l0_n271_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,271,1,-0.9732209605163527,0.9367859559488056,-0.0182175022837735,0.9550034582325791,1.9100069164651583,affine,0.27516733078326927,0.5503346615665385,1.0,0.27516733078326927,eta_l0_n271_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,272,0,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.0785284283481333,0.1570568566962666,0.9319962481713715,0.08425830951810208,eta_l0_n272_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,272,1,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.28407907522631537,0.5681581504526307,1.0,0.28407907522631537,eta_l0_n272_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,273,0,-0.8018194571083685,0.7795301587031829,-0.011144649202592838,0.7906748079057757,1.5813496158115514,affine,0.047074883359198356,0.09414976671839671,0.9319962481713715,0.050509734831617505,eta_l0_n273_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,273,1,-0.8018194571083685,0.7795301587031829,-0.011144649202592838,0.7906748079057757,1.5813496158115514,affine,0.2169475124104976,0.4338950248209952,1.0,0.2169475124104976,eta_l0_n273_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,274,0,-1.3704733539073402,1.4030091777593323,0.01626791192599608,1.3867412658333362,2.7734825316666725,affine,0.15895094587993705,0.3179018917598741,0.9319962481713715,0.17054891174917028,eta_l0_n274_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,274,1,-1.3704733539073402,1.4030091777593323,0.01626791192599608,1.3867412658333362,2.7734825316666725,affine,0.38928688741470735,0.7785737748294147,1.0,0.38928688741470735,eta_l0_n274_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,275,0,-1.054322247914134,1.0201728471958655,-0.017074700359134276,1.0372475475549998,2.0744950951099996,affine,0.08867093565940438,0.17734187131880877,0.9319962481713715,0.09514087190091344,eta_l0_n275_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,275,1,-1.054322247914134,1.0201728471958655,-0.017074700359134276,1.0372475475549998,2.0744950951099996,affine,0.3015876428654989,0.6031752857309978,1.0,0.3015876428654989,eta_l0_n275_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,276,0,-1.1254872684487631,1.1656844838811946,0.020098607716215744,1.1455858761649789,2.2911717523299577,affine,0.10957807722918857,0.21915615445837713,0.9319962481713715,0.11757351753742233,eta_l0_n276_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,276,1,-1.1254872684487631,1.1656844838811946,0.020098607716215744,1.1455858761649789,2.2911717523299577,affine,0.333009874225618,0.666019748451236,1.0,0.333009874225618,eta_l0_n276_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,277,0,-1.1043859523878041,1.1426379378077016,0.019125992709948747,1.1235119450977529,2.2470238901955057,affine,0.10522390427298968,0.21044780854597936,0.9319962481713715,0.11290163933539951,eta_l0_n277_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,277,1,-1.1043859523878041,1.1426379378077016,0.019125992709948747,1.1235119450977529,2.2470238901955057,affine,0.32692761757395183,0.6538552351479037,1.0,0.32692761757395183,eta_l0_n277_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,278,0,-0.7530154612851288,0.7784502098291963,0.012717374272033788,0.7657328355571625,1.531465671114325,affine,0.04349218747112233,0.08698437494224466,0.9319962481713715,0.04666562505638453,eta_l0_n278_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,278,1,-0.7530154612851288,0.7784502098291963,0.012717374272033788,0.7657328355571625,1.531465671114325,affine,0.20758068201875263,0.41516136403750525,1.0,0.20758068201875263,eta_l0_n278_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,279,0,-0.9664557004775315,0.9399092435870476,-0.013273228445241925,0.9531824720322896,1.9063649440645791,affine,0.07338862410217874,0.14677724820435747,0.9319962481713715,0.07874347589507072,eta_l0_n279_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,279,1,-0.9664557004775315,0.9399092435870476,-0.013273228445241925,0.9531824720322896,1.9063649440645791,affine,0.2746272259394916,0.5492544518789833,1.0,0.2746272259394916,eta_l0_n279_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,280,0,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.10125806852097143,0.20251613704194285,0.9319962481713715,0.10864643363065613,eta_l0_n280_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,280,1,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.3211816722610421,0.6423633445220842,1.0,0.3211816722610421,eta_l0_n280_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,281,0,-0.7251348527214971,0.758782585137245,0.016823866207873905,0.741958718929371,1.483917437858742,affine,0.040218829577515165,0.08043765915503033,0.9319962481713715,0.04315342433666095,eta_l0_n281_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,281,1,-0.7251348527214971,0.758782585137245,0.016823866207873905,0.741958718929371,1.483917437858742,affine,0.19853197030781827,0.39706394061563655,1.0,0.19853197030781827,eta_l0_n281_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,282,0,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.1874287668164584,0.3748575336329168,0.9319962481713715,0.20110463661651434,eta_l0_n282_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,282,1,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.41327305208332726,0.8265461041666545,1.0,0.41327305208332726,eta_l0_n282_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,283,0,-0.8267574423081061,0.7934742831840171,-0.016641579562044484,0.8101158627460616,1.6202317254921232,affine,0.04999348869782195,0.0999869773956439,0.9319962481713715,0.05364129823045099,eta_l0_n283_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,283,1,-0.8267574423081061,0.7934742831840171,-0.016641579562044484,0.8101158627460616,1.6202317254921232,affine,0.2240945199925709,0.4481890399851418,1.0,0.2240945199925709,eta_l0_n283_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,284,0,-0.8807211491905227,0.9069751074714274,0.013126979140452355,0.893848128330975,1.78769625666195,affine,0.06324291874219658,0.12648583748439315,0.9319962481713715,0.06785748211570884,eta_l0_n284_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,284,1,-0.8807211491905227,0.9069751074714274,0.013126979140452355,0.893848128330975,1.78769625666195,affine,0.25431416160486126,0.5086283232097225,1.0,0.25431416160486126,eta_l0_n284_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,285,0,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.06959998889061066,0.1391999777812213,0.9319962481713715,0.0746784002909558,eta_l0_n285_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,285,1,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.26717870611859196,0.5343574122371839,1.0,0.26717870611859196,eta_l0_n285_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,286,0,-1.3222298401611514,1.2810144283174631,-0.020607705921844133,1.3016221342393073,2.6032442684786146,affine,0.14126131622843827,0.28252263245687653,0.9319962481713715,0.15156854601679012,eta_l0_n286_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,286,1,-1.3222298401611514,1.2810144283174631,-0.020607705921844133,1.3016221342393073,2.6032442684786146,affine,0.3715097252062655,0.743019450412531,1.0,0.3715097252062655,eta_l0_n286_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,287,0,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,affine,0.08700633057614827,0.17401266115229655,0.9319962481713715,0.09335480775470882,eta_l0_n287_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,287,1,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,affine,0.29874920593166787,0.5974984118633357,1.0,0.29874920593166787,eta_l0_n287_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,288,0,-0.9609248528732853,0.9933100955407531,0.016192621333733892,0.9771174742070192,1.9542349484140384,affine,0.07765047558068246,0.15530095116136491,0.9319962481713715,0.08331629631882854,eta_l0_n288_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,288,1,-0.9609248528732853,0.9933100955407531,0.016192621333733892,0.9771174742070192,1.9542349484140384,affine,0.28250049405507804,0.5650009881101561,1.0,0.28250049405507804,eta_l0_n288_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,289,0,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.12823730959170523,0.25647461918341047,0.9319962481713715,0.13759423371426008,eta_l0_n289_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,289,1,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.3568584940492191,0.7137169880984382,1.0,0.3568584940492191,eta_l0_n289_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,290,0,-1.2209164590474624,1.265004290450195,0.022043915701366323,1.2429603747488287,2.4859207494976574,affine,0.12920111825149752,0.25840223650299504,0.9319962481713715,0.13862836734053094,eta_l0_n290_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,290,1,-1.2209164590474624,1.265004290450195,0.022043915701366323,1.2429603747488287,2.4859207494976574,affine,0.35794597035653486,0.7158919407130697,1.0,0.35794597035653486,eta_l0_n290_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,291,0,-0.8933386502203347,0.9310906530126255,0.018876001396145425,0.9122146516164801,1.8244293032329602,affine,0.0663563654821971,0.1327127309643942,0.9319962481713715,0.07119810365373463,eta_l0_n291_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,291,1,-0.8933386502203347,0.9310906530126255,0.018876001396145425,0.9122146516164801,1.8244293032329602,affine,0.260624237096595,0.52124847419319,1.0,0.260624237096595,eta_l0_n291_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,292,0,-0.8645810082391754,0.838881124514349,-0.012849941862413172,0.8517310663767622,1.7034621327535244,affine,0.05640316746034286,0.11280633492068572,0.9319962481713715,0.06051866364377433,eta_l0_n292_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,292,1,-0.8645810082391754,0.838881124514349,-0.012849941862413172,0.8517310663767622,1.7034621327535244,affine,0.23933924698846587,0.47867849397693174,1.0,0.23933924698846587,eta_l0_n292_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,293,0,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.042222681286823445,0.08444536257364689,0.9319962481713715,0.045303488473979046,eta_l0_n293_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,293,1,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.2041600550732502,0.4083201101465004,1.0,0.2041600550732502,eta_l0_n293_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,294,0,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.09129259489700804,0.18258518979401608,0.9319962481713715,0.09795382232077564,eta_l0_n294_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,294,1,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.3058624608389326,0.6117249216778652,1.0,0.3058624608389326,eta_l0_n294_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,295,0,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.16854344612361224,0.3370868922472245,0.9319962481713715,0.1808413354177164,eta_l0_n295_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,295,1,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.3979471867839069,0.7958943735678138,1.0,0.3979471867839069,eta_l0_n295_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,296,0,-0.7369816136118256,0.7101516754046736,-0.01341496910357598,0.7235666445082496,1.4471332890164992,affine,0.03773032047057046,0.07546064094114092,0.9319962481713715,0.04048333943897248,eta_l0_n296_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,296,1,-0.7369816136118256,0.7101516754046736,-0.01341496910357598,0.7235666445082496,1.4471332890164992,affine,0.19157267510947118,0.38314535021894236,1.0,0.19157267510947118,eta_l0_n296_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,297,0,-0.9825135379376,1.01382408988808,0.01565527597523997,0.99816881391284,1.99633762782568,affine,0.08144974624643898,0.16289949249287797,0.9319962481713715,0.08739278340041381,eta_l0_n297_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,297,1,-0.9825135379376,1.01382408988808,0.01565527597523997,0.99816881391284,1.99633762782568,affine,0.2893225528047418,0.5786451056094836,1.0,0.2893225528047418,eta_l0_n297_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,298,0,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,affine,0.055178454356791036,0.11035690871358207,0.9319962481713715,0.05920458850028016,eta_l0_n298_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,298,1,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,affine,0.23657700326353948,0.47315400652707895,1.0,0.23657700326353948,eta_l0_n298_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,299,0,-0.7428390480563718,0.7269071605470809,-0.007965943754645433,0.7348731043017264,1.4697462086034527,affine,0.03921345041839409,0.07842690083678817,0.9319962481713715,0.042074686990782484,eta_l0_n299_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,0,299,1,-0.7428390480563718,0.7269071605470809,-0.007965943754645433,0.7348731043017264,1.4697462086034527,affine,0.19593380084189632,0.39186760168379264,1.0,0.19593380084189632,eta_l0_n299_r1,NA,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/benchmark_metadata.json b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/benchmark_metadata.json new file mode 100644 index 0000000..f2fd730 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/benchmark_metadata.json @@ -0,0 +1,13 @@ +{ + "schema_version": "1.2", + "benchmark_level": "medium", + "problem_id": "poisson_100d_ridge", + "model_id": "pinn_100d_poisson_shallow_300_seed_20260804", + "method_id": "hybrid_pz_topk96_B_symbolic", + "git_commit": "fb576a55aed822c24764f8d4db93e38495445e37", + "dtype": "float64", + "device": "cpu", + "quantile_interpolation": "linear", + "cell_average": "volume_weighted", + "timestamp_utc": "2026-08-04T11:32:53.044341+00:00" +} diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/cell_intervals.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/cell_intervals.csv new file mode 100644 index 0000000..bce4963 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/cell_intervals.csv @@ -0,0 +1,102 @@ +run_id,split_id,cell_id,cell_weight,quantity,output_index,input_index_a,input_index_b,lower,upper,midpoint,radius,width,magnitude,mignitude,local_relative_radius,global_normalized_radius,sign_certified,status,local_relative_width +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,0,NA,-0.5385561292613027,0.8222511396975952,0.14184750521814626,0.680403634479449,1.360807268958898,0.8222511396975952,0.0,0.8274888311248624,0.6858659039545195,0,ok,1.6549776622497248 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,1,NA,-0.5747547318457272,0.8371732500059902,0.13120925908013148,0.7059639909258587,1.4119279818517174,0.8371732500059902,0.0,0.8432710802940817,0.7116314585329115,0,ok,1.6865421605881634 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,2,NA,-0.4695020822193571,0.7216311148175457,0.12606451629909432,0.5955665985184514,1.1911331970369028,0.7216311148175457,0.0,0.8253061519790926,0.6003477976282237,0,ok,1.6506123039581853 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,3,NA,-0.5474334858271525,0.8186026624416,0.13558458830722375,0.6830180741343762,1.3660361482687524,0.8186026624416,0.0,0.8343706971306162,0.6885013322891037,0,ok,1.6687413942612324 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,4,NA,-0.49404359181320395,0.7587889593361705,0.13237268376148328,0.6264162755746873,1.2528325511493745,0.7587889593361705,0.0,0.8255474303720892,0.6314451353975434,0,ok,1.6510948607441784 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,5,NA,-0.5653538919166423,0.8462472958285323,0.140446701955945,0.7058005938725873,1.4116011877451746,0.8462472958285323,0.0,0.8340358632184007,0.7114667497307147,0,ok,1.6680717264368015 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,6,NA,-0.5538921973921932,0.8065126678295728,0.12631023521868978,0.680202432610883,1.360404865221766,0.8065126678295728,0.0,0.8433871651903416,0.6856630868405766,0,ok,1.6867743303806833 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,7,NA,-0.6876719681600314,0.9505113005819048,0.1314196662109367,0.8190916343709681,1.6381832687419362,0.9505113005819048,0.0,0.8617379234413296,0.8256672888868837,0,ok,1.7234758468826592 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,8,NA,-0.5592158473012507,0.8308249810912667,0.13580456689500797,0.6950204141962587,1.3900408283925174,0.8308249810912667,0.0,0.8365425089690584,0.700600026944682,0,ok,1.6730850179381167 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,9,NA,-0.514444196889863,0.7919756376917821,0.13876572040095958,0.6532099172908226,1.3064198345816451,0.7919756376917821,0.0,0.8247853673815104,0.6584538760400447,0,ok,1.6495707347630209 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,10,NA,-0.5846954163903301,0.8643394189413512,0.1398220012755106,0.7245174176658407,1.4490348353316813,0.8643394189413512,0.0,0.8382325297083344,0.730333831885469,0,ok,1.6764650594166688 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,11,NA,-0.5809553467400683,0.8593289385520002,0.13918679590596594,0.7201421426460343,1.4402842852920685,0.8593289385520002,0.0,0.8380285014717407,0.7259234322279113,0,ok,1.6760570029434814 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,12,NA,-0.541245643764008,0.8104946220270334,0.1346244891315127,0.6758701328955207,1.3517402657910413,0.8104946220270334,0.0,0.8338983560497674,0.6812960074925185,0,ok,1.6677967120995347 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,13,NA,-0.4922326987517904,0.7742521116586023,0.14100970645340596,0.6332424052051964,1.2664848104103927,0.7742521116586023,0.0,0.8178762391085572,0.6383260650873469,0,ok,1.6357524782171144 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,14,NA,-0.6131212046857248,0.8611898650425667,0.12403433017842092,0.7371555348641458,1.4743110697282915,0.8611898650425667,0.0,0.8559733048271647,0.7430734076861336,0,ok,1.7119466096543294 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,15,NA,-0.5449766151404094,0.8140380014464283,0.13453069315300947,0.6795073082934189,1.3590146165868378,0.8140380014464283,0.0,0.8347365934833905,0.6849623820762304,0,ok,1.669473186966781 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,16,NA,-0.6015462251322486,0.8637987840704929,0.13112627946912214,0.7326725046013708,1.4653450092027416,0.8637987840704929,0.0,0.8481981198778566,0.7385543877282436,0,ok,1.6963962397557133 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,17,NA,-0.5146495501608901,0.7771104815407972,0.13123046568995356,0.6458800158508436,1.2917600317016873,0.7771104815407972,0.0,0.8311302333359866,0.6510651302687571,0,ok,1.6622604666719731 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,18,NA,-0.5765233897237163,0.8161719715481468,0.11982429091221525,0.6963476806359316,1.392695361271863,0.8161719715481468,0.0,0.8531874468993,0.701937948658065,0,ok,1.7063748937986 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,19,NA,-0.47141683805803014,0.7326747481277281,0.130628955034849,0.6020457930928791,1.2040915861857582,0.7326747481277281,0.0,0.8217094892806701,0.6068790070728747,0,ok,1.6434189785613402 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,20,NA,-0.5064480025333881,0.760194051085545,0.1268730242760785,0.6333210268094666,1.2666420536189331,0.760194051085545,0.0,0.8331044236732636,0.6384053178645964,0,ok,1.6662088473465273 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,21,NA,-0.6356443584943197,0.9017883892420131,0.13307201537384672,0.7687163738681664,1.5374327477363328,0.9017883892420131,0.0,0.852435430571801,0.7748876165999596,0,ok,1.704870861143602 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,22,NA,-0.5784297986933006,0.8218416794939308,0.12170594040031513,0.7001357390936157,1.4002714781872314,0.8218416794939308,0.0,0.8519107226646639,0.705756417588349,0,ok,1.7038214453293279 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,23,NA,-0.49529070389381086,0.7616062955793256,0.13315779584275736,0.6284484997365682,1.2568969994731365,0.7616062955793256,0.0,0.8251619023954244,0.6334936742224324,0,ok,1.6503238047908488 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,24,NA,-0.5423911993462552,0.7966333588844231,0.12712107976908393,0.6695122791153392,1.3390245582306783,0.7966333588844231,0.0,0.8404271195131726,0.6748871129640656,0,ok,1.6808542390263452 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,25,NA,-0.5902920086689517,0.849229279454032,0.12946863539254017,0.7197606440614919,1.4395212881229837,0.849229279454032,0.0,0.8475457234873304,0.7255388709788452,0,ok,1.6950914469746607 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,26,NA,-0.6079777392922127,0.8669934845266433,0.1295078726172153,0.737485611909428,1.474971223818856,0.8669934845266433,0.0,0.8506241685450228,0.7434061345846463,0,ok,1.7012483370900455 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,27,NA,-0.5272816435801113,0.7987619596917024,0.13574015805579553,0.6630218016359068,1.3260436032718137,0.7987619596917024,0.0,0.8300618145258356,0.6683445300354294,0,ok,1.6601236290516712 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,28,NA,-0.4836599875642732,0.733408898687583,0.12487445556165488,0.6085344431259281,1.2170688862518562,0.733408898687583,0.0,0.8297341963192503,0.6134197478844173,0,ok,1.6594683926385005 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,29,NA,-0.579175597019563,0.8577534189544697,0.13928891096745333,0.7184645079870163,1.4369290159740327,0.8577534189544697,0.0,0.8376119431418473,0.7242323295447326,0,ok,1.6752238862836946 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,30,NA,-0.713045458833589,0.9920359512791399,0.13949524622277543,0.8525407050563645,1.705081410112729,0.9920359512791399,0.0,0.8593848881757672,0.8593848881757672,0,ok,1.7187697763515344 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,31,NA,-0.5406162309394681,0.809074574212456,0.13422917163649395,0.674845402575962,1.349690805151924,0.809074574212456,0.0,0.8340954271524956,0.6802630506543744,0,ok,1.6681908543049913 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,32,NA,-0.5577261225572173,0.8129356306517012,0.12760475404724192,0.6853308766044592,1.3706617532089185,0.8129356306517012,0.0,0.8430321550244442,0.6908327019003571,0,ok,1.6860643100488883 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,33,NA,-0.5958895227296551,0.8712997190660167,0.13770509816818077,0.7335946208978359,1.4671892417956718,0.8712997190660167,0.0,0.8419543870439982,0.739483906759561,0,ok,1.6839087740879963 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,34,NA,-0.5094020354768468,0.7884391647824612,0.1395185646528072,0.648920600129654,1.297841200259308,0.7884391647824612,0.0,0.8230446039659866,0.6541301243094366,0,ok,1.646089207931973 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,35,NA,-0.5970152710464862,0.848631070608519,0.12580789978101636,0.7228231708275026,1.4456463416550052,0.848631070608519,0.0,0.8517519518925879,0.7286259836606607,0,ok,1.7035039037851758 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,36,NA,-0.5300435559601534,0.7964126092893965,0.13318452666462155,0.663228082624775,1.32645616524955,0.7964126092893965,0.0,0.8327694399722574,0.6685524670447707,0,ok,1.6655388799445148 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,37,NA,-0.6109869756183782,0.8995732643397686,0.14429314436069518,0.7552801199790734,1.5105602399581468,0.8995732643397686,0.0,0.8395982294264854,0.7613434966799425,0,ok,1.6791964588529709 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,38,NA,-0.5208781090556123,0.7865102802707138,0.13281608560755076,0.653694194663163,1.307388389326326,0.7865102802707138,0.0,0.8311324226279204,0.6589420411833704,0,ok,1.6622648452558408 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,39,NA,-0.4545165632126038,0.7303793449750186,0.13793139088120737,0.5924479540938112,1.1848959081876225,0.7303793449750186,0.0,0.811151024696728,0.5972041167761144,0,ok,1.622302049393456 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,40,NA,-0.5505472436428157,0.8115863630224712,0.13051955968982776,0.6810668033326435,1.362133606665287,0.8115863630224712,0.0,0.8391797033112373,0.6865343967166411,0,ok,1.6783594066224745 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,41,NA,-0.5369683120354769,0.8255402371498715,0.14428596255719728,0.6812542745926742,1.3625085491853484,0.8255402371498715,0.0,0.8252223743141388,0.6867233729929434,0,ok,1.6504447486282776 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,42,NA,-0.6215382238677585,0.8847615207293642,0.13161164843080286,0.7531498722985613,1.5062997445971227,0.8847615207293642,0.0,0.8512461885522472,0.7591961474051855,0,ok,1.7024923771044944 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,43,NA,-0.6032973440018511,0.882965520014313,0.13983408800623098,0.7431314320080821,1.4862628640161641,0.882965520014313,0.0,0.841631315338379,0.749097279236586,0,ok,1.683262630676758 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,44,NA,-0.47967586699598147,0.7602727255167319,0.1402984292603752,0.6199742962563567,1.2399485925127134,0.7602727255167319,0.0,0.8154630245810554,0.6249514399724692,0,ok,1.6309260491621107 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,45,NA,-0.5350841986876227,0.8223779667686839,0.1436468840405306,0.6787310827281533,1.3574621654563066,0.8223779667686839,0.0,0.8253274165345735,0.6841799249845648,0,ok,1.650654833069147 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,46,NA,-0.6384125828450896,0.9130678658659815,0.13732764151044596,0.7757402243555356,1.5514804487110712,0.9130678658659815,0.0,0.8495975527731444,0.7819678544464939,0,ok,1.6991951055462888 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,47,NA,-0.5117428875059774,0.7863804180125354,0.13731876525327902,0.6490616527592564,1.2981233055185128,0.7863804180125354,0.0,0.8253787071652513,0.6542723093072893,0,ok,1.6507574143305026 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,48,NA,-0.5367776725745083,0.807023390485425,0.1351228589554584,0.6719005315299666,1.3438010630599333,0.807023390485425,0.0,0.8325663660452495,0.6772945382308092,0,ok,1.665132732090499 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,49,NA,-0.5721215199845059,0.8412006179887893,0.1345395490021417,0.7066610689866476,1.4133221379732952,0.8412006179887893,0.0,0.8400624700873262,0.7123341327252027,0,ok,1.6801249401746523 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,50,NA,-0.5072481146760655,0.790910133664485,0.1418310094942098,0.6490791241702752,1.2981582483405505,0.790910133664485,0.0,0.8206736727002458,0.6542899209785158,0,ok,1.6413473454004917 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,51,NA,-0.4986587508494818,0.7599428012741477,0.13064202521233295,0.6293007760618148,1.2586015521236296,0.7599428012741477,0.0,0.8280896601779848,0.6343527926083614,0,ok,1.6561793203559696 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,52,NA,-0.5915057003895957,0.8530613457103607,0.13077782266038251,0.7222835230499782,1.4445670460999565,0.8530613457103607,0.0,0.8466958756037865,0.7280820035994254,0,ok,1.693391751207573 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,53,NA,-0.5047262490200412,0.7748939976198376,0.1350838742998982,0.6398101233199394,1.2796202466398787,0.7748939976198376,0.0,0.8256743829287342,0.6449465087378764,0,ok,1.6513487658574684 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,54,NA,-0.5419755293062319,0.8124313955740038,0.13522793313388592,0.6772034624401179,1.3544069248802357,0.8124313955740038,0.0,0.8335515664823072,0.682640040985335,0,ok,1.6671031329646144 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,55,NA,-0.5210581233314914,0.7876173514222692,0.1332796140453889,0.6543377373768803,1.3086754747537606,0.7876173514222692,0.0,0.8307812622402054,0.6595907502476814,0,ok,1.6615625244804109 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,56,NA,-0.5407793758120242,0.8070913818515993,0.13315600301978758,0.6739353788318118,1.3478707576636235,0.8070913818515993,0.0,0.8350174391475895,0.6793457212541879,0,ok,1.670034878295179 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,57,NA,-0.48265070401238136,0.7730880930097691,0.14521869449869387,0.6278693985110753,1.2557387970221505,0.7730880930097691,0.0,0.8121576366112021,0.6329099239815804,0,ok,1.6243152732224042 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,58,NA,-0.5300374232403017,0.7960358913349622,0.13299923404733027,0.663036657287632,1.326073314575264,0.7960358913349622,0.0,0.8329230685512322,0.6683595049480885,0,ok,1.6658461371024644 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,59,NA,-0.4795304594110246,0.7521909435257143,0.1363302420573448,0.6158607014683695,1.231721402936739,0.7521909435257143,0.0,0.8187558049843973,0.6208048213114387,0,ok,1.6375116099687945 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,60,NA,-0.6416364592644307,0.9129633592936653,0.13566345001461733,0.777299909279048,1.554599818558096,0.9129633592936653,0.0,0.8514031821391207,0.7835400604955804,0,ok,1.7028063642782414 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,61,NA,-0.6538692451581801,0.9187609545550567,0.13244585469843828,0.7863150998566184,1.5726301997132368,0.9187609545550567,0.0,0.8558429654179416,0.7926276248786516,0,ok,1.7116859308358832 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,62,NA,-0.5059253339255655,0.7672114034761165,0.1306430347752755,0.636568368700841,1.273136737401682,0.7672114034761165,0.0,0.8297170321200231,0.6416787293646405,0,ok,1.6594340642400462 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,63,NA,-0.4430068256812901,0.6955588176627062,0.12627599599070807,0.5692828216719982,1.1385656433439963,0.6955588176627062,0.0,0.8184538923465385,0.5738530150424084,0,ok,1.636907784693077 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,64,NA,-0.5353545503560182,0.7921130604738575,0.12837925505891967,0.6637338054149379,1.3274676108298757,0.7921130604738575,0.0,0.8379281172537154,0.6690622497693895,0,ok,1.6758562345074308 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,65,NA,-0.5716794079328733,0.8434577278461428,0.13588915995663475,0.7075685678895081,1.4151371357790161,0.8434577278461428,0.0,0.8388903729607862,0.7132489170147139,0,ok,1.6777807459215723 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,66,NA,-0.5403647348135069,0.8133989435398404,0.13651710436316677,0.6768818391766737,1.3537636783533473,0.8133989435398404,0.0,0.8321646401837499,0.6823158357354855,0,ok,1.6643292803674998 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,67,NA,-0.5663558054982991,0.8491475296307922,0.14139586206624655,0.7077516675645457,1.4155033351290913,0.8491475296307922,0.0,0.8334849279633125,0.7134334866110088,0,ok,1.666969855926625 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,68,NA,-0.49764098895724435,0.7653277399799034,0.13384337551132955,0.6314843644685739,1.2629687289371478,0.7653277399799034,0.0,0.8251162625898755,0.6365539108278611,0,ok,1.650232525179751 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,69,NA,-0.558874008538181,0.8355428208296104,0.13833440614571468,0.6972084146838957,1.3944168293677914,0.8355428208296104,0.0,0.8344376820707257,0.7028055926651742,0,ok,1.6688753641414513 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,70,NA,-0.5748994070637128,0.8375988442257112,0.1313497185809992,0.706249125644712,1.412498251289424,0.8375988442257112,0.0,0.8431830231303378,0.7119188823087189,0,ok,1.6863660462606755 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,71,NA,-0.5185238753507772,0.7756831497246987,0.12857963718696075,0.6471035125377379,1.2942070250754758,0.7756831497246987,0.0,0.8342369081594778,0.652298449167449,0,ok,1.6684738163189556 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,72,NA,-0.5215321966880009,0.7746193939910602,0.12654359865152964,0.6480757953395305,1.296151590679061,0.7746193939910602,0.0,0.8366377092632022,0.6532785374399949,0,ok,1.6732754185264045 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,73,NA,-0.4930734013322184,0.7719467610988466,0.13943667988331412,0.6325100812155325,1.265020162431065,0.7719467610988466,0.0,0.8193700823553822,0.6375878620124286,0,ok,1.6387401647107644 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,74,NA,-0.6208991680382628,0.9133795163456554,0.14624017415369628,0.7671393421919591,1.5342786843839182,0.9133795163456554,0.0,0.8398911169600242,0.7732979245387255,0,ok,1.6797822339200483 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,75,NA,-0.5276321875012563,0.8093469195661961,0.1408573660324699,0.6684895535337262,1.3369791070674524,0.8093469195661961,0.0,0.825961695007169,0.6738561769578711,0,ok,1.651923390014338 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,76,NA,-0.4857379738306019,0.7589016290729088,0.13658182762115345,0.6223198014517554,1.2446396029035107,0.7589016290729088,0.0,0.8200269674107764,0.627315774845993,0,ok,1.6400539348215528 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,77,NA,-0.5540753161071208,0.836776946793639,0.1413508153432591,0.6954261314503799,1.3908522629007598,0.836776946793639,0.0,0.8310770679272571,0.7010090012904183,0,ok,1.6621541358545142 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,78,NA,-0.6344366449934056,0.9129710305884684,0.13926719279753141,0.773703837790937,1.547407675581874,0.9129710305884684,0.0,0.8474571611458857,0.779915119803185,0,ok,1.6949143222917713 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,79,NA,-0.58574117655362,0.8724825401706733,0.14337068180852663,0.7291118583621466,1.4582237167242933,0.8724825401706733,0.0,0.8356750133011478,0.7349651566780653,0,ok,1.6713500266022956 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,80,NA,-0.6143774482567524,0.8554075576366994,0.12051505468997348,0.7348925029467259,1.4697850058934518,0.8554075576366994,0.0,0.8591138766380206,0.7407922081847428,0,ok,1.7182277532760413 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,81,NA,-0.5689004912428196,0.8446983461586725,0.13789892745792642,0.706799418700746,1.413598837401492,0.8446983461586725,0.0,0.8367477241017082,0.712473593108589,0,ok,1.6734954482034163 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,82,NA,-0.528253124148856,0.7887013829101219,0.130224129380633,0.6584772535294889,1.3169545070589779,0.7887013829101219,0.0,0.8348879154996068,0.6637634983696332,0,ok,1.6697758309992137 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,83,NA,-0.5610792381906715,0.8062592654070528,0.12259001360819066,0.6836692517988622,1.3673385035977244,0.8062592654070528,0.0,0.847952118049398,0.6891577375974459,0,ok,1.695904236098796 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,84,NA,-0.5425954025707853,0.8016792324797373,0.12954191495447598,0.6721373175252613,1.3442746350505226,0.8016792324797373,0.0,0.8384117865274124,0.6775332251402799,0,ok,1.6768235730548249 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,85,NA,-0.6405406311789529,0.9215169457105138,0.14048815726578046,0.7810287884447333,1.5620575768894667,0.9215169457105138,0.0,0.8475468542171404,0.7872988750434577,0,ok,1.6950937084342808 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,86,NA,-0.5149337065165599,0.7868626695842719,0.13596448153385599,0.6508981880504159,1.3017963761008318,0.7868626695842719,0.0,0.8272068471545474,0.6561235882743383,0,ok,1.6544136943090948 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,87,NA,-0.533988629610563,0.7908770020415273,0.12844418621548215,0.6624328158260452,1.3248656316520904,0.7908770020415273,0.0,0.8375927155753382,0.6677508158569238,0,ok,1.6751854311506764 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,88,NA,-0.5539022816629978,0.8386175215454877,0.14235761994124496,0.6962599016042428,1.3925198032084856,0.8386175215454877,0.0,0.8302472625674524,0.7018494649377164,0,ok,1.6604945251349048 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,89,NA,-0.5274398390397788,0.8105786157189017,0.14156938833956145,0.6690092273793402,1.3380184547586804,0.8105786157189017,0.0,0.8253477385237906,0.6743800227367908,0,ok,1.6506954770475812 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,90,NA,-0.43987524059489647,0.7060410778519882,0.13308291862854588,0.5729581592234423,1.1459163184468846,0.7060410778519882,0.0,0.811508249585947,0.5775578581448233,0,ok,1.623016499171894 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,91,NA,-0.6196081716524764,0.8733019846697505,0.12684690650863706,0.7464550781611135,1.492910156322227,0.8733019846697505,0.0,0.8547502367619081,0.752447607567677,0,ok,1.7095004735238162 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,92,NA,-0.5035144221376043,0.7808407401727092,0.13866315901755244,0.6421775811551568,1.2843551623103135,0.7808407401727092,0.0,0.8224181297368239,0.6473329724866598,0,ok,1.6448362594736479 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,93,NA,-0.5217337945544605,0.7887499223768668,0.13350806391120318,0.6552418584656636,1.3104837169313273,0.7887499223768668,0.0,0.8307346091282242,0.6605021296060782,0,ok,1.6614692182564483 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,94,NA,-0.5241731738244159,0.7980193614982166,0.13692309383690038,0.6610962676613162,1.3221925353226325,0.7980193614982166,0.0,0.8284213385752466,0.6664035379049448,0,ok,1.6568426771504932 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,95,NA,-0.510308144285408,0.7524467764734875,0.1210693160940397,0.6313774603794478,1.2627549207588955,0.7524467764734875,0.0,0.8390991630511615,0.6364461485144204,0,ok,1.678198326102323 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,96,NA,-0.4959827050252069,0.7678527354311492,0.13593501520297113,0.631917720228178,1.263835440456356,0.7678527354311492,0.0,0.8229673361433769,0.6369907455605593,0,ok,1.6459346722867538 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,97,NA,-0.5958534844756822,0.8773333299618036,0.14073992274306069,0.7365934072187429,1.4731868144374858,0.8773333299618036,0.0,0.839582154311648,0.7425067672890008,0,ok,1.679164308623296 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,98,NA,-0.6276478736749588,0.8963746618062841,0.13436339406566267,0.7620112677406214,1.524022535481243,0.8963746618062841,0.0,0.850103533945385,0.7681286819878629,0,ok,1.70020706789077 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,J,0,99,NA,-0.5941517101240096,0.8531878917593233,0.12951809081765686,0.7236698009416664,1.4473396018833329,0.8531878917593233,0.0,0.8481951137977556,0.729479410507815,0,ok,1.6963902275955112 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,0,1.0,Y,0,NA,NA,-2.9913949982448904,3.506439915785475,0.25752245877029223,3.2489174570151826,6.497834914030365,3.506439915785475,0.0,0.9265572874610045,0.9265572874610045,0,ok,1.853114574922009 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/complexity.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/complexity.csv new file mode 100644 index 0000000..217ff2a --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/complexity.csv @@ -0,0 +1,4 @@ +run_id,quantity,n_alpha,n_eta,n_monomials,n_mixed_monomials,max_degree,n_coefficients,status +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,Y,100.0,400.0,400.0,0.0,1.0,400.0,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,J,100.0,400.0,196.0,0.0,1.0,19600.0,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,H,NA,NA,NA,NA,NA,NA,not_implemented diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/layer_normalized_radius_Y.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/layer_normalized_radius_Y.csv new file mode 100644 index 0000000..d064387 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/layer_normalized_radius_Y.csv @@ -0,0 +1,301 @@ +neuron,layer_0 +0,0.6162421199489911 +1,0.5419218285316451 +2,0.6525327778653724 +3,0.637024294710363 +4,0.7427237147703024 +5,0.6646561232806381 +6,0.6281257110189709 +7,0.543776295629057 +8,0.8657529873940282 +9,0.651544860521241 +10,0.6165398200770281 +11,0.6251195724642008 +12,0.6912848856671185 +13,0.685035917252616 +14,0.8453483334854439 +15,0.6454932120893029 +16,0.9546592428300092 +17,0.6844219675544602 +18,0.9989769533006161 +19,0.7132754622721597 +20,0.6544510118719792 +21,0.7250216825362106 +22,0.7881857569830729 +23,0.7751149638665825 +24,0.859768585369206 +25,0.830398867531548 +26,0.7610666779300438 +27,0.7755950750012318 +28,0.6479592660850625 +29,0.8336659398768762 +30,0.577638923433512 +31,0.7251830806294721 +32,0.8600723383926094 +33,0.6476726254761521 +34,0.6909176216103874 +35,0.7047202246124595 +36,0.8552648324434805 +37,0.7914027620828362 +38,0.5933413914154003 +39,0.7468845735772127 +40,0.7235186720351032 +41,0.6355710553440279 +42,0.626288058482591 +43,0.9528482053020632 +44,0.8615678731924009 +45,0.5368203060683006 +46,0.6688985438254108 +47,0.6443708874499796 +48,0.7467942927395395 +49,0.6385533374042454 +50,0.8230485319677043 +51,0.6296994068635823 +52,0.5916061892419623 +53,0.7685769031707625 +54,0.8246169214452626 +55,0.54171952478903 +56,0.5591547684566563 +57,0.834630628282293 +58,0.5705099526169742 +59,0.6983825094353423 +60,0.9711665349434576 +61,0.7925580496557257 +62,0.7739003345523898 +63,0.9033425122231032 +64,0.6596971791975124 +65,0.6517398863272531 +66,0.7201117129977588 +67,0.6761713582560301 +68,0.7714366430971834 +69,0.9099648233044394 +70,0.704484787216771 +71,0.9159053838275886 +72,0.8244760671147741 +73,0.8255080588076058 +74,0.6846305582944902 +75,0.675608962006266 +76,0.8131197354843077 +77,0.8903252064464775 +78,0.5109893040753523 +79,0.6470779993909104 +80,0.8507588073142239 +81,0.48828299860482227 +82,0.6098286677909557 +83,0.6681325395936913 +84,0.6265530750327327 +85,0.6889388762532255 +86,0.7968470016124662 +87,0.8531743537692537 +88,0.6404709513492913 +89,0.6633758900956943 +90,0.6230628600687211 +91,0.8603494246601253 +92,0.7854578489955437 +93,0.5279852281313057 +94,0.8334946520781183 +95,0.6553095660427952 +96,0.5221521700057307 +97,0.8275395025216328 +98,0.8675218865905903 +99,0.6754901047331782 +100,0.9290957181679524 +101,0.9718747288736591 +102,0.7544446448407093 +103,0.7963318065887969 +104,0.6624648458375416 +105,0.8281177477259958 +106,0.5729591361537771 +107,0.48646887613813905 +108,0.764051344262394 +109,0.6952094840415755 +110,0.8484577743164452 +111,0.701198157325423 +112,0.6221638583532983 +113,0.6555523479516163 +114,0.5238569605481382 +115,0.5803652319175282 +116,0.6733924570906795 +117,0.9072138935425135 +118,0.5538870821400664 +119,0.8060555734457937 +120,0.5878475917970294 +121,0.5408231103857551 +122,0.6853393974816521 +123,0.7938620192397763 +124,0.5990550539427488 +125,0.9022322562957132 +126,0.8136390896551343 +127,0.8052346460926463 +128,0.6983654751243557 +129,0.6321577929598001 +130,0.7496539427504885 +131,0.7495822560561621 +132,0.7726884769714508 +133,0.8440855668704697 +134,0.7295759818673087 +135,0.8732070662815837 +136,0.6468853179208286 +137,0.806865572195593 +138,0.9320892519043357 +139,0.8131811376887522 +140,0.7411601804149776 +141,0.6620114115895328 +142,0.785938628164867 +143,0.6325580465007923 +144,0.7624114006192217 +145,0.6740164849680114 +146,0.8796752345182725 +147,0.7017082917790937 +148,0.9152976338348483 +149,0.6435628091587109 +150,0.6353369849744391 +151,0.5975395068039396 +152,0.8684341609763699 +153,0.5598472710329968 +154,0.6066249584541327 +155,0.6646263568977663 +156,0.7054739897318261 +157,0.8240778609863428 +158,0.6821186903787475 +159,0.78845386314625 +160,0.9367525217150431 +161,0.6182182494715668 +162,0.7231220561650594 +163,0.6456272556641447 +164,0.799794685950358 +165,0.7191218970415708 +166,0.8173451869250589 +167,0.7762967129222806 +168,0.7577494022439597 +169,0.6576045490629244 +170,0.6560185667358831 +171,0.801364092260314 +172,0.5199058524673331 +173,0.5296623781354116 +174,0.8489691556477169 +175,0.9117735772737228 +176,0.7521245822606443 +177,0.6476925002821613 +178,0.7563424348594338 +179,0.8367961820471885 +180,0.7736008406491968 +181,0.8019484928702707 +182,0.693601603310003 +183,0.6887513360557742 +184,0.931283851277937 +185,0.5784294415472628 +186,0.6681619265262403 +187,0.7630300157898852 +188,0.6431987744455856 +189,0.7699110668821418 +190,0.6370287818625128 +191,0.719251344513818 +192,0.4914141614607233 +193,0.5709855737214663 +194,0.7029058056270725 +195,0.932253345899195 +196,0.816999899919923 +197,0.49713666100544623 +198,0.6821086174382271 +199,0.7900017819271218 +200,0.7292898027159896 +201,0.7910902417132392 +202,0.6220698339112714 +203,0.6847364428358164 +204,0.5873884707760092 +205,0.7293810867623591 +206,0.86752801895761 +207,0.8687592485001452 +208,0.8212440643846498 +209,0.8352457761128697 +210,0.6987361037936566 +211,0.8723927041740711 +212,0.7798132270081817 +213,0.9038894681178925 +214,0.49108414803207956 +215,0.5296268453931839 +216,0.8563636346631119 +217,0.7323919388719169 +218,0.7306481333777458 +219,0.7526980757677582 +220,0.7368032693234934 +221,0.6980074007702309 +222,0.7887509191444781 +223,0.5676430654113679 +224,0.8630306925047158 +225,0.6311002366227472 +226,0.741103947737733 +227,0.8481680525951041 +228,0.7382943637762912 +229,0.6008984291499322 +230,0.9293008484031721 +231,0.7300032302497331 +232,0.738678469054186 +233,0.7652617361979039 +234,0.7185404049956695 +235,0.7264295487074044 +236,0.8172424721389996 +237,0.6867625929112331 +238,0.8597439029896542 +239,0.7084514124737799 +240,0.8097671992413316 +241,0.6293817962278792 +242,0.6368686413963487 +243,0.7878994162668858 +244,0.902222207242424 +245,0.7669338079892857 +246,0.6843791934140323 +247,0.6337265624309671 +248,0.7052846827158022 +249,0.6857994352130584 +250,0.5828781625114009 +251,0.5216217649225898 +252,0.7899318569568655 +253,0.6221052967589766 +254,0.7339873002011027 +255,0.8586812771175738 +256,0.6484040726146777 +257,0.48934342656612595 +258,0.7241077021581477 +259,0.6516994095776407 +260,0.6382654051471263 +261,0.65038165873561 +262,0.7232558861886125 +263,0.7518161929425176 +264,0.590377345136714 +265,0.6624628318580533 +266,0.7724032715998305 +267,0.6988892946404283 +268,0.66461720593681 +269,0.648338441739094 +270,0.7258076409702059 +271,0.7086962762919159 +272,0.723201435588892 +273,0.6132558048139611 +274,0.9047843465339248 +275,0.7518906902116689 +276,0.8043475276533333 +277,0.7940551159124855 +278,0.5976623127230066 +279,0.7077269709075823 +280,0.784406875698381 +281,0.5825145547288328 +282,0.9527891403677229 +283,0.6251814278730241 +284,0.674642251412045 +285,0.6956741249057677 +286,0.8717298877588612 +287,0.7472835883496265 +288,0.7206120437267526 +289,0.8454980596041871 +290,0.847466837773604 +291,0.685029461329672 +292,0.6501792735220139 +293,0.5919296981013319 +294,0.7589313102422693 +295,0.9215966179594822 +296,0.5706432778521059 +297,0.7317465260702745 +298,0.6456245621083724 +299,0.5779864956945218 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/metrics.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/metrics.csv new file mode 100644 index 0000000..c66490c --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/metrics.csv @@ -0,0 +1,79 @@ +schema_version,benchmark_level,problem_id,model_id,method_id,run_id,split_id,quantity,metric,aggregation,derivative_order,layer,neuron,output_index,input_index_a,input_index_b,cell_id,value,unit,status +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,mean_width,mean_weighted,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,max_width,max,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,q50_width,q50,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,q90_width,q90,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,q99_width,q99,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,mean_global_normalized_radius,mean_weighted,0,NA,NA,NA,NA,NA,NA,0.9265572874610045,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,Y,max_global_normalized_radius,max,0,NA,NA,NA,NA,NA,NA,0.9265572874610045,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,mean_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,1.3691779288618617,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,max_width,max,1,NA,NA,NA,NA,NA,NA,1.705081410112729,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,q50_width,q50,1,NA,NA,NA,NA,NA,NA,1.3559345451682712,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,q90_width,q90,1,NA,NA,NA,NA,NA,NA,1.5119064695104565,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,q99_width,q99,1,NA,NA,NA,NA,NA,NA,1.6388522501556444,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,mean_global_normalized_radius,mean_weighted,1,NA,NA,NA,NA,NA,NA,0.690084833667788,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,max_global_normalized_radius,max,1,NA,NA,NA,NA,NA,NA,0.8593848881757672,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,mean_frobenius_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,13.732613537547406,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,J,max_frobenius_width,max,1,NA,NA,NA,NA,NA,NA,13.732613537547406,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,mean_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,max_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,q50_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,q90_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,q99_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,mean_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,H,max_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,width,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,upper,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,width,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,L2,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,9.771853005764254e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,width,none,NA,NA,NA,NA,NA,NA,NA,9.771853005764254e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,77.08632807813812,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,77.08632807813812,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,upper,none,NA,NA,NA,NA,NA,NA,NA,9.885268335136003e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,width,none,NA,NA,NA,NA,NA,NA,NA,9.885268335136003e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,8.779882008212759,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,W12,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,8.779882008212759,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,PDE_residual,linf_upper,max,2,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,boundary_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,initial_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,196.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,runtime,seconds,onejet_construction,NA,NA,NA,NA,NA,NA,NA,0.33028991800074436,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,runtime,seconds,L2_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,0.38106552499993995,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,runtime,seconds,W12_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,1.0608345710006688,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,runtime,seconds,W12_total,NA,NA,NA,NA,NA,NA,NA,1.3911244890014132,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,memory,bytes,peak,NA,NA,NA,NA,NA,NA,NA,NA,bytes,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,soundness,failure_count,Y,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,soundness,max_violation,Y,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,soundness,failure_count,J,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_topk96_B_symbolic,pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,single_cell,soundness,max_violation,J,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/norms.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/norms.csv new file mode 100644 index 0000000..a04d11d --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/norms.csv @@ -0,0 +1,5 @@ +run_id,norm,squared,lower,upper,width,relative_width,value_contribution_upper,gradient_contribution_upper,hessian_contribution_upper,value_contribution_width,gradient_contribution_width,hessian_contribution_width,status,domain_volume,domain_volume_normalized_lower,domain_volume_normalized_upper,domain_volume_normalized_width +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,L2,1,0.0,1.244073696539206e-69,1.244073696539206e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,9.81401102413563,9.81401102413563 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,L2,0,0.0,3.527142889846123e-35,3.527142889846123e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,3.1327321979600535,3.1327321979600535 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,W12,1,0.0,9.771853005764254e-69,9.771853005764254e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,77.08632807813812,77.08632807813812 +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,W12,0,0.0,9.885268335136003e-35,9.885268335136003e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,8.779882008212759,8.779882008212759 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/soundness.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/soundness.csv new file mode 100644 index 0000000..73e862b --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/soundness.csv @@ -0,0 +1,3 @@ +run_id,quantity,sample_count,failure_count,max_violation,invalid_interval_count,nan_endpoint_count,infinite_endpoint_count,status +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,Y,16384,0,0.0,0,0,0,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,J,16384,0,0.0,0,0,0,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/timings.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/timings.csv new file mode 100644 index 0000000..d5dcae9 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_topk96_B_symbolic/timings.csv @@ -0,0 +1,5 @@ +run_id,stage,seconds,status +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,onejet_construction,0.33028991800074436,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,L2_symbolic_integration,0.38106552499993995,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,W12_symbolic_integration,1.0608345710006688,ok +pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic,W12_total,1.3911244890014132,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/activation_approximation.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/activation_approximation.csv new file mode 100644 index 0000000..dce1015 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/activation_approximation.csv @@ -0,0 +1,601 @@ +run_id,split_id,cell_id,cell_weight,layer,neuron,derivative_order,preactivation_lower,preactivation_upper,preactivation_midpoint,preactivation_radius,preactivation_width,approximation_kind,approximation_error_radius,approximation_error_diameter,activation_scale,normalized_approximation_radius,noise_symbol_id,shared_noise_group,status +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,0,0,-0.8000887741224203,0.7908767282671687,-0.004606022927625797,0.7954827511947945,1.590965502389589,affine,0.04776087567042881,0.09552175134085762,0.9319962481713715,0.051245781047014195,eta_l0_n0_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,0,1,-0.8000887741224203,0.7908767282671687,-0.004606022927625797,0.7954827511947945,1.590965502389589,affine,0.21878416980543122,0.43756833961086244,1.0,0.21878416980543122,eta_l0_n0_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,1,0,-0.6668101129690585,0.693384972556816,0.013287429793878758,0.6800975427629372,1.3601950855258744,affine,0.032203781618758816,0.06440756323751763,0.9319962481713715,0.034553552851682,eta_l0_n1_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,1,1,-0.6668101129690585,0.693384972556816,0.013287429793878758,0.6800975427629372,1.3601950855258744,affine,0.17491064771605183,0.34982129543210366,1.0,0.17491064771605183,eta_l0_n1_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,2,0,-0.8409685245821179,0.8705015267430378,0.01476650108045996,0.8557350256625779,1.7114700513251557,affine,0.05704981128360182,0.11409962256720364,0.9319962481713715,0.06121249028151854,eta_l0_n2_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,2,1,-0.8409685245821179,0.8705015267430378,0.01476650108045996,0.8557350256625779,1.7114700513251557,affine,0.24075774656914548,0.48151549313829095,1.0,0.24075774656914548,eta_l0_n2_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,3,0,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.05297381165009602,0.10594762330019204,0.9319962481713715,0.056839082511365886,eta_l0_n3_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,3,1,-0.8131646470132093,0.8461858474217637,0.016510600204277193,0.8296752472174865,1.659350494434973,affine,0.23128307788642286,0.4625661557728457,1.0,0.23128307788642286,eta_l0_n3_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,4,0,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.08534503477820339,0.17069006955640678,0.9319962481713715,0.09157229435811046,eta_l0_n4_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,4,1,-1.0376263596207114,1.0010137091612128,-0.018306325229749287,1.019320034390962,2.038640068781924,affine,0.29599361143468017,0.5919872228693603,1.0,0.29599361143468017,eta_l0_n4_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,5,0,-0.8652278170806444,0.8877201689912206,0.011246175955288096,0.8764739930359325,1.752947986071865,affine,0.06037470341247909,0.12074940682495817,0.9319962481713715,0.06477998546768575,eta_l0_n5_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,5,1,-0.8652278170806444,0.8877201689912206,0.011246175955288096,0.8764739930359325,1.752947986071865,affine,0.2482083281727395,0.496416656345479,1.0,0.2482083281727395,eta_l0_n5_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,6,0,-0.8002253266309621,0.8296484987611809,0.014711586065109361,0.8149369126960715,1.629873825392143,affine,0.05070909200326985,0.1014181840065397,0.9319962481713715,0.05440911602676932,eta_l0_n6_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,6,1,-0.8002253266309621,0.8296484987611809,0.014711586065109361,0.8149369126960715,1.629873825392143,affine,0.22589998281650622,0.45179996563301245,1.0,0.22589998281650622,eta_l0_n6_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,7,0,-0.672406545778101,0.6932924621130971,0.010442958167498073,0.6828495039455991,1.3656990078911981,affine,0.03252624948541378,0.06505249897082756,0.9319962481713715,0.034899549809596433,eta_l0_n7_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,7,1,-0.672406545778101,0.6932924621130971,0.010442958167498073,0.6828495039455991,1.3656990078911981,affine,0.1759971377883037,0.3519942755766074,1.0,0.1759971377883037,eta_l0_n7_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,8,0,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.13821186086084938,0.27642372172169877,0.9319962481713715,0.1482965850259899,eta_l0_n8_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,8,1,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,affine,0.36821782384984864,0.7364356476996973,1.0,0.36821782384984864,eta_l0_n8_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,9,0,-0.8395677251243506,0.8685446340190671,0.014488454447358246,0.8540561795717089,1.7081123591434177,affine,0.056781076836971985,0.11356215367394397,0.9319962481713715,0.06092414743984176,eta_l0_n9_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,9,1,-0.8395677251243506,0.8685446340190671,0.014488454447358246,0.8540561795717089,1.7081123591434177,affine,0.2401571303404774,0.4803142606809548,1.0,0.2401571303404774,eta_l0_n9_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,10,0,-0.80808685285236,0.7839032216460969,-0.01209181560313155,0.7959950372492285,1.591990074498457,affine,0.04786198015530158,0.09572396031060317,0.9319962481713715,0.05135426269065938,eta_l0_n10_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,10,1,-0.80808685285236,0.7839032216460969,-0.01209181560313155,0.7959950372492285,1.591990074498457,affine,0.2189189541178256,0.4378379082356512,1.0,0.2189189541178256,eta_l0_n10_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,11,0,-0.7982094929703913,0.8217535642036398,0.01177203561662421,0.8099815285870156,1.619963057174031,affine,0.049945300875641425,0.09989060175128285,0.9319962481713715,0.05358959435044603,eta_l0_n11_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,11,1,-0.7982094929703913,0.8217535642036398,0.01177203561662421,0.8099815285870156,1.619963057174031,affine,0.22410676447470176,0.44821352894940353,1.0,0.22410676447470176,eta_l0_n11_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,12,0,-0.9067744380742929,0.9399477281012677,0.01658664501348739,0.9233610830877803,1.8467221661755606,affine,0.06823703003790989,0.13647406007581978,0.9319962481713715,0.07321599220146512,eta_l0_n12_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,12,1,-0.9067744380742929,0.9399477281012677,0.01658664501348739,0.9233610830877803,1.8467221661755606,affine,0.26449501696567784,0.5289900339313557,1.0,0.26449501696567784,eta_l0_n12_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,13,0,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.06636055767389427,0.13272111534778855,0.9319962481713715,0.07120260173160287,eta_l0_n13_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,13,1,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,affine,0.26062467619872115,0.5212493523974423,1.0,0.26062467619872115,eta_l0_n13_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,14,0,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.1281480288819092,0.2562960577638184,0.9319962481713715,0.1374984385756303,eta_l0_n14_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,14,1,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,affine,0.3568019908962487,0.7136039817924974,1.0,0.3568019908962487,eta_l0_n14_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,15,0,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.055167049976694256,0.11033409995338851,0.9319962481713715,0.059192351991689963,eta_l0_n15_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,15,1,-0.858538760091075,0.829128620847134,-0.01470506962197049,0.8438336904691045,1.687667380938209,affine,0.23646121014237306,0.4729224202847461,1.0,0.23646121014237306,eta_l0_n15_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,16,0,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.18859813104338555,0.3771962620867711,0.9319962481713715,0.20235932431426154,eta_l0_n16_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,16,1,-1.5135272447646497,1.5437531980141883,0.015112976624769292,1.528640221389419,3.057280442778838,affine,0.4141731440342193,0.8283462880684386,1.0,0.4141731440342193,eta_l0_n16_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,17,0,-0.924974709822734,0.8972162631509851,-0.013879223335874435,0.9110954864868596,1.8221909729737191,affine,0.06613657047259132,0.13227314094518264,0.9319962481713715,0.07096227114900404,eta_l0_n17_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,17,1,-0.924974709822734,0.8972162631509851,-0.013879223335874435,0.9110954864868596,1.8221909729737191,affine,0.2603096831220119,0.5206193662440238,1.0,0.2603096831220119,eta_l0_n17_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,18,0,-1.6733730976128973,1.6691489651366271,-0.0021120662381350908,1.6712610313747622,3.3425220627495245,affine,0.2180433724141755,0.436086744828351,0.9319962481713715,0.23395305811797928,eta_l0_n18_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,18,1,-1.6733730976128973,1.6691489651366271,-0.0021120662381350908,1.6712610313747622,3.3425220627495245,quadratic,0.13549735134988097,0.27099470269976195,1.0,0.13549735134988097,eta_l0_n18_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,19,0,-0.980686547188127,0.9462170674146508,-0.01723473988673807,0.9634518073013889,1.9269036146027778,affine,0.07522189449895324,0.15044378899790647,0.9319962481713715,0.08071051213622671,eta_l0_n19_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,19,1,-0.980686547188127,0.9462170674146508,-0.01723473988673807,0.9634518073013889,1.9269036146027778,affine,0.27798964960046396,0.5559792992009279,1.0,0.27798964960046396,eta_l0_n19_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,20,0,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.05759508702140571,0.11519017404281141,0.9319962481713715,0.061797552441236185,eta_l0_n20_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,20,1,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,affine,0.24189208900710638,0.48378417801421275,1.0,0.24189208900710638,eta_l0_n20_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,21,0,-0.9711358295108825,0.9996589259075812,0.01426154819834935,0.9853973777092319,1.9707947554184637,affine,0.07912843252686426,0.15825686505372852,0.9319962481713715,0.08490209341734867,eta_l0_n21_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,21,1,-0.9711358295108825,0.9996589259075812,0.01426154819834935,0.9853973777092319,1.9707947554184637,affine,0.285222212816681,0.570444425633362,1.0,0.285222212816681,eta_l0_n21_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,22,0,-1.091541462437189,1.130698829634847,0.019578683598828972,1.111120146036018,2.222240292072036,affine,0.10280415066198174,0.2056083013239635,0.9319962481713715,0.1103053267260348,eta_l0_n22_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,22,1,-1.091541462437189,1.130698829634847,0.019578683598828972,1.111120146036018,2.222240292072036,affine,0.323427255172354,0.646854510344708,1.0,0.323427255172354,eta_l0_n22_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,23,0,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.09752239534692254,0.1950447906938451,0.9319962481713715,0.10463818447581405,eta_l0_n23_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,23,1,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,affine,0.31564410377407026,0.6312882075481405,1.0,0.31564410377407026,eta_l0_n23_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,24,0,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.13521015581349047,0.27042031162698094,0.9319962481713715,0.14507585849061122,eta_l0_n24_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,24,1,-1.2523324641820424,1.2922888533963237,0.019978194607140676,1.272310658789183,2.544621317578366,affine,0.36488914114320087,0.7297782822864017,1.0,0.36488914114320087,eta_l0_n24_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,25,0,-1.2223185723825796,1.1844726444174878,-0.018922963982545893,1.2033956084000337,2.4067912168000674,affine,0.12114127036711338,0.24228254073422675,0.9319962481713715,0.12998042707231847,eta_l0_n25_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,25,1,-1.2223185723825796,1.1844726444174878,-0.018922963982545893,1.2033956084000337,2.4067912168000674,affine,0.34822494018929073,0.6964498803785815,1.0,0.34822494018929073,eta_l0_n25_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,26,0,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.09212749605465377,0.18425499210930754,0.9319962481713715,0.09884964261971338,eta_l0_n26_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,26,1,-1.076035332101526,1.0349918026899472,-0.02052176470578937,1.0555135673957365,2.111027134791473,affine,0.3071097487560824,0.6142194975121648,1.0,0.3071097487560824,eta_l0_n26_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,27,0,-1.076575027201125,1.0932061682436136,0.00831557052124432,1.0848905977223693,2.1697811954447386,affine,0.09767678010090204,0.19535356020180408,0.9319962481713715,0.10480383402030782,eta_l0_n27_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,27,1,-1.076575027201125,1.0932061682436136,0.00831557052124432,1.0848905977223693,2.1697811954447386,affine,0.31598948424092793,0.6319789684818559,1.0,0.31598948424092793,eta_l0_n27_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,28,0,-0.834957147551347,0.8610004970616335,0.013021674755143264,0.8479788223064902,1.6959576446129805,affine,0.055810473951034785,0.11162094790206957,0.9319962481713715,0.059882723842009065,eta_l0_n28_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,28,1,-0.834957147551347,0.8610004970616335,0.013021674755143264,0.8479788223064902,1.6959576446129805,affine,0.2379824436138722,0.4759648872277444,1.0,0.2379824436138722,eta_l0_n28_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,29,0,-1.1914992825472055,1.2302241779957934,0.019362447724293963,1.2108617302714995,2.421723460542999,affine,0.1226544191509072,0.2453088383018144,0.9319962481713715,0.13160398380525887,eta_l0_n29_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,29,1,-1.1914992825472055,1.2302241779957934,0.019362447724293963,1.2108617302714995,2.421723460542999,affine,0.35010101640132024,0.7002020328026405,1.0,0.35010101640132024,eta_l0_n29_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,30,0,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.03916738782673008,0.07833477565346016,0.9319962481713715,0.042025263410210796,eta_l0_n30_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,30,1,-0.7475100832928717,0.7212093167911565,-0.0131503832508576,0.7343597000420141,1.4687194000840282,affine,0.1956904995319357,0.3913809990638714,1.0,0.1956904995319357,eta_l0_n30_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,31,0,-1.0033698976233736,0.9680875531177244,-0.017641172252824577,0.985728725370549,1.971457450741098,affine,0.07920689094127416,0.15841378188254832,0.9319962481713715,0.08498627660431304,eta_l0_n31_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,31,1,-1.0033698976233736,0.9680875531177244,-0.017641172252824577,0.985728725370549,1.971457450741098,affine,0.2852835771335911,0.5705671542671822,1.0,0.2852835771335911,eta_l0_n31_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,32,0,-1.2946227610182184,1.251499462932461,-0.021561649042878717,1.2730611119753397,2.5461222239506793,affine,0.1353723538326648,0.2707447076653296,0.9319962481713715,0.1452498914006069,eta_l0_n32_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,32,1,-1.2946227610182184,1.251499462932461,-0.021561649042878717,1.2730611119753397,2.5461222239506793,affine,0.3650408857692888,0.7300817715385776,1.0,0.3650408857692888,eta_l0_n32_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,33,0,-0.8320043509056438,0.8630198990126956,0.01550777405352588,0.8475121249591697,1.6950242499183394,affine,0.05575080421097712,0.11150160842195424,0.9319962481713715,0.05981870025803569,eta_l0_n33_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,33,1,-0.8320043509056438,0.8630198990126956,0.01550777405352588,0.8475121249591697,1.6950242499183394,affine,0.23778197001319681,0.47556394002639363,1.0,0.23778197001319681,eta_l0_n33_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,34,0,-0.9062201538536023,0.9391835025457023,0.016481674346049968,0.9227018281996523,1.8454036563993046,affine,0.06812369722603763,0.13624739445207526,0.9319962481713715,0.07309438998247055,eta_l0_n34_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,34,1,-0.9062201538536023,0.9391835025457023,0.016481674346049968,0.9227018281996523,1.8454036563993046,affine,0.2642706782631371,0.5285413565262742,1.0,0.2642706782631371,eta_l0_n34_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,35,0,-0.9404150670737784,0.9548630677126783,0.007224000319449919,0.9476390673932283,1.8952781347864567,affine,0.07239481230028884,0.14478962460057768,0.9319962481713715,0.07767714992665635,eta_l0_n35_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,35,1,-0.9404150670737784,0.9548630677126783,0.007224000319449919,0.9476390673932283,1.8952781347864567,affine,0.2728251996328844,0.5456503992657687,1.0,0.2728251996328844,eta_l0_n35_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,36,0,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.1329675800582422,0.2659351601164844,0.9319962481713715,0.1426696516419803,eta_l0_n36_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,36,1,-1.2437303274076266,1.2791681800370145,0.01771892631469396,1.2614492537223205,2.522898507444641,affine,0.3623911650694393,0.7247823301388786,1.0,0.3623911650694393,eta_l0_n36_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,37,0,-1.1019913101592171,1.133740148975588,0.01587441940818546,1.1178657295674026,2.235731459134805,affine,0.10410160451057877,0.20820320902115755,0.9319962481713715,0.11169745019342289,eta_l0_n37_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,37,1,-1.1019913101592171,1.133740148975588,0.01587441940818546,1.1178657295674026,2.235731459134805,affine,0.3253861193108079,0.6507722386216158,1.0,0.3253861193108079,eta_l0_n37_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,38,0,-0.7748642347846237,0.7429797929158601,-0.0159422209343818,0.7589220138502419,1.5178440277004839,affine,0.04255388603436006,0.08510777206872013,0.9319962481713715,0.04565885980534058,eta_l0_n38_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,38,1,-0.7748642347846237,0.7429797929158601,-0.0159422209343818,0.7589220138502419,1.5178440277004839,affine,0.2049704217776224,0.4099408435552448,1.0,0.2049704217776224,eta_l0_n38_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,39,0,-1.0441939899867647,1.0106362591533828,-0.016778865416690936,1.0274151245700738,2.0548302491401476,affine,0.08683626610744645,0.1736725322148929,0.9319962481713715,0.09317233441424688,eta_l0_n39_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,39,1,-1.0441939899867647,1.0106362591533828,-0.016778865416690936,1.0274151245700738,2.0548302491401476,affine,0.2985464348510089,0.5970928697020178,1.0,0.2985464348510089,eta_l0_n39_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,40,0,-1.0016438696095074,0.9635624254813991,-0.019040722064054105,0.9826031475454533,1.9652062950909066,affine,0.07865281463649128,0.15730562927298256,0.9319962481713715,0.08439177173815074,eta_l0_n40_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,40,1,-1.0016438696095074,0.9635624254813991,-0.019040722064054105,0.9826031475454533,1.9652062950909066,affine,0.2842463860855686,0.5684927721711373,1.0,0.2842463860855686,eta_l0_n40_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,41,0,-0.811573872058711,0.8429313111060707,0.01567871952367983,0.8272525915823908,1.6545051831647817,affine,0.052595132871832324,0.10519026574366465,0.9319962481713715,0.056432773173740666,eta_l0_n41_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,41,1,-0.811573872058711,0.8429313111060707,0.01567871952367983,0.8272525915823908,1.6545051831647817,affine,0.23040909991814082,0.46081819983628164,1.0,0.23040909991814082,eta_l0_n41_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,42,0,-0.8258438072260422,0.7979791433617528,-0.013932331932144715,0.8119114752938975,1.623822950587795,affine,0.05024729084278002,0.10049458168556004,0.9319962481713715,0.05391361922472113,eta_l0_n42_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,42,1,-0.8258438072260422,0.7979791433617528,-0.013932331932144715,0.8119114752938975,1.623822950587795,affine,0.2247944505364546,0.4495889010729092,1.0,0.2247944505364546,eta_l0_n42_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,43,0,-1.5065542104519747,1.5398009931369268,0.016623391342476035,1.5231776017944507,3.0463552035889014,affine,0.18746340190277452,0.37492680380554905,0.9319962481713715,0.20114179887589478,eta_l0_n43_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,43,1,-1.5065542104519747,1.5398009931369268,0.016623391342476035,1.5231776017944507,3.0463552035889014,affine,0.4133050751389223,0.8266101502778446,1.0,0.4133050751389223,eta_l0_n43_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,44,0,-1.2988952400105458,1.2544908389724558,-0.02220220051904498,1.2766930394915008,2.5533860789830016,affine,0.13612331689707471,0.27224663379414943,0.9319962481713715,0.14605564900519313,eta_l0_n44_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,44,1,-1.2988952400105458,1.2544908389724558,-0.02220220051904498,1.2766930394915008,2.5533860789830016,affine,0.3658672600173671,0.7317345200347342,1.0,0.3658672600173671,eta_l0_n44_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,45,0,-0.6773605002971812,0.6675984405014699,-0.00488102989785566,0.6724794703993255,1.344958940798651,affine,0.031247814786823872,0.062495629573647744,0.9319962481713715,0.03352783323767003,eta_l0_n45_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,45,1,-0.6773605002971812,0.6675984405014699,-0.00488102989785566,0.6724794703993255,1.344958940798651,affine,0.17204478770874512,0.34408957541749025,1.0,0.17204478770874512,eta_l0_n45_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,46,0,-0.8663927459559604,0.901336093049574,0.017471673546806787,0.8838644195027672,1.7677288390055343,affine,0.06161881063103398,0.12323762126206796,0.9319962481713715,0.06611486983121823,eta_l0_n46_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,46,1,-0.8663927459559604,0.901336093049574,0.017471673546806787,0.8838644195027672,1.7677288390055343,affine,0.25074390701983446,0.5014878140396689,1.0,0.25074390701983446,eta_l0_n46_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,47,0,-0.8527698834317738,0.8310757190829281,-0.010847082174422873,0.841922801257351,1.683845602514702,affine,0.054847789658709296,0.10969557931741859,0.9319962481713715,0.05884979662345606,eta_l0_n47_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,47,1,-0.8527698834317738,0.8310757190829281,-0.010847082174422873,0.841922801257351,1.683845602514702,affine,0.23581240905165804,0.4716248181033161,1.0,0.23581240905165804,eta_l0_n47_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,48,0,-1.0115982611523096,1.0428605240202085,0.015631131433949452,1.027229392586259,2.054458785172518,affine,0.0867955966979921,0.1735911933959842,0.9319962481713715,0.09312869753316055,eta_l0_n48_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,48,1,-1.0115982611523096,1.0428605240202085,0.015631131433949452,1.027229392586259,2.054458785172518,affine,0.2985041035683372,0.5970082071366744,1.0,0.2985041035683372,eta_l0_n48_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,49,0,-0.8165600505003641,0.8478715991858072,0.015655774342721585,0.8322158248430856,1.6644316496861713,affine,0.053361052968857854,0.10672210593771571,0.9319962481713715,0.057254579161187834,eta_l0_n49_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,49,1,-0.8165600505003641,0.8478715991858072,0.015655774342721585,0.8322158248430856,1.6644316496861713,affine,0.23222316361302534,0.46444632722605067,1.0,0.23222316361302534,eta_l0_n49_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,50,0,-1.1685921066522977,1.2049616256427138,0.018184759495208036,1.1867768661475058,2.3735537322950115,affine,0.11778720959870133,0.23557441919740266,0.9319962481713715,0.1263816349366281,eta_l0_n50_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,50,1,-1.1685921066522977,1.2049616256427138,0.018184759495208036,1.1867768661475058,2.3735537322950115,affine,0.34397954463048364,0.6879590892609673,1.0,0.34397954463048364,eta_l0_n50_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,51,0,-0.8006207796149771,0.8344702881968005,0.01692475429091167,0.8175455339058888,1.6350910678117776,affine,0.05111893061710033,0.10223786123420066,0.9319962481713715,0.054848858798947436,eta_l0_n51_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,51,1,-0.8006207796149771,0.8344702881968005,0.01692475429091167,0.8175455339058888,1.6350910678117776,affine,0.22682905751238872,0.45365811502477743,1.0,0.22682905751238872,eta_l0_n51_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,52,0,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.04215920397663842,0.08431840795327684,0.9319962481713715,0.04523537949788653,eta_l0_n52_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,52,1,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,affine,0.20395651747432283,0.40791303494864567,1.0,0.20395651747432283,eta_l0_n52_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,53,0,-1.090911376884629,1.0503864671686705,-0.020262454857979284,1.0706489220266497,2.1412978440532995,affine,0.09500119573358663,0.19000239146717326,0.9319962481713715,0.10193302378629127,eta_l0_n53_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,53,1,-1.090911376884629,1.0503864671686705,-0.020262454857979284,1.0706489220266497,2.1412978440532995,affine,0.3116520262051116,0.6233040524102232,1.0,0.3116520262051116,eta_l0_n53_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,54,0,-1.2066771566490582,1.1738953830218986,-0.016390886813579808,1.1902862698354784,2.3805725396709567,affine,0.11848491224053499,0.23696982448106998,0.9319962481713715,0.12713024593501207,eta_l0_n54_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,54,1,-1.2066771566490582,1.1738953830218986,-0.016390886813579808,1.1902862698354784,2.3805725396709567,affine,0.34490796924504924,0.6898159384900985,1.0,0.34490796924504924,eta_l0_n54_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,55,0,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.03217402595959646,0.06434805191919292,0.9319962481713715,0.03452162605023753,eta_l0_n55_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,55,1,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,affine,0.17478486104675103,0.34956972209350207,1.0,0.17478486104675103,eta_l0_n55_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,56,0,-0.7083442383527362,0.7036147618286083,-0.002364738262063959,0.7059795000906722,1.4119590001813445,affine,0.03540540227173134,0.07081080454346268,0.9319962481713715,0.03798878197331664,eta_l0_n56_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,56,1,-0.7083442383527362,0.7036147618286083,-0.002364738262063959,0.7059795000906722,1.4119590001813445,quadratic,0.01435081539743612,0.02870163079487224,1.0,0.01435081539743612,eta_l0_n56_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,57,0,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.1231062852532452,0.2462125705064904,0.9319962481713715,0.1320888206307553,eta_l0_n57_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,57,1,-1.2329296305415842,1.1932285191339407,-0.01985055570382177,1.2130790748377624,2.426158149675525,affine,0.3506497194153905,0.701299438830781,1.0,0.3506497194153905,eta_l0_n57_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,58,0,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.037684701643931236,0.07536940328786247,0.9319962481713715,0.04043439200304799,eta_l0_n58_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,58,1,-0.7334941393710497,0.7131942540621978,-0.010149942654425925,0.7233441967166238,1.4466883934332475,affine,0.19152118340627797,0.38304236681255593,1.0,0.19152118340627797,eta_l0_n58_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,59,0,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.07040104306089869,0.14080208612179737,0.9319962481713715,0.07553790393365795,eta_l0_n59_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,59,1,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,affine,0.26891234934821284,0.5378246986964257,1.0,0.26891234934821284,eta_l0_n59_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,60,0,-1.5860529702984412,1.5733837593858007,-0.0063346054563202525,1.579718364842121,3.159436729684242,affine,0.1992008157072346,0.3984016314144692,0.9319962481713715,0.21373564120894015,eta_l0_n60_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,60,1,-1.5860529702984412,1.5733837593858007,-0.0063346054563202525,1.579718364842121,3.159436729684242,quadratic,0.12300515623212518,0.24601031246425037,1.0,0.12300515623212518,eta_l0_n60_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,61,0,-1.1365766537137105,1.1040485548774308,-0.01626404941813986,1.1203126042955707,2.2406252085911413,affine,0.10458222490461823,0.20916444980923646,0.9319962481713715,0.11221313938743249,eta_l0_n61_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,61,1,-1.1365766537137105,1.1040485548774308,-0.01626404941813986,1.1203126042955707,2.2406252085911413,affine,0.3260695819670721,0.6521391639341442,1.0,0.3260695819670721,eta_l0_n61_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,62,0,-1.0643707508376146,1.0985694149464216,0.017099332054403504,1.081470082892018,2.162940165784036,affine,0.09705393191182805,0.1941078638236561,0.9319962481713715,0.1041355392816798,eta_l0_n62_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,62,1,-1.0643707508376146,1.0985694149464216,0.017099332054403504,1.081470082892018,2.162940165784036,affine,0.3148977187880975,0.629795437576195,1.0,0.3148977187880975,eta_l0_n62_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,63,0,-1.4015171493211271,1.3643005656294993,-0.018608291845813918,1.3829088574753132,2.7658177149506264,affine,0.15815871965502917,0.31631743931005835,0.9319962481713715,0.16969888018899795,eta_l0_n63_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,63,1,-1.4015171493211271,1.3643005656294993,-0.018608291845813918,1.3829088574753132,2.7658177149506264,affine,0.3885130831309808,0.7770261662619616,1.0,0.3885130831309808,eta_l0_n63_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,64,0,-0.8850519189924944,0.8509091370870554,-0.01707139095271948,0.8679805280397749,1.7359610560795498,affine,0.05902823505223593,0.11805647010447186,0.9319962481713715,0.0633352711108576,eta_l0_n64_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,64,1,-0.8850519189924944,0.8509091370870554,-0.01707139095271948,0.8679805280397749,1.7359610560795498,affine,0.2451125389514658,0.4902250779029316,1.0,0.2451125389514658,eta_l0_n64_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,65,0,-0.8429087446687176,0.865827317741177,0.01145928653622974,0.8543680312049473,1.7087360624098946,affine,0.05681525626691875,0.1136305125338375,0.9319962481713715,0.060960820795570206,eta_l0_n65_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,65,1,-0.8429087446687176,0.865827317741177,0.01145928653622974,0.8543680312049473,1.7087360624098946,affine,0.24030458215024156,0.4806091643004831,1.0,0.24030458215024156,eta_l0_n65_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,66,0,-0.9905468239121494,0.9617857365141983,-0.014380543698975568,0.9761662802131739,1.9523325604263477,affine,0.0774704270215625,0.154940854043125,0.9319962481713715,0.08312311039187527,eta_l0_n66_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,66,1,-0.9905468239121494,0.9617857365141983,-0.014380543698975568,0.9761662802131739,1.9523325604263477,affine,0.2822132019188917,0.5644264038377834,1.0,0.2822132019188917,eta_l0_n66_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,67,0,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.06370430392497341,0.12740860784994681,0.9319962481713715,0.06835253258794206,eta_l0_n67_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,67,1,-0.8810449969340859,0.912044788908644,0.015499895987279078,0.896544892921365,1.79308978584273,affine,0.2552280030625681,0.5104560061251362,1.0,0.2552280030625681,eta_l0_n67_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,68,0,-1.0577074974769156,1.0951935167927764,0.01874300965793041,1.076450507134846,2.152901014269692,affine,0.0961007655593182,0.1922015311186364,0.9319962481713715,0.10311282448601401,eta_l0_n68_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,68,1,-1.0577074974769156,1.0951935167927764,0.01874300965793041,1.076450507134846,2.152901014269692,affine,0.313395395818585,0.62679079163717,1.0,0.313395395818585,eta_l0_n68_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,69,0,-1.379178420726976,1.4222829218948303,0.02155225058392718,1.4007306713109031,2.8014613426218062,affine,0.16189286350766371,0.32378572701532743,0.9319962481713715,0.17370548843442934,eta_l0_n69_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,69,1,-1.379178420726976,1.4222829218948303,0.02155225058392718,1.4007306713109031,2.8014613426218062,affine,0.39193665167048064,0.7838733033409613,1.0,0.39193665167048064,eta_l0_n69_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,70,0,-0.9308513790946809,0.9636725950147775,0.01641060796004834,0.9472619870547292,1.8945239741094584,affine,0.07236808784580497,0.14473617569160993,0.9319962481713715,0.0776484755038394,eta_l0_n70_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,70,1,-0.9308513790946809,0.9636725950147775,0.01641060796004834,0.9472619870547292,1.8945239741094584,affine,0.2726039266590258,0.5452078533180515,1.0,0.2726039266590258,eta_l0_n70_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,71,0,-1.4388858973944985,1.394990236741566,-0.02194783032646619,1.4169380670680323,2.8338761341360645,affine,0.16528217318108282,0.33056434636216564,0.9319962481713715,0.17734210143589701,eta_l0_n71_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,71,1,-1.4388858973944985,1.394990236741566,-0.02194783032646619,1.4169380670680323,2.8338761341360645,affine,0.3949950653074476,0.7899901306148952,1.0,0.3949950653074476,eta_l0_n71_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,72,0,-1.1711538278994706,1.2088264922154854,0.018836332158007396,1.189990160057478,2.379980320114956,affine,0.11843682940143908,0.23687365880287817,0.9319962481713715,0.12707865469825522,eta_l0_n72_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,72,1,-1.1711538278994706,1.2088264922154854,0.018836332158007396,1.189990160057478,2.379980320114956,affine,0.3448007599664115,0.689601519932823,1.0,0.3448007599664115,eta_l0_n72_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,73,0,-1.174538707187349,1.210071346678575,0.017766319745613046,1.192305026932962,2.384610053865924,affine,0.11889769438589372,0.23779538877178744,0.9319962481713715,0.12757314701553532,eta_l0_n73_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,73,1,-1.174538707187349,1.210071346678575,0.017766319745613046,1.192305026932962,2.384610053865924,affine,0.3454105213869703,0.6908210427739406,1.0,0.3454105213869703,eta_l0_n73_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,74,0,-0.9300731152852422,0.8929332780733101,-0.018569918605966018,0.9115031966792762,1.8230063933585523,affine,0.06623391994248702,0.13246783988497404,0.9319962481713715,0.0710667237904033,eta_l0_n74_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,74,1,-0.9300731152852422,0.8929332780733101,-0.018569918605966018,0.9115031966792762,1.8230063933585523,affine,0.26038337545599266,0.5207667509119853,1.0,0.26038337545599266,eta_l0_n74_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,75,0,-0.8829617050674683,0.9081151180277416,0.012576706480136646,0.8955384115476049,1.7910768230952099,affine,0.06352122059880379,0.12704244119760758,0.9319962481713715,0.0681560904600592,eta_l0_n75_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,75,1,-0.8829617050674683,0.9081151180277416,0.012576706480136646,0.8955384115476049,1.7910768230952099,affine,0.25491234963215764,0.5098246992643153,1.0,0.25491234963215764,eta_l0_n75_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,76,0,-1.1805875736327947,1.1488019022058173,-0.01589283571348865,1.164694737919306,2.329389475838612,affine,0.1133540174298754,0.2267080348597508,0.9319962481713715,0.1216249718303934,eta_l0_n76_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,76,1,-1.1805875736327947,1.1488019022058173,-0.01589283571348865,1.164694737919306,2.329389475838612,affine,0.3382125951197492,0.6764251902394984,1.0,0.3382125951197492,eta_l0_n76_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,77,0,-1.3659675145450214,1.3314526620255036,-0.0172574262597589,1.3487100882852625,2.697420176570525,affine,0.1510226319798477,0.3020452639596954,0.9319962481713715,0.16204210293352844,eta_l0_n77_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,77,1,-1.3659675145450214,1.3314526620255036,-0.0172574262597589,1.3487100882852625,2.697420176570525,affine,0.3816306101926488,0.7632612203852976,1.0,0.3816306101926488,eta_l0_n77_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,78,0,-0.6125577319415737,0.6569759697827166,0.022209118920571425,0.6347668508621451,1.2695337017242903,affine,0.026981033085969944,0.05396206617193989,0.9319962481713715,0.028949722854473108,eta_l0_n78_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,78,1,-0.6125577319415737,0.6569759697827166,0.022209118920571425,0.6347668508621451,1.2695337017242903,affine,0.15734440236877792,0.31468880473755584,1.0,0.15734440236877792,eta_l0_n78_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,79,0,-0.8548400889214127,0.8380958072828195,-0.008372140819296603,0.8464679481021161,1.6929358962042322,affine,0.05555260530301887,0.11110521060603774,0.9319962481713715,0.05960603962946865,eta_l0_n79_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,79,1,-0.8548400889214127,0.8380958072828195,-0.008372140819296603,0.8464679481021161,1.6929358962042322,affine,0.237480546051433,0.474961092102866,1.0,0.237480546051433,eta_l0_n79_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,80,0,-1.2719651427464118,1.2295099902121156,-0.021227576267148107,1.2507375664792637,2.5014751329585274,affine,0.13078781192491887,0.26157562384983774,0.9319962481713715,0.14033083521691406,eta_l0_n80_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,80,1,-1.2719651427464118,1.2295099902121156,-0.021227576267148107,1.2507375664792637,2.5014751329585274,affine,0.3598172673204267,0.7196345346408534,1.0,0.3598172673204267,eta_l0_n80_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,81,0,-0.6323568266861044,0.5723896197843605,-0.02998360345087192,0.6023732232352325,1.204746446470465,affine,0.023591200034515373,0.047182400069030746,0.9319962481713715,0.025312548286329072,eta_l0_n81_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,81,1,-0.6323568266861044,0.5723896197843605,-0.02998360345087192,0.6023732232352325,1.204746446470465,affine,0.1447703056170632,0.2895406112341264,1.0,0.1447703056170632,eta_l0_n81_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,82,0,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.046288370646411676,0.09257674129282335,0.9319962481713715,0.049665833673935955,eta_l0_n82_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,82,1,-0.7707737858392829,0.7995670803645063,0.014396647262611695,0.7851704331018946,1.5703408662037892,affine,0.21485561047209326,0.4297112209441865,1.0,0.21485561047209326,eta_l0_n82_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,83,0,-0.8932013736751797,0.8717751463069091,-0.0107131136841353,0.8824882599910444,1.7649765199820888,affine,0.06135632904886263,0.12271265809772526,0.9319962481713715,0.0658332360985864,eta_l0_n83_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,83,1,-0.8932013736751797,0.8717751463069091,-0.0107131136841353,0.8824882599910444,1.7649765199820888,affine,0.25034280413540316,0.5006856082708063,1.0,0.25034280413540316,eta_l0_n83_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,84,0,-0.7976625548491153,0.8270409447667989,0.0146891949588418,0.8123517498079571,1.6247034996159142,affine,0.05031809937737942,0.10063619875475883,0.9319962481713715,0.05398959435309566,eta_l0_n84_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,84,1,-0.7976625548491153,0.8270409447667989,0.0146891949588418,0.8123517498079571,1.6247034996159142,affine,0.2249472072371165,0.449894414474233,1.0,0.2249472072371165,eta_l0_n84_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,85,0,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.06751399250682856,0.13502798501365712,0.9319962481713715,0.07244019773609044,eta_l0_n85_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,85,1,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,affine,0.2630640987443081,0.5261281974886162,1.0,0.2630640987443081,eta_l0_n85_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,86,0,-1.1100026096220488,1.1489145687805895,0.019455979579270366,1.1294585892013191,2.2589171784026383,affine,0.1063931461831313,0.2127862923662626,0.9319962481713715,0.11415619579143217,eta_l0_n86_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,86,1,-1.1100026096220488,1.1489145687805895,0.019455979579270366,1.1294585892013191,2.2589171784026383,affine,0.3285811233329349,0.6571622466658698,1.0,0.3285811233329349,eta_l0_n86_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,87,0,-1.2789544097029164,1.23404014812509,-0.022457130788913204,1.2564972789140032,2.5129945578280064,affine,0.13197455335417327,0.26394910670834654,0.9319962481713715,0.14160416805659323,eta_l0_n87_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,87,1,-1.2789544097029164,1.23404014812509,-0.022457130788913204,1.2564972789140032,2.5129945578280064,affine,0.3611638067539205,0.722327613507841,1.0,0.3611638067539205,eta_l0_n87_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,88,0,-0.8196835402297237,0.8511525501491991,0.015734504959737716,0.8354180451894614,1.6708360903789228,affine,0.053858323789688196,0.10771664757937639,0.9319962481713715,0.05778813369191263,eta_l0_n88_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,88,1,-0.8196835402297237,0.8511525501491991,0.015734504959737716,0.8354180451894614,1.6708360903789228,affine,0.23338951812693723,0.46677903625387446,1.0,0.23338951812693723,eta_l0_n88_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,89,0,-0.8579518670180815,0.8906473389988188,0.01634773599036865,0.8742996030084501,1.7485992060169002,affine,0.06004773422309601,0.12009546844619202,0.9319962481713715,0.06442915874491234,eta_l0_n89_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,89,1,-0.8579518670180815,0.8906473389988188,0.01634773599036865,0.8742996030084501,1.7485992060169002,affine,0.24737350884828138,0.49474701769656276,1.0,0.24737350884828138,eta_l0_n89_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,90,0,-0.8207744817317321,0.7924840586718497,-0.014145211529941193,0.8066292702017909,1.6132585404035817,affine,0.04945446424308742,0.09890892848617484,0.9319962481713715,0.05306294348300203,eta_l0_n90_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,90,1,-0.8207744817317321,0.7924840586718497,-0.014145211529941193,0.8066292702017909,1.6132585404035817,affine,0.22283997146248544,0.4456799429249709,1.0,0.22283997146248544,eta_l0_n90_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,91,0,-1.289402589613948,1.2579584264319819,-0.015722081590983095,1.273680508022965,2.54736101604593,affine,0.13547413367370534,0.2709482673474107,0.9319962481713715,0.14535909767825045,eta_l0_n91_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,91,1,-1.289402589613948,1.2579584264319819,-0.015722081590983095,1.273680508022965,2.54736101604593,affine,0.36525440639843126,0.7305088127968625,1.0,0.36525440639843126,eta_l0_n91_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,92,0,-1.124946905632745,1.08585988447044,-0.019543510581152557,1.1054033950515925,2.210806790103185,affine,0.10169126819839032,0.20338253639678064,0.9319962481713715,0.10911124202260926,eta_l0_n92_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,92,1,-1.124946905632745,1.08585988447044,-0.019543510581152557,1.1054033950515925,2.210806790103185,affine,0.3217988766221556,0.6435977532443112,1.0,0.3217988766221556,eta_l0_n92_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,93,0,-0.646894182625127,0.6720388239177192,0.012572320646296098,0.6594665032714231,1.3189330065428462,affine,0.02973109792160489,0.05946219584320978,0.9319962481713715,0.03190044807577172,eta_l0_n93_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,93,1,-0.646894182625127,0.6720388239177192,0.012572320646296098,0.6594665032714231,1.3189330065428462,affine,0.16698133324584755,0.3339626664916951,1.0,0.16698133324584755,eta_l0_n93_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,94,0,-1.1891179119541306,1.2318600758063354,0.021371081926102375,1.210488993880233,2.420977987760466,affine,0.12258952797033353,0.24517905594066705,0.9319962481713715,0.13153435779474543,eta_l0_n94_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,94,1,-1.1891179119541306,1.2318600758063354,0.021371081926102375,1.210488993880233,2.420977987760466,affine,0.34997880388507535,0.6999576077701507,1.0,0.34997880388507535,eta_l0_n94_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,95,0,-0.842527144443055,0.8784439097769786,0.017958382666961814,0.8604855271100168,1.7209710542200336,affine,0.05782891292611981,0.11565782585223962,0.9319962481713715,0.0620484396150557,eta_l0_n95_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,95,1,-0.842527144443055,0.8784439097769786,0.017958382666961814,0.8604855271100168,1.7209710542200336,affine,0.2424173386872114,0.4848346773744228,1.0,0.2424173386872114,eta_l0_n95_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,96,0,-0.6659994683686797,0.6358673961792188,-0.01506603609473045,0.6509334322739493,1.3018668645478986,affine,0.0287543739277934,0.0575087478555868,0.9319962481713715,0.030852456739188686,eta_l0_n96_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,96,1,-0.6659994683686797,0.6358673961792188,-0.01506603609473045,0.6509334322739493,1.3018668645478986,affine,0.1636693252431171,0.3273386504862342,1.0,0.1636693252431171,eta_l0_n96_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,97,0,-1.2151238777579998,1.1786727174263016,-0.018225580165849076,1.1968982975921507,2.3937965951843014,affine,0.11982580429260405,0.2396516085852081,0.9319962481713715,0.12856897710447757,eta_l0_n97_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,97,1,-1.2151238777579998,1.1786727174263016,-0.018225580165849076,1.1968982975921507,2.3937965951843014,affine,0.34658140653981434,0.6931628130796287,1.0,0.34658140653981434,eta_l0_n97_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,98,0,-1.273373259798392,1.3090285069017624,0.017827623551685212,1.2912008833500772,2.5824017667001544,affine,0.13909500430324453,0.27819000860648907,0.9319962481713715,0.14924416764140055,eta_l0_n98_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,98,1,-1.273373259798392,1.3090285069017624,0.017827623551685212,1.2912008833500772,2.5824017667001544,affine,0.3692182011977751,0.7384364023955502,1.0,0.3692182011977751,eta_l0_n98_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,99,0,-0.913575753766706,0.8771689762111494,-0.018203388777778273,0.8953723649889277,1.7907447299778554,affine,0.06352641759977425,0.1270528351995485,0.9319962481713715,0.06816166666380537,eta_l0_n99_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,99,1,-0.913575753766706,0.8771689762111494,-0.018203388777778273,0.8953723649889277,1.7907447299778554,affine,0.2547773127854502,0.5095546255709004,1.0,0.2547773127854502,eta_l0_n99_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,100,0,-1.4400569326913257,1.4673384738386845,0.013640770573679406,1.4536977032650051,2.9073954065300103,affine,0.17294317701844078,0.34588635403688156,0.9319962481713715,0.18556209572491833,eta_l0_n100_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,100,1,-1.4400569326913257,1.4673384738386845,0.013640770573679406,1.4536977032650051,2.9073954065300103,affine,0.4017389057098899,0.8034778114197798,1.0,0.4017389057098899,eta_l0_n100_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,101,0,-1.5837879660157446,1.5801226097140428,-0.0018326781508508638,1.5819552878648937,3.1639105757297874,affine,0.1996618192557774,0.3993236385115548,0.9319962481713715,0.2142302822007331,eta_l0_n101_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,101,1,-1.5837879660157446,1.5801226097140428,-0.0018326781508508638,1.5819552878648937,3.1639105757297874,quadratic,0.12229427866343806,0.24458855732687612,1.0,0.12229427866343806,eta_l0_n101_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,102,0,-1.058315031825715,1.0262563408124221,-0.016029345506646475,1.0422856863190686,2.084571372638137,affine,0.08960906604109521,0.17921813208219042,0.9319962481713715,0.09614745361573417,eta_l0_n102_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,102,1,-1.058315031825715,1.0262563408124221,-0.016029345506646475,1.0422856863190686,2.084571372638137,affine,0.303150340466033,0.606300680932066,1.0,0.303150340466033,eta_l0_n102_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,103,0,-1.1476452573399847,1.1090705510943306,-0.019287353122827033,1.1283579042171576,2.2567158084343153,affine,0.10617588471459573,0.21235176942919146,0.9319962481713715,0.11392308168935092,eta_l0_n103_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,103,1,-1.1476452573399847,1.1090705510943306,-0.019287353122827033,1.1283579042171576,2.2567158084343153,affine,0.3282775513507246,0.6565551027014492,1.0,0.3282775513507246,eta_l0_n103_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,104,0,-0.8577016693145988,0.8877385522479758,0.015018441466688515,0.8727201107812873,1.7454402215625746,affine,0.05978300541739091,0.11956601083478181,0.9319962481713715,0.06414511381852502,eta_l0_n104_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,104,1,-0.8577016693145988,0.8877385522479758,0.015018441466688515,0.8727201107812873,1.7454402215625746,affine,0.24683057948606554,0.4936611589721311,1.0,0.24683057948606554,eta_l0_n104_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,105,0,-1.1779127957269424,1.2185421121727302,0.02031465822289391,1.1982274539498363,2.3964549078996726,affine,0.12010463419158925,0.2402092683831785,0.9319962481713715,0.12886815202018378,eta_l0_n105_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,105,1,-1.1779127957269424,1.2185421121727302,0.02031465822289391,1.1982274539498363,2.3964549078996726,affine,0.34689200531234765,0.6937840106246953,1.0,0.34689200531234765,eta_l0_n105_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,106,0,-0.7436028604158079,0.7107002167910171,-0.016451321812395392,0.7271515386034125,1.454303077206825,affine,0.03822436614129144,0.07644873228258288,0.9319962481713715,0.04101343349427616,eta_l0_n106_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,106,1,-0.7436028604158079,0.7107002167910171,-0.016451321812395392,0.7271515386034125,1.454303077206825,affine,0.19290116812859956,0.3858023362571991,1.0,0.19290116812859956,eta_l0_n106_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,107,0,-0.636446391775374,0.5633476109147433,-0.036549390430315354,0.5998970013450586,1.1997940026901173,affine,0.023429717443423096,0.04685943488684619,0.9319962481713715,0.02513928300612101,eta_l0_n107_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,107,1,-0.636446391775374,0.5633476109147433,-0.036549390430315354,0.5998970013450586,1.1997940026901173,affine,0.14365662849300706,0.2873132569860141,1.0,0.14365662849300706,eta_l0_n107_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,108,0,-1.0426471018400014,1.0803325368651482,0.01884271751257338,1.0614898193525748,2.1229796387051496,affine,0.09324930213724093,0.18649860427448187,0.9319962481713715,0.10005330206017594,eta_l0_n108_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,108,1,-1.0426471018400014,1.0803325368651482,0.01884271751257338,1.0614898193525748,2.1229796387051496,affine,0.30893735262813343,0.6178747052562669,1.0,0.30893735262813343,eta_l0_n108_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,109,0,-0.9134574422361407,0.9473867494974705,0.016964653630664905,0.9304220958668056,1.8608441917336112,affine,0.06945072467805004,0.13890144935610008,0.9319962481713715,0.07451824491173245,eta_l0_n109_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,109,1,-0.9134574422361407,0.9473867494974705,0.016964653630664905,0.9304220958668056,1.8608441917336112,affine,0.2669010613614465,0.533802122722893,1.0,0.2669010613614465,eta_l0_n109_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,110,0,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.1296818009212755,0.259363601842551,0.9319962481713715,0.13914412335427143,eta_l0_n110_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,110,1,-1.2677004222053614,1.222903813557143,-0.022398304324109164,1.2453021178812522,2.4906042357625044,affine,0.35850263978940383,0.7170052795788077,1.0,0.35850263978940383,eta_l0_n110_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,111,0,-0.9581578905614648,0.9243831398125947,-0.016887375374435076,0.9412705151870298,1.8825410303740595,affine,0.07132673835576632,0.14265347671153264,0.9319962481713715,0.07653114322693182,eta_l0_n111_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,111,1,-0.9581578905614648,0.9243831398125947,-0.016887375374435076,0.9412705151870298,1.8825410303740595,affine,0.2705801126588765,0.541160225317753,1.0,0.2705801126588765,eta_l0_n111_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,112,0,-0.7914234807966221,0.8188916681191158,0.013734093661246893,0.805157574457869,1.610315148915738,affine,0.04923190847633619,0.09846381695267238,0.9319962481713715,0.05282414878056852,eta_l0_n112_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,112,1,-0.7914234807966221,0.8188916681191158,0.013734093661246893,0.805157574457869,1.610315148915738,affine,0.2223003589928226,0.4446007179856452,1.0,0.2223003589928226,eta_l0_n112_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,113,0,-0.8496861508209819,0.8720170059064729,0.01116542754274552,0.8608515783637274,1.7217031567274548,affine,0.05784896434081916,0.11569792868163832,0.9319962481713715,0.062069954095117924,eta_l0_n113_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,113,1,-0.8496861508209819,0.8720170059064729,0.01116542754274552,0.8608515783637274,1.7217031567274548,affine,0.24263703680786106,0.4852740736157221,1.0,0.24263703680786106,eta_l0_n113_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,114,0,-0.6630886315286886,0.6437069553096089,-0.009690838109539857,0.6533977934191487,1.3067955868382974,affine,0.029010392935394484,0.05802078587078897,0.9319962481713715,0.031127156351020178,eta_l0_n114_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,114,1,-0.6630886315286886,0.6437069553096089,-0.009690838109539857,0.6533977934191487,1.3067955868382974,affine,0.16467230951589704,0.3293446190317941,1.0,0.16467230951589704,eta_l0_n114_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,115,0,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.039720764451005815,0.07944152890201163,0.9319962481713715,0.04261901754319308,eta_l0_n115_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,115,1,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,affine,0.1973252022617025,0.394650404523405,1.0,0.1973252022617025,eta_l0_n115_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,116,0,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.0629088693592051,0.1258177387184102,0.9319962481713715,0.06749905858809604,eta_l0_n116_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,116,1,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,affine,0.25350285914249493,0.5070057182849899,1.0,0.25350285914249493,eta_l0_n116_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,117,0,-1.374050756910884,1.4125160594440571,0.019232651266586576,1.3932834081774705,2.786566816354941,affine,0.1603277382012225,0.320655476402445,0.9319962481713715,0.17202616267586318,eta_l0_n117_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,117,1,-1.374050756910884,1.4125160594440571,0.019232651266586576,1.3932834081774705,2.786566816354941,affine,0.3905296760644244,0.7810593521288488,1.0,0.3905296760644244,eta_l0_n117_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,118,0,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.03443316994739421,0.06886633989478842,0.9319962481713715,0.03694561004398248,eta_l0_n118_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,118,1,-0.7114293139660042,0.684661120813965,-0.013384096576019577,0.6980452173899846,1.3960904347799692,affine,0.18180360596996578,0.36360721193993156,1.0,0.18180360596996578,eta_l0_n118_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,119,0,-1.168735262508867,1.1298298900732502,-0.01945268621780838,1.1492825762910586,2.298565152582117,affine,0.11030672337739635,0.2206134467547927,0.9319962481713715,0.11835532985655713,eta_l0_n119_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,119,1,-1.168735262508867,1.1298298900732502,-0.01945268621780838,1.1492825762910586,2.298565152582117,affine,0.3340246272819402,0.6680492545638804,1.0,0.3340246272819402,eta_l0_n119_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,120,0,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.041338081872310646,0.08267616374462129,0.9319962481713715,0.044354343650436646,eta_l0_n120_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,120,1,-0.7639812920130188,0.7365572828456631,-0.013712004583677828,0.750269287429341,1.500538574858682,affine,0.20172634433294964,0.4034526886658993,1.0,0.20172634433294964,eta_l0_n120_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,121,0,-0.6662127062570723,0.690696927749146,0.012242110746036872,0.6784548170031092,1.3569096340062183,affine,0.03199782957754133,0.06399565915508267,0.9319962481713715,0.034332573377116975,eta_l0_n121_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,121,1,-0.6662127062570723,0.690696927749146,0.012242110746036872,0.6784548170031092,1.3569096340062183,affine,0.17429024028187626,0.3485804805637525,1.0,0.17429024028187626,eta_l0_n121_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,122,0,-0.896479261260048,0.9290067815766617,0.016263760158306884,0.9127430214183548,1.8254860428367097,affine,0.06642863621157087,0.13285727242314174,0.9319962481713715,0.0712756476669381,eta_l0_n122_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,122,1,-0.896479261260048,0.9290067815766617,0.016263760158306884,0.9127430214183548,1.8254860428367097,affine,0.26084735534552267,0.5216947106910453,1.0,0.26084735534552267,eta_l0_n122_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,123,0,-1.104167563368715,1.1420328248526197,0.018932630741952394,1.1231001941106673,2.2462003882213346,affine,0.10514210954629312,0.21028421909258624,0.9319962481713715,0.11281387639981147,eta_l0_n123_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,123,1,-1.104167563368715,1.1420328248526197,0.018932630741952394,1.1231001941106673,2.2462003882213346,affine,0.3268151844988936,0.6536303689977871,1.0,0.3268151844988936,eta_l0_n123_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,124,0,-0.7854236701342574,0.7505279392833626,-0.017447865425447406,0.76797580470881,1.53595160941762,affine,0.04383925134039441,0.08767850268078882,0.9319962481713715,0.04703801268128435,eta_l0_n124_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,124,1,-0.7854236701342574,0.7505279392833626,-0.017447865425447406,0.76797580470881,1.53595160941762,affine,0.20836219466955877,0.41672438933911754,1.0,0.20836219466955877,eta_l0_n124_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,125,0,-1.3977310453244938,1.3621603770086215,-0.01778533415793615,1.3799457111665576,2.7598914223331152,affine,0.1575371349875396,0.3150742699750792,0.9319962481713715,0.16903194116568196,eta_l0_n125_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,125,1,-1.3977310453244938,1.3621603770086215,-0.01778533415793615,1.3799457111665576,2.7598914223331152,affine,0.387937826162168,0.775875652324336,1.0,0.387937826162168,eta_l0_n125_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,126,0,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.11361256119173553,0.22722512238347106,0.9319962481713715,0.12190238041692732,eta_l0_n126_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,126,1,-1.1455060830246946,1.1862496927259263,0.02037180485061585,1.1658778878753104,2.331755775750621,affine,0.338467378434341,0.676934756868682,1.0,0.338467378434341,eta_l0_n126_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,127,0,-1.1653861727789863,1.1295881895992126,-0.017898991589886837,1.1474871811890994,2.294974362378199,affine,0.10994269110721135,0.2198853822144227,0.9319962481713715,0.11796473571961799,eta_l0_n127_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,127,1,-1.1653861727789863,1.1295881895992126,-0.017898991589886837,1.1474871811890994,2.294974362378199,affine,0.33355880708885166,0.6671176141777033,1.0,0.33355880708885166,eta_l0_n127_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,128,0,-0.918369695032508,0.9538988826345167,0.01776459380100437,0.9361342888335124,1.8722685776670247,affine,0.07044155189568786,0.14088310379137572,0.9319962481713715,0.07558136852363742,eta_l0_n128_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,128,1,-0.918369695032508,0.9538988826345167,0.01776459380100437,0.9361342888335124,1.8722685776670247,affine,0.26882962869842364,0.5376592573968473,1.0,0.26882962869842364,eta_l0_n128_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,129,0,-0.8382147723907124,0.8049901321285019,-0.01661232013110525,0.8216024522596072,1.6432049045192143,affine,0.0517349305265229,0.1034698610530458,0.9319962481713715,0.05550980556845558,eta_l0_n129_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,129,1,-0.8382147723907124,0.8049901321285019,-0.01661232013110525,0.8216024522596072,1.6432049045192143,affine,0.2283246773501362,0.4566493547002724,1.0,0.2283246773501362,eta_l0_n129_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,130,0,-1.0156344504627692,1.0500586106082936,0.017212080072762204,1.0328465305355314,2.065693061071063,affine,0.08784979328981053,0.17569958657962106,0.9319962481713715,0.0942598143095283,eta_l0_n130_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,130,1,-1.0156344504627692,1.0500586106082936,0.017212080072762204,1.0328465305355314,2.065693061071063,affine,0.3002263934712634,0.6004527869425268,1.0,0.3002263934712634,eta_l0_n130_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,131,0,-1.0474500720898494,1.017922602848839,-0.014763734620505176,1.0326863374693442,2.0653726749386885,affine,0.08780704324716622,0.17561408649433244,0.9319962481713715,0.09421394498041008,eta_l0_n131_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,131,1,-1.0474500720898494,1.017922602848839,-0.014763734620505176,1.0326863374693442,2.0653726749386885,affine,0.3002092776642887,0.6004185553285774,1.0,0.3002092776642887,eta_l0_n131_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,132,0,-1.0609350410804446,1.0970640029002232,0.018064480909889324,1.0789995219903339,2.1579990439806678,affine,0.09658521996959714,0.19317043993919428,0.9319962481713715,0.10363262744791378,eta_l0_n132_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,132,1,-1.0609350410804446,1.0970640029002232,0.018064480909889324,1.0789995219903339,2.1579990439806678,affine,0.3141573803296545,0.628314760659309,1.0,0.3141573803296545,eta_l0_n132_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,133,0,-1.2534023950114845,1.216564803864441,-0.018418795573521773,1.2349835994379628,2.4699671988759255,affine,0.12755412399079488,0.25510824798158976,0.9319962481713715,0.13686119900274618,eta_l0_n133_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,133,1,-1.2534023950114845,1.216564803864441,-0.018418795573521773,1.2349835994379628,2.4699671988759255,affine,0.3560682692072111,0.7121365384144221,1.0,0.3560682692072111,eta_l0_n133_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,134,0,-1.011433053078495,0.9766652867437465,-0.017383883167374214,0.9940491699111207,1.9880983398222414,affine,0.08071068741298588,0.16142137482597177,0.9319962481713715,0.08659979862724206,eta_l0_n134_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,134,1,-1.011433053078495,0.9766652867437465,-0.017383883167374214,0.9940491699111207,1.9880983398222414,affine,0.28797534903698724,0.5759506980739745,1.0,0.28797534903698724,eta_l0_n134_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,135,0,-1.2832724515737506,1.3273446149675858,0.02203608169691762,1.3053085332706682,2.6106170665413364,affine,0.142031026455352,0.284062052910704,0.9319962481713715,0.15239441868357817,eta_l0_n135_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,135,1,-1.2832724515737506,1.3273446149675858,0.02203608169691762,1.3053085332706682,2.6106170665413364,affine,0.3723051142259512,0.7446102284519024,1.0,0.3723051142259512,eta_l0_n135_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,136,0,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.055506724224711264,0.11101344844942253,0.9319962481713715,0.059556810806501145,eta_l0_n136_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,136,1,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,affine,0.23735487630607996,0.4747097526121599,1.0,0.23735487630607996,eta_l0_n136_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,137,0,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.1106683443332991,0.2213366886665982,0.9319962481713715,0.11874333673599712,eta_l0_n137_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,137,1,-1.1299748853383094,1.1721429880236618,0.021084051342676213,1.1510589366809856,2.302117873361971,affine,0.3344812720220745,0.668962544044149,1.0,0.3344812720220745,eta_l0_n137_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,138,0,-1.4795147536405526,1.4449804711042002,-0.01726714126817619,1.4622476123723764,2.924495224744753,affine,0.1747409545584442,0.3494819091168884,0.9319962481713715,0.18749104934842353,eta_l0_n138_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,138,1,-1.4795147536405526,1.4449804711042002,-0.01726714126817619,1.4622476123723764,2.924495224744753,affine,0.4032069921724033,0.8064139843448066,1.0,0.4032069921724033,eta_l0_n138_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,139,0,-1.1846112842973127,1.1451154059873385,-0.01974793915498707,1.1648633451423256,2.329726690284651,affine,0.11340669701661678,0.22681339403323356,0.9319962481713715,0.12168149521967178,eta_l0_n139_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,139,1,-1.1846112842973127,1.1451154059873385,-0.01974793915498707,1.1648633451423256,2.329726690284651,affine,0.33820679583469976,0.6764135916693995,1.0,0.33820679583469976,eta_l0_n139_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,140,0,-1.0016612677883299,1.0308531994790655,0.01459596584536782,1.0162572336336977,2.0325144672673954,affine,0.08475915274863734,0.16951830549727467,0.9319962481713715,0.0909436630404248,eta_l0_n140_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,140,1,-1.0016612677883299,1.0308531994790655,0.01459596584536782,1.0162572336336977,2.0325144672673954,affine,0.29508199766810767,0.5901639953362153,1.0,0.29508199766810767,eta_l0_n140_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,141,0,-0.8542154631766358,0.8897074866463955,0.017746011734879885,0.8719614749115157,1.7439229498230313,affine,0.05967715153016984,0.11935430306033969,0.9319962481713715,0.06403153622909935,eta_l0_n141_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,141,1,-0.8542154631766358,0.8897074866463955,0.017746011734879885,0.8719614749115157,1.7439229498230313,affine,0.24652077232990385,0.4930415446598077,1.0,0.24652077232990385,eta_l0_n141_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,142,0,-1.090759700233871,1.1219900064229054,0.01561515309451722,1.1063748533283881,2.2127497066567763,affine,0.10185940078171347,0.20371880156342695,0.9319962481713715,0.10929164251633769,eta_l0_n142_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,142,1,-1.090759700233871,1.1219900064229054,0.01561515309451722,1.1063748533283881,2.2127497066567763,affine,0.32213038977985814,0.6442607795597163,1.0,0.32213038977985814,eta_l0_n142_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,143,0,-0.8162896758725654,0.8281241957461959,0.0059172599368152445,0.8222069358093806,1.6444138716187613,affine,0.05177898004970977,0.10355796009941955,0.9319962481713715,0.05555706919561427,eta_l0_n143_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,143,1,-0.8162896758725654,0.8281241957461959,0.0059172599368152445,0.8222069358093806,1.6444138716187613,affine,0.22865439410150254,0.4573087882030051,1.0,0.22865439410150254,eta_l0_n143_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,144,0,-1.073717980740392,1.0426127742833178,-0.015552603228537132,1.058165377511855,2.11633075502371,affine,0.09260083445632895,0.1852016689126579,0.9319962481713715,0.0993575184857417,eta_l0_n144_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,144,1,-1.073717980740392,1.0426127742833178,-0.015552603228537132,1.058165377511855,2.11633075502371,affine,0.30798318336415115,0.6159663667283023,1.0,0.30798318336415115,eta_l0_n144_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,145,0,-0.9082466318108172,0.8772921563875725,-0.01547723771162235,0.8927693940991949,1.7855387881983897,affine,0.06307662244905878,0.12615324489811755,0.9319962481713715,0.06767905189835112,eta_l0_n145_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,145,1,-0.9082466318108172,0.8772921563875725,-0.01547723771162235,0.8927693940991949,1.7855387881983897,affine,0.2539061430843221,0.5078122861686442,1.0,0.2539061430843221,eta_l0_n145_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,146,0,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.145374579693725,0.29074915938745,0.9319962481713715,0.15598193660002174,eta_l0_n146_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,146,1,-1.3022655421739977,1.3407396287397113,0.019237043282856803,1.3215025854568545,2.643005170913709,affine,0.3758677625197134,0.7517355250394268,1.0,0.3758677625197134,eta_l0_n146_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,147,0,-0.9610675137578782,0.9233653689554443,-0.018851072401216973,0.9422164413566613,1.8844328827133225,affine,0.071503963689264,0.143007927378528,0.9319962481713715,0.07672129992964967,eta_l0_n147_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,147,1,-0.9610675137578782,0.9233653689554443,-0.018851072401216973,0.9422164413566613,1.8844328827133225,affine,0.2708685489359554,0.5417370978719108,1.0,0.2708685489359554,eta_l0_n147_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,148,0,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.16491939624199328,0.32983879248398656,0.9319962481713715,0.17695285422615628,eta_l0_n148_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,148,1,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,affine,0.39470579092113367,0.7894115818422673,1.0,0.39470579092113367,eta_l0_n148_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,149,0,-0.8559099683090499,0.8252779003869346,-0.015316033961057629,0.8405939343479922,1.6811878686959845,affine,0.05466287032502593,0.10932574065005186,0.9319962481713715,0.05865138452249945,eta_l0_n149_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,149,1,-0.8559099683090499,0.8252779003869346,-0.015316033961057629,0.8405939343479922,1.6811878686959845,affine,0.2352777172231649,0.4705554344463298,1.0,0.2352777172231649,eta_l0_n149_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,150,0,-0.8144288244005295,0.8392548862181484,0.012413030908809408,0.826841855309339,1.653683710618678,affine,0.0525136087308466,0.1050272174616932,0.9319962481713715,0.05634530056733729,eta_l0_n150_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,150,1,-0.8144288244005295,0.8392548862181484,0.012413030908809408,0.826841855309339,1.653683710618678,affine,0.23029984075508195,0.4605996815101639,1.0,0.23029984075508195,eta_l0_n150_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,151,0,-0.7794240378560413,0.7516667225555953,-0.013878657650223003,0.7655453802058183,1.5310907604116366,affine,0.04347210480581693,0.08694420961163386,0.9319962481713715,0.04664407704549414,eta_l0_n151_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,151,1,-0.7794240378560413,0.7516667225555953,-0.013878657650223003,0.7655453802058183,1.5310907604116366,affine,0.20749637537672608,0.41499275075345216,1.0,0.20749637537672608,eta_l0_n151_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,152,0,-1.272877808027193,1.3140653792633166,0.0205937856180618,1.2934715936452548,2.5869431872905095,affine,0.13957622498198222,0.27915244996396443,0.9319962481713715,0.1497605009202972,eta_l0_n152_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,152,1,-1.272877808027193,1.3140653792633166,0.0205937856180618,1.2934715936452548,2.5869431872905095,affine,0.36969406958175527,0.7393881391635105,1.0,0.36969406958175527,eta_l0_n152_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,153,0,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.03558178463885487,0.07116356927770974,0.9319962481713715,0.038178034202034944,eta_l0_n153_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,153,1,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,affine,0.18526418828007993,0.37052837656015986,1.0,0.18526418828007993,eta_l0_n153_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,154,0,-0.7966809166067101,0.7634063707671255,-0.016637272919792334,0.7800436436869178,1.5600872873738356,affine,0.04555995415804837,0.09111990831609675,0.9319962481713715,0.04888426777193528,eta_l0_n154_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,154,1,-0.7966809166067101,0.7634063707671255,-0.016637272919792334,0.7800436436869178,1.5600872873738356,affine,0.2129062840233313,0.4258125680466626,1.0,0.2129062840233313,eta_l0_n154_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,155,0,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.060409021979322915,0.12081804395864583,0.9319962481713715,0.06481680811253134,eta_l0_n155_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,155,1,-0.8940835656454459,0.858851089575522,-0.01761623803496193,0.876467327610484,1.752934655220968,affine,0.2481239651014495,0.496247930202899,1.0,0.2481239651014495,eta_l0_n155_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,156,0,-0.929226820999473,0.968979740210427,0.019876459605477015,0.94910328060495,1.8982065612099,affine,0.07271276999907768,0.14542553999815536,0.9319962481713715,0.07801830762918217,eta_l0_n156_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,156,1,-0.929226820999473,0.968979740210427,0.019876459605477015,0.94910328060495,1.8982065612099,affine,0.2731668711556356,0.5463337423112712,1.0,0.2731668711556356,eta_l0_n156_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,157,0,-1.1731237389534521,1.2050166691047846,0.015946465075666216,1.1890702040291183,2.3781404080582367,affine,0.11823823949624425,0.2364764789924885,0.9319962481713715,0.12686557454306738,eta_l0_n157_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,157,1,-1.1731237389534521,1.2050166691047846,0.015946465075666216,1.1890702040291183,2.3781404080582367,affine,0.3445996205240258,0.6891992410480516,1.0,0.3445996205240258,eta_l0_n157_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,158,0,-0.8913860173428698,0.9226653347131666,0.015639658685148383,0.9070256760280182,1.8140513520560364,affine,0.06545990610854242,0.13091981221708485,0.9319962481713715,0.07023623350091635,eta_l0_n158_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,158,1,-0.8913860173428698,0.9226653347131666,0.015639658685148383,0.9070256760280182,1.8140513520560364,affine,0.2588768989638672,0.5177537979277343,1.0,0.2588768989638672,eta_l0_n158_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,159,0,-1.1273921839493977,1.095908177981933,-0.015742002983732295,1.1116501809656654,2.2233003619313307,affine,0.10288727370103208,0.20577454740206416,0.9319962481713715,0.11039451489520762,eta_l0_n159_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,159,1,-1.1273921839493977,1.095908177981933,-0.015742002983732295,1.1116501809656654,2.2233003619313307,affine,0.3236304041241462,0.6472608082482924,1.0,0.3236304041241462,eta_l0_n159_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,160,0,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.1775418997412546,0.3550837994825092,0.9319962481713715,0.19049636743667336,eta_l0_n160_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,160,1,-1.492386044559716,1.4589190080498684,-0.01673351825492375,1.4756525263047922,2.9513050526095843,affine,0.40551713911296217,0.8110342782259243,1.0,0.40551713911296217,eta_l0_n160_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,161,0,-0.8118887083898726,0.7855637226605158,-0.01316249286467841,0.7987262155251942,1.5974524310503884,affine,0.04827148630569767,0.09654297261139534,0.9319962481713715,0.05179364874097832,eta_l0_n161_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,161,1,-0.8118887083898726,0.7855637226605158,-0.01316249286467841,0.7987262155251942,1.5974524310503884,affine,0.21992204690021314,0.4398440938004263,1.0,0.21992204690021314,eta_l0_n161_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,162,0,-0.9685612086655188,0.9950593658781902,0.013249078606335729,0.9818102872718545,1.963620574543709,affine,0.07847764604365189,0.15695529208730377,0.9319962481713715,0.08420382184759799,eta_l0_n162_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,162,1,-0.9685612086655188,0.9950593658781902,0.013249078606335729,0.9818102872718545,1.963620574543709,affine,0.2840685315409661,0.5681370630819322,1.0,0.2840685315409661,eta_l0_n162_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,163,0,-0.8271071303752253,0.8610459715485379,0.016969420586656292,0.8440765509618816,1.6881531019237632,affine,0.055219362719068694,0.11043872543813739,0.9319962481713715,0.05924848176954806,eta_l0_n163_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,163,1,-0.8271071303752253,0.8610459715485379,0.016969420586656292,0.8440765509618816,1.6881531019237632,affine,0.23651711717424592,0.47303423434849184,1.0,0.23651711717424592,eta_l0_n163_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,164,0,-1.1176997476072554,1.1538089483476914,0.018054600370218,1.1357543479774734,2.271508695954947,affine,0.10762484486853492,0.21524968973706984,0.9319962481713715,0.11547776622459678,eta_l0_n164_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,164,1,-1.1176997476072554,1.1538089483476914,0.018054600370218,1.1357543479774734,2.271508695954947,affine,0.3303440208889012,0.6606880417778024,1.0,0.3303440208889012,eta_l0_n164_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,165,0,-0.9571068847382632,0.9915651490227831,0.017229132142259962,0.9743360168805232,1.9486720337610464,affine,0.07715881571356727,0.15431763142713453,0.9319962481713715,0.08278876214894336,eta_l0_n165_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,165,1,-0.9571068847382632,0.9915651490227831,0.017229132142259962,0.9743360168805232,1.9486720337610464,affine,0.2815750736883109,0.5631501473766218,1.0,0.2815750736883109,eta_l0_n165_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,166,0,-1.1951923778641025,1.1529662485775043,-0.021113064643299095,1.1740793132208034,2.3481586264416068,affine,0.11525571941871461,0.23051143883742922,0.9319962481713715,0.12366543282213072,eta_l0_n166_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,166,1,-1.1951923778641025,1.1529662485775043,-0.021113064643299095,1.1740793132208034,2.3481586264416068,affine,0.34062469224724484,0.6812493844944897,1.0,0.34062469224724484,eta_l0_n166_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,167,0,-1.1001154688531447,1.0726088877062845,-0.01375329057343011,1.0863621782797146,2.172724356559429,affine,0.0979784771275863,0.1959569542551726,0.9319962481713715,0.10512754457952542,eta_l0_n167_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,167,1,-1.1001154688531447,1.0726088877062845,-0.01375329057343011,1.0863621782797146,2.172724356559429,affine,0.31637193416115633,0.6327438683223127,1.0,0.31637193416115633,eta_l0_n167_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,168,0,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.09087186490139544,0.18174372980279088,0.9319962481713715,0.09750239346959937,eta_l0_n168_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,168,1,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,affine,0.30510844941791904,0.6102168988358381,1.0,0.30510844941791904,eta_l0_n168_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,169,0,-0.8810525006112329,0.8477341734459161,-0.0166591635826584,0.8643933370285745,1.728786674057149,affine,0.05844734457479079,0.11689468914958158,0.9319962481713715,0.06271199555735094,eta_l0_n169_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,169,1,-0.8810525006112329,0.8477341734459161,-0.0166591635826584,0.8643933370285745,1.728786674057149,affine,0.24383710591235344,0.4876742118247069,1.0,0.24383710591235344,eta_l0_n169_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,170,0,-0.8468298613304298,0.8765102975610902,0.014840218115330206,0.86167007944576,1.72334015889152,affine,0.05799879967277095,0.1159975993455419,0.9319962481713715,0.06223072226585442,eta_l0_n170_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,170,1,-0.8468298613304298,0.8765102975610902,0.014840218115330206,0.86167007944576,1.72334015889152,affine,0.24288770303173637,0.48577540606347275,1.0,0.24288770303173637,eta_l0_n170_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,171,0,-1.156771804990858,1.1214757897709782,-0.017648007609939897,1.139123797380918,2.278247594761836,affine,0.10828744905510314,0.21657489811020628,0.9319962481713715,0.11618871778461462,eta_l0_n171_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,171,1,-1.156771804990858,1.1214757897709782,-0.017648007609939897,1.139123797380918,2.278247594761836,affine,0.33127683317116435,0.6625536663423287,1.0,0.33127683317116435,eta_l0_n171_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,172,0,-0.6602039211953793,0.6350755481873448,-0.01256418650401725,0.647639734691362,1.295279469382724,affine,0.028361496955946756,0.05672299391189351,0.9319962481713715,0.03043091322695085,eta_l0_n172_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,172,1,-0.6602039211953793,0.6350755481873448,-0.01256418650401725,0.647639734691362,1.295279469382724,affine,0.16243039250440386,0.3248607850088077,1.0,0.16243039250440386,eta_l0_n172_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,173,0,-0.6736564656359885,0.6502014587413381,-0.011727503447325205,0.6619289621886633,1.3238579243773265,affine,0.030016059137825183,0.060032118275650366,0.9319962481713715,0.032206201684522186,eta_l0_n173_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,173,1,-0.6736564656359885,0.6502014587413381,-0.011727503447325205,0.6619289621886633,1.3238579243773265,affine,0.1679374884917527,0.3358749769835054,1.0,0.1679374884917527,eta_l0_n173_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,174,0,-1.2671951827874361,1.2257910549274456,-0.020702063929995296,1.2464931188574409,2.4929862377148817,affine,0.12991639486248988,0.25983278972497975,0.9319962481713715,0.13939583460490648,eta_l0_n174_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,174,1,-1.2671951827874361,1.2257910549274456,-0.020702063929995296,1.2464931188574409,2.4929862377148817,affine,0.35881234692755687,0.7176246938551137,1.0,0.35881234692755687,eta_l0_n174_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,175,0,-1.383141385735959,1.4281571697421032,0.022507892003072127,1.405649277739031,2.811298555478062,affine,0.16292477100837002,0.32584954201674005,0.9319962481713715,0.17481268978071263,eta_l0_n175_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,175,1,-1.383141385735959,1.4281571697421032,0.022507892003072127,1.405649277739031,2.811298555478062,affine,0.39286284593744103,0.7857256918748821,1.0,0.39286284593744103,eta_l0_n175_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,176,0,-1.0572251389801486,1.0182363067147557,-0.019494416132696424,1.0377307228474522,2.0754614456949043,affine,0.08877578314687136,0.17755156629374272,0.9319962481713715,0.09525336965792983,eta_l0_n176_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,176,1,-1.0572251389801486,1.0182363067147557,-0.019494416132696424,1.0377307228474522,2.0754614456949043,affine,0.30169992359450326,0.6033998471890065,1.0,0.30169992359450326,eta_l0_n176_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,177,0,-0.8553465281258209,0.8396584283206342,-0.007844049902593353,0.8475024782232276,1.695004956446455,affine,0.05571402887752242,0.11142805775504484,0.9319962481713715,0.05977924158690171,eta_l0_n177_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,177,1,-0.8553465281258209,0.8396584283206342,-0.007844049902593353,0.8475024782232276,1.695004956446455,affine,0.23785857921394318,0.47571715842788637,1.0,0.23785857921394318,eta_l0_n177_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,178,0,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.09034133178274757,0.18068266356549514,0.9319962481713715,0.09693314963445646,eta_l0_n178_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,178,1,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,affine,0.30426139037209493,0.6085227807441899,1.0,0.30426139037209493,eta_l0_n178_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,179,0,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.12411459017998151,0.24822918035996303,0.9319962481713715,0.1331706972249097,eta_l0_n179_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,179,1,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,affine,0.3518962394926582,0.7037924789853164,1.0,0.3518962394926582,eta_l0_n179_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,180,0,-1.0633221779271933,1.098399243085133,0.017538532578969868,1.0808607105061632,2.1617214210123263,affine,0.09693932115397964,0.1938786423079593,0.9319962481713715,0.1040125658704957,eta_l0_n180_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,180,1,-1.0633221779271933,1.098399243085133,0.017538532578969868,1.0808607105061632,2.1617214210123263,affine,0.3147125704027748,0.6294251408055496,1.0,0.3147125704027748,eta_l0_n180_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,181,0,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.10855051376169962,0.21710102752339924,0.9319962481713715,0.11647097718974918,eta_l0_n181_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,181,1,-1.1207207697771528,1.1600810252341789,0.01968012772851302,1.1404008975056659,2.2808017950113317,affine,0.33159863418336927,0.6631972683667385,1.0,0.33159863418336927,eta_l0_n181_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,182,0,-0.946853930283215,0.9082390537521355,-0.01930743826553971,0.9275464920176753,1.8550929840353505,affine,0.06897207704411946,0.13794415408823893,0.9319962481713715,0.07400467242163959,eta_l0_n182_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,182,1,-0.946853930283215,0.9082390537521355,-0.01930743826553971,0.9275464920176753,1.8550929840353505,affine,0.2658832386164964,0.5317664772329928,1.0,0.2658832386164964,eta_l0_n182_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,183,0,-0.9012005330706289,0.9364696817911625,0.01763457436026683,0.9188351074308957,1.8376702148617914,affine,0.06747144526222183,0.13494289052444366,0.9319962481713715,0.07239454600230909,eta_l0_n183_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,183,1,-0.9012005330706289,0.9364696817911625,0.01763457436026683,0.9188351074308957,1.8376702148617914,affine,0.26292614612453097,0.5258522922490619,1.0,0.26292614612453097,eta_l0_n183_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,184,0,-1.4793999358375656,1.4405298344240642,-0.019435050706750667,1.459964885130815,2.91992977026163,affine,0.17427052042194865,0.3485410408438973,0.9319962481713715,0.18698628965929542,eta_l0_n184_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,184,1,-1.4793999358375656,1.4405298344240642,-0.019435050706750667,1.459964885130815,2.91992977026163,affine,0.4027895634419543,0.8055791268839086,1.0,0.4027895634419543,eta_l0_n184_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,185,0,-0.7321511044331087,0.7389444504690762,0.0033966730179837423,0.7355477774510925,1.471095554902185,affine,0.03929340910885843,0.07858681821771686,0.9319962481713715,0.04216047992248283,eta_l0_n185_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,185,1,-0.7321511044331087,0.7389444504690762,0.0033966730179837423,0.7355477774510925,1.471095554902185,affine,0.19621331094010355,0.3924266218802071,1.0,0.19621331094010355,eta_l0_n185_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,186,0,-0.8942204521232299,0.8708685501338452,-0.011675950994692319,0.8825445011285376,1.765089002257075,affine,0.06136968976427765,0.1227393795285553,0.9319962481713715,0.06584757168785647,eta_l0_n186_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,186,1,-0.8942204521232299,0.8708685501338452,-0.011675950994692319,0.8825445011285376,1.765089002257075,affine,0.2503530688112541,0.5007061376225082,1.0,0.2503530688112541,eta_l0_n186_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,187,0,-1.0714555174736882,1.0473124800913571,-0.012071518691165517,1.0593839987825227,2.1187679975650453,affine,0.09281648755288631,0.18563297510577262,0.9319962481713715,0.09958890685986925,eta_l0_n187_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,187,1,-1.0714555174736882,1.0473124800913571,-0.012071518691165517,1.0593839987825227,2.1187679975650453,affine,0.30838930232721823,0.6167786046544365,1.0,0.30838930232721823,eta_l0_n187_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,188,0,-0.8561385666094257,0.8238401922534622,-0.01614918717798175,0.8399893794314439,1.6799787588628878,affine,0.05457352556779011,0.10914705113558022,0.9319962481713715,0.058555520663164044,eta_l0_n188_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,188,1,-0.8561385666094257,0.8238401922534622,-0.01614918717798175,0.8399893794314439,1.6799787588628878,affine,0.23504641413088642,0.47009282826177284,1.0,0.23504641413088642,eta_l0_n188_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,189,0,-1.0554762744104922,1.0911960330035595,0.017859879296533654,1.0733361537070258,2.1466723074140517,affine,0.09549999712492331,0.19099999424984662,0.9319962481713715,0.10246822056665959,eta_l0_n189_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,189,1,-1.0554762744104922,1.0911960330035595,0.017859879296533654,1.0733361537070258,2.1466723074140517,affine,0.3124867202893598,0.6249734405787196,1.0,0.3124867202893598,eta_l0_n189_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,190,0,-0.8409602339792386,0.8183358148780695,-0.01131220955058454,0.829648024428654,1.659296048857308,affine,0.052940748618230955,0.10588149723646191,0.9319962481713715,0.05680360701247848,eta_l0_n190_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,190,1,-0.8409602339792386,0.8183358148780695,-0.01131220955058454,0.829648024428654,1.659296048857308,affine,0.23133788022140225,0.4626757604428045,1.0,0.23133788022140225,eta_l0_n190_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,191,0,-0.9923663640591591,0.9567994816892673,-0.017783441184945903,0.9745829228742132,1.9491658457484264,affine,0.07720636019263961,0.15441272038527923,0.9319962481713715,0.08283977574386463,eta_l0_n191_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,191,1,-0.9923663640591591,0.9567994816892673,-0.017783441184945903,0.9745829228742132,1.9491658457484264,affine,0.2816476439026048,0.5632952878052097,1.0,0.2816476439026048,eta_l0_n191_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,192,0,-0.6229070664001423,0.5904531146635353,-0.01622697586830346,0.6066800905318388,1.2133601810636776,affine,0.02390234411526149,0.04780468823052298,0.9319962481713715,0.025646395210451995,eta_l0_n192_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,192,1,-0.6229070664001423,0.5904531146635353,-0.01622697586830346,0.6066800905318388,1.2133601810636776,affine,0.14666339508907575,0.2933267901781515,1.0,0.14666339508907575,eta_l0_n192_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,193,0,-0.737441641819339,0.710743302455173,-0.013349169682082995,0.724092472137256,1.448184944274512,affine,0.037799429557977056,0.07559885911595411,0.9319962481713715,0.04055749111881259,eta_l0_n193_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,193,1,-0.737441641819339,0.710743302455173,-0.013349169682082995,0.724092472137256,1.448184944274512,affine,0.1917741690529898,0.3835483381059796,1.0,0.1917741690529898,eta_l0_n193_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,194,0,-0.92629178814389,0.9624928742496371,0.018100543052873563,0.9443923311967636,1.8887846623935272,affine,0.07187781562146196,0.14375563124292393,0.9319962481713715,0.07712243022704247,eta_l0_n194_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,194,1,-0.92629178814389,0.9624928742496371,0.018100543052873563,0.9443923311967636,1.8887846623935272,affine,0.2716137525954709,0.5432275051909418,1.0,0.2716137525954709,eta_l0_n194_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,195,0,-1.4438322648629724,1.4816268957431418,0.01889731544008466,1.462729580303057,2.925459160606114,affine,0.17484682179969427,0.34969364359938854,0.9319962481713715,0.18760464126626417,eta_l0_n195_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,195,1,-1.4438322648629724,1.4816268957431418,0.01889731544008466,1.462729580303057,2.925459160606114,affine,0.4032760717116363,0.8065521434232726,1.0,0.4032760717116363,eta_l0_n195_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,196,0,-1.152447437734088,1.1941743187475988,0.02086344050675537,1.1733108782408435,2.346621756481687,affine,0.11510051619594604,0.23020103239189207,0.9319962481713715,0.12349890508870573,eta_l0_n196_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,196,1,-1.152447437734088,1.1941743187475988,0.02086344050675537,1.1733108782408435,2.346621756481687,affine,0.3404262798220871,0.6808525596441742,1.0,0.3404262798220871,eta_l0_n196_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,197,0,-0.6221827602303872,0.6073776418168663,-0.007402559206760473,0.6147802010236267,1.2295604020472535,affine,0.024711480698816016,0.04942296139763203,0.9319962481713715,0.026514571005303203,eta_l0_n197_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,197,1,-0.6221827602303872,0.6073776418168663,-0.007402559206760473,0.6147802010236267,1.2295604020472535,affine,0.14984737210450264,0.29969474420900527,1.0,0.14984737210450264,eta_l0_n197_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,198,0,-0.8900304609633192,0.9240067976547857,0.016988168345733246,0.9070186293090524,1.8140372586181048,affine,0.06546697285142163,0.13093394570284325,0.9319962481713715,0.07024381587358476,eta_l0_n198_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,198,1,-0.8900304609633192,0.9240067976547857,0.016988168345733246,0.9070186293090524,1.8140372586181048,affine,0.25885497492854465,0.5177099498570893,1.0,0.25885497492854465,eta_l0_n198_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,199,0,-1.095266671314244,1.1346180087386017,0.01967566871217885,1.114942340026423,2.229884680052846,affine,0.10355036232782984,0.20710072465565968,0.9319962481713715,0.11110598624297191,eta_l0_n199_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,199,1,-1.095266671314244,1.1346180087386017,0.01967566871217885,1.114942340026423,2.229884680052846,affine,0.3245088156471296,0.6490176312942592,1.0,0.3245088156471296,eta_l0_n199_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,200,0,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.08060765055937986,0.16121530111875973,0.9319962481713715,0.08648924361824048,eta_l0_n200_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,200,1,-0.9767008442562579,1.0102996945497786,0.01679942514676036,0.9935002694030183,1.9870005388060366,affine,0.28780718721053394,0.5756143744210679,1.0,0.28780718721053394,eta_l0_n200_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,201,0,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.10398467023247389,0.20796934046494778,0.9319962481713715,0.11157198372471734,eta_l0_n201_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,201,1,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,affine,0.3251803898554791,0.6503607797109582,1.0,0.3251803898554791,eta_l0_n201_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,202,0,-0.7951331280804157,0.8148313974508756,0.009849134685229965,0.8049822627656457,1.6099645255312913,affine,0.049187121407205665,0.09837424281441133,0.9319962481713715,0.05277609379191551,eta_l0_n202_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,202,1,-0.7951331280804157,0.8148313974508756,0.009849134685229965,0.8049822627656457,1.6099645255312913,affine,0.22227638354605225,0.4445527670921045,1.0,0.22227638354605225,eta_l0_n202_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,203,0,-0.929105445796119,0.8942568543418767,-0.01742429572712112,0.9116811500689979,1.8233623001379957,affine,0.0662563014773734,0.1325126029547468,0.9319962481713715,0.07109073840948603,eta_l0_n203_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,203,1,-0.929105445796119,0.8942568543418767,-0.01742429572712112,0.9116811500689979,1.8233623001379957,affine,0.2604631297546389,0.5209262595092778,1.0,0.2604631297546389,eta_l0_n203_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,204,0,-0.7641244024694368,0.7349866898999596,-0.014568856284738585,0.7495555461846982,1.4991110923693964,affine,0.041244716478901274,0.08248943295780255,0.9319962481713715,0.04425416578642425,eta_l0_n204_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,204,1,-0.7641244024694368,0.7349866898999596,-0.014568856284738585,0.7495555461846982,1.4991110923693964,affine,0.2014452447000997,0.4028904894001994,1.0,0.2014452447000997,eta_l0_n204_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,205,0,-1.0117373774449652,0.9756317416300792,-0.01805281790744301,0.9936845595375221,1.9873691190750442,affine,0.08064859882773769,0.16129719765547537,0.9319962481713715,0.08653317970536333,eta_l0_n205_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,205,1,-1.0117373774449652,0.9756317416300792,-0.01805281790744301,0.9936845595375221,1.9873691190750442,affine,0.28784791784582964,0.5756958356916593,1.0,0.28784791784582964,eta_l0_n205_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,206,0,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.13911287947176756,0.27822575894353513,0.9319962481713715,0.14926334708397676,eta_l0_n206_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,206,1,-1.271032511729701,1.3114418482049401,0.020204668237619572,1.2912371799673206,2.582474359934641,affine,0.36919763717385085,0.7383952743477017,1.0,0.36919763717385085,eta_l0_n206_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,207,0,-1.3112708446293018,1.2772121752583014,-0.017029334685500164,1.2942415099438016,2.588483019887603,affine,0.13971997178468812,0.27943994356937624,0.9319962481713715,0.14991473630803395,eta_l0_n207_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,207,1,-1.3112708446293018,1.2772121752583014,-0.017029334685500164,1.2942415099438016,2.588483019887603,affine,0.3699089369805238,0.7398178739610476,1.0,0.3699089369805238,eta_l0_n207_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,208,0,-1.1651989100051081,1.2002604364732326,0.01753076323406222,1.1827296732391703,2.3654593464783407,affine,0.11697091939156241,0.23394183878312483,0.9319962481713715,0.12550578354909248,eta_l0_n208_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,208,1,-1.1651989100051081,1.2002604364732326,0.01753076323406222,1.1827296732391703,2.3654593464783407,affine,0.3429378820617445,0.685875764123489,1.0,0.3429378820617445,eta_l0_n208_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,209,0,-1.1982393510876126,1.2306822718446642,0.016221460378525787,1.2144608114661384,2.4289216229322768,affine,0.12336957207392958,0.24673914414785916,0.9319962481713715,0.13237131835668603,eta_l0_n209_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,209,1,-1.1982393510876126,1.2306822718446642,0.016221460378525787,1.2144608114661384,2.4289216229322768,affine,0.3510407017654134,0.7020814035308268,1.0,0.3510407017654134,eta_l0_n209_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,210,0,-0.9228447532260016,0.9507075443892259,0.01393139558161216,0.9367761488076137,1.8735522976152275,affine,0.07053047421855775,0.1410609484371155,0.9319962481713715,0.07567677912539075,eta_l0_n210_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,210,1,-0.9228447532260016,0.9507075443892259,0.01393139558161216,0.9367761488076137,1.8735522976152275,affine,0.26910035646862585,0.5382007129372517,1.0,0.26910035646862585,eta_l0_n210_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,211,0,-1.3219320199504638,1.2845693941220588,-0.018681312914202497,1.3032507070362613,2.6065014140725227,affine,0.14158975427736184,0.2831795085547237,0.9319962481713715,0.15192094877545786,eta_l0_n211_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,211,1,-1.3219320199504638,1.2845693941220588,-0.018681312914202497,1.3032507070362613,2.6065014140725227,affine,0.3718937045390254,0.7437874090780509,1.0,0.3718937045390254,eta_l0_n211_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,212,0,-1.1136630602458562,1.0736732395863262,-0.01999491032976497,1.0936681499160912,2.1873362998321824,affine,0.09941980308099137,0.19883960616198273,0.9319962481713715,0.10667403787950708,eta_l0_n212_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,212,1,-1.1136630602458562,1.0736732395863262,-0.01999491032976497,1.0936681499160912,2.1873362998321824,affine,0.3184140898770353,0.6368281797540706,1.0,0.3184140898770353,eta_l0_n212_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,213,0,-1.3976935250910973,1.3709670065547257,-0.013363259268185823,1.3843302658229115,2.768660531645823,affine,0.1584389554911889,0.3168779109823778,0.9319962481713715,0.1699995636270585,eta_l0_n213_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,213,1,-1.3976935250910973,1.3709670065547257,-0.013363259268185823,1.3843302658229115,2.768660531645823,affine,0.38883905368764476,0.7776781073752895,1.0,0.38883905368764476,eta_l0_n213_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,214,0,-0.6308511366333197,0.5817134666571074,-0.024568834988106136,0.6062823016452136,1.2125646032904271,affine,0.02393379841437915,0.0478675968287583,0.9319962481713715,0.02568014459429273,eta_l0_n214_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,214,1,-0.6308511366333197,0.5817134666571074,-0.024568834988106136,0.6062823016452136,1.2125646032904271,affine,0.14637875612598197,0.29275751225196395,1.0,0.14637875612598197,eta_l0_n214_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,215,0,-0.6750710083304651,0.6486981605886052,-0.013186423870929942,0.6618845844595351,1.3237691689190703,affine,0.030018619591721033,0.060037239183442066,0.9319962481713715,0.03220894896371014,eta_l0_n215_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,215,1,-0.6750710083304651,0.6486981605886052,-0.013186423870929942,0.6618845844595351,1.3237691689190703,affine,0.1679053284611978,0.3358106569223956,1.0,0.1679053284611978,eta_l0_n215_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,216,0,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.13353496019544117,0.26706992039088234,0.9319962481713715,0.1432784310639063,eta_l0_n216_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,216,1,-1.2424597126868335,1.2857833148653481,0.021661801089257304,1.2641215137760908,2.5282430275521817,affine,0.36296503509120265,0.7259300701824053,1.0,0.36296503509120265,eta_l0_n216_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,217,0,-1.0176420196916929,0.9812025096472038,-0.018219755022244544,0.9994222646694484,1.9988445293388968,affine,0.08169289794408681,0.16338579588817362,0.9319962481713715,0.0876536768301083,eta_l0_n217_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,217,1,-1.0176420196916929,0.9812025096472038,-0.018219755022244544,0.9994222646694484,1.9988445293388968,affine,0.289687170148816,0.579374340297632,1.0,0.289687170148816,eta_l0_n217_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,218,0,-0.9798393779987575,1.0123201516079219,0.016240386804582174,0.9960797648033397,1.9921595296066794,affine,0.08107290966168675,0.1621458193233735,0.9319962481713715,0.08698845067322568,eta_l0_n218_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,218,1,-0.9798393779987575,1.0123201516079219,0.016240386804582174,0.9960797648033397,1.9921595296066794,affine,0.2886444411656374,0.5772888823312748,1.0,0.2886444411656374,eta_l0_n218_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,219,0,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.08893857072936846,0.17787714145873693,0.9319962481713715,0.09542803514914452,eta_l0_n219_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,219,1,-1.050898072218128,1.026711272754142,-0.012093399731992971,1.038804672486135,2.07760934497227,affine,0.3021270990529596,0.6042541981059192,1.0,0.3021270990529596,eta_l0_n219_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,220,0,-1.0276272819085595,0.9881549383021555,-0.019736171803201974,1.0078911101053576,2.015782220210715,affine,0.08325051921200739,0.16650103842401479,0.9319962481713715,0.08932495101278524,eta_l0_n220_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,220,1,-1.0276272819085595,0.9881549383021555,-0.019736171803201974,1.0078911101053576,2.015782220210715,affine,0.2923627454325825,0.584725490865165,1.0,0.2923627454325825,eta_l0_n220_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,221,0,-0.9182714052195694,0.9526902849630151,0.017209439871722854,0.9354808450912923,1.8709616901825845,affine,0.07032498333747385,0.1406499666749477,0.9319962481713715,0.0754562944598279,eta_l0_n221_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,221,1,-0.9182714052195694,0.9526902849630151,0.017209439871722854,0.9354808450912923,1.8709616901825845,affine,0.2686165914228866,0.5372331828457731,1.0,0.2686165914228866,eta_l0_n221_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,222,0,-1.0931660046004656,1.131441604552337,0.019137799975935676,1.1123038045764013,2.2246076091528026,affine,0.1030323751704313,0.2060647503408626,0.9319962481713715,0.11055020379383132,eta_l0_n222_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,222,1,-1.0931660046004656,1.131441604552337,0.019137799975935676,1.1123038045764013,2.2246076091528026,affine,0.32376979893757785,0.6475395978751557,1.0,0.32376979893757785,eta_l0_n222_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,223,0,-0.7347884981061658,0.7031688277584786,-0.0158098351738436,0.7189786629323222,1.4379573258646443,affine,0.0371415219972684,0.0742830439945368,0.9319962481713715,0.03985157887721343,eta_l0_n223_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,223,1,-0.7347884981061658,0.7031688277584786,-0.0158098351738436,0.7189786629323222,1.4379573258646443,affine,0.18978987632131175,0.3795797526426235,1.0,0.18978987632131175,eta_l0_n223_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,224,0,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.13683906908689789,0.27367813817379577,0.9319962481713715,0.1468236265493383,eta_l0_n224_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,224,1,-1.2999700704514803,1.260481893985253,-0.019744088233113688,1.2802259822183666,2.560451964436733,affine,0.3667091474322621,0.7334182948645241,1.0,0.3667091474322621,eta_l0_n224_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,225,0,-0.801929426509088,0.8378016275339949,0.017936100512453423,0.8198655270215415,1.639731054043083,affine,0.051479085767128514,0.10295817153425703,0.9319962481713715,0.055235292918971876,eta_l0_n225_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,225,1,-0.801929426509088,0.8378016275339949,0.017936100512453423,0.8198655270215415,1.639731054043083,affine,0.22766628827824661,0.45533257655649323,1.0,0.22766628827824661,eta_l0_n225_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,226,0,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.0847732214456068,0.1695464428912136,0.9319962481713715,0.09095875826961383,eta_l0_n226_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,226,1,-1.0355052941917036,0.9968705997566258,-0.01931734721753886,1.0161879469741648,2.0323758939483296,affine,0.29499299606789253,0.5899859921357851,1.0,0.29499299606789253,eta_l0_n226_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,227,0,-1.2227325622777163,1.266492614795333,0.02188002625880836,1.2446125885365247,2.4892251770730494,affine,0.12953797657304725,0.2590759531460945,0.9319962481713715,0.1389898047628496,eta_l0_n227_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,227,1,-1.2227325622777163,1.266492614795333,0.02188002625880836,1.2446125885365247,2.4892251770730494,affine,0.3583450777565647,0.7166901555131294,1.0,0.3583450777565647,eta_l0_n227_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,228,0,-0.9930624745502177,1.0284245545367263,0.01768103999325432,1.010743514543472,2.021487029086944,affine,0.08376097929774685,0.1675219585954937,0.9319962481713715,0.08987265717226926,eta_l0_n228_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,228,1,-0.9930624745502177,1.0284245545367263,0.01768103999325432,1.010743514543472,2.021487029086944,affine,0.2932994718153777,0.5865989436307554,1.0,0.2932994718153777,eta_l0_n228_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,229,0,-0.7782243522405148,0.7634707519083996,-0.007376800166057573,0.7708475520744572,1.5416951041489144,affine,0.04419558317224093,0.08839116634448187,0.9319962481713715,0.04742034451206765,eta_l0_n229_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,229,1,-0.7782243522405148,0.7634707519083996,-0.007376800166057573,0.7708475520744572,1.5416951041489144,affine,0.20955349419685784,0.4191069883937157,1.0,0.20955349419685784,eta_l0_n229_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,230,0,-1.4736909184242268,1.4349512883955993,-0.019369815014313785,1.454321103409913,2.908642206819826,affine,0.1730901226391359,0.3461802452782718,0.9319962481713715,0.18571976333461465,eta_l0_n230_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,230,1,-1.4736909184242268,1.4349512883955993,-0.019369815014313785,1.454321103409913,2.908642206819826,affine,0.4018009870697762,0.8036019741395524,1.0,0.4018009870697762,eta_l0_n230_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,231,0,-0.9741772249753823,1.0156090479613664,0.020715911492992067,0.9948931364683744,1.9897862729367488,affine,0.08088571108096311,0.16177142216192622,0.9319962481713715,0.08678759301838969,eta_l0_n231_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,231,1,-0.9741772249753823,1.0156090479613664,0.020715911492992067,0.9948931364683744,1.9897862729367488,affine,0.2881928095706266,0.5763856191412532,1.0,0.2881928095706266,eta_l0_n231_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,232,0,-0.9915097879699292,1.0315020359202347,0.01999612397515277,1.011505911945082,2.023011823890164,affine,0.08391577855064876,0.16783155710129752,0.9319962481713715,0.09003875145988643,eta_l0_n232_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,232,1,-0.9915097879699292,1.0315020359202347,0.01999612397515277,1.011505911945082,2.023011823890164,affine,0.29350402543245907,0.5870080508649181,1.0,0.29350402543245907,eta_l0_n232_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,233,0,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.09369831588937973,0.18739663177875945,0.9319962481713715,0.10053507841176511,eta_l0_n233_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,233,1,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,affine,0.3096918945766334,0.6193837891532668,1.0,0.3096918945766334,eta_l0_n233_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,234,0,-0.9553499690208582,0.9911598413697273,0.01790493617443456,0.9732549051952928,1.9465098103905856,affine,0.0769698294054751,0.1539396588109502,0.9319962481713715,0.08258598632397307,eta_l0_n234_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,234,1,-0.9553499690208582,0.9911598413697273,0.01790493617443456,0.9732549051952928,1.9465098103905856,affine,0.28121030953724063,0.5624206190744813,1.0,0.28121030953724063,eta_l0_n234_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,235,0,-0.9690273603532124,1.0071673197752011,0.01906997971099439,0.9880973400642068,1.9761946801284136,affine,0.07964352957996403,0.15928705915992805,0.9319962481713715,0.08545477488372841,eta_l0_n235_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,235,1,-0.9690273603532124,1.0071673197752011,0.01906997971099439,0.9880973400642068,1.9761946801284136,affine,0.286028525577318,0.572057051154636,1.0,0.286028525577318,eta_l0_n235_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,236,0,-1.155883803770632,1.1917587708387822,0.017937483534075094,1.1738212873047071,2.3476425746094143,affine,0.11518710792645188,0.23037421585290377,0.9319962481713715,0.12359181504480882,eta_l0_n236_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,236,1,-1.155883803770632,1.1917587708387822,0.017937483534075094,1.1738212873047071,2.3476425746094143,affine,0.3406020962928643,0.6812041925857286,1.0,0.3406020962928643,eta_l0_n236_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,237,0,-0.9032024251467425,0.9272904688323387,0.01204402184279807,0.9152464469895406,1.8304928939790812,affine,0.06683064114202937,0.13366128228405874,0.9319962481713715,0.0717069851656107,eta_l0_n237_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,237,1,-0.9032024251467425,0.9272904688323387,0.01204402184279807,0.9152464469895406,1.8304928939790812,affine,0.26176398680269347,0.5235279736053869,1.0,0.26176398680269347,eta_l0_n237_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,238,0,-1.247516362852558,1.2970872348018665,0.024785435974654213,1.2723017988272123,2.5446035976544246,affine,0.1352337806803164,0.2704675613606328,0.9319962481713715,0.14510120716220973,eta_l0_n238_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,238,1,-1.247516362852558,1.2970872348018665,0.024785435974654213,1.2723017988272123,2.5446035976544246,affine,0.3648170558558873,0.7296341117117746,1.0,0.3648170558558873,eta_l0_n238_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,239,0,-0.9472028721092914,0.9617653086795487,0.007281218285128621,0.9544840903944201,1.9089681807888401,affine,0.07359532437748174,0.1471906487549635,0.9319962481713715,0.0789652581991396,eta_l0_n239_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,239,1,-0.9472028721092914,0.9617653086795487,0.007281218285128621,0.9544840903944201,1.9089681807888401,affine,0.2751152937909374,0.5502305875818748,1.0,0.2751152937909374,eta_l0_n239_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,240,0,-1.1390201794660133,1.1757004326923164,0.01834012661315154,1.1573603060791648,2.3147206121583297,affine,0.11190478616491507,0.22380957232983015,0.9319962481713715,0.12006999640232296,eta_l0_n240_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,240,1,-1.1390201794660133,1.1757004326923164,0.01834012661315154,1.1573603060791648,2.3147206121583297,affine,0.33622077083622676,0.6724415416724535,1.0,0.33622077083622676,eta_l0_n240_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,241,0,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.05101514148501563,0.10203028297003126,0.9319962481713715,0.054737496620946895,eta_l0_n241_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,241,1,-0.8034218143463597,0.8305734264130716,0.013575806033355953,0.8169976203797157,1.6339952407594314,affine,0.22667314579283168,0.45334629158566336,1.0,0.22667314579283168,eta_l0_n241_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,242,0,-0.8216464738019208,0.8370838065220393,0.007718666360059245,0.82936514016198,1.65873028032396,affine,0.05288344570803522,0.10576689141607044,0.9319962481713715,0.05674212295574739,eta_l0_n242_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,242,1,-0.8216464738019208,0.8370838065220393,0.007718666360059245,0.82936514016198,1.65873028032396,affine,0.23126511064546854,0.46253022129093707,1.0,0.23126511064546854,eta_l0_n242_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,243,0,-1.0947451042886664,1.1262257894330194,0.01574034257217649,1.110485446860843,2.220970893721686,affine,0.10266022844570051,0.20532045689140102,0.9319962481713715,0.11015090312554969,eta_l0_n243_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,243,1,-1.0947451042886664,1.1262257894330194,0.01574034257217649,1.110485446860843,2.220970893721686,affine,0.3232996894200661,0.6465993788401322,1.0,0.3232996894200661,eta_l0_n243_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,244,0,-1.3579344396070394,1.4019798358364985,0.022022698114729566,1.379957137721769,2.759914275443538,affine,0.15755628942850283,0.31511257885700567,0.9319962481713715,0.16905249322369809,eta_l0_n244_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,244,1,-1.3579344396070394,1.4019798358364985,0.022022698114729566,1.379957137721769,2.759914275443538,affine,0.3878926045528993,0.7757852091057986,1.0,0.3878926045528993,eta_l0_n244_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,245,0,-1.0757844619013357,1.0586892018922314,-0.008547630004552165,1.0672368318967835,2.134473663793567,affine,0.09429752179835456,0.18859504359670912,0.9319962481713715,0.10117800579494987,eta_l0_n245_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,245,1,-1.0757844619013357,1.0586892018922314,-0.008547630004552165,1.0672368318967835,2.134473663793567,affine,0.31077188650761006,0.6215437730152201,1.0,0.31077188650761006,eta_l0_n245_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,246,0,-0.8996611244567156,0.9223465961082322,0.011342735825758288,0.9110038602824739,1.8220077205649479,affine,0.06610913113535623,0.13221826227071246,0.9319962481713715,0.07093282968152073,eta_l0_n246_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,246,1,-0.8996611244567156,0.9223465961082322,0.011342735825758288,0.9110038602824739,1.8220077205649479,affine,0.2603062842228779,0.5206125684457558,1.0,0.2603062842228779,eta_l0_n246_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,247,0,-0.8285968972008306,0.819679600921353,-0.004458648139738841,0.8241382490610918,1.6482764981221836,affine,0.05207163037664759,0.10414326075329518,0.9319962481713715,0.0558710729563719,eta_l0_n247_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,247,1,-0.8285968972008306,0.819679600921353,-0.004458648139738841,0.8241382490610918,1.6482764981221836,affine,0.22936955208021273,0.45873910416042546,1.0,0.22936955208021273,eta_l0_n247_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,248,0,-0.9630373114550292,0.9343823587700506,-0.01432747634248932,0.9487098351125399,1.8974196702250798,affine,0.07260971125925632,0.14521942251851264,0.9319962481713715,0.0779077291370224,eta_l0_n248_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,248,1,-0.9630373114550292,0.9343823587700506,-0.01432747634248932,0.9487098351125399,1.8974196702250798,affine,0.2731176633837486,0.5462353267674972,1.0,0.2731176633837486,eta_l0_n248_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,249,0,-0.8971474300455686,0.9299772734367955,0.016414921695613427,0.913562351741182,1.827124703482364,affine,0.06656829091883472,0.13313658183766944,0.9319962481713715,0.07142549237665431,eta_l0_n249_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,249,1,-0.8971474300455686,0.9299772734367955,0.016414921695613427,0.913562351741182,1.827124703482364,affine,0.26112809395995745,0.5222561879199149,1.0,0.26112809395995745,eta_l0_n249_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,250,0,-0.726121059105734,0.758922912037253,0.016400926465759524,0.7425219855714935,1.485043971142987,affine,0.04029261137626906,0.08058522275253811,0.9319962481713715,0.043232589675468544,eta_l0_n250_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,250,1,-0.726121059105734,0.758922912037253,0.016400926465759524,0.7425219855714935,1.485043971142987,affine,0.19875206432670653,0.39750412865341306,1.0,0.19875206432670653,eta_l0_n250_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,251,0,-0.6364866647888499,0.6638130918043736,0.013663213507761895,0.6501498782966117,1.3002997565932235,affine,0.028655556908585626,0.05731111381717125,0.9319962481713715,0.030746429467725243,eta_l0_n251_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,251,1,-0.6364866647888499,0.6638130918043736,0.013663213507761895,0.6501498782966117,1.3002997565932235,affine,0.16338437697765587,0.32676875395531174,1.0,0.16338437697765587,eta_l0_n251_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,252,0,-1.0954772645448596,1.134105751171441,0.01931424331329068,1.1147915078581503,2.2295830157163006,affine,0.10351882023537858,0.20703764047075715,0.9319962481713715,0.11107214265989618,eta_l0_n252_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,252,1,-1.0954772645448596,1.134105751171441,0.01931424331329068,1.1147915078581503,2.2295830157163006,affine,0.3244716295031554,0.6489432590063108,1.0,0.3244716295031554,eta_l0_n252_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,253,0,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.04924698734954847,0.09849397469909695,0.9319962481713715,0.052840327894209664,eta_l0_n253_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,253,1,-0.7874311842140017,0.8227512611658138,0.017660038475906026,0.8050912226899077,1.6101824453798155,affine,0.22222072025332532,0.44444144050665063,1.0,0.22222072025332532,eta_l0_n253_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,254,0,-0.9847557236564188,1.0201811473465006,0.017712711845040863,1.0024684355014597,2.0049368710029194,affine,0.08224546002753337,0.16449092005506674,0.9319962481713715,0.08824655698872559,eta_l0_n254_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,254,1,-0.9847557236564188,1.0201811473465006,0.017712711845040863,1.0024684355014597,2.0049368710029194,affine,0.2906685717088255,0.581337143417651,1.0,0.2906685717088255,eta_l0_n254_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,255,0,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.13466273029786982,0.26932546059573964,0.9319962481713715,0.14448848969304928,eta_l0_n255_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,255,1,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,affine,0.36429317679792755,0.7285863535958551,1.0,0.36429317679792755,eta_l0_n255_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,256,0,-0.8345849622272771,0.8628897940747232,0.014152415923723072,0.8487373781510001,1.6974747563020003,affine,0.055936304533085686,0.11187260906617137,0.9319962481713715,0.0600177357396404,eta_l0_n256_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,256,1,-0.8345849622272771,0.8628897940747232,0.014152415923723072,0.8487373781510001,1.6974747563020003,affine,0.23824283895446133,0.47648567790892266,1.0,0.23824283895446133,eta_l0_n256_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,257,0,-0.6351214718940507,0.572660131382451,-0.03123067025579984,0.6038908016382508,1.2077816032765016,affine,0.023764750123252524,0.04752950024650505,0.9319962481713715,0.025498761577506655,eta_l0_n257_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,257,1,-0.6351214718940507,0.572660131382451,-0.03123067025579984,0.6038908016382508,1.2077816032765016,affine,0.14532038190436056,0.2906407638087211,1.0,0.14532038190436056,eta_l0_n257_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,258,0,-0.9998125161825588,0.9675624413404638,-0.01612503742104754,0.9836874787615113,1.9673749575230226,affine,0.07883014658578365,0.1576602931715673,0.9319962481713715,0.0845820428359586,eta_l0_n258_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,258,1,-0.9998125161825588,0.9675624413404638,-0.01612503742104754,0.9836874787615113,1.9673749575230226,affine,0.2846428283577383,0.5692856567154766,1.0,0.2846428283577383,eta_l0_n258_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,259,0,-0.8697080952856971,0.8389415568536381,-0.015383269216029505,0.8543248260696676,1.7086496521393353,affine,0.05682902813904301,0.11365805627808602,0.9319962481713715,0.06097559754188359,eta_l0_n259_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,259,1,-0.8697080952856971,0.8389415568536381,-0.015383269216029505,0.8543248260696676,1.7086496521393353,affine,0.2402419092767963,0.4804838185535926,1.0,0.2402419092767963,eta_l0_n259_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,260,0,-0.8381656949954068,0.8252083496681948,-0.006478672663606022,0.8316870223318008,1.6633740446636016,affine,0.0532387319093588,0.1064774638187176,0.9319962481713715,0.05712333286085234,eta_l0_n260_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,260,1,-0.8381656949954068,0.8252083496681948,-0.006478672663606022,0.8316870223318008,1.6633740446636016,affine,0.2321211134765583,0.4642422269531166,1.0,0.2321211134765583,eta_l0_n260_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,261,0,-0.8336237521159588,0.8706092838419,0.01849276586297055,0.8521165179789294,1.7042330359578588,affine,0.056499013590415766,0.11299802718083153,0.9319962481713715,0.06062150325311928,eta_l0_n261_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,261,1,-0.8336237521159588,0.8706092838419,0.01849276586297055,0.8521165179789294,1.7042330359578588,affine,0.23939911738456962,0.47879823476913924,1.0,0.23939911738456962,eta_l0_n261_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,262,0,-0.9956755857415507,0.9684535524140816,-0.013611016663734543,0.9820645690778161,1.9641291381556323,affine,0.07852507002697835,0.1570501400539567,0.9319962481713715,0.0842547061547178,eta_l0_n262_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,262,1,-0.9956755857415507,0.9684535524140816,-0.013611016663734543,0.9820645690778161,1.9641291381556323,affine,0.2841471210232977,0.5682942420465954,1.0,0.2841471210232977,eta_l0_n262_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,263,0,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.08863715985000005,0.1772743197000001,0.9319962481713715,0.09510463161617988,eta_l0_n263_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,263,1,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,affine,0.30155250881267864,0.6031050176253573,1.0,0.30155250881267864,eta_l0_n263_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,264,0,-0.7417572625689898,0.7667071698008695,0.012474953615939866,0.7542322161849296,1.5084644323698593,affine,0.04187984998861028,0.08375969997722056,0.9319962481713715,0.0449356422526173,eta_l0_n264_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,264,1,-0.7417572625689898,0.7667071698008695,0.012474953615939866,0.7542322161849296,1.5084644323698593,affine,0.20324099935783693,0.40648199871567386,1.0,0.20324099935783693,eta_l0_n264_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,265,0,-0.888567388759357,0.8568782922249831,-0.015844548267186953,0.8727228404921701,1.7454456809843402,affine,0.05978837364507889,0.11957674729015778,0.9319962481713715,0.06415087374266475,eta_l0_n265_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,265,1,-0.888567388759357,0.8568782922249831,-0.015844548267186953,0.8727228404921701,1.7454456809843402,affine,0.2468201722980821,0.4936403445961642,1.0,0.2468201722980821,eta_l0_n265_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,266,0,-1.0979394401052622,1.05892111862414,-0.019509160740561082,1.0784302793647011,2.1568605587294023,affine,0.09648454749320724,0.19296909498641449,0.9319962481713715,0.10352460933454968,eta_l0_n266_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,266,1,-1.0979394401052622,1.05892111862414,-0.019509160740561082,1.0784302793647011,2.1568605587294023,affine,0.31396790847240563,0.6279358169448113,1.0,0.31396790847240563,eta_l0_n266_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,267,0,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.07059108594245113,0.14118217188490226,0.9319962481713715,0.07574181342570291,eta_l0_n267_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,267,1,-0.9212656338216741,0.9528695879992136,0.015801977088769736,0.9370676109104439,1.8741352218208878,affine,0.26917471363069706,0.5383494272613941,1.0,0.26917471363069706,eta_l0_n267_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,268,0,-0.8940710602099512,0.8588320503931356,-0.0176195049084078,0.8764515553015434,1.7529031106030868,affine,0.06040647288144008,0.12081294576288017,0.9319962481713715,0.06481407301795575,eta_l0_n268_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,268,1,-0.8940710602099512,0.8588320503931356,-0.0176195049084078,0.8764515553015434,1.7529031106030868,affine,0.2481183177822283,0.4962366355644566,1.0,0.2481183177822283,eta_l0_n268_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,269,0,-0.8629278062752648,0.8343272647993645,-0.01430027073795015,0.8486275355373146,1.6972550710746293,affine,0.05591978122054115,0.1118395624410823,0.9319962481713715,0.06000000679215058,eta_l0_n269_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,269,1,-0.8629278062752648,0.8343272647993645,-0.01430027073795015,0.8486275355373146,1.6972550710746293,affine,0.23820126653711424,0.4764025330742285,1.0,0.23820126653711424,eta_l0_n269_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,270,0,-0.9706774538813652,1.0031156697628498,0.016219107940742328,0.9868965618221075,1.973793123644215,affine,0.07940926081926343,0.15881852163852686,0.9319962481713715,0.08520341254062859,eta_l0_n270_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,270,1,-0.9706774538813652,1.0031156697628498,0.016219107940742328,0.9868965618221075,1.973793123644215,affine,0.28568270709119464,0.5713654141823893,1.0,0.28568270709119464,eta_l0_n270_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,271,0,-0.973220960516353,0.936785955948806,-0.0182175022837735,0.9550034582325795,1.910006916465159,affine,0.07373664133513287,0.14747328267026574,0.9319962481713715,0.07911688644649403,eta_l0_n271_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,271,1,-0.973220960516353,0.936785955948806,-0.0182175022837735,0.9550034582325795,1.910006916465159,affine,0.27516733078326944,0.5503346615665389,1.0,0.27516733078326944,eta_l0_n271_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,272,0,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.0785284283481333,0.1570568566962666,0.9319962481713715,0.08425830951810208,eta_l0_n272_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,272,1,-0.9651068342533381,0.998866628856511,0.016879897301586455,0.9819867315549246,1.963973463109849,affine,0.2840790752263153,0.5681581504526306,1.0,0.2840790752263153,eta_l0_n272_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,273,0,-0.8018194571083687,0.779530158703183,-0.011144649202592838,0.7906748079057758,1.5813496158115516,affine,0.047074883359198356,0.09414976671839671,0.9319962481713715,0.050509734831617505,eta_l0_n273_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,273,1,-0.8018194571083687,0.779530158703183,-0.011144649202592838,0.7906748079057758,1.5813496158115516,affine,0.21694751241049756,0.4338950248209951,1.0,0.21694751241049756,eta_l0_n273_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,274,0,-1.3704733539073404,1.4030091777593325,0.01626791192599608,1.3867412658333365,2.773482531666673,affine,0.15895094587993705,0.3179018917598741,0.9319962481713715,0.17054891174917028,eta_l0_n274_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,274,1,-1.3704733539073404,1.4030091777593325,0.01626791192599608,1.3867412658333365,2.773482531666673,affine,0.38928688741470735,0.7785737748294147,1.0,0.38928688741470735,eta_l0_n274_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,275,0,-1.0543222479141343,1.0201728471958658,-0.017074700359134276,1.037247547555,2.07449509511,affine,0.08867093565940438,0.17734187131880877,0.9319962481713715,0.09514087190091344,eta_l0_n275_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,275,1,-1.0543222479141343,1.0201728471958658,-0.017074700359134276,1.037247547555,2.07449509511,affine,0.30158764286549894,0.6031752857309979,1.0,0.30158764286549894,eta_l0_n275_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,276,0,-1.1254872684487627,1.1656844838811942,0.020098607716215744,1.1455858761649784,2.291171752329957,affine,0.10957807722918857,0.21915615445837713,0.9319962481713715,0.11757351753742233,eta_l0_n276_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,276,1,-1.1254872684487627,1.1656844838811942,0.020098607716215744,1.1455858761649784,2.291171752329957,affine,0.3330098742256179,0.6660197484512358,1.0,0.3330098742256179,eta_l0_n276_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,277,0,-1.1043859523878043,1.1426379378077018,0.019125992709948747,1.123511945097753,2.247023890195506,affine,0.10522390427298968,0.21044780854597936,0.9319962481713715,0.11290163933539951,eta_l0_n277_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,277,1,-1.1043859523878043,1.1426379378077018,0.019125992709948747,1.123511945097753,2.247023890195506,affine,0.32692761757395195,0.6538552351479039,1.0,0.32692761757395195,eta_l0_n277_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,278,0,-0.7530154612851289,0.7784502098291964,0.012717374272033788,0.7657328355571626,1.5314656711143253,affine,0.04349218747112233,0.08698437494224466,0.9319962481713715,0.04666562505638453,eta_l0_n278_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,278,1,-0.7530154612851289,0.7784502098291964,0.012717374272033788,0.7657328355571626,1.5314656711143253,affine,0.2075806820187526,0.4151613640375052,1.0,0.2075806820187526,eta_l0_n278_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,279,0,-0.9664557004775317,0.9399092435870479,-0.013273228445241925,0.9531824720322898,1.9063649440645796,affine,0.07338862410217874,0.14677724820435747,0.9319962481713715,0.07874347589507072,eta_l0_n279_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,279,1,-0.9664557004775317,0.9399092435870479,-0.013273228445241925,0.9531824720322898,1.9063649440645796,affine,0.2746272259394916,0.5492544518789833,1.0,0.2746272259394916,eta_l0_n279_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,280,0,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.10125806852097143,0.20251613704194285,0.9319962481713715,0.10864643363065613,eta_l0_n280_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,280,1,-1.1218413295624685,1.0845588312120327,-0.018641249175217922,1.1032000803872506,2.206400160774501,affine,0.32118167226104205,0.6423633445220841,1.0,0.32118167226104205,eta_l0_n280_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,281,0,-0.7251348527214968,0.7587825851372446,0.016823866207873905,0.7419587189293707,1.4839174378587414,affine,0.040218829577515165,0.08043765915503033,0.9319962481713715,0.04315342433666095,eta_l0_n281_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,281,1,-0.7251348527214968,0.7587825851372446,0.016823866207873905,0.7419587189293707,1.4839174378587414,affine,0.19853197030781802,0.39706394061563605,1.0,0.19853197030781802,eta_l0_n281_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,282,0,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.1874287668164584,0.3748575336329168,0.9319962481713715,0.20110463661651434,eta_l0_n282_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,282,1,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,affine,0.4132730520833272,0.8265461041666544,1.0,0.4132730520833272,eta_l0_n282_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,283,0,-0.8267574423081063,0.7934742831840174,-0.016641579562044484,0.8101158627460618,1.6202317254921237,affine,0.04999348869782195,0.0999869773956439,0.9319962481713715,0.05364129823045099,eta_l0_n283_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,283,1,-0.8267574423081063,0.7934742831840174,-0.016641579562044484,0.8101158627460618,1.6202317254921237,affine,0.22409451999257093,0.44818903998514187,1.0,0.22409451999257093,eta_l0_n283_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,284,0,-0.8807211491905229,0.9069751074714276,0.013126979140452355,0.8938481283309753,1.7876962566619505,affine,0.06324291874219658,0.12648583748439315,0.9319962481713715,0.06785748211570884,eta_l0_n284_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,284,1,-0.8807211491905229,0.9069751074714276,0.013126979140452355,0.8938481283309753,1.7876962566619505,affine,0.2543141616048612,0.5086283232097224,1.0,0.2543141616048612,eta_l0_n284_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,285,0,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.06959998889061066,0.1391999777812213,0.9319962481713715,0.0746784002909558,eta_l0_n285_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,285,1,-0.9136703395627072,0.9488611758931843,0.017595418165238574,0.9312657577279457,1.8625315154558915,affine,0.2671787061185919,0.5343574122371838,1.0,0.2671787061185919,eta_l0_n285_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,286,0,-1.3222298401611516,1.2810144283174634,-0.020607705921844133,1.3016221342393075,2.603244268478615,affine,0.14126131622843827,0.28252263245687653,0.9319962481713715,0.15156854601679012,eta_l0_n286_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,286,1,-1.3222298401611516,1.2810144283174634,-0.020607705921844133,1.3016221342393075,2.603244268478615,affine,0.3715097252062655,0.743019450412531,1.0,0.3715097252062655,eta_l0_n286_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,287,0,-1.0082309842020034,1.0482187913995948,0.019993903598795715,1.0282248878007991,2.0564497756015983,affine,0.08700633057614827,0.17401266115229655,0.9319962481713715,0.09335480775470882,eta_l0_n287_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,287,1,-1.0082309842020034,1.0482187913995948,0.019993903598795715,1.0282248878007991,2.0564497756015983,affine,0.29874920593166776,0.5974984118633355,1.0,0.29874920593166776,eta_l0_n287_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,288,0,-0.9609248528732854,0.9933100955407532,0.016192621333733892,0.9771174742070193,1.9542349484140387,affine,0.07765047558068246,0.15530095116136491,0.9319962481713715,0.08331629631882854,eta_l0_n288_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,288,1,-0.9609248528732854,0.9933100955407532,0.016192621333733892,0.9771174742070193,1.9542349484140387,affine,0.2825004940550781,0.5650009881101562,1.0,0.2825004940550781,eta_l0_n288_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,289,0,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.12823730959170523,0.25647461918341047,0.9319962481713715,0.13759423371426008,eta_l0_n289_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,289,1,-1.2578650512751817,1.2187445857963901,-0.019560232739395778,1.238304818535786,2.476609637071572,affine,0.35685849404921904,0.7137169880984381,1.0,0.35685849404921904,eta_l0_n289_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,290,0,-1.2209164590474622,1.2650042904501948,0.022043915701366323,1.2429603747488285,2.485920749497657,affine,0.12920111825149752,0.25840223650299504,0.9319962481713715,0.13862836734053094,eta_l0_n290_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,290,1,-1.2209164590474622,1.2650042904501948,0.022043915701366323,1.2429603747488285,2.485920749497657,affine,0.3579459703565347,0.7158919407130694,1.0,0.3579459703565347,eta_l0_n290_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,291,0,-0.8933386502203348,0.9310906530126256,0.018876001396145425,0.9122146516164802,1.8244293032329604,affine,0.0663563654821971,0.1327127309643942,0.9319962481713715,0.07119810365373463,eta_l0_n291_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,291,1,-0.8933386502203348,0.9310906530126256,0.018876001396145425,0.9122146516164802,1.8244293032329604,affine,0.26062423709659494,0.5212484741931899,1.0,0.26062423709659494,eta_l0_n291_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,292,0,-0.8645810082391748,0.8388811245143485,-0.012849941862413172,0.8517310663767617,1.7034621327535233,affine,0.05640316746034286,0.11280633492068572,0.9319962481713715,0.06051866364377433,eta_l0_n292_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,292,1,-0.8645810082391748,0.8388811245143485,-0.012849941862413172,0.8517310663767617,1.7034621327535233,affine,0.23933924698846556,0.47867849397693113,1.0,0.23933924698846556,eta_l0_n292_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,293,0,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.042222681286823445,0.08444536257364689,0.9319962481713715,0.045303488473979046,eta_l0_n293_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,293,1,-0.7696583947245351,0.7436963641755313,-0.012981015274501906,0.7566773794500332,1.5133547589000664,affine,0.20416005507325016,0.4083201101465003,1.0,0.20416005507325016,eta_l0_n293_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,294,0,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.09129259489700804,0.18258518979401608,0.9319962481713715,0.09795382232077564,eta_l0_n294_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,294,1,-1.0343795818103405,1.068042105369411,0.0168312617795352,1.0512108435898757,2.1024216871797514,affine,0.30586246083893254,0.6117249216778651,1.0,0.30586246083893254,eta_l0_n294_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,295,0,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.16854344612361224,0.3370868922472245,0.9319962481713715,0.1808413354177164,eta_l0_n295_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,295,1,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,affine,0.39794718678390684,0.7958943735678137,1.0,0.39794718678390684,eta_l0_n295_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,296,0,-0.7369816136118259,0.7101516754046739,-0.01341496910357598,0.7235666445082499,1.4471332890164998,affine,0.03773032047057046,0.07546064094114092,0.9319962481713715,0.04048333943897248,eta_l0_n296_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,296,1,-0.7369816136118259,0.7101516754046739,-0.01341496910357598,0.7235666445082499,1.4471332890164998,affine,0.19157267510947115,0.3831453502189423,1.0,0.19157267510947115,eta_l0_n296_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,297,0,-0.9825135379376001,1.0138240898880801,0.015655275975240024,0.9981688139128402,1.9963376278256804,affine,0.08144974624643898,0.16289949249287797,0.9319962481713715,0.08739278340041381,eta_l0_n297_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,297,1,-0.9825135379376001,1.0138240898880801,0.015655275975240024,0.9981688139128402,1.9963376278256804,affine,0.2893225528047419,0.5786451056094838,1.0,0.2893225528047419,eta_l0_n297_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,298,0,-0.8332058737903767,0.8548558364943839,0.010824981352003604,0.8440308551423803,1.6880617102847606,affine,0.055178454356791036,0.11035690871358207,0.9319962481713715,0.05920458850028016,eta_l0_n298_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,298,1,-0.8332058737903767,0.8548558364943839,0.010824981352003604,0.8440308551423803,1.6880617102847606,affine,0.23657700326353934,0.4731540065270787,1.0,0.23657700326353934,eta_l0_n298_r1,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,299,0,-0.742839048056372,0.7269071605470812,-0.007965943754645433,0.7348731043017266,1.4697462086034532,affine,0.03921345041839409,0.07842690083678817,0.9319962481713715,0.042074686990782484,eta_l0_n299_r0,NA,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,0,299,1,-0.742839048056372,0.7269071605470812,-0.007965943754645433,0.7348731043017266,1.4697462086034532,affine,0.1959338008418963,0.3918676016837926,1.0,0.1959338008418963,eta_l0_n299_r1,NA,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/benchmark_metadata.json b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/benchmark_metadata.json new file mode 100644 index 0000000..166909e --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/benchmark_metadata.json @@ -0,0 +1,13 @@ +{ + "schema_version": "1.2", + "benchmark_level": "medium", + "problem_id": "poisson_100d_ridge", + "model_id": "pinn_100d_poisson_shallow_300_seed_20260804", + "method_id": "hybrid_pz_uncompressed_reverse_symbolic", + "git_commit": "fb576a55aed822c24764f8d4db93e38495445e37", + "dtype": "float64", + "device": "cpu", + "quantile_interpolation": "linear", + "cell_average": "volume_weighted", + "timestamp_utc": "2026-08-04T11:32:53.179489+00:00" +} diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/cell_intervals.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/cell_intervals.csv new file mode 100644 index 0000000..4196993 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/cell_intervals.csv @@ -0,0 +1,102 @@ +run_id,split_id,cell_id,cell_weight,quantity,output_index,input_index_a,input_index_b,lower,upper,midpoint,radius,width,magnitude,mignitude,local_relative_radius,global_normalized_radius,sign_certified,status,local_relative_width +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,0,NA,-0.5272538421472067,0.8113414367336492,0.14204379729322125,0.669297639440428,1.338595278880856,0.8113414367336492,0.0,0.824927224393875,0.6841121914419067,0,ok,1.64985444878775 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,1,NA,-0.5663608647526251,0.8288645655721248,0.13125185040974985,0.697612715162375,1.39522543032475,0.8288645655721248,0.0,0.8416486168410963,0.7130540065052017,0,ok,1.6832972336821925 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,2,NA,-0.46483715676336507,0.7170338114646269,0.1260983273506309,0.590935484113996,1.181870968227992,0.7170338114646269,0.0,0.8241389383116258,0.6040155309317989,0,ok,1.6482778766232515 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,3,NA,-0.5412675716244109,0.8126447086457349,0.13568856851066202,0.6769561401350729,1.3539122802701458,0.8126447086457349,0.0,0.8330284230401428,0.6919402090302449,0,ok,1.6660568460802856 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,4,NA,-0.48964037368015495,0.7546070329923743,0.13248332965610968,0.6221237033362647,1.2442474066725293,0.7546070329923743,0.0,0.8244340115268334,0.6358940850189688,0,ok,1.6488680230536668 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,5,NA,-0.5579795466454754,0.8391015858354322,0.14056101959497835,0.6985405662404538,1.3970811324809076,0.8391015858354322,0.0,0.8324862901372877,0.7140023950799573,0,ok,1.6649725802745754 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,6,NA,-0.5434173401158277,0.7959725062783635,0.1262775830812679,0.6696949231970956,1.3393898463941911,0.7959725062783635,0.0,0.8413543406521798,0.6845182688660335,0,ok,1.6827086813043597 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,7,NA,-0.6748002540119225,0.937821460190289,0.13151060308918328,0.8063108571011057,1.6126217142022115,0.937821460190289,0.0,0.8597701069215252,0.8241581247709395,0,ok,1.7195402138430504 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,8,NA,-0.5483443497560938,0.8202229440865556,0.1359392971652309,0.6842836469213247,1.3685672938426494,0.8202229440865556,0.0,0.8342654297282305,0.6994299063337334,0,ok,1.668530859456461 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,9,NA,-0.5064375729973408,0.7842765125167536,0.13891946975970637,0.6453570427570472,1.2907140855140944,0.7842765125167536,0.0,0.8228692718159926,0.6596416822149692,0,ok,1.6457385436319851 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,10,NA,-0.5768042355554577,0.856688710870182,0.13994223765736213,0.7167464732128198,1.4334929464256396,0.856688710870182,0.0,0.8366475058189856,0.732611280248693,0,ok,1.6732950116379712 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,11,NA,-0.5668718072339385,0.8455585199267361,0.13934335634639883,0.7062151635803373,1.4124303271606746,0.8455585199267361,0.0,0.8352055439539864,0.7218468656043289,0,ok,1.6704110879079728 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,12,NA,-0.5302846772540564,0.7996953608232202,0.13470534178458193,0.6649900190386383,1.3299800380772766,0.7996953608232202,0.0,0.8315541787738847,0.6797092241231635,0,ok,1.6631083575477694 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,13,NA,-0.48144569329211645,0.7638894737842583,0.14122189024607093,0.6226675835381874,1.2453351670763748,0.7638894737842583,0.0,0.815127848867367,0.6364500037237327,0,ok,1.630255697734734 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,14,NA,-0.6069882704461806,0.8549128194792248,0.12396227451652209,0.7309505449627027,1.4619010899254055,0.8549128194792248,0.0,0.855000098615863,0.7471297516724599,0,ok,1.710000197231726 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,15,NA,-0.5376437395931263,0.8069440786199519,0.1346501695134128,0.6722939091065391,1.3445878182130782,0.8069440786199519,0.0,0.8331356867458602,0.6871747819646343,0,ok,1.6662713734917205 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,16,NA,-0.5924006453494112,0.8547488394753252,0.13117409706295702,0.7235747424123682,1.4471494848247364,0.8547488394753252,0.0,0.846534922301297,0.7395906896034964,0,ok,1.693069844602594 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,17,NA,-0.5052755390061582,0.767776524664667,0.13125049282925438,0.6365260318354126,1.2730520636708251,0.767776524664667,0.0,0.8290511775069195,0.6506152015011005,0,ok,1.658102355013839 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,18,NA,-0.5722460114848205,0.8117798019377576,0.11976689522646855,0.692012906711289,1.384025813422578,0.8117798019377576,0.0,0.8524638147677742,0.7073302492328315,0,ok,1.7049276295355484 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,19,NA,-0.4622768057194338,0.7236191838716167,0.13067118907609146,0.5929479947955253,1.1858959895910506,0.7236191838716167,0.0,0.8194199490718934,0.6060725874810979,0,ok,1.6388398981437868 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,20,NA,-0.502282490943987,0.7561033057672936,0.12691040741165327,0.6291928983556403,1.2583857967112806,0.7561033057672936,0.0,0.8321520267883704,0.6431197529601834,0,ok,1.6643040535767408 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,21,NA,-0.630006474079431,0.8962600750351252,0.13312680047784708,0.7631332745572781,1.5262665491145562,0.8962600750351252,0.0,0.8514640959850523,0.7800248290971082,0,ok,1.7029281919701047 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,22,NA,-0.5652372995168381,0.8085787431569992,0.12167072182008054,0.6869080213369186,1.3738160426738373,0.8085787431569992,0.0,0.8495252035132265,0.7021123698997442,0,ok,1.699050407026453 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,23,NA,-0.4875419046618377,0.7540019218219639,0.1332300085800631,0.6207719132419008,1.2415438264838017,0.7540019218219639,0.0,0.823302826260539,0.634512373760286,0,ok,1.646605652521078 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,24,NA,-0.5383736523965369,0.792557030860921,0.12709168923219205,0.665465341628729,1.330930683257458,0.792557030860921,0.0,0.8396434776508919,0.6801950677293364,0,ok,1.6792869553017837 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,25,NA,-0.584436743073778,0.8433847944126911,0.12947402566945654,0.7139107687432346,1.4278215374864691,0.8433847944126911,0.0,0.8464828551247257,0.7297128089488517,0,ok,1.6929657102494513 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,26,NA,-0.6010952517485425,0.8601444993074262,0.12952462377944185,0.7306198755279844,1.4612397510559687,0.8601444993074262,0.0,0.8494152739641619,0.7467917630433394,0,ok,1.6988305479283239 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,27,NA,-0.5232774413891257,0.7948888331530439,0.13580569588195912,0.6590831372710848,1.3181662745421696,0.7948888331530439,0.0,0.8291513351077461,0.6736715966276179,0,ok,1.6583026702154922 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,28,NA,-0.47536461881495057,0.7249567042626736,0.1247960427238615,0.6001606615388121,1.2003213230776242,0.7249567042626736,0.0,0.827857247212043,0.6134449028175384,0,ok,1.655714494424086 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,29,NA,-0.5658189144281208,0.8448718022817442,0.13952644392681168,0.7053453583549325,1.410690716709865,0.8448718022817442,0.0,0.8348548933104493,0.7209578076967337,0,ok,1.6697097866208985 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,30,NA,-0.6991298691308708,0.9783448501769073,0.13960749052301824,0.8387373596538891,1.6774747193077781,0.9783448501769073,0.0,0.8573023709402937,0.8573023709402937,0,ok,1.7146047418805874 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,31,NA,-0.5293330060333251,0.7980067165851962,0.13433685527593553,0.6636698613092606,1.3273397226185213,0.7980067165851962,0.0,0.8316594929792254,0.6783598453952651,0,ok,1.6633189859584507 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,32,NA,-0.5516602138683092,0.8069481055270312,0.127643945829361,0.6793041596976702,1.3586083193953404,0.8069481055270312,0.0,0.8418188915060472,0.6943402007736192,0,ok,1.6836377830120943 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,33,NA,-0.5815299168620396,0.8572519680706034,0.13786102560428193,0.7193909424663215,1.438781884932643,0.8572519680706034,0.0,0.8391826082189552,0.735314283441303,0,ok,1.6783652164379104 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,34,NA,-0.4970698377923927,0.7765229645455209,0.1397265633765641,0.6367964011689569,1.2735928023379137,0.7765229645455209,0.0,0.8200612605728377,0.650891555317953,0,ok,1.6401225211456754 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,35,NA,-0.5901737207986902,0.8418051180826555,0.1258156986419826,0.7159894194406728,1.4319788388813457,0.8418051180826555,0.0,0.8505405871984387,0.7318374694886015,0,ok,1.7010811743968774 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,36,NA,-0.5204179354486919,0.7869631432120799,0.13327260388169404,0.6536905393303859,1.3073810786607718,0.7869631432120799,0.0,0.8306494973351272,0.6681596363615382,0,ok,1.6612989946702543 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,37,NA,-0.5966743050272215,0.8857533182722327,0.1445395066225056,0.7412138116497271,1.4824276232994542,0.8857533182722327,0.0,0.836817425754106,0.7576201903814371,0,ok,1.673634851508212 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,38,NA,-0.5156017410727135,0.7815370878736497,0.1329676734004681,0.6485694144731816,1.2971388289463632,0.7815370878736497,0.0,0.8298638983823058,0.662925158093187,0,ok,1.6597277967646116 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,39,NA,-0.4454926678669671,0.7216642801423404,0.13808580613768665,0.5835784740046538,1.1671569480093076,0.7216642801423404,0.0,0.8086564487985318,0.5964956772646469,0,ok,1.6173128975970636 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,40,NA,-0.539813167678067,0.8010698247859344,0.1306283285539337,0.6704414962320007,1.3408829924640013,0.8010698247859344,0.0,0.8369326561653464,0.6852813669032646,0,ok,1.6738653123306928 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,41,NA,-0.5240478513217143,0.8130940669773855,0.14452310782783562,0.6685709591495499,1.3371419182990998,0.8130940669773855,0.0,0.8222553654030595,0.683369426464152,0,ok,1.644510730806119 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,42,NA,-0.6117033270234389,0.8750037380244289,0.131650205500495,0.7433535325239339,1.4867070650478678,0.8750037380244289,0.0,0.8495432650405209,0.7598072728542685,0,ok,1.6990865300810418 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,43,NA,-0.594836064355267,0.8748039503687897,0.1399839430067613,0.7348200073620283,1.4696400147240567,0.8748039503687897,0.0,0.8399824978525205,0.7510848625912999,0,ok,1.679964995705041 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,44,NA,-0.4746279735788499,0.7554900193382158,0.14043102287968295,0.6150589964585329,1.2301179929170658,0.7554900193382158,0.0,0.8141192877667718,0.6286730045620581,0,ok,1.6282385755335436 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,45,NA,-0.5232206318406037,0.8109453194563153,0.14386234380785579,0.6670829756484595,1.334165951296919,0.8109453194563153,0.0,0.8225992056969934,0.681848507229159,0,ok,1.6451984113939868 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,46,NA,-0.63285072348017,0.9076589931662119,0.13740413484302094,0.770254858323191,1.540509716646382,0.9076589931662119,0.0,0.8486170071827193,0.7873040453822711,0,ok,1.6972340143654385 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,47,NA,-0.5030292296076135,0.7779397595129807,0.13745526495268356,0.6404844945602971,1.2809689891205942,0.7779397595129807,0.0,0.8233086003487781,0.6546612827209983,0,ok,1.6466172006975561 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,48,NA,-0.5307506249047741,0.801196327511403,0.13522285130331446,0.6659734762080886,1.331946952416177,0.801196327511403,0.0,0.8312238253471151,0.680714449600941,0,ok,1.6624476506942303 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,49,NA,-0.5585299467375412,0.8277387445187735,0.13460439889061615,0.6931343456281573,1.3862686912563147,0.8277387445187735,0.0,0.8373829909715392,0.7084765106115933,0,ok,1.6747659819430785 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,50,NA,-0.49811031100783615,0.7821612980994519,0.14202549354580787,0.640135804553644,1.280271609107288,0.7821612980994519,0.0,0.8184191753147196,0.6543048746440406,0,ok,1.6368383506294393 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,51,NA,-0.49085080011803106,0.7521378665655106,0.13064353322373978,0.6214943333417708,1.2429886666835417,0.7521378665655106,0.0,0.8263037415995318,0.6352507842499404,0,ok,1.6526074831990636 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,52,NA,-0.5809956938180829,0.8425447839955518,0.13077454508873443,0.7117702389068173,1.4235404778136347,0.8425447839955518,0.0,0.8447862385800197,0.7275248996078559,0,ok,1.6895724771600393 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,53,NA,-0.49329042779999227,0.7636684007428929,0.1351889864714503,0.6284794142714426,1.2569588285428852,0.7636684007428929,0.0,0.8229742302549914,0.6423904762802186,0,ok,1.6459484605099828 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,54,NA,-0.5386035604854037,0.809238511990913,0.13531747575275466,0.6739210362381584,1.3478420724763167,0.809238511990913,0.0,0.8327841869267412,0.6888379247013953,0,ok,1.6655683738534823 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,55,NA,-0.5159916086967601,0.7827654731252988,0.13338693221426934,0.6493785409110294,1.2987570818220588,0.7827654731252988,0.0,0.829595278798254,0.6637521941201069,0,ok,1.659190557596508 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,56,NA,-0.5317375710496752,0.7981787586517457,0.13322059380103524,0.6649581648507105,1.329916329701421,0.7981787586517457,0.0,0.8330942882693764,0.679676664859503,0,ok,1.6661885765387527 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,57,NA,-0.4736611366724964,0.7644831975864347,0.14541103045696915,0.6190721671294656,1.2381443342589311,0.7644831975864347,0.0,0.8097917247677264,0.6327750046596791,0,ok,1.6195834495354529 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,58,NA,-0.5174926943431536,0.7836504145042223,0.13307886008053438,0.650571554423688,1.301143108847376,0.7836504145042223,0.0,0.8301808336760379,0.6649716143607743,0,ok,1.6603616673520758 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,59,NA,-0.47084207706099185,0.7438143499095412,0.13648613642427468,0.6073282134852666,1.214656426970533,0.7438143499095412,0.0,0.8165051044781895,0.6207711047647949,0,ok,1.633010208956379 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,60,NA,-0.632495237576023,0.9041003738068182,0.1358025681153976,0.7682978056914206,1.5365956113828412,0.9041003738068182,0.0,0.8497925982005899,0.785303674417558,0,ok,1.6995851964011799 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,61,NA,-0.6376875534862348,0.9026636505807056,0.1324880485472354,0.7701756020334702,1.5403512040669405,0.9026636505807056,0.0,0.8532254528450297,0.7872230347961711,0,ok,1.7064509056900594 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,62,NA,-0.49811690331836594,0.7595243183300916,0.13070370750586283,0.6288206108242288,1.2576412216484576,0.7595243183300916,0.0,0.8279137292230075,0.6427392250396408,0,ok,1.655827458446015 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,63,NA,-0.4414528785442063,0.6940331193637247,0.1262901204097592,0.5677429989539655,1.135485997907931,0.6940331193637247,0.0,0.8180344469359915,0.5803096922841721,0,ok,1.636068893871983 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,64,NA,-0.5261381707506022,0.7829535581801707,0.12840769371478422,0.6545458644653864,1.3090917289307729,0.7829535581801707,0.0,0.8359957721972119,0.6690338936695271,0,ok,1.6719915443944238 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,65,NA,-0.5647360235063547,0.8367483495428917,0.1360061630182685,0.7007421865246232,1.4014843730492463,0.8367483495428917,0.0,0.8374587017798512,0.716252747073706,0,ok,1.6749174035597023 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,66,NA,-0.5298130481090955,0.8031579271559026,0.13667243952340358,0.666485487632499,1.332970975264998,0.8031579271559026,0.0,0.829831176531644,0.6812377941295271,0,ok,1.659662353063288 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,67,NA,-0.5549878586118452,0.8381480805252773,0.14158011095671608,0.6965679695685613,1.3931359391371225,0.8381480805252773,0.0,0.8310798363124734,0.7119861360159515,0,ok,1.6621596726249468 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,68,NA,-0.49328859770322847,0.7610814891123326,0.1338964457045521,0.6271850434077806,1.2543700868155612,0.7610814891123326,0.0,0.8240708155171155,0.6410674551967755,0,ok,1.648141631034231 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,69,NA,-0.5446384964560594,0.8215960304899569,0.13847876701694872,0.6831172634730082,1.3662345269460163,0.8215960304899569,0.0,0.8314515140313333,0.6982377055999066,0,ok,1.6629030280626667 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,70,NA,-0.5681413030158012,0.8310310342197312,0.131444865601965,0.6995861686177662,1.3991723372355325,0.8310310342197312,0.0,0.8418291734130233,0.7150711413171592,0,ok,1.6836583468260466 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,71,NA,-0.5153644363492513,0.7726556449190602,0.12864560428490446,0.6440100406341558,1.2880200812683116,0.7726556449190602,0.0,0.8335020197796125,0.6582648649069947,0,ok,1.667004039559225 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,72,NA,-0.5149559996109415,0.7680282786257747,0.1265361395074166,0.6414921391183581,1.2829842782367162,0.7680282786257747,0.0,0.8352454681306443,0.65569123096254,0,ok,1.6704909362612885 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,73,NA,-0.4820442053816752,0.7612712066524926,0.1396135006354087,0.6216577060170839,1.2433154120341678,0.7612712066524926,0.0,0.8166047797219003,0.6354177730936835,0,ok,1.6332095594438005 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,74,NA,-0.6079070213128196,0.9008459260861726,0.14646945238667652,0.7543764736994961,1.5087529473989922,0.9008459260861726,0.0,0.8374089862147353,0.7710742010479101,0,ok,1.6748179724294705 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,75,NA,-0.5170731072741043,0.7990951862495507,0.1410110394877232,0.6580841467618275,1.316168293523655,0.7990951862495507,0.0,0.8235366175216996,0.6726504939877086,0,ok,1.6470732350433992 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,76,NA,-0.47890290587430756,0.7523978401651547,0.13674746714542355,0.6156503730197311,1.2313007460394623,0.7523978401651547,0.0,0.8182511168354673,0.629277470933084,0,ok,1.6365022336709345 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,77,NA,-0.5403191011022396,0.8235058228268702,0.14159336086231533,0.6819124619645549,1.3638249239291098,0.8235058228268702,0.0,0.8280602796756626,0.6970062364423438,0,ok,1.6561205593513253 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,78,NA,-0.619402934069117,0.8984045861826007,0.13950082605674186,0.7589037601258588,1.5178075202517176,0.8984045861826007,0.0,0.8447238268790535,0.7757016965833995,0,ok,1.689447653758107 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,79,NA,-0.5787578812877671,0.8658319144229886,0.14353701656761075,0.7222948978553778,1.4445897957107556,0.8658319144229886,0.0,0.8342206909024977,0.7382825163588996,0,ok,1.6684413818049955 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,80,NA,-0.605434722177461,0.8463914687926568,0.12047837330759792,0.7259130954850589,1.4518261909701178,0.8463914687926568,0.0,0.8576564417887441,0.7419808008942829,0,ok,1.7153128835774882 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,81,NA,-0.5601108539474196,0.8362075932330983,0.1380483696428394,0.698159223590259,1.396318447180518,0.8362075932330983,0.0,0.8349113655987126,0.7136126116103291,0,ok,1.6698227311974252 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,82,NA,-0.5241928461574229,0.7846140322195646,0.13021059303107085,0.6544034391884938,1.3088068783769875,0.7846140322195646,0.0,0.8340450365605582,0.6688883158838752,0,ok,1.6680900731211163 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,83,NA,-0.5568863849689263,0.8019417277470615,0.12252767138906762,0.6794140563579939,1.3588281127159878,0.8019417277470615,0.0,0.8472112534494355,0.6944525299388453,0,ok,1.694422506898871 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,84,NA,-0.5347321691744481,0.7938279250476271,0.1295478779365895,0.6642800471110376,1.3285600942220752,0.7938279250476271,0.0,0.8368060963226797,0.6789835373395388,0,ok,1.6736121926453593 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,85,NA,-0.6277294427627641,0.9089323334926601,0.140601445364948,0.7683308881277121,1.5366617762554242,0.9089323334926601,0.0,0.8453114272823001,0.7853374891162151,0,ok,1.6906228545646003 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,86,NA,-0.5037223169843337,0.7759119754371742,0.13609482922642024,0.6398171462107539,1.2796342924215078,0.7759119754371742,0.0,0.8246001691754532,0.6539791629659626,0,ok,1.6492003383509064 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,87,NA,-0.5299052398339988,0.7866288998343746,0.12836183000018786,0.6582670698341867,1.3165341396683734,0.7866288998343746,0.0,0.8368203481626284,0.6728374659662765,0,ok,1.6736406963252568 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,88,NA,-0.540136365285783,0.8253002698491435,0.14258195228168025,0.6827183175674633,1.3654366351349265,0.8253002698491435,0.0,0.8272362708572205,0.6978299292361094,0,ok,1.654472541714441 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,89,NA,-0.5164160032885852,0.7999190497942996,0.14175152325285723,0.6581675265414424,1.3163350530828848,0.7999190497942996,0.0,0.8227926647211254,0.6727357193349875,0,ok,1.6455853294422509 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,90,NA,-0.436318621129639,0.7027226223886369,0.13320200062949897,0.569520621759138,1.139041243518276,0.7027226223886369,0.0,0.8104486800542586,0.5821266618372403,0,ok,1.6208973601085173 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,91,NA,-0.6100670712758738,0.8638089713776858,0.12687095005090598,0.7369380213267798,1.4738760426535595,0.8638089713776858,0.0,0.8531261491200306,0.753249757683627,0,ok,1.7062522982400612 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,92,NA,-0.4940820746537219,0.7717977780074216,0.13885785167684983,0.6329399263305717,1.2658798526611434,0.7717977780074216,0.0,0.8200851885900161,0.6469497194328989,0,ok,1.6401703771800322 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,93,NA,-0.5117285030495521,0.7789362073330882,0.13360385214176806,0.6453323551913202,1.2906647103826403,0.7789362073330882,0.0,0.8284790835449809,0.6596164482029309,0,ok,1.6569581670899618 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,94,NA,-0.5147296182111293,0.7889598228664766,0.13711510232767365,0.6518447205388029,1.3036894410776059,0.7889598228664766,0.0,0.8262077505677003,0.6662729613396896,0,ok,1.6524155011354007 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,95,NA,-0.5090520784048606,0.7511394697103949,0.12104369565276718,0.6300957740576277,1.2601915481152555,0.7511394697103949,0.0,0.8388532349399292,0.644042613342005,0,ok,1.6777064698798585 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,96,NA,-0.48694996210354824,0.7590388437263919,0.1360444408114218,0.6229944029149701,1.2459888058299402,0.7590388437263919,0.0,0.8207674851743672,0.6367840570759057,0,ok,1.6415349703487343 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,97,NA,-0.5872859604881931,0.8691418741390008,0.14092795682540382,0.728213917313597,1.456427834627194,0.8691418741390008,0.0,0.8378539096795774,0.7443325502065239,0,ok,1.6757078193591548 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,98,NA,-0.6191320320679213,0.8881319107747324,0.13449993935340554,0.7536319714213269,1.5072639428426537,0.8881319107747324,0.0,0.8485586006744438,0.7703132196025285,0,ok,1.6971172013488875 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,J,0,99,NA,-0.5857154006941845,0.8448701776711862,0.12957738848850087,0.7152927891826854,1.4305855783653707,0.8448701776711862,0.0,0.846630415047114,0.7311254196854452,0,ok,1.693260830094228 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,0,1.0,Y,0,NA,NA,-2.99139499824489,3.5064399157854753,0.2575224587702927,3.2489174570151826,6.497834914030365,3.5064399157854753,0.0,0.9265572874610044,0.9265572874610044,0,ok,1.8531145749220088 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/complexity.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/complexity.csv new file mode 100644 index 0000000..817a00e --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/complexity.csv @@ -0,0 +1,4 @@ +run_id,quantity,n_alpha,n_eta,n_monomials,n_mixed_monomials,max_degree,n_coefficients,status +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,Y,100.0,600.0,400.0,0.0,1.0,400.0,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,J,100.0,600.0,5450.0,0.0,2.0,545000.0,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,H,NA,NA,NA,NA,NA,NA,not_implemented diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/layer_normalized_radius_Y.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/layer_normalized_radius_Y.csv new file mode 100644 index 0000000..d064387 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/layer_normalized_radius_Y.csv @@ -0,0 +1,301 @@ +neuron,layer_0 +0,0.6162421199489911 +1,0.5419218285316451 +2,0.6525327778653724 +3,0.637024294710363 +4,0.7427237147703024 +5,0.6646561232806381 +6,0.6281257110189709 +7,0.543776295629057 +8,0.8657529873940282 +9,0.651544860521241 +10,0.6165398200770281 +11,0.6251195724642008 +12,0.6912848856671185 +13,0.685035917252616 +14,0.8453483334854439 +15,0.6454932120893029 +16,0.9546592428300092 +17,0.6844219675544602 +18,0.9989769533006161 +19,0.7132754622721597 +20,0.6544510118719792 +21,0.7250216825362106 +22,0.7881857569830729 +23,0.7751149638665825 +24,0.859768585369206 +25,0.830398867531548 +26,0.7610666779300438 +27,0.7755950750012318 +28,0.6479592660850625 +29,0.8336659398768762 +30,0.577638923433512 +31,0.7251830806294721 +32,0.8600723383926094 +33,0.6476726254761521 +34,0.6909176216103874 +35,0.7047202246124595 +36,0.8552648324434805 +37,0.7914027620828362 +38,0.5933413914154003 +39,0.7468845735772127 +40,0.7235186720351032 +41,0.6355710553440279 +42,0.626288058482591 +43,0.9528482053020632 +44,0.8615678731924009 +45,0.5368203060683006 +46,0.6688985438254108 +47,0.6443708874499796 +48,0.7467942927395395 +49,0.6385533374042454 +50,0.8230485319677043 +51,0.6296994068635823 +52,0.5916061892419623 +53,0.7685769031707625 +54,0.8246169214452626 +55,0.54171952478903 +56,0.5591547684566563 +57,0.834630628282293 +58,0.5705099526169742 +59,0.6983825094353423 +60,0.9711665349434576 +61,0.7925580496557257 +62,0.7739003345523898 +63,0.9033425122231032 +64,0.6596971791975124 +65,0.6517398863272531 +66,0.7201117129977588 +67,0.6761713582560301 +68,0.7714366430971834 +69,0.9099648233044394 +70,0.704484787216771 +71,0.9159053838275886 +72,0.8244760671147741 +73,0.8255080588076058 +74,0.6846305582944902 +75,0.675608962006266 +76,0.8131197354843077 +77,0.8903252064464775 +78,0.5109893040753523 +79,0.6470779993909104 +80,0.8507588073142239 +81,0.48828299860482227 +82,0.6098286677909557 +83,0.6681325395936913 +84,0.6265530750327327 +85,0.6889388762532255 +86,0.7968470016124662 +87,0.8531743537692537 +88,0.6404709513492913 +89,0.6633758900956943 +90,0.6230628600687211 +91,0.8603494246601253 +92,0.7854578489955437 +93,0.5279852281313057 +94,0.8334946520781183 +95,0.6553095660427952 +96,0.5221521700057307 +97,0.8275395025216328 +98,0.8675218865905903 +99,0.6754901047331782 +100,0.9290957181679524 +101,0.9718747288736591 +102,0.7544446448407093 +103,0.7963318065887969 +104,0.6624648458375416 +105,0.8281177477259958 +106,0.5729591361537771 +107,0.48646887613813905 +108,0.764051344262394 +109,0.6952094840415755 +110,0.8484577743164452 +111,0.701198157325423 +112,0.6221638583532983 +113,0.6555523479516163 +114,0.5238569605481382 +115,0.5803652319175282 +116,0.6733924570906795 +117,0.9072138935425135 +118,0.5538870821400664 +119,0.8060555734457937 +120,0.5878475917970294 +121,0.5408231103857551 +122,0.6853393974816521 +123,0.7938620192397763 +124,0.5990550539427488 +125,0.9022322562957132 +126,0.8136390896551343 +127,0.8052346460926463 +128,0.6983654751243557 +129,0.6321577929598001 +130,0.7496539427504885 +131,0.7495822560561621 +132,0.7726884769714508 +133,0.8440855668704697 +134,0.7295759818673087 +135,0.8732070662815837 +136,0.6468853179208286 +137,0.806865572195593 +138,0.9320892519043357 +139,0.8131811376887522 +140,0.7411601804149776 +141,0.6620114115895328 +142,0.785938628164867 +143,0.6325580465007923 +144,0.7624114006192217 +145,0.6740164849680114 +146,0.8796752345182725 +147,0.7017082917790937 +148,0.9152976338348483 +149,0.6435628091587109 +150,0.6353369849744391 +151,0.5975395068039396 +152,0.8684341609763699 +153,0.5598472710329968 +154,0.6066249584541327 +155,0.6646263568977663 +156,0.7054739897318261 +157,0.8240778609863428 +158,0.6821186903787475 +159,0.78845386314625 +160,0.9367525217150431 +161,0.6182182494715668 +162,0.7231220561650594 +163,0.6456272556641447 +164,0.799794685950358 +165,0.7191218970415708 +166,0.8173451869250589 +167,0.7762967129222806 +168,0.7577494022439597 +169,0.6576045490629244 +170,0.6560185667358831 +171,0.801364092260314 +172,0.5199058524673331 +173,0.5296623781354116 +174,0.8489691556477169 +175,0.9117735772737228 +176,0.7521245822606443 +177,0.6476925002821613 +178,0.7563424348594338 +179,0.8367961820471885 +180,0.7736008406491968 +181,0.8019484928702707 +182,0.693601603310003 +183,0.6887513360557742 +184,0.931283851277937 +185,0.5784294415472628 +186,0.6681619265262403 +187,0.7630300157898852 +188,0.6431987744455856 +189,0.7699110668821418 +190,0.6370287818625128 +191,0.719251344513818 +192,0.4914141614607233 +193,0.5709855737214663 +194,0.7029058056270725 +195,0.932253345899195 +196,0.816999899919923 +197,0.49713666100544623 +198,0.6821086174382271 +199,0.7900017819271218 +200,0.7292898027159896 +201,0.7910902417132392 +202,0.6220698339112714 +203,0.6847364428358164 +204,0.5873884707760092 +205,0.7293810867623591 +206,0.86752801895761 +207,0.8687592485001452 +208,0.8212440643846498 +209,0.8352457761128697 +210,0.6987361037936566 +211,0.8723927041740711 +212,0.7798132270081817 +213,0.9038894681178925 +214,0.49108414803207956 +215,0.5296268453931839 +216,0.8563636346631119 +217,0.7323919388719169 +218,0.7306481333777458 +219,0.7526980757677582 +220,0.7368032693234934 +221,0.6980074007702309 +222,0.7887509191444781 +223,0.5676430654113679 +224,0.8630306925047158 +225,0.6311002366227472 +226,0.741103947737733 +227,0.8481680525951041 +228,0.7382943637762912 +229,0.6008984291499322 +230,0.9293008484031721 +231,0.7300032302497331 +232,0.738678469054186 +233,0.7652617361979039 +234,0.7185404049956695 +235,0.7264295487074044 +236,0.8172424721389996 +237,0.6867625929112331 +238,0.8597439029896542 +239,0.7084514124737799 +240,0.8097671992413316 +241,0.6293817962278792 +242,0.6368686413963487 +243,0.7878994162668858 +244,0.902222207242424 +245,0.7669338079892857 +246,0.6843791934140323 +247,0.6337265624309671 +248,0.7052846827158022 +249,0.6857994352130584 +250,0.5828781625114009 +251,0.5216217649225898 +252,0.7899318569568655 +253,0.6221052967589766 +254,0.7339873002011027 +255,0.8586812771175738 +256,0.6484040726146777 +257,0.48934342656612595 +258,0.7241077021581477 +259,0.6516994095776407 +260,0.6382654051471263 +261,0.65038165873561 +262,0.7232558861886125 +263,0.7518161929425176 +264,0.590377345136714 +265,0.6624628318580533 +266,0.7724032715998305 +267,0.6988892946404283 +268,0.66461720593681 +269,0.648338441739094 +270,0.7258076409702059 +271,0.7086962762919159 +272,0.723201435588892 +273,0.6132558048139611 +274,0.9047843465339248 +275,0.7518906902116689 +276,0.8043475276533333 +277,0.7940551159124855 +278,0.5976623127230066 +279,0.7077269709075823 +280,0.784406875698381 +281,0.5825145547288328 +282,0.9527891403677229 +283,0.6251814278730241 +284,0.674642251412045 +285,0.6956741249057677 +286,0.8717298877588612 +287,0.7472835883496265 +288,0.7206120437267526 +289,0.8454980596041871 +290,0.847466837773604 +291,0.685029461329672 +292,0.6501792735220139 +293,0.5919296981013319 +294,0.7589313102422693 +295,0.9215966179594822 +296,0.5706432778521059 +297,0.7317465260702745 +298,0.6456245621083724 +299,0.5779864956945218 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/metrics.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/metrics.csv new file mode 100644 index 0000000..10f1a4c --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/metrics.csv @@ -0,0 +1,84 @@ +schema_version,benchmark_level,problem_id,model_id,method_id,run_id,split_id,quantity,metric,aggregation,derivative_order,layer,neuron,output_index,input_index_a,input_index_b,cell_id,value,unit,status +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,mean_width,mean_weighted,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,max_width,max,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,q50_width,q50,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,q90_width,q90,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,q99_width,q99,0,NA,NA,NA,NA,NA,NA,6.497834914030365,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,mean_global_normalized_radius,mean_weighted,0,NA,NA,NA,NA,NA,NA,0.9265572874610044,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,Y,max_global_normalized_radius,max,0,NA,NA,NA,NA,NA,NA,0.9265572874610044,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,mean_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,1.3517955728399362,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,max_width,max,1,NA,NA,NA,NA,NA,NA,1.6774747193077781,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,q50_width,q50,1,NA,NA,NA,NA,NA,NA,1.3356539347980094,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,q90_width,q90,1,NA,NA,NA,NA,NA,NA,1.4887627528273466,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,q99_width,q99,1,NA,NA,NA,NA,NA,NA,1.6132702442532674,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,mean_global_normalized_radius,mean_weighted,1,NA,NA,NA,NA,NA,NA,0.6908584292110806,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,max_global_normalized_radius,max,1,NA,NA,NA,NA,NA,NA,0.8573023709402937,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,mean_frobenius_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,13.557321792556515,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,J,max_frobenius_width,max,1,NA,NA,NA,NA,NA,NA,13.557321792556515,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,mean_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,max_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,q50_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,q90_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,q99_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,mean_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,H,max_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,width,none,NA,NA,NA,NA,NA,NA,NA,1.244073696539206e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,9.81401102413563,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,upper,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,width,none,NA,NA,NA,NA,NA,NA,NA,3.527142889846123e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,L2,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,3.1327321979600535,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,2.883487903430552e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,width,none,NA,NA,NA,NA,NA,NA,NA,2.883487903430552e-69,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,22.7467087769405,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,22.7467087769405,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,upper,none,NA,NA,NA,NA,NA,NA,NA,5.369811824850617e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,width,none,NA,NA,NA,NA,NA,NA,NA,5.369811824850617e-35,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,4.769350980682853,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,W12,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,4.769350980682853,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,PDE_residual,linf_upper,max,2,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,boundary_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,initial_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,600.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,400.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,100.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,600.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,5450.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,2.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,preactivation_preparation,NA,NA,NA,NA,NA,NA,NA,0.0008144900002662325,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,activation_certification,NA,NA,NA,NA,NA,NA,NA,0.004760411000461318,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,value_construction,NA,NA,NA,NA,NA,NA,NA,5.4009999985282775e-05,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,reverse_jacobian_construction,NA,NA,NA,NA,NA,NA,NA,0.49466390700035845,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,onejet_construction,NA,NA,NA,NA,NA,NA,NA,0.5002928180010713,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,onejet_construction_measured,NA,NA,NA,NA,NA,NA,NA,0.5093309129997579,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,L2_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,0.03405151700007991,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,W12_symbolic_integration,NA,NA,NA,NA,NA,NA,NA,0.016671393000251555,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,runtime,seconds,W12_total,NA,NA,NA,NA,NA,NA,NA,0.5260023060000094,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,memory,bytes,peak,NA,NA,NA,NA,NA,NA,NA,NA,bytes,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,soundness,failure_count,Y,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,soundness,max_violation,Y,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,soundness,failure_count,J,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,hybrid_pz_uncompressed_reverse_symbolic,pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,single_cell,soundness,max_violation,J,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/norms.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/norms.csv new file mode 100644 index 0000000..f3b67ab --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/norms.csv @@ -0,0 +1,5 @@ +run_id,norm,squared,lower,upper,width,relative_width,value_contribution_upper,gradient_contribution_upper,hessian_contribution_upper,value_contribution_width,gradient_contribution_width,hessian_contribution_width,status,domain_volume,domain_volume_normalized_lower,domain_volume_normalized_upper,domain_volume_normalized_width +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,L2,1,0.0,1.244073696539206e-69,1.244073696539206e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,9.81401102413563,9.81401102413563 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,L2,0,0.0,3.527142889846123e-35,3.527142889846123e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,3.1327321979600535,3.1327321979600535 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,W12,1,0.0,2.883487903430552e-69,2.883487903430552e-69,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,22.7467087769405,22.7467087769405 +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,W12,0,0.0,5.369811824850617e-35,5.369811824850617e-35,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,4.769350980682853,4.769350980682853 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/soundness.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/soundness.csv new file mode 100644 index 0000000..04aacca --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/soundness.csv @@ -0,0 +1,3 @@ +run_id,quantity,sample_count,failure_count,max_violation,invalid_interval_count,nan_endpoint_count,infinite_endpoint_count,status +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,Y,16384,0,0.0,0,0,0,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,J,16384,0,0.0,0,0,0,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/timings.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/timings.csv new file mode 100644 index 0000000..120d7ce --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/hybrid_pz_uncompressed_reverse_symbolic/timings.csv @@ -0,0 +1,10 @@ +run_id,stage,seconds,status +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,preactivation_preparation,0.0008144900002662325,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,activation_certification,0.004760411000461318,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,value_construction,5.4009999985282775e-05,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,reverse_jacobian_construction,0.49466390700035845,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,onejet_construction,0.5002928180010713,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,onejet_construction_measured,0.5093309129997579,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,L2_symbolic_integration,0.03405151700007991,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,W12_symbolic_integration,0.016671393000251555,ok +pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic,W12_total,0.5260023060000094,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/activation_approximation.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/activation_approximation.csv new file mode 100644 index 0000000..262b50b --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/activation_approximation.csv @@ -0,0 +1,601 @@ +run_id,split_id,cell_id,cell_weight,layer,neuron,derivative_order,preactivation_lower,preactivation_upper,preactivation_midpoint,preactivation_radius,preactivation_width,approximation_kind,approximation_error_radius,approximation_error_diameter,activation_scale,normalized_approximation_radius,noise_symbol_id,shared_noise_group,status +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,0,0,-0.8000887741224206,0.790876728267169,-0.004606022927625797,0.7954827511947948,1.5909655023895897,interval,0.6614959053390281,1.3229918106780563,0.9319962481713716,0.7097624122810793,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,0,1,-0.8000887741224206,0.790876728267169,-0.004606022927625797,0.7954827511947948,1.5909655023895897,interval,0.22050537132306086,0.4410107426461217,1.0,0.22050537132306086,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,1,0,-0.6668101129690586,0.6933849725568161,0.013287429793878758,0.6800975427629373,1.3601950855258746,interval,0.5915149146867266,1.1830298293734531,0.9319962481713716,0.6346752101709762,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,1,1,-0.6668101129690586,0.6933849725568161,0.013287429793878758,0.6800975427629373,1.3601950855258746,interval,0.1800913106760792,0.3601826213521584,1.0,0.1800913106760792,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,2,0,-0.8409685245821179,0.8705015267430378,0.01476650108045996,0.8557350256625779,1.7114700513251557,interval,0.6939752851026396,1.3879505702052792,0.9319962481713716,0.7446116724871561,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,2,1,-0.8409685245821179,0.8705015267430378,0.01476650108045996,0.8557350256625779,1.7114700513251557,interval,0.24614152450842008,0.49228304901684017,1.0,0.24614152450842008,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,3,0,-0.8131646470132096,0.846185847421764,0.016510600204277193,0.8296752472174868,1.6593504944349735,interval,0.6802019744005069,1.3604039488010138,0.9319962481713716,0.729833382629062,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,3,1,-0.8131646470132096,0.846185847421764,0.016510600204277193,0.8296752472174868,1.6593504944349735,interval,0.23740984357244665,0.4748196871448933,1.0,0.23740984357244665,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,4,0,-1.0376263596207116,1.001013709161213,-0.018306325229749287,1.0193200343909623,2.0386400687819246,interval,0.7694842832193629,1.5389685664387258,0.9319962481713716,0.8256302369554961,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,4,1,-1.0376263596207116,1.001013709161213,-0.018306325229749287,1.0193200343909623,2.0386400687819246,interval,0.3018248799079793,0.6036497598159586,1.0,0.3018248799079793,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,5,0,-0.8652278170806444,0.8877201689912206,0.011246175955288096,0.8764739930359325,1.752947986071865,interval,0.7046036179578139,1.4092072359156278,0.9319962481713716,0.7560155090112062,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,5,1,-0.8652278170806444,0.8877201689912206,0.011246175955288096,0.8764739930359325,1.752947986071865,interval,0.25223879079626843,0.5044775815925369,1.0,0.25223879079626843,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,6,0,-0.8002253266309624,0.8296484987611811,0.014711586065109361,0.8149369126960717,1.6298738253921434,interval,0.6722249714837691,1.3444499429675383,0.9319962481713716,0.7212743321689458,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,6,1,-0.8002253266309624,0.8296484987611811,0.014711586065109361,0.8149369126960717,1.6298738253921434,interval,0.23139535188838034,0.4627907037767607,1.0,0.23139535188838034,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,7,0,-0.672406545778101,0.6932924621130971,0.010442958167498073,0.6828495039455991,1.3656990078911981,interval,0.5933268243989807,1.1866536487979613,0.9319962481713716,0.6366193271304695,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,7,1,-0.672406545778101,0.6932924621130971,0.010442958167498073,0.6828495039455991,1.3656990078911981,interval,0.18005578757535584,0.3601115751507117,1.0,0.18005578757535584,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,8,0,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,interval,0.8582165835941257,1.7164331671882513,0.9319962481713716,0.9208369510907305,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,8,1,-1.306841361512731,1.2669144201791418,-0.019963470666794603,1.2868778908459364,2.573755781691873,interval,0.37279367753664217,0.7455873550732843,1.0,0.37279367753664217,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,9,0,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,interval,0.6931069874433504,1.3862139748867008,0.9319962481713716,0.7436800188876992,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,9,1,-0.8395677251243508,0.8685446340190673,0.014488454447358246,0.8540561795717091,1.7081123591434182,interval,0.24544396021159154,0.4908879204231831,1.0,0.24544396021159154,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,10,0,-0.80808685285236,0.7839032216460969,-0.01209181560313155,0.7959950372492285,1.591990074498457,interval,0.6617374353928898,1.3234748707857795,0.9319962481713716,0.7100215657426254,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,10,1,-0.80808685285236,0.7839032216460969,-0.01209181560313155,0.7959950372492285,1.591990074498457,interval,0.2234685348258767,0.4469370696517534,1.0,0.2234685348258767,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,11,0,-0.7982094929703916,0.82175356420364,0.01177203561662421,0.8099815285870158,1.6199630571740316,interval,0.669528882013166,1.339057764026332,0.9319962481713716,0.7183815206625764,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,11,1,-0.7982094929703916,0.82175356420364,0.01177203561662421,0.8099815285870158,1.6199630571740316,interval,0.2285036675507498,0.4570073351014996,1.0,0.2285036675507498,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,12,0,-0.9067744380742931,0.9399477281012679,0.01658664501348739,0.9233610830877805,1.846722166175561,interval,0.7273893591633587,1.4547787183267173,0.9319962481713716,0.7804638276071787,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,12,1,-0.9067744380742931,0.9399477281012679,0.01658664501348739,0.9233610830877805,1.846722166175561,interval,0.27025822293309093,0.5405164458661819,1.0,0.27025822293309093,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,13,0,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,interval,0.7220736531954765,1.444147306390953,0.9319962481713716,0.7747602574712346,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,13,1,-0.931364154693545,0.8930929315848278,-0.01913561155435861,0.9122285431391864,1.8244570862783729,interval,0.2673481268852028,0.5346962537704056,1.0,0.2673481268852028,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,14,0,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,interval,0.8447959138158468,1.6895918276316937,0.9319962481713716,0.9064370328457688,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,14,1,-1.2211529111329582,1.2547055639860074,0.0167763264265246,1.2379292375594828,2.4758584751189656,interval,0.3609082910041765,0.721816582008353,1.0,0.3609082910041765,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,15,0,-0.8585387600910745,0.8291286208471336,-0.01470506962197049,0.8438336904691041,1.6876673809382081,interval,0.687755939136431,1.375511878272862,0.9319962481713716,0.7379385276344688,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,15,1,-0.8585387600910745,0.8291286208471336,-0.01470506962197049,0.8438336904691041,1.6876673809382081,interval,0.2418629339756192,0.4837258679512384,1.0,0.2418629339756192,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,16,0,-1.51352724476465,1.5437531980141885,0.015112976624769292,1.5286402213894192,3.0572804427788385,interval,0.9101559456604009,1.8203118913208018,0.9319962481713716,0.9765661046878433,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,16,1,-1.51352724476465,1.5437531980141885,0.015112976624769292,1.5286402213894192,3.0572804427788385,interval,0.4165552668259878,0.8331105336519756,1.0,0.4165552668259878,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,17,0,-0.924974709822734,0.8972162631509851,-0.013879223335874435,0.9110954864868596,1.8221909729737191,interval,0.7215910220522693,1.4431820441045387,0.9319962481713716,0.7742424108123515,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,17,1,-0.924974709822734,0.8972162631509851,-0.013879223335874435,0.9110954864868596,1.8221909729737191,interval,0.26516844439101567,0.5303368887820313,1.0,0.26516844439101567,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,18,0,-1.6733730976128975,1.6691489651366274,-0.0021120662381350908,1.6712610313747625,3.342522062749525,interval,0.931717663490474,1.863435326980948,0.9319962481713716,0.999701088194889,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,18,1,-1.6733730976128975,1.6691489651366274,-0.0021120662381350908,1.6712610313747625,3.342522062749525,interval,0.43430850330275644,0.8686170066055129,1.0,0.43430850330275644,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,19,0,-0.9806865471881271,0.946217067414651,-0.01723473988673807,0.963451807301389,1.926903614602778,interval,0.7457142959484946,1.4914285918969892,0.9319962481713716,0.8001258560982701,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,19,1,-0.9806865471881271,0.946217067414651,-0.01723473988673807,0.963451807301389,1.926903614602778,interval,0.28377786861302506,0.5675557372260501,1.0,0.28377786861302506,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,20,0,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,interval,0.6956377787345464,1.3912755574690927,0.9319962481713716,0.746395471118501,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,20,1,-0.8770094683630917,0.84103632101463,-0.017986573674230844,0.8590228946888608,1.7180457893777217,interval,0.24845468020899553,0.49690936041799105,1.0,0.24845468020899553,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,21,0,-0.9711358295108827,0.9996589259075814,0.01426154819834935,0.9853973777092321,1.9707947554184642,interval,0.7553269350488825,1.510653870097765,0.9319962481713716,0.8104398880691589,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,21,1,-0.9711358295108827,0.9996589259075814,0.01426154819834935,0.9853973777092321,1.9707947554184642,interval,0.2899037185665899,0.5798074371331798,1.0,0.2899037185665899,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,22,0,-1.091541462437189,1.130698829634847,0.019578683598828972,1.111120146036018,2.222240292072036,interval,0.8043491819949763,1.6086983639899526,0.9319962481713716,0.8630390772207013,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,22,1,-1.091541462437189,1.130698829634847,0.019578683598828972,1.111120146036018,2.222240292072036,interval,0.3290700142194459,0.6581400284388919,1.0,0.3290700142194459,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,23,0,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,interval,0.7945872582901568,1.5891745165803135,0.9319962481713716,0.8525648679908112,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,23,1,-1.0995505666712748,1.068345283359666,-0.015602641655804383,1.0839479250154704,2.167895850030941,interval,0.32027007826374626,0.6405401565274925,1.0,0.32027007826374626,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,24,0,-1.2523324641820428,1.2922888533963242,0.019978194607140676,1.2723106587891835,2.544621317578367,interval,0.8543306099532068,1.7086612199064135,0.9319962481713716,0.9166674346913423,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,24,1,-1.2523324641820428,1.2922888533963242,0.019978194607140676,1.2723106587891835,2.544621317578367,interval,0.3695633788115916,0.7391267576231833,1.0,0.3695633788115916,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,25,0,-1.22231857238258,1.1844726444174882,-0.018922963982545893,1.2033956084000341,2.4067912168000682,interval,0.8345967564137813,1.6691935128275626,0.9319962481713716,0.8954936868590474,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,25,1,-1.22231857238258,1.1844726444174882,-0.018922963982545893,1.2033956084000341,2.4067912168000682,interval,0.3530829519317226,0.7061659038634452,1.0,0.3530829519317226,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,26,0,-1.0760353321015264,1.0349918026899476,-0.02052176470578937,1.055513567395737,2.111027134791474,interval,0.7838134444427352,1.5676268888854703,0.9319962481713716,0.8410049353531419,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,26,1,-1.0760353321015264,1.0349918026899476,-0.02052176470578937,1.055513567395737,2.111027134791474,interval,0.3134136268415227,0.6268272536830454,1.0,0.3134136268415227,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,27,0,-1.0765750272011252,1.0932061682436138,0.00831557052124432,1.0848905977223695,2.169781195444739,interval,0.7949854519251977,1.5899709038503953,0.9319962481713716,0.852992116100256,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,27,1,-1.0765750272011252,1.0932061682436138,0.00831557052124432,1.0848905977223695,2.169781195444739,interval,0.3184381999710487,0.6368763999420974,1.0,0.3184381999710487,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,28,0,-0.8349571475513472,0.8610004970616337,0.013021674755143264,0.8479788223064905,1.695957644612981,interval,0.6899507916482427,1.3799015832964854,0.9319962481713716,0.7402935290801486,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,28,1,-0.8349571475513472,0.8610004970616337,0.013021674755143264,0.8479788223064905,1.695957644612981,interval,0.24274616394735976,0.4854923278947195,1.0,0.24274616394735976,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,29,0,-1.1914992825472055,1.2302241779957934,0.019362447724293963,1.2108617302714995,2.421723460542999,interval,0.8368438069736551,1.6736876139473102,0.9319962481713716,0.8979046950195231,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,29,1,-1.1914992825472055,1.2302241779957934,0.019362447724293963,1.2108617302714995,2.421723460542999,interval,0.3550247405736481,0.7100494811472962,1.0,0.3550247405736481,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,30,0,-0.7475100832928718,0.7212093167911566,-0.0131503832508576,0.7343597000420142,1.4687194000840285,interval,0.6256594881001101,1.2513189762002201,0.9319962481713716,0.6713111660349371,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,30,1,-0.7475100832928718,0.7212093167911566,-0.0131503832508576,0.7343597000420142,1.4687194000840285,interval,0.200763228780994,0.401526457561988,1.0,0.200763228780994,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,31,0,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,interval,0.7554342359137922,1.5108684718275844,0.9319962481713716,0.8105550182160026,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,31,1,-1.0033698976233738,0.9680875531177247,-0.017641172252824577,0.9857287253705492,1.9714574507410985,interval,0.29108892402513853,0.5821778480502771,1.0,0.29108892402513853,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,32,0,-1.2946227610182184,1.251499462932461,-0.021561649042878717,1.2730611119753397,2.5461222239506793,interval,0.8545180129830796,1.7090360259661592,0.9319962481713716,0.9168685117131012,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,32,1,-1.2946227610182184,1.251499462932461,-0.021561649042878717,1.2730611119753397,2.5461222239506793,interval,0.3700859612061639,0.7401719224123278,1.0,0.3700859612061639,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,33,0,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,interval,0.6896805556759176,1.3793611113518351,0.9319962481713716,0.7400035751529142,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,33,1,-0.8320043509056442,0.8630198990126959,0.01550777405352588,0.84751212495917,1.69502424991834,interval,0.24346962939953165,0.4869392587990633,1.0,0.24346962939953165,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,34,0,-0.9062201538536026,0.9391835025457025,0.016481674346049968,0.9227018281996525,1.845403656399305,interval,0.727079993632769,1.454159987265538,0.9319962481713716,0.7801318890063563,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,34,1,-0.9062201538536026,0.9391835025457025,0.016481674346049968,0.9227018281996525,1.845403656399305,interval,0.26999997539257886,0.5399999507851577,1.0,0.26999997539257886,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,35,0,-0.9404150670737782,0.954863067712678,0.007224000319449919,0.9476390673932281,1.8952781347864562,interval,0.7386948257502022,1.4773896515004044,0.9319962481713716,0.7925942053946703,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,35,1,-0.9404150670737782,0.954863067712678,0.007224000319449919,0.9476390673932281,1.8952781347864562,interval,0.27526475243379783,0.5505295048675957,1.0,0.27526475243379783,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,36,0,-1.243730327407627,1.279168180037015,0.01771892631469396,1.261449253722321,2.522898507444642,interval,0.85138964307293,1.70277928614586,0.9319962481713716,0.9135118781254793,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,36,1,-1.243730327407627,1.279168180037015,0.01771892631469396,1.261449253722321,2.522898507444642,interval,0.3665932762430341,0.7331865524860682,1.0,0.3665932762430341,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,37,0,-1.1019913101592174,1.1337401489755883,0.01587441940818546,1.1178657295674028,2.2357314591348056,interval,0.8067543031695504,1.6135086063391009,0.9319962481713716,0.8656196897277721,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,37,1,-1.1019913101592174,1.1337401489755883,0.01587441940818546,1.1178657295674028,2.2357314591348056,interval,0.32991194308669003,0.6598238861733801,1.0,0.32991194308669003,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,38,0,-0.7748642347846241,0.7429797929158605,-0.0159422209343818,0.7589220138502423,1.5178440277004845,interval,0.6403455648337384,1.2806911296674768,0.9319962481713716,0.6870688225302751,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,38,1,-0.7748642347846241,0.7429797929158605,-0.0159422209343818,0.7589220138502423,1.5178440277004845,interval,0.21108689572503808,0.42217379145007616,1.0,0.21108689572503808,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,39,0,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,interval,0.772781944482967,1.545563888965934,0.9319962481713716,0.8291685143574431,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,39,1,-1.0441939899867645,1.0106362591533826,-0.016778865416690936,1.0274151245700736,2.054830249140147,interval,0.30384039895806364,0.6076807979161273,1.0,0.30384039895806364,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,40,0,-1.0016438696095076,0.9635624254813994,-0.019040722064054105,0.9826031475454535,1.965206295090907,interval,0.7540726802880209,1.5081453605760418,0.9319962481713716,0.8090941157408663,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,40,1,-1.0016438696095076,0.9635624254813994,-0.019040722064054105,0.9826031475454535,1.965206295090907,interval,0.29053820069259123,0.5810764013851825,1.0,0.29053820069259123,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,41,0,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,interval,0.6789080640427207,1.3578161280854415,0.9319962481713716,0.7284450612057464,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,41,1,-0.8115738720587108,0.8429313111060704,0.01567871952367983,0.8272525915823906,1.6545051831647812,interval,0.23623089392476404,0.4724617878495281,1.0,0.23623089392476404,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,42,0,-0.8258438072260424,0.797979143361753,-0.013932331932144715,0.8119114752938977,1.6238229505877955,interval,0.6705717475909982,1.3411434951819965,0.9319962481713716,0.7195004796496738,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,42,1,-0.8258438072260424,0.797979143361753,-0.013932331932144715,0.8119114752938977,1.6238229505877955,interval,0.2300034003394853,0.4600068006789706,1.0,0.2300034003394853,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,43,0,-1.5065542104519745,1.5398009931369265,0.016623391342476035,1.5231776017944505,3.046355203588901,interval,0.9092062819856284,1.8184125639712567,0.9319962481713716,0.9755471481452223,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,43,1,-1.5065542104519745,1.5398009931369265,0.016623391342476035,1.5231776017944505,3.046355203588901,interval,0.41595127731099285,0.8319025546219857,1.0,0.41595127731099285,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,44,0,-1.2988952400105458,1.2544908389724558,-0.02220220051904498,1.2766930394915008,2.5533860789830016,interval,0.855488318497447,1.710976636994894,0.9319962481713716,0.9179096162414415,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,44,1,-1.2988952400105458,1.2544908389724558,-0.02220220051904498,1.2766930394915008,2.5533860789830016,interval,0.37103813303386945,0.7420762660677389,1.0,0.37103813303386945,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,45,0,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,interval,0.5865993429083203,1.1731986858166406,0.9319962481713716,0.6294009702928105,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,45,1,-0.6773605002971811,0.6675984405014698,-0.00488102989785566,0.6724794703993254,1.3449589407986509,interval,0.17393247099641473,0.34786494199282947,1.0,0.17393247099641473,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,46,0,-0.8663927459559605,0.9013360930495741,0.017471673546806787,0.8838644195027673,1.7677288390055346,interval,0.7082422758763172,1.4164845517526343,0.9319962481713716,0.7599196641251805,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,46,1,-0.8663927459559605,0.9013360930495741,0.017471673546806787,0.8838644195027673,1.7677288390055346,interval,0.25700708449506665,0.5140141689901333,1.0,0.25700708449506665,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,47,0,-0.8527698834317741,0.8310757190829283,-0.010847082174422873,0.8419228012573512,1.6838456025147024,interval,0.6867834728178142,1.3735669456356283,0.9319962481713716,0.7368951046373002,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,47,1,-0.8527698834317741,0.8310757190829283,-0.010847082174422873,0.8419228012573512,1.6838456025147024,interval,0.23978764856123647,0.47957529712247293,1.0,0.23978764856123647,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,48,0,-1.0115982611523096,1.0428605240202085,0.015631131433949452,1.027229392586259,2.054458785172518,interval,0.7727187062005546,1.5454374124011092,0.9319962481713716,0.8291006618499501,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,48,1,-1.0115982611523096,1.0428605240202085,0.015631131433949452,1.027229392586259,2.054458785172518,interval,0.3034323006665872,0.6068646013331744,1.0,0.3034323006665872,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,49,0,-0.8165600505003643,0.8478715991858075,0.015655774342721585,0.8322158248430859,1.6644316496861717,interval,0.6815745671858776,1.3631491343717552,0.9319962481713716,0.7313061275977932,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,49,1,-0.8165600505003643,0.8478715991858075,0.015655774342721585,0.8322158248430859,1.6644316496861717,interval,0.2380195775569216,0.4760391551138432,1.0,0.2380195775569216,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,50,0,-1.168592106652298,1.204961625642714,0.018184759495208036,1.186776866147506,2.373553732295012,interval,0.8294910329957721,1.6589820659915442,0.9319962481713716,0.8900154207951798,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,50,1,-1.168592106652298,1.204961625642714,0.018184759495208036,1.186776866147506,2.373553732295012,interval,0.3487475790813658,0.6974951581627316,1.0,0.3487475790813658,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,51,0,-0.8006207796149772,0.8344702881968006,0.01692475429091167,0.8175455339058889,1.6350910678117778,interval,0.6736263600043797,1.3472527200087594,0.9319962481713716,0.722777973973685,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,51,1,-0.8006207796149772,0.8344702881968006,0.01692475429091167,0.8175455339058889,1.6350910678117778,interval,0.2331550647635034,0.4663101295270068,1.0,0.2331550647635034,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,52,0,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,interval,0.6387431359052212,1.2774862718104425,0.9319962481713716,0.6853494712650086,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,52,1,-0.7420477599644351,0.7703028121308593,0.014127526083212105,0.7561752860476472,1.5123505720952943,interval,0.20937276137287864,0.41874552274575727,1.0,0.20937276137287864,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,53,0,-1.090911376884629,1.0503864671686705,-0.020262454857979284,1.0706489220266497,2.1412978440532995,interval,0.7895835521350493,1.5791671042700985,0.9319962481713716,0.8471960629500989,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,53,1,-1.090911376884629,1.0503864671686705,-0.020262454857979284,1.0706489220266497,2.1412978440532995,interval,0.3177723239805786,0.6355446479611572,1.0,0.3177723239805786,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,54,0,-1.2066771566490584,1.1738953830218988,-0.016390886813579808,1.1902862698354786,2.380572539670957,interval,0.8305984501862445,1.661196900372489,0.9319962481713716,0.8912036414480474,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,54,1,-1.2066771566490584,1.1738953830218988,-0.016390886813579808,1.1902862698354786,2.380572539670957,interval,0.34918050541633616,0.6983610108326723,1.0,0.34918050541633616,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,55,0,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,interval,0.5913118311798722,1.1826236623597444,0.9319962481713716,0.6344573085353711,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,55,1,-0.6654554261844666,0.6941495666632106,0.014347070239371984,0.6798024964238386,1.3596049928476772,interval,0.1803848903042865,0.360769780608573,1.0,0.1803848903042865,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,56,0,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,interval,0.608147348057122,1.216294696114244,0.9319962481713716,0.6525212405632972,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,56,1,-0.7083442383527363,0.7036147618286084,-0.002364738262063959,0.7059795000906723,1.4119590001813447,interval,0.18582893954535834,0.3716578790907167,1.0,0.18582893954535834,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,57,0,-1.2329296305415844,1.193228519133941,-0.01985055570382177,1.2130790748377627,2.4261581496755253,interval,0.8375022378973618,1.6750044757947236,0.9319962481713716,0.8986111688116638,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,57,1,-1.2329296305415844,1.193228519133941,-0.01985055570382177,1.2130790748377627,2.4261581496755253,interval,0.3556845518822672,0.7113691037645344,1.0,0.3556845518822672,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,58,0,-0.7334941393710498,0.7131942540621979,-0.010149942654425925,0.7233441967166239,1.4466883934332477,interval,0.6189371673066469,1.2378743346132939,0.9319962481713716,0.664098346448321,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,58,1,-0.7334941393710498,0.7131942540621979,-0.010149942654425925,0.7233441967166239,1.4466883934332477,interval,0.1954365041477666,0.3908730082955332,1.0,0.1954365041477666,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,59,0,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,interval,0.733394274591874,1.466788549183748,0.9319962481713716,0.7869068958494568,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,59,1,-0.9466935748263166,0.9255364543021678,-0.010578560262074377,0.9361150145642422,1.8722300291284844,interval,0.2725305235001287,0.5450610470002574,1.0,0.2725305235001287,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,60,0,-1.5860529702984412,1.5733837593858007,-0.0063346054563202525,1.579718364842121,3.159436729684242,interval,0.9185521390163409,1.8371042780326818,0.9319962481713716,0.9855749321079256,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,60,1,-1.5860529702984412,1.5733837593858007,-0.0063346054563202525,1.579718364842121,3.159436729684242,interval,0.4227786990702158,0.8455573981404316,1.0,0.4227786990702158,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,61,0,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,interval,0.8076033477214184,1.6152066954428368,0.9319962481713716,0.8665306854034885,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,61,1,-1.1365766537137107,1.104048554877431,-0.01626404941813986,1.120312604295571,2.240625208591142,interval,0.33069439554739516,0.6613887910947903,1.0,0.33069439554739516,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,62,0,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,interval,0.7936577580846368,1.5873155161692736,0.9319962481713716,0.8515675461589436,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,62,1,-1.0643707508376148,1.0985694149464218,0.017099332054403504,1.0814700828920183,2.1629401657840366,interval,0.31998765206375646,0.6399753041275129,1.0,0.31998765206375646,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,63,0,-1.4015171493211271,1.3643005656294993,-0.018608291845813918,1.3829088574753132,2.7658177149506264,interval,0.8815329561125858,1.7630659122251715,0.9319962481713716,0.9458546188809261,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,63,1,-1.4015171493211271,1.3643005656294993,-0.018608291845813918,1.3829088574753132,2.7658177149506264,interval,0.39221377271476693,0.7844275454295339,1.0,0.39221377271476693,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,64,0,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,interval,0.7002426466547398,1.4004852933094796,0.9319962481713716,0.7513363364162192,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,64,1,-0.8850519189924945,0.8509091370870555,-0.01707139095271948,0.867980528039775,1.73596105607955,interval,0.2512987896364209,0.5025975792728418,1.0,0.2512987896364209,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,65,0,-0.8429087446687177,0.8658273177411772,0.01145928653622974,0.8543680312049474,1.7087360624098948,interval,0.6932972707219847,1.3865945414439693,0.9319962481713716,0.7438841863175656,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,65,1,-0.8429087446687177,0.8658273177411772,0.01145928653622974,0.8543680312049474,1.7087360624098948,interval,0.24447380433468513,0.48894760866937026,1.0,0.24447380433468513,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,66,0,-0.9905468239121497,0.9617857365141985,-0.014380543698975568,0.9761662802131741,1.9523325604263482,interval,0.7513338675798401,1.5026677351596802,0.9319962481713716,0.8061554636662958,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,66,1,-0.9905468239121497,0.9617857365141985,-0.014380543698975568,0.9761662802131741,1.9523325604263482,interval,0.2869753909958087,0.5739507819916174,1.0,0.2869753909958087,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,67,0,-0.881044996934086,0.9120447889086442,0.015499895987279078,0.8965448929213651,1.7930897858427302,interval,0.7145273447120282,1.4290546894240563,0.9319962481713716,0.7666633273620688,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,67,1,-0.881044996934086,0.9120447889086442,0.015499895987279078,0.8965448929213651,1.7930897858427302,interval,0.26072304463092766,0.5214460892618553,1.0,0.26072304463092766,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,68,0,-1.0577074974769158,1.0951935167927767,0.01874300965793041,1.0764505071348462,2.1529010142696925,interval,0.7917753670898736,1.5835507341797472,0.9319962481713716,0.8495478052013415,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,68,1,-1.0577074974769158,1.0951935167927767,0.01874300965793041,1.0764505071348462,2.1529010142696925,interval,0.3190134587926411,0.6380269175852822,1.0,0.3190134587926411,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,69,0,-1.3791784207269766,1.422282921894831,0.02155225058392718,1.4007306713109038,2.8014613426218076,interval,0.8854206850407025,1.770841370081405,0.9319962481713716,0.9500260186432585,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,69,1,-1.3791784207269766,1.422282921894831,0.02155225058392718,1.4007306713109038,2.8014613426218076,interval,0.3961160472232093,0.7922320944464186,1.0,0.3961160472232093,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,70,0,-0.930851379094681,0.9636725950147776,0.01641060796004834,0.9472619870547293,1.8945239741094586,interval,0.73845057624171,1.47690115248342,0.9319962481713716,0.7923321340516026,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,70,1,-0.930851379094681,0.9636725950147776,0.01641060796004834,0.9472619870547293,1.8945239741094586,interval,0.27819128368256396,0.5563825673651279,1.0,0.27819128368256396,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,71,0,-1.438885897394499,1.394990236741567,-0.021947830326465967,1.416938067068033,2.833876134136066,interval,0.8888685951833994,1.777737190366799,0.9319962481713716,0.9537255079378366,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,71,1,-1.438885897394499,1.394990236741567,-0.021947830326465967,1.416938067068033,2.833876134136066,interval,0.39914720640412554,0.7982944128082511,1.0,0.39914720640412554,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,72,0,-1.1711538278994706,1.2088264922154854,0.018836332158007396,1.189990160057478,2.379980320114956,interval,0.8304844183535263,1.6609688367070525,0.9319962481713716,0.8910812892036667,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,72,1,-1.1711538278994706,1.2088264922154854,0.018836332158007396,1.189990160057478,2.379980320114956,interval,0.34972153348182755,0.6994430669636551,1.0,0.34972153348182755,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,73,0,-1.174538707187349,1.210071346678575,0.017766319745613046,1.192305026932962,2.384610053865924,interval,0.8312113123397611,1.6624226246795222,0.9319962481713716,0.8918612215131164,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,73,1,-1.174538707187349,1.210071346678575,0.017766319745613046,1.192305026932962,2.384610053865924,interval,0.3500341892277142,0.7000683784554284,1.0,0.3500341892277142,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,74,0,-0.9300731152852421,0.89293327807331,-0.018569918605966018,0.911503196679276,1.823006393358552,interval,0.721733747695763,1.443467495391526,0.9319962481713716,0.7743955505312867,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,74,1,-0.9300731152852421,0.89293327807331,-0.018569918605966018,0.911503196679276,1.823006393358552,interval,0.26690862477248034,0.5338172495449607,1.0,0.26690862477248034,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,75,0,-0.8829617050674684,0.9081151180277417,0.012576706480136646,0.895538411547605,1.79107682309521,interval,0.7140631445422361,1.4281262890844721,0.9319962481713716,0.7661652565053428,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,75,1,-0.8829617050674684,0.9081151180277417,0.012576706480136646,0.895538411547605,1.79107682309521,interval,0.2593629905902751,0.5187259811805502,1.0,0.2593629905902751,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,76,0,-1.1805875736327949,1.1488019022058176,-0.01589283571348865,1.1646947379193062,2.3293894758386124,interval,0.8224967910357934,1.6449935820715869,0.9319962481713716,0.8825108391257774,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,76,1,-1.1805875736327949,1.1488019022058176,-0.01589283571348865,1.1646947379193062,2.3293894758386124,interval,0.34249133380857305,0.6849826676171461,1.0,0.34249133380857305,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,77,0,-1.3659675145450219,1.331452662025504,-0.0172574262597589,1.348710088285263,2.697420176570526,interval,0.8736869254065871,1.7473738508131742,0.9319962481713716,0.9374360971096283,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,77,1,-1.3659675145450219,1.331452662025504,-0.0172574262597589,1.348710088285263,2.697420176570526,interval,0.38524002231589133,0.7704800446317827,1.0,0.38524002231589133,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,78,0,-0.6125577319415738,0.6569759697827167,0.022209118920571425,0.6347668508621452,1.2695337017242905,interval,0.5611362299286811,1.1222724598573621,0.9319962481713716,0.6020799236366686,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,78,1,-0.6125577319415738,0.6569759697827167,0.022209118920571425,0.6347668508621452,1.2695337017242905,interval,0.16608817077507965,0.3321763415501593,1.0,0.16608817077507965,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,79,0,-0.8548400889214127,0.8380958072828195,-0.008372140819296603,0.8464679481021161,1.6929358962042322,interval,0.6891943765294223,1.3783887530588446,0.9319962481713716,0.7394819216082252,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,79,1,-0.8548400889214127,0.8380958072828195,-0.008372140819296603,0.8464679481021161,1.6929358962042322,interval,0.2405332627728336,0.4810665255456672,1.0,0.2405332627728336,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,80,0,-1.2719651427464118,1.2295099902121156,-0.021227576267148107,1.2507375664792637,2.5014751329585274,interval,0.8483832534042048,1.6967665068084097,0.9319962481713716,0.9102861251520914,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,80,1,-1.2719651427464118,1.2295099902121156,-0.021227576267148107,1.2507375664792637,2.5014751329585274,interval,0.3649393380360813,0.7298786760721626,1.0,0.3649393380360813,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,81,0,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,interval,0.5383925018272909,1.0767850036545819,0.9319962481713716,0.5776766836600973,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,81,1,-0.6323568266861045,0.5723896197843606,-0.02998360345087192,0.6023732232352326,1.2047464464704651,interval,0.15661689948538382,0.31323379897076764,1.0,0.15661689948538382,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,82,0,-0.770773785839283,0.7995670803645064,0.014396647262611695,0.7851704331018947,1.5703408662037894,interval,0.6555869213581343,1.3111738427162687,0.9319962481713716,0.7034222751909488,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,82,1,-0.770773785839283,0.7995670803645064,0.014396647262611695,0.7851704331018947,1.5703408662037894,interval,0.22031168509811438,0.44062337019622877,1.0,0.22031168509811438,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,83,0,-0.89320137367518,0.8717751463069094,-0.0107131136841353,0.8824882599910447,1.7649765199820895,interval,0.7076231336972404,1.4152462673944808,0.9319962481713716,0.7592553458081364,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,83,1,-0.89320137367518,0.8717751463069094,-0.0107131136841353,0.8824882599910447,1.7649765199820895,interval,0.2541640748483256,0.5083281496966512,1.0,0.2541640748483256,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,84,0,-0.7976625548491154,0.827040944766799,0.0146891949588418,0.8123517498079572,1.6247034996159144,interval,0.6708059568070246,1.3416119136140492,0.9319962481713716,0.7197517781033809,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,84,1,-0.7976625548491154,0.827040944766799,0.0146891949588418,0.8123517498079572,1.6247034996159144,interval,0.23044169737247155,0.4608833947449431,1.0,0.23044169737247155,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,85,0,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,interval,0.725412284139243,1.450824568278486,0.9319962481713716,0.7783424939344361,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,85,1,-0.9348748672197601,0.9034342047802039,-0.015720331219778116,0.919154535999982,1.838309071999964,interval,0.2685408887151122,0.5370817774302244,1.0,0.2685408887151122,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,86,0,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,interval,0.8107287463734445,1.621457492746889,0.9319962481713716,0.8698841309330797,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,86,1,-1.1100026096220486,1.1489145687805893,0.019455979579270366,1.129458589201319,2.258917178402638,interval,0.33406662183766556,0.6681332436753311,1.0,0.33406662183766556,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,87,0,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,interval,0.8499766573879501,1.6999533147759003,0.9319962481713716,0.9119957929612394,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,87,1,-1.2789544097029169,1.2340401481250904,-0.022457130788913204,1.2564972789140036,2.5129945578280073,interval,0.36654443042086376,0.7330888608417275,1.0,0.36654443042086376,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,88,0,-0.8196835402297237,0.8511525501491991,0.015734504959737716,0.8354180451894614,1.6708360903789228,interval,0.6832843520820353,1.3665687041640706,0.9319962481713716,0.7331406681332432,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,88,1,-0.8196835402297237,0.8511525501491991,0.015734504959737716,0.8354180451894614,1.6708360903789228,interval,0.239204461552884,0.478408923105768,1.0,0.239204461552884,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,89,0,-0.8579518670180815,0.8906473389988188,0.01634773599036865,0.8742996030084501,1.7485992060169002,interval,0.7034571174860031,1.4069142349720063,0.9319962481713716,0.7547853533383048,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,89,1,-0.8579518670180815,0.8906473389988188,0.01634773599036865,0.8742996030084501,1.7485992060169002,interval,0.2532679232281045,0.506535846456209,1.0,0.2532679232281045,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,90,0,-0.8207744817317321,0.7924840586718497,-0.014145211529941193,0.8066292702017909,1.6132585404035817,interval,0.6676525691668744,1.3353051383337489,0.9319962481713716,0.7163683013497596,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,90,1,-0.8207744817317321,0.7924840586718497,-0.014145211529941193,0.8066292702017909,1.6132585404035817,interval,0.228144173822866,0.456288347645732,1.0,0.228144173822866,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,91,0,-1.289402589613948,1.2579584264319819,-0.015722081590983095,1.273680508022965,2.54736101604593,interval,0.8547351423950711,1.7094702847901422,0.9319962481713716,0.9171014841230412,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,91,1,-1.289402589613948,1.2579584264319819,-0.015722081590983095,1.273680508022965,2.54736101604593,interval,0.36891472628940897,0.7378294525788179,1.0,0.36891472628940897,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,92,0,-1.124946905632745,1.08585988447044,-0.019543510581152557,1.1054033950515925,2.210806790103185,interval,0.8023224101319772,1.6046448202639545,0.9319962481713716,0.860864420544802,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,92,1,-1.124946905632745,1.08585988447044,-0.019543510581152557,1.1054033950515925,2.210806790103185,interval,0.32746928486190574,0.6549385697238115,1.0,0.32746928486190574,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,93,0,-0.6468941826251272,0.6720388239177194,0.012572320646296098,0.6594665032714233,1.3189330065428466,interval,0.5779474284276658,1.1558948568553316,0.9319962481713716,0.6201177628790145,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,93,1,-0.6468941826251272,0.6720388239177194,0.012572320646296098,0.6594665032714233,1.3189330065428466,interval,0.17188523042428866,0.3437704608485773,1.0,0.17188523042428866,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,94,0,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,interval,0.836711556432103,1.673423112864206,0.9319962481713716,0.8977627947255985,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,94,1,-1.1891179119541309,1.2318600758063356,0.021371081926102375,1.2104889938802332,2.4209779877604665,interval,0.3554239931944218,0.7108479863888436,1.0,0.3554239931944218,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,95,0,-0.8425271444430551,0.8784439097769787,0.017958382666961814,0.8604855271100169,1.7209710542200338,interval,0.6963920958791352,1.3927841917582704,0.9319962481713716,0.7472048275360498,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,95,1,-0.8425271444430551,0.8784439097769787,0.017958382666961814,0.8604855271100169,1.7209710542200338,interval,0.24896313277562554,0.4979262655512511,1.0,0.24896313277562554,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,96,0,-0.6659994683686796,0.6358673961792187,-0.01506603609473045,0.6509334322739492,1.3018668645478984,interval,0.5722106612535437,1.1444213225070874,0.9319962481713716,0.613962408514254,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,96,1,-0.6659994683686796,0.6358673961792187,-0.01506603609473045,0.6509334322739492,1.3018668645478984,interval,0.16956120745919218,0.33912241491838435,1.0,0.16956120745919218,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,97,0,-1.215123877758,1.1786727174263019,-0.018225580165849076,1.196898297592151,2.393796595184302,interval,0.8326212692135757,1.6652425384271514,0.9319962481713716,0.8933740568668864,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,97,1,-1.215123877758,1.1786727174263019,-0.018225580165849076,1.196898297592151,2.393796595184302,interval,0.35129792757793404,0.7025958551558681,1.0,0.35129792757793404,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,98,0,-1.273373259798392,1.3090285069017624,0.017827623551685212,1.2912008833500772,2.5824017667001544,interval,0.8593693343392095,1.718738668678419,0.9319962481713716,0.9220738130924238,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,98,1,-1.273373259798392,1.3090285069017624,0.017827623551685212,1.2912008833500772,2.5824017667001544,interval,0.3732733945915383,0.7465467891830766,1.0,0.3732733945915383,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,99,0,-0.9135757537667062,0.8771689762111496,-0.018203388777778273,0.8953723649889279,1.7907447299778558,interval,0.7139211501296949,1.4278423002593899,0.9319962481713716,0.7660129013721331,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,99,1,-0.9135757537667062,0.8771689762111496,-0.018203388777778273,0.8953723649889279,1.7907447299778558,interval,0.26125178274412253,0.5225035654882451,1.0,0.26125178274412253,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,100,0,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,interval,0.8963888549627822,1.7927777099255644,0.9319962481713716,0.9617944886812011,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,100,1,-1.4400569326913262,1.467338473838685,0.013640770573679406,1.4536977032650056,2.907395406530011,interval,0.4041621040797618,0.8083242081595235,1.0,0.4041621040797618,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,101,0,-1.5837879660157446,1.5801226097140428,-0.0018326781508508638,1.5819552878648937,3.1639105757297874,interval,0.918906223127348,1.837812446254696,0.9319962481713716,0.9859548522113615,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,101,1,-1.5837879660157446,1.5801226097140428,-0.0018326781508508638,1.5819552878648937,3.1639105757297874,interval,0.422456421834918,0.844912843669836,1.0,0.422456421834918,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,102,0,-1.058315031825715,1.0262563408124221,-0.016029345506646475,1.0422856863190686,2.084571372638137,interval,0.7787103216272394,1.5574206432544788,0.9319962481713716,0.8355294596465515,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,102,1,-1.058315031825715,1.0262563408124221,-0.016029345506646475,1.0422856863190686,2.084571372638137,interval,0.3081267189017766,0.6162534378035532,1.0,0.3081267189017766,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,103,0,-1.1476452573399851,1.109070551094331,-0.019287353122827033,1.128357904217158,2.256715808434316,interval,0.8103530500543654,1.6207061001087308,0.9319962481713716,0.8694810216718395,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,103,1,-1.1476452573399851,1.109070551094331,-0.019287353122827033,1.128357904217158,2.256715808434316,interval,0.33372203233631603,0.6674440646726321,1.0,0.33372203233631603,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,104,0,-0.8577016693145989,0.8877385522479759,0.015018441466688515,0.8727201107812874,1.7454402215625748,interval,0.7026732903800352,1.4053465807600705,0.9319962481713716,0.7539443337446039,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,104,1,-0.8577016693145989,0.8877385522479759,0.015018441466688515,0.8727201107812874,1.7454402215625748,interval,0.25224526078926535,0.5044905215785307,1.0,0.25224526078926535,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,105,0,-1.177912795726943,1.218542112172731,0.02031465822289391,1.198227453949837,2.396454907899674,interval,0.8330079639825803,1.6660159279651605,0.9319962481713716,0.8937889670875694,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,105,1,-1.177912795726943,1.218542112172731,0.02031465822289391,1.198227453949837,2.396454907899674,interval,0.3521481252889635,0.704296250577927,1.0,0.3521481252889635,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,106,0,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,interval,0.6212163624347173,1.2424327248694347,0.9319962481713716,0.6665438446276778,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,106,1,-0.7436028604158077,0.7107002167910169,-0.016451321812395392,0.7271515386034123,1.4543030772068246,interval,0.19928057231551816,0.3985611446310363,1.0,0.19928057231551816,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,107,0,-0.6364463917753741,0.5633476109147434,-0.036549390430315354,0.5998970013450587,1.1997940026901175,interval,0.5364660410282376,1.0729320820564752,0.9319962481713716,0.5756096573144085,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,107,1,-0.6364463917753741,0.5633476109147434,-0.036549390430315354,0.5998970013450587,1.1997940026901175,interval,0.15818911070264985,0.3163782214052997,1.0,0.15818911070264985,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,108,0,-1.0426471018400014,1.0803325368651482,0.01884271751257338,1.0614898193525748,2.1229796387051496,interval,0.7861268057509216,1.5722536115018433,0.9319962481713716,0.8434870926716133,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,108,1,-1.0426471018400014,1.0803325368651482,0.01884271751257338,1.0614898193525748,2.1229796387051496,interval,0.3146802000445956,0.6293604000891913,1.0,0.3146802000445956,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,109,0,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,interval,0.7306926358883902,1.4613852717767803,0.9319962481713716,0.7840081302065858,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,109,1,-0.9134574422361408,0.9473867494974706,0.016964653630664905,0.9304220958668057,1.8608441917336114,interval,0.27276327302913006,0.5455265460582601,1.0,0.27276327302913006,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,110,0,-1.2677004222053616,1.2229038135571433,-0.022398304324109164,1.2453021178812524,2.490604235762505,interval,0.846840916941537,1.693681833883074,0.9319962481713716,0.90863125104107,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,110,1,-1.2677004222053616,1.2229038135571433,-0.022398304324109164,1.2453021178812524,2.490604235762505,interval,0.3639522329250414,0.7279044658500828,1.0,0.3639522329250414,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,111,0,-0.9581578905614647,0.9243831398125946,-0.016887375374435076,0.9412705151870296,1.8825410303740593,interval,0.7357092168554062,1.4714184337108125,0.9319962481713716,0.7893907494787759,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,111,1,-0.9581578905614647,0.9243831398125946,-0.016887375374435076,0.9412705151870296,1.8825410303740593,interval,0.27636197241514915,0.5527239448302983,1.0,0.27636197241514915,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,112,0,-0.7914234807966224,0.8188916681191162,0.013734093661246893,0.8051575744578693,1.6103151489157386,interval,0.666840429248015,1.33368085849603,0.9319962481713716,0.7154969030791626,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,112,1,-0.7914234807966224,0.8188916681191162,0.013734093661246893,0.8051575744578693,1.6103151489157386,interval,0.2274523138389941,0.4549046276779882,1.0,0.2274523138389941,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,113,0,-0.849686150820982,0.872017005906473,0.01116542754274552,0.8608515783637275,1.721703156727455,interval,0.6966514715103378,1.3933029430206756,0.9319962481713716,0.747483128689849,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,113,1,-0.849686150820982,0.872017005906473,0.01116542754274552,0.8608515783637275,1.721703156727455,interval,0.24668110347759487,0.49336220695518973,1.0,0.24668110347759487,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,114,0,-0.6630886315286886,0.6437069553096089,-0.009690838109539857,0.6533977934191487,1.3067955868382974,interval,0.5739167513151886,1.1478335026303772,0.9319962481713716,0.6157929846190316,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,114,1,-0.6630886315286886,0.6437069553096089,-0.009690838109539857,0.6533977934191487,1.3067955868382974,interval,0.16844090943864065,0.3368818188772813,1.0,0.16844090943864065,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,115,0,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,interval,0.6282439248722511,1.2564878497445022,0.9319962481713716,0.6740841780263607,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,115,1,-0.7285481849997572,0.7485975228590082,0.010024668929625502,0.7385728539293827,1.4771457078587653,interval,0.20117554522378533,0.40235109044757067,1.0,0.20117554522378533,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,116,0,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,interval,0.7121244349396276,1.4242488698792553,0.9319962481713716,0.7640850876136629,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,116,1,-0.8744698086050322,0.9089145327376147,0.01722236206629124,0.8916921706713234,1.783384341342647,interval,0.2596400031108217,0.5192800062216434,1.0,0.2596400031108217,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,117,0,-1.3740507569108844,1.4125160594440576,0.019232651266586576,1.393283408177471,2.786566816354942,interval,0.8838196593740678,1.7676393187481356,0.9319962481713716,0.9483081730298497,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,117,1,-1.3740507569108844,1.4125160594440576,0.019232651266586576,1.393283408177471,2.786566816354942,interval,0.3942961868391139,0.7885923736782278,1.0,0.3942961868391139,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,118,0,-0.7114293139660043,0.6846611208139651,-0.013384096576019577,0.6980452173899847,1.3960904347799694,interval,0.6030567973951524,1.2061135947903048,0.9319962481713716,0.6470592543461235,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,118,1,-0.7114293139660043,0.6846611208139651,-0.013384096576019577,0.6980452173899847,1.3960904347799694,interval,0.18701036102920576,0.3740207220584115,1.0,0.18701036102920576,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,119,0,-1.168735262508867,1.1298298900732502,-0.01945268621780838,1.1492825762910586,2.298565152582117,interval,0.8174136690297589,1.6348273380595177,0.9319962481713716,0.8770568236015648,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,119,1,-1.168735262508867,1.1298298900732502,-0.01945268621780838,1.1492825762910586,2.298565152582117,interval,0.3393778432135923,0.6787556864271846,1.0,0.3393778432135923,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,120,0,-0.763981292013019,0.7365572828456634,-0.013712004583677828,0.7502692874293412,1.5005385748586824,interval,0.635238343505159,1.270476687010318,0.9319962481713716,0.6815889492597549,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,120,1,-0.763981292013019,0.7365572828456634,-0.013712004583677828,0.7502692874293412,1.5005385748586824,interval,0.20699210071157514,0.4139842014231503,1.0,0.20699210071157514,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,121,0,-0.6662127062570722,0.6906969277491459,0.012242110746036872,0.6784548170031091,1.3569096340062181,interval,0.5904563097134787,1.1809126194269575,0.9319962481713716,0.6335393633525744,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,121,1,-0.6662127062570722,0.6906969277491459,0.012242110746036872,0.6784548170031091,1.3569096340062181,interval,0.1790589526951929,0.3581179053903858,1.0,0.1790589526951929,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,122,0,-0.8964792612600481,0.9290067815766618,0.016263760158306884,0.912743021418355,1.82548604283671,interval,0.7223548617373069,1.4447097234746138,0.9319962481713716,0.7750619845891089,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,122,1,-0.8964792612600481,0.9290067815766618,0.016263760158306884,0.912743021418355,1.82548604283671,interval,0.2665452656406132,0.5330905312812264,1.0,0.2665452656406132,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,123,0,-1.104167563368715,1.1420328248526197,0.018932630741952394,1.1231001941106673,2.2462003882213346,interval,0.8085442518765142,1.6170885037530285,0.9319962481713716,0.8675402432820121,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,123,1,-1.104167563368715,1.1420328248526197,0.018932630741952394,1.1231001941106673,2.2462003882213346,interval,0.3321919459926812,0.6643838919853624,1.0,0.3321919459926812,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,124,0,-0.7854236701342574,0.7505279392833626,-0.017447865425447406,0.76797580470881,1.53595160941762,interval,0.6456362737049398,1.2912725474098796,0.9319962481713716,0.692745571639064,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,124,1,-0.7854236701342574,0.7505279392833626,-0.017447865425447406,0.76797580470881,1.53595160941762,interval,0.2150425514318801,0.4300851028637602,1.0,0.2150425514318801,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,125,0,-1.3977310453244938,1.3621603770086215,-0.01778533415793615,1.3799457111665576,2.7598914223331152,interval,0.8808767044886803,1.7617534089773605,0.9319962481713716,0.945150483402706,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,125,1,-1.3977310453244938,1.3621603770086215,-0.01778533415793615,1.3799457111665576,2.7598914223331152,interval,0.39148880599503544,0.7829776119900709,1.0,0.39148880599503544,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,126,0,-1.145506083024695,1.1862496927259267,0.02037180485061585,1.1658778878753109,2.3317557757506218,interval,0.8228359919841374,1.6456719839682747,0.9319962481713716,0.882874790106277,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,126,1,-1.145506083024695,1.1862496927259267,0.02037180485061585,1.1658778878753109,2.3317557757506218,interval,0.34396223580338464,0.6879244716067693,1.0,0.34396223580338464,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,127,0,-1.1653861727789863,1.1295881895992126,-0.017898991589886837,1.1474871811890994,2.294974362378199,interval,0.8168328644037188,1.6336657288074377,0.9319962481713716,0.8764336401636704,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,127,1,-1.1653861727789863,1.1295881895992126,-0.017898991589886837,1.1474871811890994,2.294974362378199,interval,0.33848959803104195,0.6769791960620839,1.0,0.33848959803104195,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,128,0,-0.918369695032508,0.9538988826345167,0.01776459380100437,0.9361342888335124,1.8722685776670247,interval,0.7333341603127539,1.4666683206255078,0.9319962481713716,0.7868423952903204,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,128,1,-0.918369695032508,0.9538988826345167,0.01776459380100437,0.9361342888335124,1.8722685776670247,interval,0.274943063867085,0.54988612773417,1.0,0.274943063867085,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,129,0,-0.8382147723907124,0.8049901321285019,-0.01661232013110525,0.8216024522596072,1.6432049045192143,interval,0.6758398113003152,1.3516796226006305,0.9319962481713716,0.7251529312766554,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,129,1,-0.8382147723907124,0.8049901321285019,-0.01661232013110525,0.8216024522596072,1.6432049045192143,interval,0.23451820544726687,0.46903641089453374,1.0,0.23451820544726687,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,130,0,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,interval,0.7749557909021136,1.5499115818042273,0.9319962481713716,0.8315009769863558,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,130,1,-1.0156344504627695,1.0500586106082939,0.017212080072762204,1.0328465305355317,2.0656930610710633,interval,0.30562840452193735,0.6112568090438747,1.0,0.30562840452193735,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,131,0,-1.0474500720898499,1.0179226028488395,-0.014763734620505176,1.0326863374693447,2.0653726749386894,interval,0.7749160339683391,1.5498320679366782,0.9319962481713716,0.8314583191603694,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,131,1,-1.0474500720898499,1.0179226028488395,-0.014763734620505176,1.0326863374693447,2.0653726749386894,interval,0.3048344872991245,0.609668974598249,1.0,0.3048344872991245,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,132,0,-1.0609350410804448,1.0970640029002234,0.018064480909889324,1.078999521990334,2.157999043980668,interval,0.7927316964807651,1.5854633929615303,0.9319962481713716,0.8505739138286755,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,132,1,-1.0609350410804448,1.0970640029002234,0.018064480909889324,1.078999521990334,2.157999043980668,interval,0.319553696781892,0.639107393563784,1.0,0.319553696781892,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,133,0,-1.2534023950114845,1.216564803864441,-0.018418795573521773,1.2349835994379628,2.4699671988759255,interval,0.8439364519936033,1.6878729039872067,0.9319962481713716,0.9055148597963281,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,133,1,-1.2534023950114845,1.216564803864441,-0.018418795573521773,1.2349835994379628,2.4699671988759255,interval,0.36060001942466396,0.7212000388493279,1.0,0.36060001942466396,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,134,0,-1.0114330530784952,0.9766652867437469,-0.01738388316737416,0.994049169911121,1.988098339822242,interval,0.7589863986022531,1.5179727972045063,0.9319962481713716,0.8143663669155606,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,134,1,-1.0114330530784952,0.9766652867437469,-0.01738388316737416,0.994049169911121,1.988098339822242,interval,0.2936492964689251,0.5872985929378503,1.0,0.2936492964689251,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,135,0,-1.2832724515737506,1.3273446149675858,0.02203608169691762,1.3053085332706682,2.6106170665413364,interval,0.8629766043713951,1.7259532087427902,0.9319962481713716,0.9259442900812135,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,135,1,-1.2832724515737506,1.3273446149675858,0.02203608169691762,1.3053085332706682,2.6106170665413364,interval,0.3772319743846195,0.754463948769239,1.0,0.3772319743846195,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,136,0,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,interval,0.6890184930856266,1.3780369861712531,0.9319962481713716,0.739293204707121,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,136,1,-0.8365248390947787,0.8557722052771414,0.009623683091181334,0.84614852218596,1.69229704437192,interval,0.24086865646321104,0.4817373129264221,1.0,0.24086865646321104,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,137,0,-1.1299748853383091,1.1721429880236616,0.021084051342676213,1.1510589366809854,2.3021178733619707,interval,0.8179843068197402,1.6359686136394804,0.9319962481713716,0.877669098373165,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,137,1,-1.1299748853383091,1.1721429880236616,0.021084051342676213,1.1510589366809854,2.3021178733619707,interval,0.34027780728420504,0.6805556145684101,1.0,0.34027780728420504,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,138,0,-1.4795147536405533,1.444980471104201,-0.01726714126817619,1.462247612372377,2.924495224744754,interval,0.8980364506502465,1.796072901300493,0.9319962481713716,0.9635623023292679,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,138,1,-1.4795147536405533,1.444980471104201,-0.01726714126817619,1.462247612372377,2.924495224744754,interval,0.4062402791572797,0.8124805583145595,1.0,0.4062402791572797,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,139,0,-1.1846112842973127,1.1451154059873385,-0.01974793915498707,1.1648633451423256,2.329726690284651,interval,0.8225147816101163,1.6450295632202325,0.9319962481713716,0.8825301423948175,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,139,1,-1.1846112842973127,1.1451154059873385,-0.01974793915498707,1.1648633451423256,2.329726690284651,interval,0.34353770580701426,0.6870754116140285,1.0,0.34353770580701426,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,140,0,-1.0016612677883299,1.0308531994790655,0.01459596584536782,1.0162572336336977,2.0325144672673954,interval,0.7682706333637228,1.5365412667274456,0.9319962481713716,0.8243280322974609,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,140,1,-1.0016612677883299,1.0308531994790655,0.01459596584536782,1.0162572336336977,2.0325144672673954,interval,0.2997317659842733,0.5994635319685466,1.0,0.2997317659842733,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,141,0,-0.854215463176636,0.8897074866463958,0.017746011734879885,0.8719614749115159,1.7439229498230318,interval,0.7022572693341231,1.4045145386682463,0.9319962481713716,0.7534979574349048,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,141,1,-0.854215463176636,0.8897074866463958,0.017746011734879885,0.8719614749115159,1.7439229498230318,interval,0.2529377292215802,0.5058754584431604,1.0,0.2529377292215802,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,142,0,-1.0907597002338711,1.1219900064229056,0.01561515309451722,1.1063748533283884,2.2127497066567767,interval,0.8027076248045013,1.6054152496090026,0.9319962481713716,0.8612777426727396,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,142,1,-1.0907597002338711,1.1219900064229056,0.01561515309451722,1.1063748533283884,2.2127497066567767,interval,0.3266421088358624,0.6532842176717248,1.0,0.3266421088358624,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,143,0,-0.8162896758725655,0.828124195746196,0.0059172599368152445,0.8222069358093808,1.6444138716187615,interval,0.6762564289871127,1.3525128579742254,0.9319962481713716,0.7255999477615552,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,143,1,-0.8162896758725655,0.828124195746196,0.0059172599368152445,0.8222069358093808,1.6444138716187615,interval,0.2308380434058983,0.4616760868117966,1.0,0.2308380434058983,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,144,0,-1.0737179807403923,1.042612774283318,-0.015552603228537132,1.0581653775118551,2.1163307550237103,interval,0.7848877991555059,1.5697755983110118,0.9319962481713716,0.8421576811017204,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,144,1,-1.0737179807403923,1.042612774283318,-0.015552603228537132,1.0581653775118551,2.1163307550237103,interval,0.31272808366406246,0.6254561673281249,1.0,0.31272808366406246,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,145,0,-0.9082466318108172,0.8772921563875725,-0.01547723771162235,0.8927693940991949,1.7855387881983897,interval,0.7126748995587046,1.425349799117409,0.9319962481713716,0.76467571726497,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,145,1,-0.9082466318108172,0.8772921563875725,-0.01547723771162235,0.8927693940991949,1.7855387881983897,interval,0.25940857440140686,0.5188171488028137,1.0,0.25940857440140686,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,146,0,-1.3022655421739981,1.3407396287397118,0.019237043282856803,1.321502585456855,2.64300517091371,interval,0.8670775149794099,1.7341550299588198,0.9319962481713716,0.9303444264723856,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,146,1,-1.3022655421739981,1.3407396287397118,0.019237043282856803,1.321502585456855,2.64300517091371,interval,0.38006102142303877,0.7601220428460775,1.0,0.38006102142303877,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,147,0,-0.9610675137578782,0.9233653689554443,-0.018851072401216973,0.9422164413566613,1.8844328827133225,interval,0.736119124904681,1.472238249809362,0.9319962481713716,0.7898305667528036,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,147,1,-0.9610675137578782,0.9233653689554443,-0.018851072401216973,0.9422164413566613,1.8844328827133225,interval,0.2773282591178785,0.554656518235757,1.0,0.2773282591178785,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,148,0,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,interval,0.8885318892576286,1.7770637785152572,0.9319962481713716,0.9533642340309605,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,148,1,-1.4347906560759482,1.3957044259669074,-0.019543115054520444,1.4152475410214278,2.8304950820428556,interval,0.398406805419174,0.796813610838348,1.0,0.398406805419174,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,149,0,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,interval,0.686038343735274,1.372076687470548,0.9319962481713716,0.736095606695112,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,149,1,-0.85590996830905,0.8252779003869347,-0.015316033961057629,0.8405939343479923,1.6811878686959847,interval,0.2409182093072293,0.4818364186144586,1.0,0.2409182093072293,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,150,0,-0.8144288244005298,0.8392548862181486,0.012413030908809408,0.8268418553093392,1.6536837106186784,interval,0.6787201875836991,1.3574403751673982,0.9319962481713716,0.7282434762107528,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,150,1,-0.8144288244005298,0.8392548862181486,0.012413030908809408,0.8268418553093392,1.6536837106186784,interval,0.23489631219170992,0.46979262438341984,1.0,0.23489631219170992,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,151,0,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,interval,0.6442591186892941,1.2885182373785882,0.9319962481713716,0.6912679315537656,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,151,1,-0.7794240378560415,0.7516667225555955,-0.013878657650223003,0.7655453802058185,1.531090760411637,interval,0.21279721903629478,0.42559443807258956,1.0,0.21279721903629478,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,152,0,-1.2728778080271934,1.314065379263317,0.0205937856180618,1.2934715936452552,2.5869431872905104,interval,0.8599380853733902,1.7198761707467805,0.9319962481713716,0.9226840634397794,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,152,1,-1.2728778080271934,1.314065379263317,0.0205937856180618,1.2934715936452552,2.5869431872905104,interval,0.3743724433947929,0.7487448867895858,1.0,0.3743724433947929,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,153,0,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,interval,0.6087679935662764,1.2175359871325528,0.9319962481713716,0.6531871719019395,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,153,1,-0.6937322806667295,0.7204075004186045,0.013337609875937462,0.707069890542667,1.414139781085334,interval,0.19044425390502961,0.38088850781005923,1.0,0.19044425390502961,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,154,0,-0.7966809166067104,0.7634063707671257,-0.016637272919792334,0.780043643686918,1.560087287373836,interval,0.6526280654287782,1.3052561308575563,0.9319962481713716,0.7002475242891488,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,154,1,-0.7966809166067104,0.7634063707671257,-0.016637272919792334,0.780043643686918,1.560087287373836,interval,0.21923927424675393,0.43847854849350787,1.0,0.21923927424675393,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,155,0,-0.8940835656454462,0.8588510895755224,-0.01761623803496193,0.8764673276104843,1.7529346552209686,interval,0.7045350401149252,1.4090700802298504,0.9319962481713716,0.7559419273385081,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,155,1,-0.8940835656454462,0.8588510895755224,-0.01761623803496193,0.8764673276104843,1.7529346552209686,interval,0.2544732249111251,0.5089464498222502,1.0,0.2544732249111251,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,156,0,-0.9292268209994731,0.9689797402104271,0.019876459605477015,0.9491032806049501,1.8982065612099002,interval,0.7392444053470522,1.4784888106941043,0.9319962481713716,0.7931838854474905,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,156,1,-0.9292268209994731,0.9689797402104271,0.019876459605477015,0.9491032806049501,1.8982065612099002,interval,0.27994322108698116,0.5598864421739623,1.0,0.27994322108698116,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,157,0,-1.1731237389534523,1.2050166691047848,0.015946465075666216,1.1890702040291186,2.378140408058237,interval,0.8302246974946781,1.6604493949893562,0.9319962481713716,0.8908026176324476,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,157,1,-1.1731237389534523,1.2050166691047848,0.015946465075666216,1.1890702040291186,2.378140408058237,interval,0.3487614847871168,0.6975229695742337,1.0,0.3487614847871168,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,158,0,-0.8913860173428699,0.9226653347131667,0.015639658685148383,0.9070256760280183,1.8140513520560366,interval,0.7196167560392228,1.4392335120784456,0.9319962481713716,0.7721240911121164,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,158,1,-0.8913860173428699,0.9226653347131667,0.015639658685148383,0.9070256760280183,1.8140513520560366,interval,0.26437783311990976,0.5287556662398195,1.0,0.26437783311990976,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,159,0,-1.1273921839493974,1.0959081779819329,-0.015742002983732295,1.1116501809656651,2.2233003619313303,interval,0.8045746326611436,1.6091492653222872,0.9319962481713716,0.8632809780508921,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,159,1,-1.1273921839493974,1.0959081779819329,-0.015742002983732295,1.1116501809656651,2.2233003619313303,interval,0.3281511389425814,0.6563022778851628,1.0,0.3281511389425814,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,160,0,-1.4923860445597164,1.4589190080498688,-0.01673351825492375,1.4756525263047926,2.951305052609585,interval,0.9006026422272458,1.8012052844544917,0.9319962481713716,0.966315737852248,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,160,1,-1.4923860445597164,1.4589190080498688,-0.01673351825492375,1.4756525263047926,2.951305052609585,interval,0.4083936356497787,0.8167872712995574,1.0,0.4083936356497787,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,161,0,-0.8118887083898725,0.7855637226605157,-0.01316249286467841,0.7987262155251941,1.5974524310503881,interval,0.6632596982517602,1.3265193965035205,0.9319962481713716,0.7116549015654436,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,161,1,-0.8118887083898725,0.7855637226605157,-0.01316249286467841,0.7987262155251941,1.5974524310503881,interval,0.2248728659832353,0.4497457319664706,1.0,0.2248728659832353,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,162,0,-0.9685612086655188,0.9950593658781902,0.013249078606335729,0.9818102872718545,1.963620574543709,interval,0.7537913684064195,1.507582736812839,0.9319962481713716,0.808792277742963,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,162,1,-0.9685612086655188,0.9950593658781902,0.013249078606335729,0.9818102872718545,1.963620574543709,interval,0.28842878064565625,0.5768575612913125,1.0,0.28842878064565625,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,163,0,-0.8271071303752253,0.8610459715485379,0.016969420586656292,0.8440765509618816,1.6881531019237632,interval,0.6878579023049842,1.3757158046099685,0.9319962481713716,0.7380479306162441,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,163,1,-0.8271071303752253,0.8610459715485379,0.016969420586656292,0.8440765509618816,1.6881531019237632,interval,0.24276246607836216,0.4855249321567243,1.0,0.24276246607836216,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,164,0,-1.1176997476072557,1.1538089483476917,0.018054600370218,1.1357543479774737,2.2715086959549473,interval,0.8128896510775949,1.6257793021551898,0.9319962481713716,0.8722027075457971,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,164,1,-1.1176997476072557,1.1538089483476917,0.018054600370218,1.1357543479774737,2.2715086959549473,interval,0.3353903081046323,0.6707806162092645,1.0,0.3353903081046323,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,165,0,-0.957106884738263,0.9915651490227829,0.017229132142259962,0.974336016880523,1.948672033761046,interval,0.750506261596203,1.501012523192406,0.9319962481713716,0.8052674708387915,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,165,1,-0.957106884738263,0.9915651490227829,0.017229132142259962,0.974336016880523,1.948672033761046,interval,0.2873039192480161,0.5746078384960323,1.0,0.2873039192480161,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,166,0,-1.1951923778641023,1.152966248577504,-0.021113064643299095,1.1740793132208032,2.3481586264416063,interval,0.8254583237888886,1.6509166475777772,0.9319962481713716,0.8856884621676147,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,166,1,-1.1951923778641023,1.152966248577504,-0.021113064643299095,1.1740793132208032,2.3481586264416063,interval,0.34626368738910285,0.6925273747782057,1.0,0.34626368738910285,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,167,0,-1.1001154688531447,1.0726088877062845,-0.01375329057343011,1.0863621782797146,2.172724356559429,interval,0.7954912969947248,1.5909825939894495,0.9319962481713716,0.853534870505673,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,167,1,-1.1001154688531447,1.0726088877062845,-0.01375329057343011,1.0863621782797146,2.172724356559429,interval,0.3204325416599745,0.640865083319949,1.0,0.3204325416599745,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,168,0,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,interval,0.7812510952946343,1.5625021905892686,0.9319962481713716,0.8382556226245463,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,168,1,-1.0289821496950908,1.0687833262456736,0.019900588275291398,1.0488827379703822,2.0977654759407645,interval,0.3112623915859444,0.6225247831718888,1.0,0.3112623915859444,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,169,0,-0.8810525006112329,0.8477341734459161,-0.0166591635826584,0.8643933370285745,1.728786674057149,interval,0.6984150485445756,1.3968300970891512,0.9319962481713716,0.7493753863439951,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,169,1,-0.8810525006112329,0.8477341734459161,-0.0166591635826584,0.8643933370285745,1.728786674057149,interval,0.24988646592851105,0.4997729318570221,1.0,0.24988646592851105,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,170,0,-0.8468298613304298,0.8765102975610902,0.014840218115330206,0.86167007944576,1.72334015889152,interval,0.6970382253264997,1.3940764506529995,0.9319962481713716,0.7478981022661061,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,170,1,-0.8468298613304298,0.8765102975610902,0.014840218115330206,0.86167007944576,1.72334015889152,interval,0.24827762471194087,0.49655524942388174,1.0,0.24827762471194087,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,171,0,-1.156771804990858,1.1214757897709782,-0.017648007609939897,1.139123797380918,2.278247594761836,interval,0.8140333370348873,1.6280666740697747,0.9319962481713716,0.8734298433411786,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,171,1,-1.156771804990858,1.1214757897709782,-0.017648007609939897,1.139123797380918,2.278247594761836,interval,0.3361877340231146,0.6723754680462292,1.0,0.3361877340231146,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,172,0,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,interval,0.5700181648878115,1.140036329775623,0.9319962481713716,0.6116099351324845,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,172,1,-0.6602039211953794,0.6350755481873449,-0.01256418650401725,0.6476397346913622,1.2952794693827243,interval,0.16733060766540886,0.3346612153308177,1.0,0.16733060766540886,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,173,0,-0.6736564656359888,0.6502014587413384,-0.011727503447325205,0.6619289621886636,1.3238579243773272,interval,0.5795927637210074,1.1591855274420149,0.9319962481713716,0.6218831512017357,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,173,1,-0.6736564656359888,0.6502014587413384,-0.011727503447325205,0.6619289621886636,1.3238579243773272,interval,0.1725076065144594,0.3450152130289188,1.0,0.1725076065144594,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,174,0,-1.2671951827874366,1.225791054927446,-0.020702063929995296,1.2464931188574413,2.4929862377148826,interval,0.8471948914625302,1.6943897829250605,0.9319962481713716,0.9090110535581808,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,174,1,-1.2671951827874366,1.225791054927446,-0.020702063929995296,1.2464931188574413,2.4929862377148826,interval,0.3638349032048351,0.7276698064096702,1.0,0.3638349032048351,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,175,0,-1.3831413857359594,1.4281571697421036,0.022507892003072127,1.4056492777390315,2.811298555478063,interval,0.8864705421197983,1.7729410842395965,0.9319962481713716,0.9511524792713518,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,175,1,-1.3831413857359594,1.4281571697421036,0.022507892003072127,1.4056492777390315,2.811298555478063,interval,0.39719743953234826,0.7943948790646965,1.0,0.39719743953234826,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,176,0,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,interval,0.7768733551762651,1.5537467103525302,0.9319962481713716,0.8335584576660408,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,176,1,-1.0572251389801488,1.018236306714756,-0.019494416132696424,1.0377307228474524,2.0754614456949048,interval,0.3077981974496608,0.6155963948993216,1.0,0.3077981974496608,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,177,0,-0.8553465281258208,0.8396584283206341,-0.007844049902593353,0.8475024782232274,1.6950049564464549,interval,0.6897402056887825,1.379480411377565,0.9319962481713716,0.7400675775702864,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,177,1,-0.8553465281258208,0.8396584283206341,-0.007844049902593353,0.8475024782232274,1.6950049564464549,interval,0.2407155143330154,0.4814310286660308,1.0,0.2407155143330154,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,178,0,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,interval,0.7801622954825553,1.5603245909651107,0.9319962481713716,0.8370873777799825,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,178,1,-1.0655132612430596,1.026646201899785,-0.01943352967163725,1.0460797315714223,2.0921594631428446,interval,0.3102867218469211,0.6205734436938422,1.0,0.3102867218469211,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,179,0,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,interval,0.8389863521718962,1.6779727043437924,0.9319962481713716,0.9002035725122649,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,179,1,-1.2375301238723018,1.1985866600574022,-0.019471731907449774,1.218058391964852,2.436116783929704,interval,0.3568010299551185,0.713602059910237,1.0,0.3568010299551185,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,180,0,-1.0633221779271935,1.0983992430851333,0.017538532578969868,1.0808607105061634,2.161721421012327,interval,0.7934276697788627,1.5868553395577254,0.9319962481713716,0.8513206693005598,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,180,1,-1.0633221779271935,1.0983992430851333,0.017538532578969868,1.0808607105061634,2.161721421012327,interval,0.3199386353559291,0.6398772707118582,1.0,0.3199386353559291,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,181,0,-1.120720769777153,1.160081025234179,0.01968012772851302,1.140400897505666,2.280801795011332,interval,0.8144428814895603,1.6288857629791207,0.9319962481713716,0.8738692704906725,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,181,1,-1.120720769777153,1.160081025234179,0.01968012772851302,1.140400897505666,2.280801795011332,interval,0.337074922325155,0.67414984465031,1.0,0.337074922325155,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,182,0,-0.9468539302832151,0.9082390537521356,-0.01930743826553971,0.9275464920176754,1.8550929840353507,interval,0.7293207116853668,1.4586414233707337,0.9319962481713716,0.7825361026037762,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,182,1,-0.9468539302832151,0.9082390537521356,-0.01930743826553971,0.9275464920176754,1.8550929840353507,interval,0.2725843789277421,0.5451687578554842,1.0,0.2725843789277421,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,183,0,-0.9012005330706289,0.9364696817911625,0.01763457436026683,0.9188351074308957,1.8376702148617914,interval,0.7252389841503541,1.4504779683007083,0.9319962481713716,0.7781565489918155,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,183,1,-0.9012005330706289,0.9364696817911625,0.01763457436026683,0.9188351074308957,1.8376702148617914,interval,0.26908157512926484,0.5381631502585297,1.0,0.26908157512926484,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,184,0,-1.4793999358375653,1.440529834424064,-0.019435050706750667,1.4599648851308147,2.9199297702616294,interval,0.897579918198107,1.795159836396214,0.9319962481713716,0.9630724586705243,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,184,1,-1.4793999358375653,1.440529834424064,-0.019435050706750667,1.4599648851308147,2.9199297702616294,interval,0.40622087021884185,0.8124417404376837,1.0,0.40622087021884185,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,185,0,-0.7321511044331087,0.7389444504690762,0.0033966730179837423,0.7355477774510925,1.471095554902185,interval,0.6264433045498379,1.2528866090996758,0.9319962481713716,0.672152174194858,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,185,1,-0.7321511044331087,0.7389444504690762,0.0033966730179837423,0.7355477774510925,1.471095554902185,interval,0.1975105243813362,0.3950210487626724,1.0,0.1975105243813362,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,186,0,-0.8942204521232301,0.8708685501338455,-0.011675950994692319,0.8825445011285378,1.7650890022570755,interval,0.7076435955338851,1.4152871910677702,0.9319962481713716,0.7592773006568655,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,186,1,-0.8942204521232301,0.8708685501338455,-0.011675950994692319,0.8825445011285378,1.7650890022570755,interval,0.25452117655436113,0.5090423531087223,1.0,0.25452117655436113,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,187,0,-1.0714555174736884,1.0473124800913574,-0.012071518691165517,1.0593839987825229,2.1187679975650457,interval,0.7853841340753871,1.5707682681507742,0.9319962481713716,0.8426902314428351,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,187,1,-1.0714555174736884,1.0473124800913574,-0.012071518691165517,1.0593839987825229,2.1187679975650457,interval,0.31205707750163003,0.6241141550032601,1.0,0.31205707750163003,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,188,0,-0.8561385666094257,0.8238401922534622,-0.01614918717798175,0.8399893794314439,1.6799787588628878,interval,0.6857087069259329,1.3714174138518658,0.9319962481713716,0.7357419177076425,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,188,1,-0.8561385666094257,0.8238401922534622,-0.01614918717798175,0.8399893794314439,1.6799787588628878,interval,0.24100042558773194,0.4820008511754639,1.0,0.24100042558773194,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,189,0,-1.0554762744104922,1.0911960330035595,0.017859879296533654,1.0733361537070258,2.1466723074140517,interval,0.7906202911240923,1.5812405822481845,0.9319962481713716,0.848308448317612,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,189,1,-1.0554762744104922,1.0911960330035595,0.017859879296533654,1.0733361537070258,2.1466723074140517,interval,0.31785501675906824,0.6357100335181365,1.0,0.31785501675906824,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,190,0,-0.8409602339792387,0.8183358148780696,-0.01131220955058454,0.8296480244286542,1.6592960488573083,interval,0.6802402018685354,1.3604804037370708,0.9319962481713716,0.7298743993907748,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,190,1,-0.8409602339792387,0.8183358148780696,-0.01131220955058454,0.8296480244286542,1.6592960488573083,interval,0.2355157391293331,0.4710314782586662,1.0,0.2355157391293331,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,191,0,-0.9923663640591591,0.9567994816892673,-0.017783441184945903,0.9745829228742132,1.9491658457484264,interval,0.7506077031966338,1.5012154063932677,0.9319962481713716,0.8053763141958649,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,191,1,-0.9923663640591591,0.9567994816892673,-0.017783441184945903,0.9745829228742132,1.9491658457484264,interval,0.28756217983474036,0.5751243596694807,1.0,0.28756217983474036,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,192,0,-0.6229070664001423,0.5904531146635356,-0.01622697586830335,0.6066800905318389,1.2133601810636778,interval,0.541685133626351,1.083370267252702,0.9319962481713716,0.5812095646191359,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,192,1,-0.6229070664001423,0.5904531146635356,-0.01622697586830335,0.6066800905318389,1.2133601810636778,interval,0.1529868268347906,0.3059736536695812,1.0,0.1529868268347906,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,193,0,-0.737441641819339,0.710743302455173,-0.013349169682082995,0.724092472137256,1.448184944274512,interval,0.6193698500330489,1.2387397000660978,0.9319962481713716,0.6645626001695683,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,193,1,-0.737441641819339,0.710743302455173,-0.013349169682082995,0.724092472137256,1.448184944274512,interval,0.1969389804558423,0.3938779609116846,1.0,0.1969389804558423,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,194,0,-0.9262917881438902,0.9624928742496374,0.018100543052873563,0.9443923311967638,1.8887846623935276,interval,0.7371235377640917,1.4742470755281833,0.9319962481713716,0.790908267292244,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,194,1,-0.9262917881438902,0.9624928742496374,0.018100543052873563,0.9443923311967638,1.8887846623935276,interval,0.2778007088991379,0.5556014177982758,1.0,0.2778007088991379,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,195,0,-1.4438322648629727,1.481626895743142,0.01889731544008466,1.4627295803030573,2.9254591606061147,interval,0.8981194455052559,1.7962388910105118,0.9319962481713716,0.9636513529613624,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,195,1,-1.4438322648629727,1.481626895743142,0.01889731544008466,1.4627295803030573,2.9254591606061147,interval,0.40659668471103166,0.8131933694220633,1.0,0.40659668471103166,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,196,0,-1.1524474377340883,1.194174318747599,0.02086344050675537,1.1733108782408437,2.3466217564816874,interval,0.8252161229039865,1.650432245807973,0.9319962481713716,0.8854285889273764,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,196,1,-1.1524474377340883,1.194174318747599,0.02086344050675537,1.1733108782408437,2.3466217564816874,interval,0.34600302155351115,0.6920060431070223,1.0,0.34600302155351115,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,197,0,-0.6221827602303873,0.6073776418168664,-0.007402559206760473,0.6147802010236268,1.2295604020472537,interval,0.5474622420835513,1.0949244841671026,0.9319962481713716,0.5874082037966383,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,197,1,-0.6221827602303873,0.6073776418168664,-0.007402559206760473,0.6147802010236268,1.2295604020472537,interval,0.15270878083270878,0.30541756166541756,1.0,0.15270878083270878,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,198,0,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,interval,0.7195980959612935,1.439196191922587,0.9319962481713716,0.7721040694886755,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,198,1,-0.8900304609633193,0.9240067976547858,0.016988168345733246,0.9070186293090525,1.814037258618105,interval,0.26483726084892223,0.5296745216978445,1.0,0.26483726084892223,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,199,0,-1.095266671314244,1.1346180087386017,0.01967566871217885,1.114942340026423,2.229884680052846,interval,0.8056931033537187,1.6113862067074374,0.9319962481713716,0.8644810587323,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,199,1,-1.095266671314244,1.1346180087386017,0.01967566871217885,1.114942340026423,2.229884680052846,interval,0.3301543879870156,0.6603087759740311,1.0,0.3301543879870156,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,200,0,-0.9767008442562581,1.0102996945497789,0.01679942514676036,0.9935002694030185,1.987000538806037,interval,0.7587600608527837,1.5175201217055674,0.9319962481713716,0.8141235142754202,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,200,1,-0.9767008442562581,1.0102996945497789,0.01679942514676036,0.9935002694030185,1.987000538806037,interval,0.29329064116050324,0.5865812823210065,1.0,0.29329064116050324,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,201,0,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,interval,0.8065115457223337,1.6130230914446675,0.9319962481713716,0.8653592193152646,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,201,1,-1.09950069888472,1.1349408402969055,0.017720070706092717,1.1172207695908127,2.2344415391816255,interval,0.33024348214164084,0.6604869642832817,1.0,0.33024348214164084,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,202,0,-0.7951331280804159,0.8148313974508758,0.009849134685229965,0.8049822627656459,1.6099645255312918,interval,0.6667769999669,1.3335539999338,0.9319962481713716,0.7154288456366144,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,202,1,-0.7951331280804159,0.8148313974508758,0.009849134685229965,0.8049822627656459,1.6099645255312918,interval,0.2259579335961155,0.451915867192231,1.0,0.2259579335961155,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,203,0,-0.9291054457961191,0.8942568543418769,-0.01742429572712112,0.911681150068998,1.823362300137996,interval,0.7218332327560163,1.4436664655120326,0.9319962481713716,0.7745022945878732,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,203,1,-0.9291054457961191,0.8942568543418769,-0.01742429572712112,0.911681150068998,1.823362300137996,interval,0.266578899388455,0.53315779877691,1.0,0.266578899388455,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,204,0,-0.7641244024694368,0.7349866898999596,-0.014568856284738585,0.7495555461846982,1.4991110923693964,interval,0.634803288839437,1.269606577678874,0.9319962481713716,0.6811221505289923,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,204,1,-0.7641244024694368,0.7349866898999596,-0.014568856284738585,0.7495555461846982,1.4991110923693964,interval,0.20704605933785514,0.4140921186757103,1.0,0.20704605933785514,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,205,0,-1.0117373774449654,0.9756317416300795,-0.018052817907442953,0.9936845595375224,1.9873691190750449,interval,0.7588241749078044,1.5176483498156088,0.9319962481713716,0.8141923064568763,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,205,1,-1.0117373774449654,0.9756317416300795,-0.018052817907442953,0.9936845595375224,1.9873691190750449,interval,0.29374553222135164,0.5874910644427033,1.0,0.29374553222135164,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,206,0,-1.2710325117297012,1.3114418482049404,0.020204668237619572,1.2912371799673208,2.5824743599346416,interval,0.8593585171479842,1.7187170342959683,0.9319962481713716,0.9220622066173477,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,206,1,-1.2710325117297012,1.3114418482049404,0.020204668237619572,1.2912371799673208,2.5824743599346416,interval,0.37380098020882946,0.7476019604176589,1.0,0.37380098020882946,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,207,0,-1.3112708446293022,1.2772121752583019,-0.017029334685500164,1.294241509943802,2.588483019887604,interval,0.8601684943175822,1.7203369886351645,0.9319962481713716,0.9229312843321854,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,207,1,-1.3112708446293022,1.2772121752583019,-0.017029334685500164,1.294241509943802,2.588483019887604,interval,0.37376365693276586,0.7475273138655317,1.0,0.37376365693276586,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,208,0,-1.1651989100051083,1.2002604364732328,0.01753076323406222,1.1827296732391706,2.365459346478341,interval,0.8282304919900442,1.6564609839800883,0.9319962481713716,0.8886629035418098,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,208,1,-1.1651989100051083,1.2002604364732328,0.01753076323406222,1.1827296732391706,2.365459346478341,interval,0.3475562148050939,0.6951124296101878,1.0,0.3475562148050939,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,209,0,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,interval,0.837946948397919,1.675893896795838,0.9319962481713716,0.8990883279219389,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,209,1,-1.198239351087613,1.2306822718446646,0.016221460378525787,1.2144608114661388,2.4289216229322776,interval,0.3551366300474823,0.7102732600949646,1.0,0.3551366300474823,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,210,0,-0.9228447532260018,0.9507075443892261,0.01393139558161216,0.936776148807614,1.873552297615228,interval,0.7336718087944889,1.4673436175889778,0.9319962481713716,0.7872046805274096,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,210,1,-0.9228447532260018,0.9507075443892261,0.01393139558161216,0.936776148807614,1.873552297615228,interval,0.2738763761915111,0.5477527523830222,1.0,0.2738763761915111,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,211,0,-1.321932019950464,1.284569394122059,-0.018681312914202497,1.3032507070362616,2.606501414072523,interval,0.8624805958471207,1.7249611916942413,0.9319962481713716,0.9254120899513871,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,211,1,-1.321932019950464,1.284569394122059,-0.018681312914202497,1.3032507070362616,2.606501414072523,interval,0.37607306776507016,0.7521461355301403,1.0,0.37607306776507016,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,212,0,-1.1136630602458562,1.0736732395863262,-0.01999491032976497,1.0936681499160912,2.1873362998321824,interval,0.7980972624876542,1.5961945249753084,0.9319962481713716,0.8563309820759101,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,212,1,-1.1136630602458562,1.0736732395863262,-0.01999491032976497,1.0936681499160912,2.1873362998321824,interval,0.3242970616249836,0.6485941232499672,1.0,0.3242970616249836,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,213,0,-1.397693525091098,1.3709670065547264,-0.013363259268185823,1.3843302658229122,2.7686605316458244,interval,0.8818822328733968,1.7637644657467937,0.9319962481713716,0.946229380862529,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,213,1,-1.397693525091098,1.3709670065547264,-0.013363259268185823,1.3843302658229122,2.7686605316458244,interval,0.3914816006105234,0.7829632012210468,1.0,0.3914816006105234,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,214,0,-0.6308511366333197,0.5817134666571074,-0.024568834988106136,0.6062823016452136,1.2125646032904271,interval,0.5412738541667765,1.082547708333553,0.9319962481713716,0.580768275868906,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,214,1,-0.6308511366333197,0.5817134666571074,-0.024568834988106136,0.6062823016452136,1.2125646032904271,interval,0.15603821361353043,0.31207642722706086,1.0,0.15603821361353043,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,215,0,-0.6750710083304652,0.6486981605886053,-0.013186423870929942,0.6618845844595352,1.3237691689190705,interval,0.57954930726608,1.15909861453216,0.9319962481713716,0.6218365239164727,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,215,1,-0.6750710083304652,0.6486981605886053,-0.013186423870929942,0.6618845844595352,1.3237691689190705,interval,0.17305179269644722,0.34610358539289443,1.0,0.17305179269644722,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,216,0,-1.2424597126868333,1.285783314865348,0.021661801089257304,1.2641215137760906,2.528243027552181,interval,0.8520869167139018,1.7041738334278036,0.9319962481713716,0.9142600288206564,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,216,1,-1.2424597126868333,1.285783314865348,0.021661801089257304,1.2641215137760906,2.528243027552181,interval,0.36809759259862157,0.7361951851972431,1.0,0.36809759259862157,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,217,0,-1.0176420196916929,0.9812025096472038,-0.018219755022244544,0.9994222646694484,1.9988445293388968,interval,0.7612451814013625,1.522490362802725,0.9319962481713716,0.8167899633662344,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,217,1,-1.0176420196916929,0.9812025096472038,-0.018219755022244544,0.9994222646694484,1.9988445293388968,interval,0.2956069671477636,0.5912139342955272,1.0,0.2956069671477636,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,218,0,-0.9798393779987575,1.0123201516079219,0.016240386804582174,0.9960797648033397,1.9921595296066794,interval,0.7598581568044667,1.5197163136089333,0.9319962481713716,0.8153017335588535,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,218,1,-0.9798393779987575,1.0123201516079219,0.016240386804582174,0.9960797648033397,1.9921595296066794,interval,0.2939297401650445,0.587859480330089,1.0,0.2939297401650445,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,219,0,-1.0508980722181285,1.0267112727541425,-0.012093399731992971,1.0388046724861355,2.077609344972271,interval,0.7773706233504556,1.5547412467009112,0.9319962481713716,0.8340920093570118,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,219,1,-1.0508980722181285,1.0267112727541425,-0.012093399731992971,1.0388046724861355,2.077609344972271,interval,0.3058834284997655,0.611766856999531,1.0,0.3058834284997655,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,220,0,-1.0276272819085597,0.9881549383021557,-0.019736171803201974,1.0078911101053576,2.015782220210715,interval,0.7647647301852281,1.5295294603704561,0.9319962481713716,0.820566318465057,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,220,1,-1.0276272819085597,0.9881549383021557,-0.019736171803201974,1.0078911101053576,2.015782220210715,interval,0.29872969648748143,0.5974593929749629,1.0,0.29872969648748143,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,221,0,-0.9182714052195694,0.9526902849630151,0.017209439871722854,0.9354808450912923,1.8709616901825845,interval,0.7330386073110371,1.4660772146220742,0.9319962481713716,0.7865252770591078,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,221,1,-0.9182714052195694,0.9526902849630151,0.017209439871722854,0.9354808450912923,1.8709616901825845,interval,0.2745394461256135,0.549078892251227,1.0,0.2745394461256135,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,222,0,-1.0931660046004659,1.1314416045523372,0.019137799975935676,1.1123038045764015,2.224607609152803,interval,0.8047714243766837,1.6095428487533674,0.9319962481713716,0.8634921288102714,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,222,1,-1.0931660046004659,1.1314416045523372,0.019137799975935676,1.1123038045764015,2.224607609152803,interval,0.3292759210988318,0.6585518421976636,1.0,0.3292759210988318,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,223,0,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,interval,0.6161807360939711,1.2323614721879421,0.9319962481713716,0.6611407903228708,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,223,1,-0.7347884981061659,0.7031688277584787,-0.0158098351738436,0.7189786629323223,1.4379573258646445,interval,0.19592934004425883,0.39185868008851765,1.0,0.19592934004425883,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,224,0,-1.2999700704514805,1.2604818939852531,-0.019744088233113688,1.2802259822183668,2.5604519644367336,interval,0.8564561813723275,1.712912362744655,0.9319962481713716,0.9189480999014128,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,224,1,-1.2999700704514805,1.2604818939852531,-0.019744088233113688,1.2802259822183668,2.5604519644367336,interval,0.3712767620511541,0.7425535241023082,1.0,0.3712767620511541,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,225,0,-0.8019294265090882,0.837801627533995,0.017936100512453423,0.8198655270215416,1.6397310540430832,interval,0.6748784737557538,1.3497569475115077,0.9319962481713716,0.7241214490722498,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,225,1,-0.8019294265090882,0.837801627533995,0.017936100512453423,0.8198655270215416,1.6397310540430832,interval,0.23436795226932694,0.4687359045386539,1.0,0.23436795226932694,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,226,0,-1.0355052941917033,0.9968705997566256,-0.01931734721753886,1.0161879469741644,2.0323758939483287,interval,0.7681918443135693,1.5363836886271387,0.9319962481713716,0.8242434943496869,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,226,1,-1.0355052941917033,0.9968705997566256,-0.01931734721753886,1.0161879469741644,2.0323758939483287,interval,0.3011709880446853,0.6023419760893706,1.0,0.3011709880446853,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,227,0,-1.2227325622777168,1.2664926147953335,0.02188002625880836,1.2446125885365251,2.4892251770730502,interval,0.8466512894064633,1.6933025788129266,0.9319962481713716,0.9084277872015474,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,227,1,-1.2227325622777168,1.2664926147953335,0.02188002625880836,1.2446125885365251,2.4892251770730502,interval,0.3636716119668499,0.7273432239336998,1.0,0.3636716119668499,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,228,0,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,interval,0.7659704281119591,1.5319408562239183,0.9319962481713716,0.8218599909761822,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,228,1,-0.9930624745502175,1.028424554536726,0.01768103999325432,1.0107435145434718,2.0214870290869436,interval,0.29897766309344787,0.5979553261868957,1.0,0.29897766309344787,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,229,0,-0.778224352240515,0.7634707519083999,-0.007376800166057573,0.7708475520744574,1.5416951041489149,interval,0.6474015533685031,1.2948031067370063,0.9319962481713716,0.6946396561560638,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,229,1,-0.778224352240515,0.7634707519083999,-0.007376800166057573,0.7708475520744574,1.5416951041489149,interval,0.21234754866793204,0.4246950973358641,1.0,0.21234754866793204,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,230,0,-1.4736909184242273,1.4349512883955997,-0.019369815014313785,1.4543211034099135,2.908642206819827,interval,0.8964780016272622,1.7929560032545244,0.9319962481713716,0.9618901399938056,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,230,1,-1.4736909184242273,1.4349512883955997,-0.019369815014313785,1.4543211034099135,2.908642206819827,interval,0.40525132035880546,0.8105026407176109,1.0,0.40525132035880546,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,231,0,-0.9741772249753827,1.0156090479613669,0.020715911492992067,0.9948931364683749,1.9897862729367497,interval,0.7593031115465589,1.5186062230931179,0.9319962481713716,0.8147061890392304,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,231,1,-0.9741772249753827,1.0156090479613669,0.020715911492992067,0.9948931364683749,1.9897862729367497,interval,0.29496731563828993,0.5899346312765799,1.0,0.29496731563828993,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,232,0,-0.9915097879699296,1.0315020359202351,0.01999612397515277,1.0115059119450824,2.023011823890165,interval,0.7662577104113392,1.5325154208226783,0.9319962481713716,0.8221682350275329,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,232,1,-0.9915097879699296,1.0315020359202351,0.01999612397515277,1.0115059119450824,2.023011823890165,interval,0.2999329128856286,0.5998658257712572,1.0,0.2999329128856286,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,233,0,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,interval,0.7870708786771334,1.5741417573542669,0.9319962481713716,0.8445000505328323,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,233,1,-1.08080947479655,1.0470137954558827,-0.016897839670333648,1.0639116351262163,2.1278232702524327,interval,0.3148203998560853,0.6296407997121706,1.0,0.3148203998560853,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,234,0,-0.9553499690208583,0.9911598413697275,0.01790493617443456,0.9732549051952929,1.9465098103905858,interval,0.7500259863213763,1.5000519726427526,0.9319962481713716,0.804752151945857,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,234,1,-0.9553499690208583,0.9911598413697275,0.01790493617443456,0.9732549051952929,1.9465098103905858,interval,0.2871731986340159,0.5743463972680318,1.0,0.2871731986340159,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,235,0,-0.9690273603532125,1.0071673197752011,0.019069979710994334,0.9880973400642068,1.9761946801284136,interval,0.756432205374689,1.512864410749378,0.9319962481713716,0.8116258052098932,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,235,1,-0.9690273603532125,1.0071673197752011,0.019069979710994334,0.9880973400642068,1.9761946801284136,interval,0.2922972930370976,0.5845945860741952,1.0,0.2922972930370976,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,236,0,-1.1558838037706325,1.1917587708387827,0.017937483534075094,1.1738212873047076,2.347642574609415,interval,0.8254087168793831,1.6508174337587662,0.9319962481713716,0.885635235655595,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,236,1,-1.1558838037706325,1.1917587708387827,0.017937483534075094,1.1738212873047076,2.347642574609415,interval,0.34538316546999365,0.6907663309399873,1.0,0.34538316546999365,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,237,0,-0.9032024251467428,0.9272904688323389,0.01204402184279807,0.9152464469895408,1.8304928939790817,interval,0.7235908672194047,1.4471817344388094,0.9319962481713716,0.7763881760673716,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,237,1,-0.9032024251467428,0.9272904688323389,0.01204402184279807,0.9152464469895408,1.8304928939790817,interval,0.26595975466700533,0.5319195093340107,1.0,0.26595975466700533,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,238,0,-1.2475163628525585,1.297087234801867,0.024785435974654213,1.2723017988272127,2.5446035976544255,interval,0.8542785771854895,1.708557154370979,0.9319962481713716,0.9166116053166861,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,238,1,-1.2475163628525585,1.297087234801867,0.024785435974654213,1.2723017988272127,2.5446035976544255,interval,0.37063590348916114,0.7412718069783223,1.0,0.37063590348916114,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,239,0,-0.9472028721092913,0.9617653086795486,0.007281218285128621,0.95448409039442,1.90896818078884,interval,0.7417886814323581,1.4835773628647162,0.9319962481713716,0.7959138063997454,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,239,1,-0.9472028721092913,0.9617653086795486,0.007281218285128621,0.95448409039442,1.90896818078884,interval,0.27755962526177946,0.5551192505235589,1.0,0.27755962526177946,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,240,0,-1.1390201794660135,1.1757004326923166,0.01834012661315154,1.157360306079165,2.31472061215833,interval,0.8200874592669462,1.6401749185338923,0.9319962481713716,0.8799257087955057,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,240,1,-1.1390201794660135,1.1757004326923166,0.01834012661315154,1.157360306079165,2.31472061215833,interval,0.34121318898571984,0.6824263779714397,1.0,0.34121318898571984,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,241,0,-0.80342181434636,0.8305734264130719,0.013575806033355953,0.8169976203797159,1.6339952407594318,interval,0.6733645988279076,1.3467291976558151,0.9319962481713716,0.7224971132116532,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,241,1,-0.80342181434636,0.8305734264130719,0.013575806033355953,0.8169976203797159,1.6339952407594318,interval,0.2317332834599305,0.463466566919861,1.0,0.2317332834599305,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,242,0,-0.8216464738019208,0.8370838065220393,0.007718666360059245,0.82936514016198,1.65873028032396,interval,0.680113192950857,1.360226385901714,0.9319962481713716,0.7297381231794408,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,242,1,-0.8216464738019208,0.8370838065220393,0.007718666360059245,0.82936514016198,1.65873028032396,interval,0.23410680642078796,0.4682136128415759,1.0,0.23410680642078796,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,243,0,-1.0947451042886664,1.1262257894330194,0.01574034257217649,1.110485446860843,2.220970893721686,interval,0.804163543656955,1.60832708731391,0.9319962481713716,0.8628398936527572,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,243,1,-1.0947451042886664,1.1262257894330194,0.01574034257217649,1.110485446860843,2.220970893721686,interval,0.3278261437909269,0.6556522875818538,1.0,0.3278261437909269,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,244,0,-1.3579344396070399,1.401979835836499,0.022022698114729566,1.3799571377217694,2.759914275443539,interval,0.8808459842054002,1.7616919684108003,0.9319962481713716,0.9451175215926769,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,244,1,-1.3579344396070399,1.401979835836499,0.022022698114729566,1.3799571377217694,2.759914275443539,interval,0.3923020813160433,0.7846041626320867,1.0,0.3923020813160433,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,245,0,-1.075784461901336,1.0586892018922316,-0.008547630004552165,1.0672368318967838,2.1344736637935675,interval,0.7883961245671858,1.5767922491343715,0.9319962481713716,0.8459219939072317,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,245,1,-1.075784461901336,1.0586892018922316,-0.008547630004552165,1.0672368318967838,2.1344736637935675,interval,0.31333949692570723,0.6266789938514145,1.0,0.31333949692570723,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,246,0,-0.8996611244567158,0.9223465961082323,0.011342735825758288,0.911003860282474,1.822007720564948,interval,0.7215692309844932,1.4431384619689864,0.9319962481713716,0.7742190297442206,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,246,1,-0.8996611244567158,0.9223465961082323,0.011342735825758288,0.911003860282474,1.822007720564948,interval,0.2642685975413804,0.5285371950827608,1.0,0.2642685975413804,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,247,0,-0.8285968972008306,0.819679600921353,-0.004458648139738841,0.8241382490610918,1.6482764981221836,interval,0.67730866863177,1.35461733726354,0.9319962481713716,0.7267289648007567,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,247,1,-0.8285968972008306,0.819679600921353,-0.004458648139738841,0.8241382490610918,1.6482764981221836,interval,0.23101092194822404,0.46202184389644807,1.0,0.23101092194822404,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,248,0,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,interval,0.7391295833538607,1.4782591667077214,0.9319962481713716,0.7930606853880302,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,248,1,-0.9630373114550294,0.9343823587700507,-0.01432747634248932,0.94870983511254,1.89741967022508,interval,0.277981009150684,0.555962018301368,1.0,0.277981009150684,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,249,0,-0.8971474300455687,0.9299772734367956,0.016414921695613427,0.9135623517411822,1.8271247034823643,interval,0.7227446821664513,1.4454893643329025,0.9319962481713716,0.7754802485358889,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,249,1,-0.8971474300455687,0.9299772734367956,0.016414921695613427,0.9135623517411822,1.8271247034823643,interval,0.26687597916056977,0.5337519583211395,1.0,0.26687597916056977,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,250,0,-0.7261210591057341,0.7589229120372532,0.016400926465759524,0.7425219855714936,1.4850439711429873,interval,0.6305643065146452,1.2611286130292905,0.9319962481713716,0.6765738679225881,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,250,1,-0.7261210591057341,0.7589229120372532,0.016400926465759524,0.7425219855714936,1.4850439711429873,interval,0.20508303938985106,0.4101660787797021,1.0,0.20508303938985106,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,251,0,-0.63648666478885,0.6638130918043738,0.013663213507761895,0.6501498782966119,1.3002997565932237,interval,0.5716990146421077,1.1433980292842154,0.9319962481713716,0.6134134292534041,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,251,1,-0.63648666478885,0.6638130918043738,0.013663213507761895,0.6501498782966119,1.3002997565932237,interval,0.16871974057148242,0.33743948114296485,1.0,0.16871974057148242,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,252,0,-1.0954772645448596,1.134105751171441,0.01931424331329068,1.1147915078581503,2.2295830157163006,interval,0.8056441661058092,1.6112883322116185,0.9319962481713716,0.8644285507441987,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,252,1,-1.0954772645448596,1.134105751171441,0.01931424331329068,1.1147915078581503,2.2295830157163006,interval,0.3300129452993124,0.6600258905986248,1.0,0.3300129452993124,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,253,0,-0.7874311842140019,0.822751261165814,0.017660038475906026,0.805091222689908,1.610182445379816,interval,0.6667579495812916,1.3335158991625833,0.9319962481713716,0.7154084052264242,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,253,1,-0.7874311842140019,0.822751261165814,0.017660038475906026,0.805091222689908,1.610182445379816,interval,0.228869797367434,0.457739594734868,1.0,0.228869797367434,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,254,0,-0.9847557236564188,1.0201811473465006,0.017712711845040863,1.0024684355014597,2.0049368710029194,interval,0.7625287820926692,1.5250575641853383,0.9319962481713716,0.8181672228711145,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,254,1,-0.9847557236564188,1.0201811473465006,0.017712711845040863,1.0024684355014597,2.0049368710029194,interval,0.2964040388903658,0.5928080777807316,1.0,0.2964040388903658,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,255,0,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,interval,0.8536266105409407,1.7072532210818814,0.9319962481713716,0.9159120674743095,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,255,1,-1.288565530322774,1.2507822038897218,-0.018891663216526133,1.269673867106248,2.539347734212496,interval,0.36872611188030885,0.7374522237606177,1.0,0.36872611188030885,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,256,0,-0.8345849622272772,0.8628897940747233,0.014152415923723072,0.8487373781510003,1.6974747563020005,interval,0.6903369060260611,1.3806738120521223,0.9319962481713716,0.7407078165610006,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,256,1,-0.8345849622272772,0.8628897940747233,0.014152415923723072,0.8487373781510003,1.6974747563020005,interval,0.24342304734780995,0.4868460946956199,1.0,0.24342304734780995,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,257,0,-0.6351214718940507,0.5726601313824508,-0.031230670255799953,0.6038908016382507,1.2077816032765014,interval,0.5394394391054169,1.0788788782108338,0.9319962481713716,0.5788000114419205,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,257,1,-0.6351214718940507,0.5726601313824508,-0.031230670255799953,0.6038908016382507,1.2077816032765014,interval,0.15767968336803823,0.31535936673607645,1.0,0.15767968336803823,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,258,0,-0.999812516182559,0.9675624413404639,-0.01612503742104754,0.9836874787615114,1.9673749575230228,interval,0.7545732857848453,1.5091465715696906,0.9319962481713716,0.8096312482645289,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,258,1,-0.999812516182559,0.9675624413404639,-0.01612503742104754,0.9836874787615114,1.9673749575230228,interval,0.2899528570309791,0.5799057140619582,1.0,0.2899528570309791,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,259,0,-0.8697080952856973,0.8389415568536382,-0.015383269216029505,0.8543248260696678,1.7086496521393355,interval,0.6932369126003877,1.3864738252007753,0.9319962481713716,0.743819424123817,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,259,1,-0.8697080952856973,0.8389415568536382,-0.015383269216029505,0.8543248260696678,1.7086496521393355,interval,0.24585880517855296,0.4917176103571059,1.0,0.24585880517855296,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,260,0,-0.838165694995407,0.825208349668195,-0.006478672663606022,0.831687022331801,1.663374044663602,interval,0.6813654959253415,1.362730991850683,0.9319962481713716,0.7310818013079112,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,260,1,-0.838165694995407,0.825208349668195,-0.006478672663606022,0.831687022331801,1.663374044663602,interval,0.2345003588320943,0.4690007176641886,1.0,0.2345003588320943,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,261,0,-0.8336237521159587,0.8706092838418998,0.01849276586297055,0.8521165179789293,1.7042330359578586,interval,0.6920502769718595,1.384100553943719,0.9319962481713716,0.7425462048046874,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,261,1,-0.8336237521159587,0.8706092838418998,0.01849276586297055,0.8521165179789293,1.7042330359578586,interval,0.24617990929141698,0.49235981858283395,1.0,0.24617990929141698,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,262,0,-0.9956755857415509,0.9684535524140818,-0.013611016663734543,0.9820645690778164,1.9641291381556327,interval,0.753897974549621,1.507795949099242,0.9319962481713716,0.8089066624772479,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,262,1,-0.9956755857415509,0.9684535524140818,-0.013611016663734543,0.9820645690778164,1.9641291381556327,interval,0.2886267636052713,0.5772535272105426,1.0,0.2886267636052713,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,263,0,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,interval,0.7766570413339962,1.5533140826679923,0.9319962481713716,0.8333263603344331,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,263,1,-1.0209221961890607,1.0532642598531714,0.01617103183205537,1.037093228021116,2.074186456042232,interval,0.30660103344906114,0.6132020668981223,1.0,0.30660103344906114,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,264,0,-0.7417572625689902,0.76670716980087,0.012474953615939866,0.7542322161849301,1.5084644323698602,interval,0.6376081659023431,1.2752163318046863,0.9319962481713716,0.6841316873896925,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,264,1,-0.7417572625689902,0.76670716980087,0.012474953615939866,0.7542322161849301,1.5084644323698602,interval,0.20801936766531537,0.41603873533063074,1.0,0.20801936766531537,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,265,0,-0.8885673887593574,0.8568782922249835,-0.015844548267186953,0.8727228404921704,1.7454456809843408,interval,0.702665604185817,1.405331208371634,0.9319962481713716,0.7539360867219004,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,265,1,-0.8885673887593574,0.8568782922249835,-0.015844548267186953,0.8727228404921704,1.7454456809843408,interval,0.25253688066400265,0.5050737613280053,1.0,0.25253688066400265,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,266,0,-1.0979394401052625,1.0589211186241403,-0.019509160740561082,1.0784302793647014,2.1568605587294027,interval,0.7925041151146257,1.5850082302292514,0.9319962481713716,0.8503297268305133,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,266,1,-1.0979394401052625,1.0589211186241403,-0.019509160740561082,1.0784302793647014,2.1568605587294027,interval,0.31980614464730556,0.6396122892946111,1.0,0.31980614464730556,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,267,0,-0.9212656338216744,0.9528695879992138,0.015801977088769736,0.9370676109104441,1.8741352218208882,interval,0.7337875103139531,1.4675750206279061,0.9319962481713716,0.7873288242883862,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,267,1,-0.9212656338216744,0.9528695879992138,0.015801977088769736,0.9370676109104441,1.8741352218208882,interval,0.2745993522977662,0.5491987045955324,1.0,0.2745993522977662,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,268,0,-0.8940710602099512,0.8588320503931356,-0.0176195049084078,0.8764515553015434,1.7529031106030868,interval,0.7045270570195337,1.4090541140390673,0.9319962481713716,0.7559333617510315,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,268,1,-0.8940710602099512,0.8588320503931356,-0.0176195049084078,0.8764515553015434,1.7529031106030868,interval,0.2544688439870598,0.5089376879741196,1.0,0.2544688439870598,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,269,0,-0.8629278062752651,0.8343272647993648,-0.01430027073795015,0.848627535537315,1.69725507107463,interval,0.6902778922371247,1.3805557844742493,0.9319962481713716,0.7406444967900764,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,269,1,-0.8629278062752651,0.8343272647993648,-0.01430027073795015,0.848627535537315,1.69725507107463,interval,0.24343665743937826,0.4868733148787565,1.0,0.24343665743937826,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,270,0,-0.9706774538813653,1.00311566976285,0.016219107940742383,0.9868965618221077,1.9737931236442154,interval,0.7559506965773626,1.5119013931547252,0.9319962481713716,0.8111091627896355,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,270,1,-0.9706774538813653,1.00311566976285,0.016219107940742383,0.9868965618221077,1.9737931236442154,interval,0.2910078659199701,0.5820157318399402,1.0,0.2910078659199701,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,271,0,-0.9732209605163529,0.9367859559488059,-0.0182175022837735,0.9550034582325794,1.9100069164651587,interval,0.7419291876701919,1.4838583753403838,0.9319962481713716,0.7960645647725494,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,271,1,-0.9732209605163529,0.9367859559488059,-0.0182175022837735,0.9550034582325794,1.9100069164651587,interval,0.28133723320683734,0.5626744664136747,1.0,0.28133723320683734,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,272,0,-0.9651068342533383,0.9988666288565112,0.016879897301586455,0.9819867315549248,1.9639734631098495,interval,0.7538319469369148,1.5076638938738296,0.9319962481713716,0.8088358171140441,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,272,1,-0.9651068342533383,0.9988666288565112,0.016879897301586455,0.9819867315549248,1.9639734631098495,interval,0.28965012088428127,0.5793002417685625,1.0,0.28965012088428127,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,273,0,-0.8018194571083688,0.7795301587031831,-0.011144649202592838,0.7906748079057759,1.5813496158115519,interval,0.6587448325890022,1.3174896651780044,0.9319962481713716,0.706810605602218,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,273,1,-0.8018194571083688,0.7795301587031831,-0.011144649202592838,0.7906748079057759,1.5813496158115519,interval,0.22114755925244545,0.4422951185048909,1.0,0.22114755925244545,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,274,0,-1.3704733539073404,1.4030091777593325,0.01626791192599608,1.3867412658333365,2.773482531666673,interval,0.8824001925062456,1.7648003850124911,0.9319962481713716,0.9467851337787719,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,274,1,-1.3704733539073404,1.4030091777593325,0.01626791192599608,1.3867412658333365,2.773482531666673,interval,0.3924983181923667,0.7849966363847334,1.0,0.3924983181923667,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,275,0,-1.0543222479141343,1.0201728471958658,-0.017074700359134276,1.037247547555,2.07449509511,interval,0.7767090072997833,1.5534180145995666,0.9319962481713716,0.8333821180329101,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,275,1,-1.0543222479141343,1.0201728471958658,-0.017074700359134276,1.037247547555,2.07449509511,interval,0.30692130640613546,0.6138426128122709,1.0,0.30692130640613546,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,276,0,-1.125487268448763,1.1656844838811944,0.020098607716215744,1.1455858761649786,2.2911717523299573,interval,0.8161764595161093,1.6323529190322186,0.9319962481713716,0.875729340238754,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,276,1,-1.125487268448763,1.1656844838811944,0.020098607716215744,1.1455858761649786,2.2911717523299573,interval,0.3385688675823769,0.6771377351647538,1.0,0.3385688675823769,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,277,0,-1.1043859523878046,1.142637937807702,0.019125992709948747,1.1235119450977533,2.2470238901955066,interval,0.8086846995446106,1.6173693990892213,0.9319962481713716,0.8676909388114973,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,277,1,-1.1043859523878046,1.142637937807702,0.019125992709948747,1.1235119450977533,2.2470238901955066,interval,0.33235741957262144,0.6647148391452429,1.0,0.33235741957262144,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,278,0,-0.7530154612851289,0.7784502098291964,0.012717374272033788,0.7657328355571626,1.5314656711143253,interval,0.6443803780842384,1.2887607561684769,0.9319962481713716,0.6913980387244568,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,278,1,-0.7530154612851289,0.7784502098291964,0.012717374272033788,0.7657328355571626,1.5314656711143253,interval,0.21243222279884333,0.42486444559768666,1.0,0.21243222279884333,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,279,0,-0.9664557004775319,0.9399092435870481,-0.013273228445241925,0.95318247203229,1.90636494406458,interval,0.7411615940183135,1.482323188036627,0.9319962481713716,0.7952409631181603,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,279,1,-0.9664557004775319,0.9399092435870481,-0.013273228445241925,0.95318247203229,1.90636494406458,interval,0.27911105909809497,0.5582221181961899,1.0,0.27911105909809497,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,280,0,-1.121841329562469,1.0845588312120331,-0.018641249175217922,1.103200080387251,2.206400160774502,interval,0.8015460041794957,1.6030920083589915,0.9319962481713716,0.8600313635941921,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,280,1,-1.121841329562469,1.0845588312120331,-0.018641249175217922,1.103200080387251,2.206400160774502,interval,0.3266004404752712,0.6532008809505424,1.0,0.3266004404752712,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,281,0,-0.7251348527214967,0.7587825851372445,0.016823866207873905,0.7419587189293706,1.4839174378587412,interval,0.6302195981176367,1.2604391962352735,0.9319962481713716,0.6762040076386172,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,281,1,-0.7251348527214967,0.7587825851372445,0.016823866207873905,0.7419587189293706,1.4839174378587412,interval,0.20503002893763023,0.41006005787526045,1.0,0.20503002893763023,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,282,0,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,interval,0.909172936965028,1.818345873930056,0.9319962481713716,0.9755113700820961,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,282,1,-1.5402310170390952,1.5057766624553988,-0.017227177291848195,1.523003839747247,3.046007679494494,interval,0.4160171851100182,0.8320343702200363,1.0,0.4160171851100182,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,283,0,-0.8267574423081064,0.7934742831840175,-0.016641579562044484,0.810115862746062,1.620231725492124,interval,0.6695518859624171,1.3391037719248342,0.9319962481713716,0.7184062031109193,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,283,1,-0.8267574423081064,0.7934742831840175,-0.016641579562044484,0.810115862746062,1.620231725492124,interval,0.23033792786576274,0.4606758557315255,1.0,0.23033792786576274,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,284,0,-0.8807211491905231,0.9069751074714278,0.013126979140452355,0.8938481283309755,1.787696256661951,interval,0.7132288219578495,1.426457643915699,0.9319962481713716,0.7652700569956629,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,284,1,-0.8807211491905231,0.9069751074714278,0.013126979140452355,0.8938481283309755,1.787696256661951,interval,0.2589676592488249,0.5179353184976498,1.0,0.2589676592488249,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,285,0,-0.9136703395627072,0.9488611758931845,0.017595418165238685,0.9312657577279458,1.8625315154558917,interval,0.7310781447349803,1.4621562894699607,0.9319962481713716,0.7844217679732041,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,285,1,-0.9136703395627072,0.9488611758931845,0.017595418165238685,0.9312657577279458,1.8625315154558917,interval,0.2732578834949018,0.5465157669898036,1.0,0.2732578834949018,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,286,0,-1.3222298401611523,1.281014428317464,-0.020607705921844133,1.3016221342393082,2.6032442684786163,interval,0.8620461729655207,1.7240923459310413,0.9319962481713716,0.9249459691033125,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,286,1,-1.3222298401611523,1.281014428317464,-0.020607705921844133,1.3016221342393082,2.6032442684786163,interval,0.3761370717892764,0.7522741435785528,1.0,0.3761370717892764,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,287,0,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,interval,0.7730711214778007,1.5461422429556013,0.9319962481713716,0.8294787913520136,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,287,1,-1.0082309842020036,1.048218791399595,0.019993903598795715,1.0282248878007993,2.0564497756015987,interval,0.3050686785231899,0.6101373570463798,1.0,0.3050686785231899,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,288,0,-0.9609248528732857,0.9933100955407534,0.016192621333733892,0.9771174742070196,1.954234948414039,interval,0.7517296689193206,1.5034593378386412,0.9319962481713716,0.8065801449246774,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,288,1,-0.9609248528732857,0.9933100955407534,0.016192621333733892,0.9771174742070196,1.954234948414039,interval,0.28786612485259233,0.5757322497051847,1.0,0.28786612485259233,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,289,0,-1.2578650512751823,1.2187445857963908,-0.019560232739395778,1.2383048185357866,2.476609637071573,interval,0.8448789586605285,1.689757917321057,0.9319962481713716,0.9065261371150667,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,289,1,-1.2578650512751823,1.2187445857963908,-0.019560232739395778,1.2383048185357866,2.476609637071573,interval,0.3616533996562892,0.7233067993125784,1.0,0.3616533996562892,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,290,0,-1.2209164590474622,1.2650042904501948,0.022043915701366323,1.2429603747488285,2.485920749497657,interval,0.8461810925933617,1.6923621851867234,0.9319962481713716,0.907923282152279,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,290,1,-1.2209164590474622,1.2650042904501948,0.022043915701366323,1.2429603747488285,2.485920749497657,interval,0.3633251692820505,0.726650338564101,1.0,0.3633251692820505,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,291,0,-0.8933386502203349,0.9310906530126257,0.018876001396145425,0.9122146516164803,1.8244293032329606,interval,0.7220704150169062,1.4441408300338123,0.9319962481713716,0.7747567830167218,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,291,1,-0.8933386502203349,0.9310906530126257,0.018876001396145425,0.9122146516164803,1.8244293032329606,interval,0.26725505914430636,0.5345101182886127,1.0,0.26725505914430636,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,292,0,-0.8645810082391747,0.8388811245143484,-0.012849941862413172,0.8517310663767615,1.703462132753523,interval,0.6919131901678273,1.3838263803356545,0.9319962481713716,0.7423991153670407,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,292,1,-0.8645810082391747,0.8388811245143484,-0.012849941862413172,0.8517310663767615,1.703462132753523,interval,0.24402824771483522,0.48805649542967044,1.0,0.24402824771483522,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,293,0,-0.7696583947245352,0.7436963641755314,-0.012981015274501906,0.7566773794500333,1.5133547589000667,interval,0.6390519973962911,1.2781039947925823,0.9319962481713716,0.6856808690487184,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,293,1,-0.7696583947245352,0.7436963641755314,-0.012981015274501906,0.7566773794500333,1.5133547589000667,interval,0.20913034386681462,0.41826068773362923,1.0,0.20913034386681462,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,294,0,-1.034379581810341,1.0680421053694114,0.0168312617795352,1.0512108435898762,2.1024216871797523,interval,0.7821906687174249,1.5643813374348499,0.9319962481713716,0.8392637526729603,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,294,1,-1.034379581810341,1.0680421053694114,0.0168312617795352,1.0512108435898762,2.1024216871797523,interval,0.3110415446724509,0.6220830893449018,1.0,0.3110415446724509,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,295,0,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,interval,0.8921575827142894,1.7843151654285787,0.9319962481713716,0.9572544787221537,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,295,1,-1.4165490101664338,1.4487144266980425,0.01608270826580438,1.4326317184322381,2.8652634368644763,interval,0.4009049453219005,0.801809890643801,1.0,0.4009049453219005,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,296,0,-0.7369816136118259,0.7101516754046739,-0.01341496910357598,0.7235666445082499,1.4471332890164998,interval,0.6190449976261072,1.2380899952522144,0.9319962481713716,0.6642140446817333,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,296,1,-0.7369816136118259,0.7101516754046739,-0.01341496910357598,0.7235666445082499,1.4471332890164998,interval,0.19676397393240463,0.39352794786480927,1.0,0.19676397393240463,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,297,0,-0.9825135379376003,1.0138240898880804,0.015655275975240024,0.9981688139128404,1.9963376278256808,interval,0.7607455026573765,1.521491005314753,0.9319962481713716,0.816253825216573,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,297,1,-0.9825135379376003,1.0138240898880804,0.015655275975240024,0.9981688139128404,1.9963376278256808,interval,0.2944046231149402,0.5888092462298804,1.0,0.2944046231149402,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,298,0,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,interval,0.6878957105502226,1.3757914211004452,0.9319962481713716,0.7380884975662855,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,298,1,-0.8332058737903768,0.854855836494384,0.010824981352003604,0.8440308551423804,1.6880617102847608,interval,0.24053893071877808,0.48107786143755615,1.0,0.24053893071877808,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,299,0,-0.7428390480563722,0.7269071605470814,-0.007965943754645433,0.7348731043017268,1.4697462086034536,interval,0.6260134598050827,1.2520269196101654,0.9319962481713716,0.6716909655305543,NA,NA,ok +pinn100d_shallow300_medium_interval,single_cell,0,1.0,0,299,1,-0.7428390480563722,0.7269071605470814,-0.007965943754645433,0.7348731043017268,1.4697462086034536,interval,0.19899051961512126,0.39798103923024253,1.0,0.19899051961512126,NA,NA,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/benchmark_metadata.json b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/benchmark_metadata.json new file mode 100644 index 0000000..223218f --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/benchmark_metadata.json @@ -0,0 +1,13 @@ +{ + "schema_version": "1.2", + "benchmark_level": "medium", + "problem_id": "poisson_100d_ridge", + "model_id": "pinn_100d_poisson_shallow_300_seed_20260804", + "method_id": "interval", + "git_commit": "fb576a55aed822c24764f8d4db93e38495445e37", + "dtype": "float64", + "device": "cpu", + "quantile_interpolation": "linear", + "cell_average": "volume_weighted", + "timestamp_utc": "2026-08-04T11:32:52.956751+00:00" +} diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/cell_intervals.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/cell_intervals.csv new file mode 100644 index 0000000..10e04ec --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/cell_intervals.csv @@ -0,0 +1,102 @@ +run_id,split_id,cell_id,cell_weight,quantity,output_index,input_index_a,input_index_b,lower,upper,midpoint,radius,width,magnitude,mignitude,local_relative_radius,global_normalized_radius,sign_certified,status,local_relative_width +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,0,NA,-0.5378221490875416,0.7987630720909168,0.13047046150168762,0.6682926105892292,1.3365852211784583,0.7987630720909168,0.0,0.8366593723966783,0.6860086284677644,0,ok,1.6733187447933566 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,1,NA,-0.5678543858611317,0.8248456126486182,0.12849561339374327,0.696349999254875,1.39269999850975,0.8248456126486182,0.0,0.8442185890022039,0.7148098009061911,0,ok,1.6884371780044078 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,2,NA,-0.4690172946593083,0.7165531062929995,0.1237679058168456,0.5927852004761539,1.1855704009523078,0.7165531062929995,0.0,0.8272732268831492,0.6084995642793198,0,ok,1.6545464537662984 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,3,NA,-0.5513010579655157,0.8097652521305833,0.1292320970825338,0.6805331550480495,1.361066310096099,0.8097652521305833,0.0,0.8404079494118701,0.6985736620815437,0,ok,1.6808158988237403 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,4,NA,-0.49210891794497696,0.7441765477464936,0.1260338149007583,0.6181427328457353,1.2362854656914706,0.7441765477464936,0.0,0.8306398995206012,0.6345293089249264,0,ok,1.6612797990412025 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,5,NA,-0.5693683720222141,0.8361692822239019,0.13340045510084386,0.702768827123058,1.405537654246116,0.8361692822239019,0.0,0.8404623825141628,0.7213987878745498,0,ok,1.6809247650283257 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,6,NA,-0.5454761824837298,0.8003497707769929,0.12743679414663156,0.6729129766303613,1.3458259532607226,0.8003497707769929,0.0,0.8407736232336097,0.6907514775142345,0,ok,1.6815472464672194 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,7,NA,-0.682885839509648,0.9347848163688026,0.12594948842957732,0.8088353279392253,1.6176706558784506,0.9347848163688026,0.0,0.8652636561654568,0.8302770450905315,0,ok,1.7305273123309135 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,8,NA,-0.5550913141179801,0.8110221656986065,0.12796542579031323,0.6830567399082933,1.3661134798165866,0.8110221656986065,0.0,0.8422171042882841,0.7011641455933568,0,ok,1.6844342085765682 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,9,NA,-0.5157645072497946,0.7751783759539667,0.12970693435208602,0.6454714416018806,1.2909428832037613,0.7751783759539667,0.0,0.8326747257462345,0.6625824846065617,0,ok,1.665349451492469 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,10,NA,-0.5846437964054533,0.8501100977728039,0.1327331506836753,0.7173769470891286,1.4347538941782572,0.8501100977728039,0.0,0.8438635759868965,0.7363941599370674,0,ok,1.687727151973793 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,11,NA,-0.5741758631931335,0.8343703689931594,0.13009725290001295,0.7042731160931465,1.408546232186293,0.8343703689931594,0.0,0.8440773333585633,0.7229429546015763,0,ok,1.6881546667171266 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,12,NA,-0.5373932984053661,0.796141280140623,0.12937399086762846,0.6667672892729946,1.3335345785459891,0.796141280140623,0.0,0.8374987026865671,0.6844428718402893,0,ok,1.6749974053731342 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,13,NA,-0.4899048077876636,0.7482390442578352,0.12916711823508578,0.6190719260227494,1.2381438520454988,0.7482390442578352,0.0,0.8273718549889303,0.6354831344301689,0,ok,1.6547437099778606 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,14,NA,-0.6086819931339233,0.8627051070744738,0.12701155697027522,0.7356935501041986,1.4713871002083971,0.8627051070744738,0.0,0.8527752346326254,0.7551963246078363,0,ok,1.7055504692652508 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,15,NA,-0.5463226041107937,0.8005679374460158,0.12712266666761107,0.6734452707784048,1.3468905415568095,0.8005679374460158,0.0,0.8412093955783944,0.6912978824462279,0,ok,1.6824187911567887 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,16,NA,-0.5964926149731888,0.8524321867192572,0.1279697858730342,0.724462400846223,1.448924801692446,0.8524321867192572,0.0,0.8498768724752762,0.74366744462847,0,ok,1.6997537449505524 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,17,NA,-0.5072729273834199,0.7666021039204053,0.1296645882684927,0.6369375156519126,1.2738750313038252,0.7666021039204053,0.0,0.8308580323411746,0.6538223296330968,0,ok,1.6617160646823492 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,18,NA,-0.5748990871085911,0.8191164514339098,0.12210868216265935,0.6970077692712504,1.3940155385425008,0.8191164514339098,0.0,0.8509263463712624,0.7154850079930734,0,ok,1.7018526927425248 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,19,NA,-0.46818890719214074,0.7232739940824648,0.12754254344516205,0.5957314506373028,1.1914629012746056,0.7232739940824648,0.0,0.8236594368266197,0.6115239176840218,0,ok,1.6473188736532394 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,20,NA,-0.5088617477996571,0.7570231898307176,0.12408072101553025,0.6329424688151873,1.2658849376303747,0.7570231898307176,0.0,0.8360938968814461,0.649721376610872,0,ok,1.6721877937628922 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,21,NA,-0.6398053968052432,0.8985138016056035,0.12935420240018014,0.7691595992054233,1.5383191984108466,0.8985138016056035,0.0,0.856035375117188,0.7895495376770697,0,ok,1.712070750234376 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,22,NA,-0.5648313210296605,0.8100681049253136,0.12261839194782653,0.687449712977487,1.374899425954974,0.8100681049253136,0.0,0.8486319962453877,0.7056735736228488,0,ok,1.6972639924907753 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,23,NA,-0.4917750439945124,0.7493548509581743,0.12878990348183098,0.6205649474763434,1.2411298949526868,0.7493548509581743,0.0,0.8281322883048642,0.6370157349459069,0,ok,1.6562645766097284 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,24,NA,-0.5415835326846522,0.7971994327213897,0.12780795001836875,0.669391482703021,1.338782965406042,0.7971994327213897,0.0,0.8396788246799519,0.6871366309919569,0,ok,1.6793576493599038 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,25,NA,-0.5887887297086892,0.8456378689699117,0.12842456963061122,0.7172132993393004,1.434426598678601,0.8456378689699117,0.0,0.848132901395431,0.7362261739880496,0,ok,1.696265802790862 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,26,NA,-0.6071703436785154,0.86323763667236,0.12803364649692228,0.7352039901754377,1.4704079803508754,0.86323763667236,0.0,0.8516820385746027,0.7546937867524709,0,ok,1.7033640771492053 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,27,NA,-0.5326131860748671,0.7956181083606135,0.13150246114287323,0.6641156472177403,1.3282312944354806,0.7956181083606135,0.0,0.8347166061694642,0.6817209364175579,0,ok,1.6694332123389284 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,28,NA,-0.47390706245265873,0.7304558431701872,0.12827439035876423,0.602181452811423,1.204362905622846,0.7304558431701872,0.0,0.8243913146042452,0.6181449053696121,0,ok,1.6487826292084904 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,29,NA,-0.5769414088549379,0.8285909368366399,0.12582476399085096,0.7027661728457889,1.4055323456915778,0.8285909368366399,0.0,0.848146101535675,0.7213960632340554,0,ok,1.69629220307135 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,30,NA,-0.7085553529499188,0.9741752258741921,0.13280993646213668,0.8413652894120555,1.682730578824111,0.9741752258741921,0.0,0.8636693554355609,0.8636693554355609,0,ok,1.7273387108711218 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,31,NA,-0.5371576892865085,0.7921524875809397,0.12749739914721558,0.6646550884337241,1.3293101768674482,0.7921524875809397,0.0,0.8390494240110705,0.6822746778817794,0,ok,1.678098848022141 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,32,NA,-0.5597969501903903,0.8091164731599592,0.12465976148478441,0.6844567116751747,1.3689134233503495,0.8091164731599592,0.0,0.8459310054608916,0.7026012297335588,0,ok,1.6918620109217832 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,33,NA,-0.5892572044737424,0.846873078082209,0.1288079368042333,0.7180651412779757,1.4361302825559514,0.846873078082209,0.0,0.8479017220668698,0.7371005977221482,0,ok,1.6958034441337395 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,34,NA,-0.5074785237577598,0.7630879226975124,0.12780469946987627,0.6352832232276361,1.2705664464552722,0.7630879226975124,0.0,0.8325164169574494,0.6521241829544108,0,ok,1.6650328339148988 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,35,NA,-0.5961112317779353,0.8454681239217059,0.12467844607188527,0.7207896778498206,1.4415793556996412,0.8454681239217059,0.0,0.8525332386352261,0.7398973600493773,0,ok,1.7050664772704522 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,36,NA,-0.5278061597526885,0.7834981620162008,0.12784600113175615,0.6556521608844447,1.3113043217688893,0.7834981620162008,0.0,0.8368266738459654,0.6730330883708159,0,ok,1.6736533476919309 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,37,NA,-0.6087256718580557,0.8695197089529422,0.13039701854744323,0.7391226904054989,1.4782453808109979,0.8695197089529422,0.0,0.8500355803269087,0.7587163692673821,0,ok,1.7000711606538175 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,38,NA,-0.521099058250826,0.7692757592091978,0.12408835047918587,0.6451874087300119,1.2903748174600238,0.7692757592091978,0.0,0.8386945786427138,0.662290922201437,0,ok,1.6773891572854276 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,39,NA,-0.4548706629905295,0.7128436947051847,0.12898651585732762,0.583857178847857,1.167714357695714,0.7128436947051847,0.0,0.8190535782031804,0.5993348663983148,0,ok,1.6381071564063607 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,40,NA,-0.5468678323397077,0.7950967680050668,0.12411446783267954,0.6709823001723872,1.3419646003447745,0.7950967680050668,0.0,0.8439001731272431,0.6887696200344966,0,ok,1.6878003462544862 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,41,NA,-0.5342843786663115,0.7963574481004562,0.13103653471707233,0.6653209133833838,1.3306418267667677,0.7963574481004562,0.0,0.8354551275565608,0.6829581534331743,0,ok,1.6709102551131216 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,42,NA,-0.6140218491877717,0.8719734303298478,0.12897579057103803,0.7429976397588097,1.4859952795176194,0.8719734303298478,0.0,0.8520874764243109,0.7626940410972457,0,ok,1.7041749528486219 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,43,NA,-0.6056739392904643,0.867993502238449,0.13115978147399232,0.7368337207644566,1.4736674415289133,0.867993502238449,0.0,0.8488931298036825,0.7563667204770547,0,ok,1.697786259607365 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,44,NA,-0.48077934011208584,0.7464046880081996,0.13281267394805688,0.6135920140601427,1.2271840281202855,0.7464046880081996,0.0,0.8220634515272527,0.6298579534390498,0,ok,1.6441269030545054 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,45,NA,-0.5332309717406042,0.7959747080610194,0.13137186816020763,0.6646028399008118,1.3292056798016236,0.7959747080610194,0.0,0.8349547205083598,0.6822210442730358,0,ok,1.6699094410167197 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,46,NA,-0.6374626690473589,0.9030729914986907,0.13280516122566588,0.7702678302730248,1.5405356605460496,0.9030729914986907,0.0,0.8529408337134857,0.7906871472550663,0,ok,1.7058816674269714 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,47,NA,-0.5134057215582655,0.7718731968719983,0.1292337376568664,0.6426394592151319,1.2852789184302638,0.7718731968719983,0.0,0.83257128479058,0.6596754281432776,0,ok,1.66514256958116 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,48,NA,-0.5357621171018417,0.7941649579485596,0.12920142042335891,0.6649635375252007,1.3299270750504013,0.7941649579485596,0.0,0.8373116074560826,0.6825913037651771,0,ok,1.6746232149121651 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,49,NA,-0.5647199698042719,0.8250401346524571,0.1301600824240926,0.6948800522283645,1.389760104456729,0.8250401346524571,0.0,0.842237877944057,0.7133008865061237,0,ok,1.684475755888114 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,50,NA,-0.5062922184537869,0.7679629283035944,0.13083535492490372,0.6371275733786906,1.2742551467573813,0.7679629283035944,0.0,0.8296332412634619,0.6540174256709862,0,ok,1.6592664825269239 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,51,NA,-0.49444823170493407,0.7543978685303252,0.12997481841269556,0.6244230501176297,1.2488461002352593,0.7543978685303252,0.0,0.8277105174410089,0.6409761134679784,0,ok,1.6554210348820178 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,52,NA,-0.584038733455528,0.8439993564907676,0.1299803115176198,0.7140190449731478,1.4280380899462957,0.8439993564907676,0.0,0.8459947741452554,0.7329472419424453,0,ok,1.6919895482905107 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,53,NA,-0.49779881663691233,0.7556978521953613,0.1289495177792245,0.6267483344161369,1.2534966688322737,0.7556978521953613,0.0,0.8293636571751315,0.6433630395945595,0,ok,1.658727314350263 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,54,NA,-0.5459358285492476,0.8052228537201401,0.12964351258544626,0.6755793411346939,1.3511586822693877,0.8052228537201401,0.0,0.8389967299282534,0.6934885256689336,0,ok,1.6779934598565067 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,55,NA,-0.5240776255617824,0.7773456014464762,0.12663398794234693,0.6507116135041293,1.3014232270082586,0.7773456014464762,0.0,0.837094353262297,0.6679615701787146,0,ok,1.674188706524594 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,56,NA,-0.5391468869563267,0.79737591314864,0.12911451309615662,0.6682614000524834,1.3365228001049667,0.79737591314864,0.0,0.838075729443199,0.6859765905592682,0,ok,1.676151458886398 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,57,NA,-0.4815223325051528,0.7499670169323257,0.13422234221358645,0.6157446747187393,1.2314893494374786,0.7499670169323257,0.0,0.8210290063653584,0.6320676797813152,0,ok,1.6420580127307167 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,58,NA,-0.5240173665962202,0.7802525062492605,0.12811756982652012,0.6521349364227403,1.3042698728454807,0.7802525062492605,0.0,0.8357998611982779,0.66942262449503,0,ok,1.6715997223965557 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,59,NA,-0.47992832823439857,0.7345279714796257,0.12729982162261355,0.6072281498570121,1.2144562997140242,0.7345279714796257,0.0,0.82669166244795,0.6233253871880245,0,ok,1.6533833248959 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,60,NA,-0.6395601383275863,0.8945719421563004,0.12750590191435707,0.7670660402419434,1.5341320804838867,0.8945719421563004,0.0,0.8574671349438778,0.7874004797787831,0,ok,1.7149342698877557 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,61,NA,-0.6419612802509167,0.9001684943506123,0.1291036070498478,0.7710648873007645,1.542129774601529,0.9001684943506123,0.0,0.8565783985330613,0.7915053337646076,0,ok,1.7131567970661226 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,62,NA,-0.5054246248693198,0.7588786757952594,0.12672702546296977,0.6321516503322896,1.2643033006645792,0.7588786757952594,0.0,0.8330075287328802,0.6489095940260828,0,ok,1.6660150574657604 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,63,NA,-0.4487343729770295,0.6985863516325034,0.12492598932773694,0.5736603623047665,1.147320724609533,0.6985863516325034,0.0,0.8211731605752081,0.5888677386452554,0,ok,1.6423463211504161 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,64,NA,-0.5316549860812189,0.7834132228417713,0.1258791183802762,0.6575341044614951,1.3150682089229901,0.7834132228417713,0.0,0.8393196403761741,0.6749649210915275,0,ok,1.6786392807523483 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,65,NA,-0.5752826742026228,0.8327949406978762,0.1287561332476267,0.7040388074502495,1.408077614900499,0.8327949406978762,0.0,0.8453927528189232,0.7227024345835409,0,ok,1.6907855056378465 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,66,NA,-0.5407912466212637,0.7954465562270385,0.12732765480288744,0.6681189014241511,1.3362378028483022,0.7954465562270385,0.0,0.8399293405620764,0.6858303143816901,0,ok,1.6798586811241527 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,67,NA,-0.5663900745918101,0.8282965121266185,0.13095321876740418,0.6973432933592143,1.3946865867184286,0.8282965121266185,0.0,0.8419005551149951,0.7158294266141307,0,ok,1.6838011102299901 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,68,NA,-0.4983219097442333,0.7586991990629192,0.13018864465934293,0.6285105544035763,1.2570211088071526,0.7586991990629192,0.0,0.8284054539399266,0.6451719749284037,0,ok,1.6568109078798532 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,69,NA,-0.5517315443149626,0.8118581345011333,0.13006329509308534,0.681794839408048,1.363589678816096,0.8118581345011333,0.0,0.8397955386959249,0.6998687929024557,0,ok,1.6795910773918499 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,70,NA,-0.5702286714251712,0.8217060415942441,0.12573868508453645,0.6959673565097076,1.3919347130194153,0.8217060415942441,0.0,0.8469785072523225,0.7144170145418859,0,ok,1.693957014504645 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,71,NA,-0.5223847588303095,0.7711898628315622,0.12440255200062633,0.6467873108309359,1.2935746216618718,0.7711898628315622,0.0,0.8386875165295093,0.6639332367034182,0,ok,1.6773750330590187 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,72,NA,-0.5165640448181227,0.7683389655406961,0.12588746036128673,0.6424515051794094,1.2849030103588188,0.7683389655406961,0.0,0.8361563502474496,0.659482491563974,0,ok,1.6723127004948992 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,73,NA,-0.4927004209008391,0.7511840562278554,0.12924181766350817,0.6219422385643473,1.2438844771286945,0.7511840562278554,0.0,0.8279492002099876,0.6384295371566622,0,ok,1.6558984004199753 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,74,NA,-0.6195924370120028,0.8859868000948393,0.13319718154141824,0.7527896185534211,1.5055792371068422,0.8859868000948393,0.0,0.8496623408755521,0.7727455991070733,0,ok,1.6993246817511043 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,75,NA,-0.5247194136555597,0.7888944416840937,0.132087514014267,0.6568069276698267,1.3136138553396535,0.7888944416840937,0.0,0.8325663016052022,0.674218467299279,0,ok,1.6651326032104043 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,76,NA,-0.4850790487617744,0.7396115041239228,0.12726622768107423,0.6123452764428486,1.2246905528856973,0.7396115041239228,0.0,0.8279282745448608,0.6285781655896151,0,ok,1.6558565490897217 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,77,NA,-0.5510496958740324,0.8061852256431631,0.12756776488456534,0.6786174607585977,1.3572349215171955,0.8061852256431631,0.0,0.8417637028974407,0.6966071839382173,0,ok,1.6835274057948815 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,78,NA,-0.6324720411929537,0.8846463259003071,0.12608714235367668,0.7585591835466304,1.5171183670932609,0.8846463259003071,0.0,0.8574716938711553,0.7786681116489335,0,ok,1.7149433877423106 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,79,NA,-0.5864668269508401,0.8540427629612605,0.1337879680052102,0.7202547949560503,1.4405095899121005,0.8540427629612605,0.0,0.8433474600951816,0.7393482977456317,0,ok,1.6866949201903632 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,80,NA,-0.6082356217317733,0.8513406857365582,0.12155253200239247,0.7297881537341657,1.4595763074683314,0.8513406857365582,0.0,0.8572222213282004,0.7491343798845617,0,ok,1.7144444426564007 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,81,NA,-0.5702549573427841,0.8289139321173743,0.1293294873872951,0.6995844447300792,1.3991688894601584,0.8289139321173743,0.0,0.8439771822185008,0.7181299895019354,0,ok,1.6879543644370016 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,82,NA,-0.5294688012656474,0.7900005475967602,0.13026587316555638,0.6597346744312038,1.3194693488624076,0.7900005475967602,0.0,0.8351066039614342,0.6772238267906886,0,ok,1.6702132079228684 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,83,NA,-0.5599632663402906,0.8105805102556615,0.12530862195768544,0.685271888297976,1.370543776595952,0.8105805102556615,0.0,0.8454087899076645,0.7034380161772601,0,ok,1.690817579815329 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,84,NA,-0.5373276552870966,0.7935708725336373,0.12812160862327038,0.665449263910367,1.330898527820734,0.7935708725336373,0.0,0.8385505150734478,0.683089906452112,0,ok,1.6771010301468956 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,85,NA,-0.6360024888903933,0.9033991339871202,0.13369832254836345,0.7697008114387568,1.5394016228775136,0.9033991339871202,0.0,0.8520052571245108,0.7901050971071997,0,ok,1.7040105142490216 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,86,NA,-0.5103036895035521,0.7670623091698544,0.12837930983315116,0.6386829993367032,1.2773659986734065,0.7670623091698544,0.0,0.8326350958736476,0.6556140849954079,0,ok,1.6652701917472952 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,87,NA,-0.5296857901868345,0.793953878945742,0.13213404437945375,0.6618198345662882,1.3236396691325765,0.793953878945742,0.0,0.8335746598342602,0.6793642632129074,0,ok,1.6671493196685203 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,88,NA,-0.5517052744370504,0.8113545682238165,0.12982464689338302,0.6815299213304334,1.3630598426608669,0.8113545682238165,0.0,0.8399902435040334,0.69959685201279,0,ok,1.6799804870080668 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,89,NA,-0.5273206423318214,0.7895421489625415,0.13111075331536004,0.6584313956471814,1.316862791294363,0.7895421489625415,0.0,0.8339407801247348,0.6758859989036646,0,ok,1.6678815602494697 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,90,NA,-0.43966295777889186,0.6918586778621681,0.12609786004163814,0.56576081782053,1.13152163564106,0.6918586778621681,0.0,0.8177404373516302,0.5807587821922234,0,ok,1.6354808747032603 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,91,NA,-0.616863034774476,0.8665840200090463,0.12486049261728516,0.7417235273917612,1.4834470547835223,0.8665840200090463,0.0,0.8559164607997483,0.7613861528106131,0,ok,1.7118329215994965 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,92,NA,-0.5018425472378379,0.757107787898624,0.12763262033039302,0.629475167568231,1.258950335136462,0.757107787898624,0.0,0.8314208064288424,0.6461621593829396,0,ok,1.6628416128576848 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,93,NA,-0.5177152564547641,0.7734121383833882,0.12784844096431203,0.6455636974190762,1.2911273948381523,0.7734121383833882,0.0,0.83469558516169,0.6626771860675978,0,ok,1.66939117032338 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,94,NA,-0.5227752943039102,0.7749886383809103,0.1261066720385,0.6488819663424102,1.2977639326848205,0.7749886383809103,0.0,0.8372793279886537,0.6660834202184985,0,ok,1.6745586559773074 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,95,NA,-0.5140123652594732,0.757198091444248,0.12159286309238737,0.6356052283518606,1.2712104567037212,0.757198091444248,0.0,0.8394173671773707,0.6524547242324807,0,ok,1.6788347343547414 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,96,NA,-0.4940115317056089,0.752656258515822,0.12932236340510653,0.6233338951107155,1.246667790221431,0.752656258515822,0.0,0.8281787177853009,0.6398580856450713,0,ok,1.6563574355706019 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,97,NA,-0.5963250463863758,0.8565804873176073,0.13012772046561571,0.7264527668519916,1.4529055337039831,0.8565804873176073,0.0,0.8480846547495936,0.7457105739884703,0,ok,1.6961693094991872 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,98,NA,-0.6305035423205515,0.8831614050229689,0.1263289313512087,0.7568324736717602,1.5136649473435204,0.8831614050229689,0.0,0.8569582744074701,0.7768956277784668,0,ok,1.7139165488149402 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,J,0,99,NA,-0.5908747755827785,0.8418703853662518,0.12549780489173668,0.7163725804745151,1.4327451609490303,0.8418703853662518,0.0,0.8509297784157838,0.7353631681935521,0,ok,1.7018595568315675 +pinn100d_shallow300_medium_interval,single_cell,0,1.0,Y,0,NA,NA,-15.591980666867151,15.858944103993526,0.13348171856318736,15.725462385430339,31.450924770860677,15.858944103993526,0.0,0.9915831900479696,0.9915831900479696,0,ok,1.9831663800959392 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/complexity.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/complexity.csv new file mode 100644 index 0000000..ac330cb --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/complexity.csv @@ -0,0 +1,4 @@ +run_id,quantity,n_alpha,n_eta,n_monomials,n_mixed_monomials,max_degree,n_coefficients,status +pinn100d_shallow300_medium_interval,Y,NA,NA,NA,NA,NA,NA,not_implemented +pinn100d_shallow300_medium_interval,J,NA,NA,NA,NA,NA,NA,not_implemented +pinn100d_shallow300_medium_interval,H,NA,NA,NA,NA,NA,NA,not_implemented diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/layer_normalized_radius_Y.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/layer_normalized_radius_Y.csv new file mode 100644 index 0000000..84450a4 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/layer_normalized_radius_Y.csv @@ -0,0 +1,301 @@ +neuron,layer_0 +0,0.7097624355322355 +1,0.6346752490022738 +2,0.7446117218373166 +3,0.7298334203423916 +4,0.8256302656708558 +5,0.7560155683859542 +6,0.721274363895353 +7,0.6366193466575827 +8,0.9208369778968852 +9,0.7436800438057513 +10,0.7100215760720421 +11,0.7183815445377337 +12,0.7804638666825527 +13,0.7747603125879564 +14,0.906437058683245 +15,0.7379385331686141 +16,0.9765661358698409 +17,0.7742424472076296 +18,0.999701112213034 +19,0.8001258993254352 +20,0.7463954874755241 +21,0.8104399102524427 +22,0.8630391084132867 +23,0.8525649253535187 +24,0.9166674500999373 +25,0.8954937110134026 +26,0.8410049677343807 +27,0.8529921363125286 +28,0.7402935655570245 +29,0.8979047348980127 +30,0.6713112108405677 +31,0.8105550589547926 +32,0.916868520647112 +33,0.7400036312931516 +34,0.780131914813879 +35,0.7925942276764527 +36,0.9135119087932743 +37,0.865619705629905 +38,0.6870688678775283 +39,0.8291685381128775 +40,0.8090941637546931 +41,0.7284450806562139 +42,0.7195005110862843 +43,0.9755471609881184 +44,0.9179096554984897 +45,0.6294010163399878 +46,0.7599196881769303 +47,0.7368951599364633 +48,0.8291006831985797 +49,0.7313061469837212 +50,0.8900154339552003 +51,0.7227779801913534 +52,0.6853495036518543 +53,0.8471960731382604 +54,0.891203662400249 +55,0.6344573266455632 +56,0.6525212833238139 +57,0.8986111998469203 +58,0.6640983805443013 +59,0.7869069178308965 +60,0.985574946967963 +61,0.8665306946277519 +62,0.8515675668346924 +63,0.9458546243315955 +64,0.7513363612869359 +65,0.7438842161091404 +66,0.8061555215449585 +67,0.7666633538631222 +68,0.8495478438861155 +69,0.9500260387637666 +70,0.7923321772416408 +71,0.9537255065983308 +72,0.8910813189025565 +73,0.8918612666945633 +74,0.7743955524525202 +75,0.7661652821642047 +76,0.8825108467135742 +77,0.9374361381971174 +78,0.6020799481821246 +79,0.739481928688642 +80,0.9102861462840672 +81,0.5776767372696731 +82,0.7034223179225934 +83,0.7592553687402962 +84,0.7197517853165163 +85,0.7783425532237384 +86,0.8698841727485072 +87,0.91199582151868 +88,0.7331407238654815 +89,0.754785386289841 +90,0.7163683448702676 +91,0.9171015041087398 +92,0.8608644575383324 +93,0.6201178117359327 +94,0.8977628215569883 +95,0.7472048539862651 +96,0.6139624564706888 +97,0.8933740923286022 +98,0.9220738431584499 +99,0.7660129123872823 +100,0.9617945175403675 +101,0.985954896115789 +102,0.8355294726637108 +103,0.8694810403712437 +104,0.7539443626795925 +105,0.8937889921933593 +106,0.6665439074696877 +107,0.575609688556176 +108,0.8434871401507121 +109,0.784008150775794 +110,0.9086312794015312 +111,0.7893907530443737 +112,0.7154969432352392 +113,0.7474831486701172 +114,0.6157930042670573 +115,0.6740842128730813 +116,0.76408512293475 +117,0.9483082094277656 +118,0.6470592824915199 +119,0.8770568400983276 +120,0.6815889919759166 +121,0.6335394306441874 +122,0.7750620143393231 +123,0.8675402683102266 +124,0.692745625356582 +125,0.9451504936940657 +126,0.8828748074273055 +127,0.8764336748910532 +128,0.7868424204877585 +129,0.725152966145514 +130,0.8315009948323462 +131,0.8314583696670498 +132,0.8505739496094441 +133,0.9055149097584001 +134,0.8143663818742832 +135,0.9259442913072227 +136,0.7392932331894566 +137,0.8776691331704176 +138,0.9635623267213483 +139,0.8825301607419606 +140,0.8243280716675803 +141,0.7534979975765371 +142,0.8612777905365482 +143,0.7255999707859332 +144,0.8421577018557391 +145,0.7646757676670758 +146,0.9303444362775523 +147,0.7898305948663185 +148,0.9533642639183956 +149,0.7360956423011245 +150,0.7282434984791482 +151,0.6912679742775514 +152,0.9226840897109758 +153,0.6531872016038573 +154,0.7002475585181485 +155,0.7559419576353822 +156,0.7931839131027328 +157,0.8908026724731543 +158,0.7721241075975568 +159,0.8632810134211393 +160,0.9663157589105302 +161,0.711654954499155 +162,0.8087923021189856 +163,0.7380479580115581 +164,0.8722027515071817 +165,0.8052674919384717 +166,0.8856884840361561 +167,0.853534911673175 +168,0.8382556605611953 +169,0.7493754118220912 +170,0.7478981444654789 +171,0.8734298638239487 +172,0.6116099502548652 +173,0.6218831906749313 +174,0.9090110686650162 +175,0.9511525198763416 +176,0.8335584824622546 +177,0.7400676170063986 +178,0.837087417658765 +179,0.9002035839290145 +180,0.8513207054122705 +181,0.873869289946607 +182,0.7825361596024328 +183,0.7781565757583491 +184,0.9630724730775555 +185,0.6721522344508162 +186,0.7592773368487423 +187,0.842690276595433 +188,0.7357419461620819 +189,0.8483084844288321 +190,0.7298744146874104 +191,0.8053763731746565 +192,0.5812096056209707 +193,0.6645626526935428 +194,0.7909083112549047 +195,0.9636513503223876 +196,0.8854286080283811 +197,0.587408257565743 +198,0.7721041220549384 +199,0.8644811052842862 +200,0.8141235495372532 +201,0.8653592540385026 +202,0.715428864482864 +203,0.7745023232154046 +204,0.6811221936540882 +205,0.8141923317807285 +206,0.9220622035784289 +207,0.9229313028789478 +208,0.8886629403382629 +209,0.899088358674042 +210,0.787204718404344 +211,0.9254121322668153 +212,0.8563310010045213 +213,0.9462293932267749 +214,0.5807683248399573 +215,0.6218365684011112 +216,0.9142600396014327 +217,0.8167900046449599 +218,0.8153017692225586 +219,0.8340920484848624 +220,0.8205663768475195 +221,0.7865253058859246 +222,0.8634921566817937 +223,0.6611408080285985 +224,0.9189481042929393 +225,0.7241214883083297 +226,0.8242435248280876 +227,0.9084278105892423 +228,0.8218600330269885 +229,0.6946397111900429 +230,0.9618901923299901 +231,0.8147062000525316 +232,0.8221682580594658 +233,0.8445001033812148 +234,0.8047521847076 +235,0.8116258363629817 +236,0.885635274527489 +237,0.7763881909937407 +238,0.9166116504649469 +239,0.7959138422937954 +240,0.8799257407585412 +241,0.7224971593349144 +242,0.7297381612520556 +243,0.8628399245013352 +244,0.9451175575198306 +245,0.8459220187789914 +246,0.7742190721169832 +247,0.7267290100479634 +248,0.7930607062295991 +249,0.7754802717752399 +250,0.6765738998534372 +251,0.6134134775976137 +252,0.8644285672898511 +253,0.7154084632409592 +254,0.8181672483578759 +255,0.9159120925195681 +256,0.740707857861286 +257,0.5788000526722973 +258,0.8096312792096699 +259,0.7438194629510569 +260,0.7310818292533727 +261,0.7425462399903608 +262,0.808906683376499 +263,0.833326362376068 +264,0.6841317285805987 +265,0.7539361126475996 +266,0.8503297742439501 +267,0.7873288526066552 +268,0.7559333878347074 +269,0.7406445436622711 +270,0.8111092180805143 +271,0.7960645812504401 +272,0.8088358546134594 +273,0.706810650829675 +274,0.9467851511959061 +275,0.8333821300341903 +276,0.8757293523923142 +277,0.867690975290003 +278,0.6913980561773457 +279,0.7952409850333547 +280,0.8600313961682631 +281,0.676204055395959 +282,0.9755114108494828 +283,0.7184062626568442 +284,0.7652700897392434 +285,0.7844218035426916 +286,0.9249460054592188 +287,0.8294788416417871 +288,0.8065801743545133 +289,0.9065261782148888 +290,0.9079232795629504 +291,0.7747568271093237 +292,0.7423991463966898 +293,0.6856809119137689 +294,0.8392637952846015 +295,0.9572545057485775 +296,0.6642141048302787 +297,0.8162538484960288 +298,0.7380885366572905 +299,0.6716910001040527 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/metrics.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/metrics.csv new file mode 100644 index 0000000..a488d5c --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/metrics.csv @@ -0,0 +1,78 @@ +schema_version,benchmark_level,problem_id,model_id,method_id,run_id,split_id,quantity,metric,aggregation,derivative_order,layer,neuron,output_index,input_index_a,input_index_b,cell_id,value,unit,status +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,mean_width,mean_weighted,0,NA,NA,NA,NA,NA,NA,31.450924770860677,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,max_width,max,0,NA,NA,NA,NA,NA,NA,31.450924770860677,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,q50_width,q50,0,NA,NA,NA,NA,NA,NA,31.450924770860677,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,q90_width,q90,0,NA,NA,NA,NA,NA,NA,31.450924770860677,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,q99_width,q99,0,NA,NA,NA,NA,NA,NA,31.450924770860677,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,mean_global_normalized_radius,mean_weighted,0,NA,NA,NA,NA,NA,NA,0.9915831900479696,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,Y,max_global_normalized_radius,max,0,NA,NA,NA,NA,NA,NA,0.9915831900479696,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,mean_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,1.3535239727895385,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,max_width,max,1,NA,NA,NA,NA,NA,NA,1.682730578824111,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,q50_width,q50,1,NA,NA,NA,NA,NA,NA,1.3365540106417124,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,q90_width,q90,1,NA,NA,NA,NA,NA,NA,1.4879536752765419,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,q99_width,q99,1,NA,NA,NA,NA,NA,NA,1.6183212551079076,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,mean_global_normalized_radius,mean_weighted,1,NA,NA,NA,NA,NA,NA,0.6947025221129658,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,max_global_normalized_radius,max,1,NA,NA,NA,NA,NA,NA,0.8636693554355609,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,mean_frobenius_width,mean_weighted,1,NA,NA,NA,NA,NA,NA,13.57521459475381,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,J,max_frobenius_width,max,1,NA,NA,NA,NA,NA,NA,13.57521459475381,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,mean_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,max_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,q50_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,q90_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,q99_width,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,mean_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,H,max_global_normalized_radius,none,2,NA,NA,NA,NA,NA,NA,NA,NA,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,3.1882186888590865e-68,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,width,none,NA,NA,NA,NA,NA,NA,NA,3.1882186888590865e-68,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,251.50610809359125,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,251.50610809359125,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,upper,none,NA,NA,NA,NA,NA,NA,NA,1.7855583689308748e-34,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,width,none,NA,NA,NA,NA,NA,NA,NA,1.7855583689308748e-34,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,15.858944103993533,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,L2,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,15.858944103993533,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,upper,none,NA,NA,NA,NA,NA,NA,NA,4.013416448073769e-68,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,width,none,NA,NA,NA,NA,NA,NA,NA,4.013416448073769e-68,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,316.6027332256354,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12_sq,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,316.6027332256354,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,upper,none,NA,NA,NA,NA,NA,NA,NA,2.0033513042084675e-34,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,width,none,NA,NA,NA,NA,NA,NA,NA,2.0033513042084675e-34,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,relative_norm_width,none,NA,NA,NA,NA,NA,NA,NA,1.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,domain_volume_normalized_lower,none,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,domain_volume_normalized_upper,none,NA,NA,NA,NA,NA,NA,NA,17.793333954760566,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,W12,domain_volume_normalized_width,none,NA,NA,NA,NA,NA,NA,NA,17.793333954760566,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,PDE_residual,linf_upper,max,2,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,boundary_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,initial_residual,linf_upper,max,0,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_alpha,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_eta,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,n_mixed_monomials,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,complexity,max_degree,none,NA,NA,NA,NA,NA,NA,NA,NA,dimensionless,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,runtime,seconds,L2_total,NA,NA,NA,NA,NA,NA,NA,0.008936213000197313,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,runtime,seconds,W12_total,NA,NA,NA,NA,NA,NA,NA,0.5348716300004526,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,runtime,seconds,final_enclosures,NA,NA,NA,NA,NA,NA,NA,0.9840491009999823,seconds,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,memory,bytes,peak,NA,NA,NA,NA,NA,NA,NA,NA,bytes,not_implemented +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,soundness,failure_count,Y,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,soundness,max_violation,Y,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,soundness,failure_count,J,NA,NA,NA,NA,NA,NA,NA,0,dimensionless,ok +1.2,medium,poisson_100d_ridge,pinn_100d_poisson_shallow_300_seed_20260804,interval,pinn100d_shallow300_medium_interval,single_cell,soundness,max_violation,J,NA,NA,NA,NA,NA,NA,NA,0.0,dimensionless,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/norms.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/norms.csv new file mode 100644 index 0000000..2fd77eb --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/norms.csv @@ -0,0 +1,5 @@ +run_id,norm,squared,lower,upper,width,relative_width,value_contribution_upper,gradient_contribution_upper,hessian_contribution_upper,value_contribution_width,gradient_contribution_width,hessian_contribution_width,status,domain_volume,domain_volume_normalized_lower,domain_volume_normalized_upper,domain_volume_normalized_width +pinn100d_shallow300_medium_interval,L2,1,0.0,3.1882186888590865e-68,3.1882186888590865e-68,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,251.50610809359125,251.50610809359125 +pinn100d_shallow300_medium_interval,L2,0,0.0,1.7855583689308748e-34,1.7855583689308748e-34,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,15.858944103993533,15.858944103993533 +pinn100d_shallow300_medium_interval,W12,1,0.0,4.013416448073769e-68,4.013416448073769e-68,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,316.6027332256354,316.6027332256354 +pinn100d_shallow300_medium_interval,W12,0,0.0,2.0033513042084675e-34,2.0033513042084675e-34,1.0,NA,NA,NA,NA,NA,NA,ok,1.2676506002282363e-70,0.0,17.793333954760566,17.793333954760566 diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/soundness.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/soundness.csv new file mode 100644 index 0000000..c85b99e --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/soundness.csv @@ -0,0 +1,3 @@ +run_id,quantity,sample_count,failure_count,max_violation,invalid_interval_count,nan_endpoint_count,infinite_endpoint_count,status +pinn100d_shallow300_medium_interval,Y,16384,0,0.0,0,0,0,ok +pinn100d_shallow300_medium_interval,J,16384,0,0.0,0,0,0,ok diff --git a/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/timings.csv b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/timings.csv new file mode 100644 index 0000000..4191f59 --- /dev/null +++ b/notebooks/benchmark_outputs/pinn_100d_poisson_shallow_300_medium/interval/timings.csv @@ -0,0 +1,4 @@ +run_id,stage,seconds,status +pinn100d_shallow300_medium_interval,L2_total,0.008936213000197313,ok +pinn100d_shallow300_medium_interval,W12_total,0.5348716300004526,ok +pinn100d_shallow300_medium_interval,final_enclosures,0.9840491009999823,ok diff --git a/notebooks/certified_support_reduction_variants_benchmarks.ipynb b/notebooks/certified_support_reduction_variants_benchmarks.ipynb new file mode 100644 index 0000000..b3bdffd --- /dev/null +++ b/notebooks/certified_support_reduction_variants_benchmarks.ipynb @@ -0,0 +1,4776 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "1d1fdcdf", + "metadata": {}, + "source": [ + "# Certified support-reduction variants A, B, and C\n", + "\n", + "This notebook implements and benchmarks the three reductions specified in\n", + "`certified_polynomial_support_reduction_variants.tex` on the saved\n", + "100-dimensional Poisson PINN. The network is loaded from its checkpoint and\n", + "is never retrained.\n", + "\n", + "The two approximation choices are fixed across all reduction runs:\n", + "\n", + "- function values use the certified affine approximation of $\\tanh$;\n", + "- $\\tanh'$ uses the certified endpoint-midpoint quadratic whenever the\n", + " preactivation interval crosses zero and the affine approximation is flat.\n", + "\n", + "Support selection and enclosure of discarded support are separate choices.\n", + "All main runs use coefficient-norm Top-$K$ support selection. Variant A\n", + "replaces discarded terms by a symmetric coefficient-sum box. Variant B uses\n", + "the exact $[0,1]$ range of componentwise-even monomials and adds the resulting\n", + "midpoint correction. Variant C starts from B, retains a budget of correlated\n", + "coefficient-space generators as fresh approximation-noise symbols, and boxes\n", + "only the remaining directions. The compact pointwise remainder tensor used\n", + "internally represents the axis-aligned generators of the final box without\n", + "allocating thousands of explicit zero-heavy coefficient tensors." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "71df79e2", + "metadata": { + "lines_to_next_cell": 1 + }, + "outputs": [], + "source": [ + "from math import sqrt\n", + "from pathlib import Path\n", + "from time import perf_counter\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "\n", + "from intervalnets import (\n", + " IntervalTensor,\n", + " PZIntegrationCell,\n", + " enable_interval_eval,\n", + " integrate_pz_onejet_squared,\n", + " integrate_pz_value_squared,\n", + " load_tanh_mlp_checkpoint,\n", + ")\n", + "\n", + "torch.set_num_threads(1)\n", + "pd.set_option(\"display.max_rows\", 500)\n", + "pd.set_option(\"display.max_columns\", 40)\n", + "torch.set_default_dtype(torch.float64)\n", + "enable_interval_eval()\n", + "\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / \"notebooks\" / \"checkpoints\").exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "checkpoint = repo_root / \"notebooks\" / \"checkpoints\" / \"pinn_100d_poisson.pt\"\n", + "domain_volume = 0.2 ** 100\n", + "normalization = sqrt(domain_volume)" + ] + }, + { + "cell_type": "markdown", + "id": "8fb64a72", + "metadata": {}, + "source": [ + "## Benchmark and diagnostic helpers\n", + "\n", + "Widths below refer to the final norm interval, not the squared-norm interval.\n", + "The normalized width divides by $|\\Omega|^{1/2}$, making the extremely small\n", + "100-dimensional volume invisible in the scale of the comparison." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "3fa081d4", + "metadata": { + "lines_to_next_cell": 1 + }, + "outputs": [], + "source": [ + "def norm_interval(squared):\n", + " return sqrt(max(0.0, float(squared.lower))), sqrt(max(0.0, float(squared.upper)))\n", + "\n", + "\n", + "def norm_metrics(squared, prefix):\n", + " lower, upper = norm_interval(squared)\n", + " width = upper - lower\n", + " scale = max(abs(lower), abs(upper))\n", + " return {\n", + " f\"{prefix}_lower\": lower,\n", + " f\"{prefix}_upper\": upper,\n", + " f\"{prefix}_width\": width,\n", + " f\"{prefix}_normalized_lower\": lower / normalization,\n", + " f\"{prefix}_normalized_upper\": upper / normalization,\n", + " f\"{prefix}_normalized_width\": width / normalization,\n", + " f\"{prefix}_relative_width\": width / scale if scale else 0.0,\n", + " }\n", + "\n", + "\n", + "def jacobian_metrics(enclosure):\n", + " lower = torch.as_tensor(enclosure.lower)\n", + " upper = torch.as_tensor(enclosure.upper)\n", + " widths = upper - lower\n", + " scales = torch.maximum(lower.abs(), upper.abs())\n", + " relative = torch.where(scales > 0, widths / scales, 0.0)\n", + " return {\n", + " \"J_mean_width\": float(widths.mean()),\n", + " \"J_max_width\": float(widths.max()),\n", + " \"J_mean_relative_width\": float(relative.mean()),\n", + " }\n", + "\n", + "\n", + "def benchmark_pz(model, box, label, **kwargs):\n", + " cell = PZIntegrationCell.from_affine_box(box)\n", + " start = perf_counter()\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain,\n", + " reduction_strategy=\"topk\",\n", + " derivative_enclosure=\"quadratic_flat\",\n", + " derivative_flatness_threshold=1.0,\n", + " quadratic_compression_guard=False,\n", + " return_trace=True,\n", + " **kwargs,\n", + " )\n", + " forward_s = perf_counter() - start\n", + "\n", + " start = perf_counter()\n", + " l2_pz = integrate_pz_value_squared(traced.final.Y, cell, output=\"pz\")\n", + " l2_squared = l2_pz.interval_enclosure()\n", + " l2_s = perf_counter() - start\n", + "\n", + " start = perf_counter()\n", + " w12_pz = integrate_pz_onejet_squared(traced.final, cell, output=\"pz\")\n", + " w12_squared = w12_pz.interval_enclosure()\n", + " w12_s = perf_counter() - start\n", + "\n", + " row = {\n", + " \"method\": label,\n", + " **kwargs,\n", + " \"forward_s\": forward_s,\n", + " \"L2_integration_s\": l2_s,\n", + " \"W12_integration_s\": w12_s,\n", + " \"total_s\": forward_s + l2_s + w12_s,\n", + " \"J_terms\": len(traced.final.J.terms),\n", + " \"J_degree\": max(map(sum, traced.final.J.terms), default=0),\n", + " \"J_noise_symbols\": traced.final.J.num_noise,\n", + " \"L2_integrated_terms\": len(l2_pz.terms),\n", + " \"W12_integrated_terms\": len(w12_pz.terms),\n", + " **jacobian_metrics(traced.final.J.interval_enclosure()),\n", + " **norm_metrics(l2_squared, \"L2\"),\n", + " **norm_metrics(w12_squared, \"W12\"),\n", + " }\n", + " return row, traced.records\n", + "\n", + "\n", + "def benchmark_interval(model, box):\n", + " start = perf_counter()\n", + " l2 = model.lpnorm(box, p=2.0, method=\"interval\")\n", + " w12 = model.sobolev_norm(box, p=2.0, order=1, method=\"interval\")\n", + " elapsed = perf_counter() - start\n", + " jacobian = model.eval_jacobian(box)\n", + " def direct_metrics(bounds, prefix):\n", + " lower, upper = float(bounds.lower), float(bounds.upper)\n", + " width = upper - lower\n", + " scale = max(abs(lower), abs(upper))\n", + " return {\n", + " f\"{prefix}_lower\": lower,\n", + " f\"{prefix}_upper\": upper,\n", + " f\"{prefix}_width\": width,\n", + " f\"{prefix}_normalized_lower\": lower / normalization,\n", + " f\"{prefix}_normalized_upper\": upper / normalization,\n", + " f\"{prefix}_normalized_width\": width / normalization,\n", + " f\"{prefix}_relative_width\": width / scale if scale else 0.0,\n", + " }\n", + " return {\n", + " \"method\": \"interval\",\n", + " \"total_s\": elapsed,\n", + " **jacobian_metrics(jacobian),\n", + " **direct_metrics(l2, \"L2\"),\n", + " **direct_metrics(w12, \"W12\"),\n", + " }\n", + "\n", + "\n", + "def activation_rows(method, trace):\n", + " rows = []\n", + " activations = [record for record in trace if record.layer_type == \"Tanh\"]\n", + " for layer, record in enumerate(activations, start=1):\n", + " summary = record.summary\n", + " lower = summary[\"preactivation_lower\"]\n", + " upper = summary[\"preactivation_upper\"]\n", + " d_lo = 1.0 - torch.tanh(lower) ** 2\n", + " d_hi = 1.0 - torch.tanh(upper) ** 2\n", + " d_max = torch.where((lower <= 0) & (upper >= 0), torch.ones_like(lower), torch.maximum(d_lo, d_hi))\n", + " interval_radius = 0.5 * (d_max - torch.minimum(d_lo, d_hi))\n", + " for neuron in range(lower.numel()):\n", + " rows.append({\n", + " \"method\": method,\n", + " \"layer\": layer,\n", + " \"neuron\": neuron,\n", + " \"preactivation_lower\": float(lower[neuron]),\n", + " \"preactivation_upper\": float(upper[neuron]),\n", + " \"preactivation_width\": float(upper[neuron] - lower[neuron]),\n", + " \"crosses_zero\": bool(lower[neuron] <= 0 <= upper[neuron]),\n", + " \"relative_affine_slope\": float(summary[\"tanh_prime_relative_slopes\"][neuron]),\n", + " \"tanh_affine_rho\": float(summary[\"tanh_approximation_radii\"][neuron]),\n", + " \"tanh_prime_interval_radius\": float(interval_radius[neuron]),\n", + " \"tanh_prime_affine_rho\": float(summary[\"tanh_prime_affine_radii\"][neuron]),\n", + " \"tanh_prime_selected_rho\": float(summary[\"tanh_prime_approximation_radii\"][neuron]),\n", + " \"quadratic_core_box_radius\": float(summary[\"tanh_prime_polynomial_reduction_radii\"][neuron]),\n", + " \"approximation_degree\": int(summary[\"tanh_prime_approximation_degrees\"][neuron]),\n", + " \"propagated_J_remainder_mean\": summary[\"J\"][\"remainder_mean_radius\"],\n", + " \"J_terms_after_activation\": summary[\"J\"][\"term_count\"],\n", + " \"J_degree_after_activation\": summary[\"J\"][\"max_degree\"],\n", + " \"J_noise_after_activation\": summary[\"J\"][\"num_noise\"],\n", + " \"activation_runtime_s\": record.elapsed_s,\n", + " })\n", + " return rows" + ] + }, + { + "cell_type": "markdown", + "id": "8555fc50", + "metadata": {}, + "source": [ + "## Structural and small-network regression checks\n", + "\n", + "These checks complement the unit tests. They verify the expected ordering\n", + "for a discarded square and test sampled Jacobians against each certified\n", + "enclosure on a tractable network where an unreduced quadratic reference is\n", + "still feasible." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "6b50001d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " variant generator_budget center \\\n", + "0 A 0 [0.0, 0.0] \n", + "1 B 0 [1.0, -2.0] \n", + "2 C 0 [1.0, -2.0] \n", + "3 C 1 [1.0, -2.0] \n", + "\n", + " box_radius noise_symbols terms \n", + "0 [2.0000000000000004, 4.000000000000001] 1 1 \n", + "1 [1.0000000000000002, 2.0000000000000004] 1 1 \n", + "2 [1.0000000000000002, 2.0000000000000004] 1 1 \n", + "3 [0.0, 0.0] 2 2 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from intervalnets.pytorch import PZReductionConfig, _CertifiedTermReducer\n", + "\n", + "\n", + "def square_reduction(variant, generator_budget=0):\n", + " center = torch.zeros(2)\n", + " reducer = _CertifiedTermReducer(\n", + " PZReductionConfig(\n", + " strategy=\"topk\", max_terms=1,\n", + " reduction_variant=variant, generator_budget=generator_budget,\n", + " ), tuple(center.shape), center,\n", + " )\n", + " reducer.offer((1,), torch.tensor([10.0, 0.0]))\n", + " reducer.offer((2,), torch.tensor([2.0, -4.0]))\n", + " return reducer.finish(center, num_noise=1, noise_kinds=(\"domain\",))\n", + "\n", + "\n", + "structural = []\n", + "for variant, budget in [(\"A\", 0), (\"B\", 0), (\"C\", 0), (\"C\", 1)]:\n", + " reduced, radius = square_reduction(variant, budget)\n", + " structural.append({\n", + " \"variant\": variant, \"generator_budget\": budget,\n", + " \"center\": reduced.center.tolist(), \"box_radius\": radius.tolist(),\n", + " \"noise_symbols\": reduced.num_noise, \"terms\": len(reduced.terms),\n", + " })\n", + "structural_table = pd.DataFrame(structural)\n", + "assert np.allclose(structural_table.loc[1, \"box_radius\"], [1.0, 2.0])\n", + "assert np.allclose(structural_table.loc[2, \"box_radius\"], [1.0, 2.0])\n", + "display(structural_table)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "45aa106b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " sample_violations J_terms noise J_mean_width J_max_width \\\n", + "method \n", + "unreduced 0 1895 28 0.062576 0.087104 \n", + "A 0 30 28 0.073462 0.096885 \n", + "B 0 30 28 0.068265 0.091551 \n", + "C-0 0 30 28 0.068265 0.091551 \n", + "C-8 0 38 39 0.064584 0.088583 \n", + "\n", + " J_mean_relative_width \n", + "method \n", + "unreduced 0.848773 \n", + "A 0.907624 \n", + "B 0.879850 \n", + "C-0 0.879850 \n", + "C-8 0.859350 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "torch.manual_seed(20260802)\n", + "small_model = torch.nn.Sequential(\n", + " torch.nn.Linear(6, 8), torch.nn.Tanh(),\n", + " torch.nn.Linear(8, 8), torch.nn.Tanh(),\n", + " torch.nn.Linear(8, 1),\n", + ").double()\n", + "small_box = IntervalTensor.from_bounds([-0.3] * 6, [0.3] * 6)\n", + "small_cell = PZIntegrationCell.from_affine_box(small_box)\n", + "small_configs = [\n", + " (\"unreduced\", dict(reduce=False)),\n", + " (\"A\", dict(max_terms=24, reduction_variant=\"A\")),\n", + " (\"B\", dict(max_terms=24, reduction_variant=\"B\")),\n", + " (\"C-0\", dict(max_terms=24, reduction_variant=\"C\", generator_budget=0)),\n", + " (\"C-8\", dict(max_terms=24, reduction_variant=\"C\", generator_budget=8)),\n", + "]\n", + "small_rows = []\n", + "for label, kwargs in small_configs:\n", + " traced = small_model.eval_pz_onejet(\n", + " small_cell.domain,\n", + " reduction_strategy=\"topk\",\n", + " derivative_enclosure=\"quadratic_flat\",\n", + " derivative_flatness_threshold=1.0,\n", + " quadratic_compression_guard=False,\n", + " return_trace=True,\n", + " **kwargs,\n", + " )\n", + " enclosure = traced.final.J.interval_enclosure()\n", + " violations = 0\n", + " for _ in range(64):\n", + " point = torch.empty(6).uniform_(-0.3, 0.3).requires_grad_(True)\n", + " gradient = torch.autograd.grad(small_model(point).sum(), point)[0]\n", + " lo = torch.as_tensor(enclosure.lower)[0]\n", + " hi = torch.as_tensor(enclosure.upper)[0]\n", + " violations += int(torch.any((gradient < lo) | (gradient > hi)))\n", + " small_rows.append({\n", + " \"method\": label, \"sample_violations\": violations,\n", + " \"J_terms\": len(traced.final.J.terms), \"noise\": traced.final.J.num_noise,\n", + " **jacobian_metrics(enclosure),\n", + " })\n", + "small_reference_table = pd.DataFrame(small_rows).set_index(\"method\")\n", + "assert small_reference_table[\"sample_violations\"].sum() == 0\n", + "display(small_reference_table)" + ] + }, + { + "cell_type": "markdown", + "id": "be0b5913", + "metadata": {}, + "source": [ + "## Saved 100D Poisson PINN benchmark\n", + "\n", + "The parameter sweep compares two retained-support budgets for A and B and\n", + "three structured choices for C. An affine-derivative Top-96 run is included\n", + "to distinguish the gain of the quadratic approximation from the behavior of\n", + "the reduction operator. C with generator budget zero is included as the\n", + "required numerical identity check against B." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "f941ea66", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "running A-K96 ...\n", + "running A-K192 ...\n", + "running B-K96 ...\n", + "running B-K192 ...\n", + "running C-K96-G0 ...\n", + "running C-K96-G8 ...\n", + "running C-K96-G32 ...\n", + "running C-K192-G32 ...\n" + ] + }, + { + "data": { + "text/plain": [ + " reduction_variant max_terms generator_budget total_s \\\n", + "method \n", + "interval NaN NaN NaN 0.214725 \n", + "affine-A-K96 A 96.0 NaN 2.794268 \n", + "A-K96 A 96.0 NaN 10.921846 \n", + "A-K192 A 192.0 NaN 21.279968 \n", + "B-K96 B 96.0 NaN 8.400954 \n", + "B-K192 B 192.0 NaN 16.254778 \n", + "C-K96-G0 C 96.0 0.0 10.784935 \n", + "C-K96-G8 C 96.0 8.0 10.287620 \n", + "C-K96-G32 C 96.0 32.0 14.119499 \n", + "C-K192-G32 C 192.0 32.0 31.528201 \n", + "\n", + " J_terms J_degree J_noise_symbols J_mean_width \\\n", + "method \n", + "interval NaN NaN NaN 17.548815 \n", + "affine-A-K96 196.0 1.0 350.0 16.809482 \n", + "A-K96 196.0 2.0 350.0 95.380557 \n", + "A-K192 292.0 2.0 350.0 93.641813 \n", + "B-K96 196.0 2.0 350.0 94.099304 \n", + "B-K192 292.0 2.0 350.0 92.380192 \n", + "C-K96-G0 196.0 2.0 350.0 94.099304 \n", + "C-K96-G8 204.0 2.0 374.0 93.950519 \n", + "C-K96-G32 228.0 2.0 446.0 93.503561 \n", + "C-K192-G32 324.0 2.0 446.0 91.812086 \n", + "\n", + " L2_normalized_width L2_relative_width W12_normalized_width \\\n", + "method \n", + "interval 7.979523 1.0 88.846805 \n", + "affine-A-K96 2.676831 1.0 85.167723 \n", + "A-K96 2.676831 1.0 480.616605 \n", + "A-K192 2.676831 1.0 471.885901 \n", + "B-K96 2.676831 1.0 474.166433 \n", + "B-K192 2.676831 1.0 465.534292 \n", + "C-K96-G0 2.676831 1.0 474.166433 \n", + "C-K96-G8 2.676831 1.0 473.419414 \n", + "C-K96-G32 2.676831 1.0 471.174670 \n", + "C-K192-G32 2.676831 1.0 462.682211 \n", + "\n", + " W12_relative_width \n", + "method \n", + "interval 1.0 \n", + "affine-A-K96 1.0 \n", + "A-K96 1.0 \n", + "A-K192 1.0 \n", + "B-K96 1.0 \n", + "B-K192 1.0 \n", + "C-K96-G0 1.0 \n", + "C-K96-G8 1.0 \n", + "C-K96-G32 1.0 \n", + "C-K192-G32 1.0 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "model = load_tanh_mlp_checkpoint(checkpoint)\n", + "assert sum(parameter.numel() for parameter in model.parameters()) == 10201\n", + "box = IntervalTensor.from_bounds([-0.1] * 100, [0.1] * 100)\n", + "\n", + "configs = [\n", + " (\"A-K96\", dict(max_terms=96, reduction_variant=\"A\")),\n", + " (\"A-K192\", dict(max_terms=192, reduction_variant=\"A\")),\n", + " (\"B-K96\", dict(max_terms=96, reduction_variant=\"B\")),\n", + " (\"B-K192\", dict(max_terms=192, reduction_variant=\"B\")),\n", + " (\"C-K96-G0\", dict(max_terms=96, reduction_variant=\"C\", generator_budget=0)),\n", + " (\"C-K96-G8\", dict(max_terms=96, reduction_variant=\"C\", generator_budget=8)),\n", + " (\"C-K96-G32\", dict(max_terms=96, reduction_variant=\"C\", generator_budget=32)),\n", + " (\"C-K192-G32\", dict(max_terms=192, reduction_variant=\"C\", generator_budget=32)),\n", + "]\n", + "\n", + "interval_row = benchmark_interval(model, box)\n", + "benchmark_rows = [interval_row]\n", + "traces = {}\n", + "for label, kwargs in configs:\n", + " print(f\"running {label} ...\", flush=True)\n", + " row, trace = benchmark_pz(model, box, label, **kwargs)\n", + " benchmark_rows.append(row)\n", + " traces[label] = trace\n", + "\n", + "# Affine derivative control with otherwise identical A-K96 reduction.\n", + "cell = PZIntegrationCell.from_affine_box(box)\n", + "start = perf_counter()\n", + "affine_trace = model.eval_pz_onejet(\n", + " cell.domain, reduction_strategy=\"topk\", max_terms=96,\n", + " reduction_variant=\"A\", derivative_enclosure=\"affine\", return_trace=True,\n", + ")\n", + "affine_forward_s = perf_counter() - start\n", + "start = perf_counter()\n", + "affine_l2_pz = integrate_pz_value_squared(affine_trace.final.Y, cell, output=\"pz\")\n", + "affine_l2_squared = affine_l2_pz.interval_enclosure()\n", + "affine_l2_s = perf_counter() - start\n", + "start = perf_counter()\n", + "affine_w12_pz = integrate_pz_onejet_squared(affine_trace.final, cell, output=\"pz\")\n", + "affine_w12_squared = affine_w12_pz.interval_enclosure()\n", + "affine_w12_s = perf_counter() - start\n", + "benchmark_rows.insert(1, {\n", + " \"method\": \"affine-A-K96\", \"max_terms\": 96, \"reduction_variant\": \"A\",\n", + " \"forward_s\": affine_forward_s, \"L2_integration_s\": affine_l2_s,\n", + " \"W12_integration_s\": affine_w12_s,\n", + " \"total_s\": affine_forward_s + affine_l2_s + affine_w12_s,\n", + " \"J_terms\": len(affine_trace.final.J.terms),\n", + " \"J_degree\": max(map(sum, affine_trace.final.J.terms), default=0),\n", + " \"J_noise_symbols\": affine_trace.final.J.num_noise,\n", + " **jacobian_metrics(affine_trace.final.J.interval_enclosure()),\n", + " **norm_metrics(affine_l2_squared, \"L2\"),\n", + " **norm_metrics(affine_w12_squared, \"W12\"),\n", + "})\n", + "traces[\"affine-A-K96\"] = affine_trace.records\n", + "\n", + "benchmark_table = pd.DataFrame(benchmark_rows).set_index(\"method\")\n", + "display_columns = [\n", + " \"reduction_variant\", \"max_terms\", \"generator_budget\", \"total_s\",\n", + " \"J_terms\", \"J_degree\", \"J_noise_symbols\", \"J_mean_width\",\n", + " \"L2_normalized_width\", \"L2_relative_width\",\n", + " \"W12_normalized_width\", \"W12_relative_width\",\n", + "]\n", + "display(benchmark_table.reindex(columns=display_columns))" + ] + }, + { + "cell_type": "markdown", + "id": "114278f4", + "metadata": {}, + "source": [ + "### Exact consistency checks\n", + "\n", + "Function-value propagation is deliberately identical in every PZ run, so\n", + "all PZ $L^2$ bounds must agree. Variant C with generator budget zero is\n", + "Variant B by definition and must reproduce it exactly at a fixed support\n", + "budget." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cb782077", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " check passed\n", + "0 common affine-value L2 True\n", + "1 C(K=0) equals B: J width True\n", + "2 C(K=0) equals B: W12 width True" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "pz_only = benchmark_table.drop(index=\"interval\")\n", + "assert np.allclose(\n", + " pz_only[\"L2_normalized_width\"],\n", + " pz_only[\"L2_normalized_width\"].iloc[0],\n", + " rtol=0.0, atol=1e-12,\n", + ")\n", + "for column in [\"J_mean_width\", \"W12_normalized_width\"]:\n", + " assert np.isclose(\n", + " benchmark_table.loc[\"B-K96\", column],\n", + " benchmark_table.loc[\"C-K96-G0\", column],\n", + " rtol=1e-12, atol=1e-12,\n", + " )\n", + "consistency_table = pd.DataFrame({\n", + " \"check\": [\"common affine-value L2\", \"C(K=0) equals B: J width\", \"C(K=0) equals B: W12 width\"],\n", + " \"passed\": [True, True, True],\n", + "})\n", + "display(consistency_table)" + ] + }, + { + "cell_type": "markdown", + "id": "d4c6f621", + "metadata": {}, + "source": [ + "## Layerwise and per-neuron diagnostics\n", + "\n", + "The full table contains one row for every hidden neuron and every method.\n", + "Approximation error is denoted by $\\rho$. The quadratic-core box radius is a\n", + "separate support-reduction contribution; it is not the activation\n", + "approximation error." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "4f38e005", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " neurons zero_crossing_fraction mean_tanh_affine_rho \\\n", + "method layer \n", + "A-K96 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "A-K192 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "B-K96 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "B-K192 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "C-K96-G0 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "C-K96-G8 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "C-K96-G32 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "C-K192-G32 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "affine-A-K96 1 50 1.0 0.074179 \n", + " 2 50 1.0 0.092422 \n", + " 3 50 1.0 0.133023 \n", + "\n", + " mean_tanh_prime_interval_radius \\\n", + "method layer \n", + "A-K96 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "A-K192 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "B-K96 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "B-K192 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "C-K96-G0 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "C-K96-G8 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "C-K96-G32 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "C-K192-G32 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "affine-A-K96 1 0.279480 \n", + " 2 0.312394 \n", + " 3 0.366982 \n", + "\n", + " mean_tanh_prime_affine_rho mean_tanh_prime_selected_rho \\\n", + "method layer \n", + "A-K96 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "A-K192 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "B-K96 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "B-K192 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "C-K96-G0 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "C-K96-G8 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "C-K96-G32 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "C-K192-G32 1 0.275129 0.038511 \n", + " 2 0.305928 0.052023 \n", + " 3 0.359351 0.082131 \n", + "affine-A-K96 1 0.275129 0.275129 \n", + " 2 0.305928 0.305928 \n", + " 3 0.359351 0.359351 \n", + "\n", + " max_tanh_prime_selected_rho \\\n", + "method layer \n", + "A-K96 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "A-K192 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "B-K96 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "B-K192 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "C-K96-G0 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "C-K96-G8 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "C-K96-G32 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "C-K192-G32 1 0.055347 \n", + " 2 0.076258 \n", + " 3 0.127976 \n", + "affine-A-K96 1 0.321017 \n", + " 2 0.356997 \n", + " 3 0.423880 \n", + "\n", + " mean_quadratic_core_box_radius \\\n", + "method layer \n", + "A-K96 1 0.544921 \n", + " 2 0.612695 \n", + " 3 0.716975 \n", + "A-K192 1 0.531821 \n", + " 2 0.602778 \n", + " 3 0.703161 \n", + "B-K96 1 0.541197 \n", + " 2 0.609649 \n", + " 3 0.713878 \n", + "B-K192 1 0.528097 \n", + " 2 0.599732 \n", + " 3 0.700132 \n", + "C-K96-G0 1 0.541197 \n", + " 2 0.609649 \n", + " 3 0.713878 \n", + "C-K96-G8 1 0.540093 \n", + " 2 0.608785 \n", + " 3 0.712663 \n", + "C-K96-G32 1 0.536726 \n", + " 2 0.606271 \n", + " 3 0.709092 \n", + "C-K192-G32 1 0.523829 \n", + " 2 0.596516 \n", + " 3 0.695709 \n", + "affine-A-K96 1 0.000000 \n", + " 2 0.000000 \n", + " 3 0.000000 \n", + "\n", + " max_quadratic_core_box_radius \\\n", + "method layer \n", + "A-K96 1 0.630954 \n", + " 2 0.709562 \n", + " 3 0.831610 \n", + "A-K192 1 0.615779 \n", + " 2 0.697297 \n", + " 3 0.816813 \n", + "B-K96 1 0.626968 \n", + " 2 0.706212 \n", + " 3 0.828240 \n", + "B-K192 1 0.611792 \n", + " 2 0.693947 \n", + " 3 0.813506 \n", + "C-K96-G0 1 0.626968 \n", + " 2 0.706212 \n", + " 3 0.828240 \n", + "C-K96-G8 1 0.625377 \n", + " 2 0.705079 \n", + " 3 0.827011 \n", + "C-K96-G32 1 0.621877 \n", + " 2 0.702134 \n", + " 3 0.822975 \n", + "C-K192-G32 1 0.606270 \n", + " 2 0.690812 \n", + " 3 0.807691 \n", + "affine-A-K96 1 0.000000 \n", + " 2 0.000000 \n", + " 3 0.000000 \n", + "\n", + " mean_propagated_J_remainder quadratic_neurons \\\n", + "method layer \n", + "A-K96 1 0.056034 50 \n", + " 2 0.581434 50 \n", + " 3 6.093022 50 \n", + "A-K192 1 0.054775 50 \n", + " 2 0.570604 50 \n", + " 3 5.981726 50 \n", + "B-K96 1 0.055677 50 \n", + " 2 0.575656 50 \n", + " 3 6.010999 50 \n", + "B-K192 1 0.054418 50 \n", + " 2 0.564866 50 \n", + " 3 5.900971 50 \n", + "C-K96-G0 1 0.055677 50 \n", + " 2 0.575656 50 \n", + " 3 6.010999 50 \n", + "C-K96-G8 1 0.055571 50 \n", + " 2 0.574725 50 \n", + " 3 6.001467 50 \n", + "C-K96-G32 1 0.055247 50 \n", + " 2 0.571931 50 \n", + " 3 5.972857 50 \n", + "C-K192-G32 1 0.054007 50 \n", + " 2 0.561316 50 \n", + " 3 5.864610 50 \n", + "affine-A-K96 1 0.026447 0 \n", + " 2 0.173416 0 \n", + " 3 1.065381 0 \n", + "\n", + " J_terms_after_activation J_degree_after_activation \\\n", + "method layer \n", + "A-K96 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + "A-K192 1 192 2 \n", + " 2 192 2 \n", + " 3 192 2 \n", + "B-K96 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + "B-K192 1 192 2 \n", + " 2 192 2 \n", + " 3 192 2 \n", + "C-K96-G0 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + "C-K96-G8 1 104 2 \n", + " 2 104 2 \n", + " 3 104 2 \n", + "C-K96-G32 1 128 2 \n", + " 2 128 2 \n", + " 3 128 2 \n", + "C-K192-G32 1 224 2 \n", + " 2 224 2 \n", + " 3 224 2 \n", + "affine-A-K96 1 96 1 \n", + " 2 96 1 \n", + " 3 96 1 \n", + "\n", + " J_noise_after_activation activation_runtime_s \n", + "method layer \n", + "A-K96 1 150 0.146239 \n", + " 2 200 0.912937 \n", + " 3 250 1.090658 \n", + "A-K192 1 150 0.167736 \n", + " 2 200 2.780127 \n", + " 3 250 2.440535 \n", + "B-K96 1 150 0.326040 \n", + " 2 200 0.805258 \n", + " 3 250 1.360132 \n", + "B-K192 1 150 0.227467 \n", + " 2 200 2.750027 \n", + " 3 250 2.482119 \n", + "C-K96-G0 1 150 0.167491 \n", + " 2 200 1.014192 \n", + " 3 250 1.230961 \n", + "C-K96-G8 1 158 0.181101 \n", + " 2 216 1.141357 \n", + " 3 274 1.451618 \n", + "C-K96-G32 1 182 0.184889 \n", + " 2 264 1.464546 \n", + " 3 346 1.695728 \n", + "C-K192-G32 1 182 0.206389 \n", + " 2 264 4.905035 \n", + " 3 346 7.972779 \n", + "affine-A-K96 1 150 0.033760 \n", + " 2 200 0.764419 \n", + " 3 250 0.864134 " + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + " preactivation_lower preactivation_upper \\\n", + "method layer neuron \n", + "A-K96 1 0 -1.050857 1.035540 \n", + " 1 -0.875598 0.908855 \n", + " 2 -1.020552 1.044043 \n", + " 3 -0.916223 0.938538 \n", + " 4 -0.994526 0.943069 \n", + " 5 -0.927317 0.952750 \n", + " 6 -1.052830 1.066374 \n", + " 7 -0.992329 0.970296 \n", + " 8 -0.999898 0.973126 \n", + " 9 -1.054855 1.034176 \n", + " 10 -0.981444 1.002705 \n", + " 11 -1.011183 1.034997 \n", + " 12 -0.934483 0.913232 \n", + " 13 -0.889433 0.834707 \n", + " 14 -0.927744 0.941738 \n", + " 15 -0.955251 0.934459 \n", + " 16 -0.882165 0.849501 \n", + " 17 -1.011610 1.033830 \n", + " 18 -1.028520 1.006007 \n", + " 19 -0.935727 0.902334 \n", + " 20 -0.906291 0.927878 \n", + " 21 -0.967293 0.908834 \n", + " 22 -0.905860 0.862763 \n", + " 23 -0.921374 0.905636 \n", + " 24 -0.975794 0.949279 \n", + " 25 -0.955553 0.984535 \n", + " 26 -1.031405 1.007670 \n", + " 27 -0.993490 0.961355 \n", + " 28 -0.954089 0.931932 \n", + " 29 -0.955465 0.979020 \n", + " 30 -0.834076 0.814742 \n", + " 31 -0.975156 0.992306 \n", + " 32 -1.037018 1.012538 \n", + " 33 -0.922759 0.901525 \n", + " 34 -0.980898 0.931827 \n", + " 35 -0.985529 0.961815 \n", + " 36 -0.939032 0.912356 \n", + " 37 -0.907336 0.895302 \n", + " 38 -0.981192 1.004048 \n", + " 39 -1.112073 1.092490 \n", + " 40 -0.853549 0.821142 \n", + " 41 -0.941923 0.962062 \n", + " 42 -0.879293 0.894091 \n", + " 43 -0.995571 0.989681 \n", + " 44 -0.980030 0.989806 \n", + " 45 -0.968446 0.947372 \n", + " 46 -0.985826 0.970922 \n", + " 47 -0.971530 0.926372 \n", + " 48 -0.833705 0.871394 \n", + " 49 -0.933434 0.961565 \n", + " 2 0 -1.006688 0.982892 \n", + " 1 -0.987402 0.995564 \n", + " 2 -1.014677 1.100242 \n", + " 3 -0.955748 0.951760 \n", + " 4 -1.028939 0.961956 \n", + " 5 -0.904774 0.956719 \n", + " 6 -0.978616 1.029530 \n", + " 7 -0.885356 0.900714 \n", + " 8 -1.090571 1.081573 \n", + " 9 -1.155747 1.065222 \n", + " 10 -0.993981 1.024611 \n", + " 11 -1.206621 1.236036 \n", + " 12 -1.031230 0.984579 \n", + " 13 -0.945045 1.016436 \n", + " 14 -1.007499 1.074294 \n", + " 15 -0.882242 0.931403 \n", + " 16 -0.993391 1.006453 \n", + " 17 -1.083633 1.076336 \n", + " 18 -1.175003 1.082563 \n", + " 19 -1.173840 1.103729 \n", + " 20 -0.933227 0.969163 \n", + " 21 -1.148679 1.068715 \n", + " 22 -1.060708 1.128207 \n", + " 23 -0.999670 1.021817 \n", + " 24 -1.074007 1.028155 \n", + " 25 -0.976696 1.000548 \n", + " 26 -1.137867 1.169255 \n", + " 27 -1.086920 1.133352 \n", + " 28 -1.081501 1.043582 \n", + " 29 -1.061011 1.017984 \n", + " 30 -0.976146 1.038496 \n", + " 31 -1.084374 1.058687 \n", + " 32 -1.063355 1.110603 \n", + " 33 -1.041825 1.007932 \n", + " 34 -1.194127 1.177067 \n", + " 35 -1.071807 1.063662 \n", + " 36 -1.081111 1.056036 \n", + " 37 -1.175077 1.149365 \n", + " 38 -0.981007 1.017197 \n", + " 39 -0.970758 0.921286 \n", + " 40 -1.259215 1.218623 \n", + " 41 -1.185432 1.129496 \n", + " 42 -1.089903 1.149481 \n", + " 43 -0.975454 1.026467 \n", + " 44 -1.127519 1.101622 \n", + " 45 -0.993248 0.943067 \n", + " 46 -1.103131 1.175911 \n", + " 47 -1.008887 1.085485 \n", + " 48 -1.110034 1.097652 \n", + " 49 -1.137119 1.148303 \n", + " 3 0 -1.204723 1.282823 \n", + " 1 -1.348960 1.318333 \n", + " 2 -1.089671 1.042597 \n", + " 3 -1.333177 1.289966 \n", + " 4 -1.172737 1.252828 \n", + " 5 -1.283960 1.204341 \n", + " 6 -1.074066 1.188734 \n", + " 7 -1.235234 1.176280 \n", + " 8 -1.421050 1.368052 \n", + " 9 -1.360121 1.254428 \n", + " 10 -1.306300 1.352195 \n", + " 11 -1.190463 1.161080 \n", + " 12 -1.203654 1.316097 \n", + " 13 -1.173404 1.095758 \n", + " 14 -1.147816 1.171743 \n", + " 15 -1.120560 1.159553 \n", + " 16 -1.227813 1.132744 \n", + " 17 -1.234890 1.277278 \n", + " 18 -1.142835 1.249586 \n", + " 19 -1.351964 1.307669 \n", + " 20 -1.357727 1.295594 \n", + " 21 -1.285285 1.209156 \n", + " 22 -1.167000 1.207041 \n", + " 23 -1.132733 1.153355 \n", + " 24 -1.181066 1.246684 \n", + " 25 -1.075825 1.123324 \n", + " 26 -1.385363 1.463517 \n", + " 27 -1.189060 1.263463 \n", + " 28 -1.272024 1.323552 \n", + " 29 -1.366311 1.288889 \n", + " 30 -1.334087 1.413285 \n", + " 31 -1.575940 1.612754 \n", + " 32 -1.205338 1.264695 \n", + " 33 -1.416301 1.367477 \n", + " 34 -1.263130 1.338502 \n", + " 35 -1.201269 1.291761 \n", + " 36 -1.199249 1.083916 \n", + " 37 -1.356922 1.344892 \n", + " 38 -0.891020 0.942126 \n", + " 39 -1.187956 1.223552 \n", + " 40 -1.256209 1.305335 \n", + " 41 -1.285986 1.351607 \n", + " 42 -1.524039 1.523854 \n", + " 43 -1.159181 1.128643 \n", + " 44 -1.099479 1.221816 \n", + " 45 -1.437527 1.380822 \n", + " 46 -1.157194 1.253030 \n", + " 47 -1.232943 1.129137 \n", + " 48 -1.422158 1.402718 \n", + " 49 -1.461363 1.498992 \n", + "B-K96 1 0 -1.050857 1.035540 \n", + " 1 -0.875598 0.908855 \n", + " 2 -1.020552 1.044043 \n", + " 3 -0.916223 0.938538 \n", + " 4 -0.994526 0.943069 \n", + " 5 -0.927317 0.952750 \n", + " 6 -1.052830 1.066374 \n", + " 7 -0.992329 0.970296 \n", + " 8 -0.999898 0.973126 \n", + " 9 -1.054855 1.034176 \n", + " 10 -0.981444 1.002705 \n", + " 11 -1.011183 1.034997 \n", + " 12 -0.934483 0.913232 \n", + " 13 -0.889433 0.834707 \n", + " 14 -0.927744 0.941738 \n", + " 15 -0.955251 0.934459 \n", + " 16 -0.882165 0.849501 \n", + " 17 -1.011610 1.033830 \n", + " 18 -1.028520 1.006007 \n", + " 19 -0.935727 0.902334 \n", + " 20 -0.906291 0.927878 \n", + " 21 -0.967293 0.908834 \n", + " 22 -0.905860 0.862763 \n", + " 23 -0.921374 0.905636 \n", + " 24 -0.975794 0.949279 \n", + " 25 -0.955553 0.984535 \n", + " 26 -1.031405 1.007670 \n", + " 27 -0.993490 0.961355 \n", + " 28 -0.954089 0.931932 \n", + " 29 -0.955465 0.979020 \n", + " 30 -0.834076 0.814742 \n", + " 31 -0.975156 0.992306 \n", + " 32 -1.037018 1.012538 \n", + " 33 -0.922759 0.901525 \n", + " 34 -0.980898 0.931827 \n", + " 35 -0.985529 0.961815 \n", + " 36 -0.939032 0.912356 \n", + " 37 -0.907336 0.895302 \n", + " 38 -0.981192 1.004048 \n", + " 39 -1.112073 1.092490 \n", + " 40 -0.853549 0.821142 \n", + " 41 -0.941923 0.962062 \n", + " 42 -0.879293 0.894091 \n", + " 43 -0.995571 0.989681 \n", + " 44 -0.980030 0.989806 \n", + " 45 -0.968446 0.947372 \n", + " 46 -0.985826 0.970922 \n", + " 47 -0.971530 0.926372 \n", + " 48 -0.833705 0.871394 \n", + " 49 -0.933434 0.961565 \n", + " 2 0 -1.006688 0.982892 \n", + " 1 -0.987402 0.995564 \n", + " 2 -1.014677 1.100242 \n", + " 3 -0.955748 0.951760 \n", + " 4 -1.028939 0.961956 \n", + " 5 -0.904774 0.956719 \n", + " 6 -0.978616 1.029530 \n", + " 7 -0.885356 0.900714 \n", + " 8 -1.090571 1.081573 \n", + " 9 -1.155747 1.065222 \n", + " 10 -0.993981 1.024611 \n", + " 11 -1.206621 1.236036 \n", + " 12 -1.031230 0.984579 \n", + " 13 -0.945045 1.016436 \n", + " 14 -1.007499 1.074294 \n", + " 15 -0.882242 0.931403 \n", + " 16 -0.993391 1.006453 \n", + " 17 -1.083633 1.076336 \n", + " 18 -1.175003 1.082563 \n", + " 19 -1.173840 1.103729 \n", + " 20 -0.933227 0.969163 \n", + " 21 -1.148679 1.068715 \n", + " 22 -1.060708 1.128207 \n", + " 23 -0.999670 1.021817 \n", + " 24 -1.074007 1.028155 \n", + " 25 -0.976696 1.000548 \n", + " 26 -1.137867 1.169255 \n", + " 27 -1.086920 1.133352 \n", + " 28 -1.081501 1.043582 \n", + " 29 -1.061011 1.017984 \n", + " 30 -0.976146 1.038496 \n", + " 31 -1.084374 1.058687 \n", + " 32 -1.063355 1.110603 \n", + " 33 -1.041825 1.007932 \n", + " 34 -1.194127 1.177067 \n", + " 35 -1.071807 1.063662 \n", + " 36 -1.081111 1.056036 \n", + " 37 -1.175077 1.149365 \n", + " 38 -0.981007 1.017197 \n", + " 39 -0.970758 0.921286 \n", + " 40 -1.259215 1.218623 \n", + " 41 -1.185432 1.129496 \n", + " 42 -1.089903 1.149481 \n", + " 43 -0.975454 1.026467 \n", + " 44 -1.127519 1.101622 \n", + " 45 -0.993248 0.943067 \n", + " 46 -1.103131 1.175911 \n", + " 47 -1.008887 1.085485 \n", + " 48 -1.110034 1.097652 \n", + " 49 -1.137119 1.148303 \n", + " 3 0 -1.204723 1.282823 \n", + " 1 -1.348960 1.318333 \n", + " 2 -1.089671 1.042597 \n", + " 3 -1.333177 1.289966 \n", + " 4 -1.172737 1.252828 \n", + " 5 -1.283960 1.204341 \n", + " 6 -1.074066 1.188734 \n", + " 7 -1.235234 1.176280 \n", + " 8 -1.421050 1.368052 \n", + " 9 -1.360121 1.254428 \n", + " 10 -1.306300 1.352195 \n", + " 11 -1.190463 1.161080 \n", + " 12 -1.203654 1.316097 \n", + " 13 -1.173404 1.095758 \n", + " 14 -1.147816 1.171743 \n", + " 15 -1.120560 1.159553 \n", + " 16 -1.227813 1.132744 \n", + " 17 -1.234890 1.277278 \n", + " 18 -1.142835 1.249586 \n", + " 19 -1.351964 1.307669 \n", + " 20 -1.357727 1.295594 \n", + " 21 -1.285285 1.209156 \n", + " 22 -1.167000 1.207041 \n", + " 23 -1.132733 1.153355 \n", + " 24 -1.181066 1.246684 \n", + " 25 -1.075825 1.123324 \n", + " 26 -1.385363 1.463517 \n", + " 27 -1.189060 1.263463 \n", + " 28 -1.272024 1.323552 \n", + " 29 -1.366311 1.288889 \n", + " 30 -1.334087 1.413285 \n", + " 31 -1.575940 1.612754 \n", + " 32 -1.205338 1.264695 \n", + " 33 -1.416301 1.367477 \n", + " 34 -1.263130 1.338502 \n", + " 35 -1.201269 1.291761 \n", + " 36 -1.199249 1.083916 \n", + " 37 -1.356922 1.344892 \n", + " 38 -0.891020 0.942126 \n", + " 39 -1.187956 1.223552 \n", + " 40 -1.256209 1.305335 \n", + " 41 -1.285986 1.351607 \n", + " 42 -1.524039 1.523854 \n", + " 43 -1.159181 1.128643 \n", + " 44 -1.099479 1.221816 \n", + " 45 -1.437527 1.380822 \n", + " 46 -1.157194 1.253030 \n", + " 47 -1.232943 1.129137 \n", + " 48 -1.422158 1.402718 \n", + " 49 -1.461363 1.498992 \n", + "C-K96-G32 1 0 -1.050857 1.035540 \n", + " 1 -0.875598 0.908855 \n", + " 2 -1.020552 1.044043 \n", + " 3 -0.916223 0.938538 \n", + " 4 -0.994526 0.943069 \n", + " 5 -0.927317 0.952750 \n", + " 6 -1.052830 1.066374 \n", + " 7 -0.992329 0.970296 \n", + " 8 -0.999898 0.973126 \n", + " 9 -1.054855 1.034176 \n", + " 10 -0.981444 1.002705 \n", + " 11 -1.011183 1.034997 \n", + " 12 -0.934483 0.913232 \n", + " 13 -0.889433 0.834707 \n", + " 14 -0.927744 0.941738 \n", + " 15 -0.955251 0.934459 \n", + " 16 -0.882165 0.849501 \n", + " 17 -1.011610 1.033830 \n", + " 18 -1.028520 1.006007 \n", + " 19 -0.935727 0.902334 \n", + " 20 -0.906291 0.927878 \n", + " 21 -0.967293 0.908834 \n", + " 22 -0.905860 0.862763 \n", + " 23 -0.921374 0.905636 \n", + " 24 -0.975794 0.949279 \n", + " 25 -0.955553 0.984535 \n", + " 26 -1.031405 1.007670 \n", + " 27 -0.993490 0.961355 \n", + " 28 -0.954089 0.931932 \n", + " 29 -0.955465 0.979020 \n", + " 30 -0.834076 0.814742 \n", + " 31 -0.975156 0.992306 \n", + " 32 -1.037018 1.012538 \n", + " 33 -0.922759 0.901525 \n", + " 34 -0.980898 0.931827 \n", + " 35 -0.985529 0.961815 \n", + " 36 -0.939032 0.912356 \n", + " 37 -0.907336 0.895302 \n", + " 38 -0.981192 1.004048 \n", + " 39 -1.112073 1.092490 \n", + " 40 -0.853549 0.821142 \n", + " 41 -0.941923 0.962062 \n", + " 42 -0.879293 0.894091 \n", + " 43 -0.995571 0.989681 \n", + " 44 -0.980030 0.989806 \n", + " 45 -0.968446 0.947372 \n", + " 46 -0.985826 0.970922 \n", + " 47 -0.971530 0.926372 \n", + " 48 -0.833705 0.871394 \n", + " 49 -0.933434 0.961565 \n", + " 2 0 -1.006688 0.982892 \n", + " 1 -0.987402 0.995564 \n", + " 2 -1.014677 1.100242 \n", + " 3 -0.955748 0.951760 \n", + " 4 -1.028939 0.961956 \n", + " 5 -0.904774 0.956719 \n", + " 6 -0.978616 1.029530 \n", + " 7 -0.885356 0.900714 \n", + " 8 -1.090571 1.081573 \n", + " 9 -1.155747 1.065222 \n", + " 10 -0.993981 1.024611 \n", + " 11 -1.206621 1.236036 \n", + " 12 -1.031230 0.984579 \n", + " 13 -0.945045 1.016436 \n", + " 14 -1.007499 1.074294 \n", + " 15 -0.882242 0.931403 \n", + " 16 -0.993391 1.006453 \n", + " 17 -1.083633 1.076336 \n", + " 18 -1.175003 1.082563 \n", + " 19 -1.173840 1.103729 \n", + " 20 -0.933227 0.969163 \n", + " 21 -1.148679 1.068715 \n", + " 22 -1.060708 1.128207 \n", + " 23 -0.999670 1.021817 \n", + " 24 -1.074007 1.028155 \n", + " 25 -0.976696 1.000548 \n", + " 26 -1.137867 1.169255 \n", + " 27 -1.086920 1.133352 \n", + " 28 -1.081501 1.043582 \n", + " 29 -1.061011 1.017984 \n", + " 30 -0.976146 1.038496 \n", + " 31 -1.084374 1.058687 \n", + " 32 -1.063355 1.110603 \n", + " 33 -1.041825 1.007932 \n", + " 34 -1.194127 1.177067 \n", + " 35 -1.071807 1.063662 \n", + " 36 -1.081111 1.056036 \n", + " 37 -1.175077 1.149365 \n", + " 38 -0.981007 1.017197 \n", + " 39 -0.970758 0.921286 \n", + " 40 -1.259215 1.218623 \n", + " 41 -1.185432 1.129496 \n", + " 42 -1.089903 1.149481 \n", + " 43 -0.975454 1.026467 \n", + " 44 -1.127519 1.101622 \n", + " 45 -0.993248 0.943067 \n", + " 46 -1.103131 1.175911 \n", + " 47 -1.008887 1.085485 \n", + " 48 -1.110034 1.097652 \n", + " 49 -1.137119 1.148303 \n", + " 3 0 -1.204723 1.282823 \n", + " 1 -1.348960 1.318333 \n", + " 2 -1.089671 1.042597 \n", + " 3 -1.333177 1.289966 \n", + " 4 -1.172737 1.252828 \n", + " 5 -1.283960 1.204341 \n", + " 6 -1.074066 1.188734 \n", + " 7 -1.235234 1.176280 \n", + " 8 -1.421050 1.368052 \n", + " 9 -1.360121 1.254428 \n", + " 10 -1.306300 1.352195 \n", + " 11 -1.190463 1.161080 \n", + " 12 -1.203654 1.316097 \n", + " 13 -1.173404 1.095758 \n", + " 14 -1.147816 1.171743 \n", + " 15 -1.120560 1.159553 \n", + " 16 -1.227813 1.132744 \n", + " 17 -1.234890 1.277278 \n", + " 18 -1.142835 1.249586 \n", + " 19 -1.351964 1.307669 \n", + " 20 -1.357727 1.295594 \n", + " 21 -1.285285 1.209156 \n", + " 22 -1.167000 1.207041 \n", + " 23 -1.132733 1.153355 \n", + " 24 -1.181066 1.246684 \n", + " 25 -1.075825 1.123324 \n", + " 26 -1.385363 1.463517 \n", + " 27 -1.189060 1.263463 \n", + " 28 -1.272024 1.323552 \n", + " 29 -1.366311 1.288889 \n", + " 30 -1.334087 1.413285 \n", + " 31 -1.575940 1.612754 \n", + " 32 -1.205338 1.264695 \n", + " 33 -1.416301 1.367477 \n", + " 34 -1.263130 1.338502 \n", + " 35 -1.201269 1.291761 \n", + " 36 -1.199249 1.083916 \n", + " 37 -1.356922 1.344892 \n", + " 38 -0.891020 0.942126 \n", + " 39 -1.187956 1.223552 \n", + " 40 -1.256209 1.305335 \n", + " 41 -1.285986 1.351607 \n", + " 42 -1.524039 1.523854 \n", + " 43 -1.159181 1.128643 \n", + " 44 -1.099479 1.221816 \n", + " 45 -1.437527 1.380822 \n", + " 46 -1.157194 1.253030 \n", + " 47 -1.232943 1.129137 \n", + " 48 -1.422158 1.402718 \n", + " 49 -1.461363 1.498992 \n", + "\n", + " preactivation_width crosses_zero \\\n", + "method layer neuron \n", + "A-K96 1 0 2.086397 True \n", + " 1 1.784453 True \n", + " 2 2.064595 True \n", + " 3 1.854761 True \n", + " 4 1.937594 True \n", + " 5 1.880067 True \n", + " 6 2.119204 True \n", + " 7 1.962625 True \n", + " 8 1.973024 True \n", + " 9 2.089031 True \n", + " 10 1.984150 True \n", + " 11 2.046180 True \n", + " 12 1.847716 True \n", + " 13 1.724140 True \n", + " 14 1.869481 True \n", + " 15 1.889710 True \n", + " 16 1.731667 True \n", + " 17 2.045439 True \n", + " 18 2.034527 True \n", + " 19 1.838061 True \n", + " 20 1.834169 True \n", + " 21 1.876128 True \n", + " 22 1.768623 True \n", + " 23 1.827009 True \n", + " 24 1.925073 True \n", + " 25 1.940087 True \n", + " 26 2.039075 True \n", + " 27 1.954845 True \n", + " 28 1.886022 True \n", + " 29 1.934485 True \n", + " 30 1.648818 True \n", + " 31 1.967462 True \n", + " 32 2.049556 True \n", + " 33 1.824283 True \n", + " 34 1.912724 True \n", + " 35 1.947345 True \n", + " 36 1.851388 True \n", + " 37 1.802637 True \n", + " 38 1.985241 True \n", + " 39 2.204562 True \n", + " 40 1.674692 True \n", + " 41 1.903985 True \n", + " 42 1.773384 True \n", + " 43 1.985253 True \n", + " 44 1.969837 True \n", + " 45 1.915818 True \n", + " 46 1.956749 True \n", + " 47 1.897902 True \n", + " 48 1.705099 True \n", + " 49 1.894999 True \n", + " 2 0 1.989579 True \n", + " 1 1.982965 True \n", + " 2 2.114919 True \n", + " 3 1.907508 True \n", + " 4 1.990896 True \n", + " 5 1.861493 True \n", + " 6 2.008146 True \n", + " 7 1.786069 True \n", + " 8 2.172145 True \n", + " 9 2.220969 True \n", + " 10 2.018592 True \n", + " 11 2.442657 True \n", + " 12 2.015810 True \n", + " 13 1.961481 True \n", + " 14 2.081794 True \n", + " 15 1.813645 True \n", + " 16 1.999844 True \n", + " 17 2.159969 True \n", + " 18 2.257566 True \n", + " 19 2.277569 True \n", + " 20 1.902391 True \n", + " 21 2.217394 True \n", + " 22 2.188915 True \n", + " 23 2.021487 True \n", + " 24 2.102162 True \n", + " 25 1.977244 True \n", + " 26 2.307121 True \n", + " 27 2.220272 True \n", + " 28 2.125083 True \n", + " 29 2.078995 True \n", + " 30 2.014643 True \n", + " 31 2.143061 True \n", + " 32 2.173958 True \n", + " 33 2.049757 True \n", + " 34 2.371194 True \n", + " 35 2.135469 True \n", + " 36 2.137147 True \n", + " 37 2.324442 True \n", + " 38 1.998204 True \n", + " 39 1.892044 True \n", + " 40 2.477838 True \n", + " 41 2.314928 True \n", + " 42 2.239384 True \n", + " 43 2.001921 True \n", + " 44 2.229141 True \n", + " 45 1.936316 True \n", + " 46 2.279042 True \n", + " 47 2.094371 True \n", + " 48 2.207686 True \n", + " 49 2.285421 True \n", + " 3 0 2.487546 True \n", + " 1 2.667293 True \n", + " 2 2.132268 True \n", + " 3 2.623142 True \n", + " 4 2.425565 True \n", + " 5 2.488301 True \n", + " 6 2.262800 True \n", + " 7 2.411514 True \n", + " 8 2.789102 True \n", + " 9 2.614549 True \n", + " 10 2.658495 True \n", + " 11 2.351543 True \n", + " 12 2.519751 True \n", + " 13 2.269162 True \n", + " 14 2.319559 True \n", + " 15 2.280113 True \n", + " 16 2.360557 True \n", + " 17 2.512168 True \n", + " 18 2.392422 True \n", + " 19 2.659633 True \n", + " 20 2.653322 True \n", + " 21 2.494442 True \n", + " 22 2.374041 True \n", + " 23 2.286087 True \n", + " 24 2.427750 True \n", + " 25 2.199150 True \n", + " 26 2.848880 True \n", + " 27 2.452523 True \n", + " 28 2.595575 True \n", + " 29 2.655201 True \n", + " 30 2.747372 True \n", + " 31 3.188695 True \n", + " 32 2.470033 True \n", + " 33 2.783777 True \n", + " 34 2.601631 True \n", + " 35 2.493030 True \n", + " 36 2.283166 True \n", + " 37 2.701814 True \n", + " 38 1.833146 True \n", + " 39 2.411508 True \n", + " 40 2.561544 True \n", + " 41 2.637593 True \n", + " 42 3.047893 True \n", + " 43 2.287825 True \n", + " 44 2.321295 True \n", + " 45 2.818349 True \n", + " 46 2.410224 True \n", + " 47 2.362080 True \n", + " 48 2.824875 True \n", + " 49 2.960355 True \n", + "B-K96 1 0 2.086397 True \n", + " 1 1.784453 True \n", + " 2 2.064595 True \n", + " 3 1.854761 True \n", + " 4 1.937594 True \n", + " 5 1.880067 True \n", + " 6 2.119204 True \n", + " 7 1.962625 True \n", + " 8 1.973024 True \n", + " 9 2.089031 True \n", + " 10 1.984150 True \n", + " 11 2.046180 True \n", + " 12 1.847716 True \n", + " 13 1.724140 True \n", + " 14 1.869481 True \n", + " 15 1.889710 True \n", + " 16 1.731667 True \n", + " 17 2.045439 True \n", + " 18 2.034527 True \n", + " 19 1.838061 True \n", + " 20 1.834169 True \n", + " 21 1.876128 True \n", + " 22 1.768623 True \n", + " 23 1.827009 True \n", + " 24 1.925073 True \n", + " 25 1.940087 True \n", + " 26 2.039075 True \n", + " 27 1.954845 True \n", + " 28 1.886022 True \n", + " 29 1.934485 True \n", + " 30 1.648818 True \n", + " 31 1.967462 True \n", + " 32 2.049556 True \n", + " 33 1.824283 True \n", + " 34 1.912724 True \n", + " 35 1.947345 True \n", + " 36 1.851388 True \n", + " 37 1.802637 True \n", + " 38 1.985241 True \n", + " 39 2.204562 True \n", + " 40 1.674692 True \n", + " 41 1.903985 True \n", + " 42 1.773384 True \n", + " 43 1.985253 True \n", + " 44 1.969837 True \n", + " 45 1.915818 True \n", + " 46 1.956749 True \n", + " 47 1.897902 True \n", + " 48 1.705099 True \n", + " 49 1.894999 True \n", + " 2 0 1.989579 True \n", + " 1 1.982965 True \n", + " 2 2.114919 True \n", + " 3 1.907508 True \n", + " 4 1.990896 True \n", + " 5 1.861493 True \n", + " 6 2.008146 True \n", + " 7 1.786069 True \n", + " 8 2.172145 True \n", + " 9 2.220969 True \n", + " 10 2.018592 True \n", + " 11 2.442657 True \n", + " 12 2.015810 True \n", + " 13 1.961481 True \n", + " 14 2.081794 True \n", + " 15 1.813645 True \n", + " 16 1.999844 True \n", + " 17 2.159969 True \n", + " 18 2.257566 True \n", + " 19 2.277569 True \n", + " 20 1.902391 True \n", + " 21 2.217394 True \n", + " 22 2.188915 True \n", + " 23 2.021487 True \n", + " 24 2.102162 True \n", + " 25 1.977244 True \n", + " 26 2.307121 True \n", + " 27 2.220272 True \n", + " 28 2.125083 True \n", + " 29 2.078995 True \n", + " 30 2.014643 True \n", + " 31 2.143061 True \n", + " 32 2.173958 True \n", + " 33 2.049757 True \n", + " 34 2.371194 True \n", + " 35 2.135469 True \n", + " 36 2.137147 True \n", + " 37 2.324442 True \n", + " 38 1.998204 True \n", + " 39 1.892044 True \n", + " 40 2.477838 True \n", + " 41 2.314928 True \n", + " 42 2.239384 True \n", + " 43 2.001921 True \n", + " 44 2.229141 True \n", + " 45 1.936316 True \n", + " 46 2.279042 True \n", + " 47 2.094371 True \n", + " 48 2.207686 True \n", + " 49 2.285421 True \n", + " 3 0 2.487546 True \n", + " 1 2.667293 True \n", + " 2 2.132268 True \n", + " 3 2.623142 True \n", + " 4 2.425565 True \n", + " 5 2.488301 True \n", + " 6 2.262800 True \n", + " 7 2.411514 True \n", + " 8 2.789102 True \n", + " 9 2.614549 True \n", + " 10 2.658495 True \n", + " 11 2.351543 True \n", + " 12 2.519751 True \n", + " 13 2.269162 True \n", + " 14 2.319559 True \n", + " 15 2.280113 True \n", + " 16 2.360557 True \n", + " 17 2.512168 True \n", + " 18 2.392422 True \n", + " 19 2.659633 True \n", + " 20 2.653322 True \n", + " 21 2.494442 True \n", + " 22 2.374041 True \n", + " 23 2.286087 True \n", + " 24 2.427750 True \n", + " 25 2.199150 True \n", + " 26 2.848880 True \n", + " 27 2.452523 True \n", + " 28 2.595575 True \n", + " 29 2.655201 True \n", + " 30 2.747372 True \n", + " 31 3.188695 True \n", + " 32 2.470033 True \n", + " 33 2.783777 True \n", + " 34 2.601631 True \n", + " 35 2.493030 True \n", + " 36 2.283166 True \n", + " 37 2.701814 True \n", + " 38 1.833146 True \n", + " 39 2.411508 True \n", + " 40 2.561544 True \n", + " 41 2.637593 True \n", + " 42 3.047893 True \n", + " 43 2.287825 True \n", + " 44 2.321295 True \n", + " 45 2.818349 True \n", + " 46 2.410224 True \n", + " 47 2.362080 True \n", + " 48 2.824875 True \n", + " 49 2.960355 True \n", + "C-K96-G32 1 0 2.086397 True \n", + " 1 1.784453 True \n", + " 2 2.064595 True \n", + " 3 1.854761 True \n", + " 4 1.937594 True \n", + " 5 1.880067 True \n", + " 6 2.119204 True \n", + " 7 1.962625 True \n", + " 8 1.973024 True \n", + " 9 2.089031 True \n", + " 10 1.984150 True \n", + " 11 2.046180 True \n", + " 12 1.847716 True \n", + " 13 1.724140 True \n", + " 14 1.869481 True \n", + " 15 1.889710 True \n", + " 16 1.731667 True \n", + " 17 2.045439 True \n", + " 18 2.034527 True \n", + " 19 1.838061 True \n", + " 20 1.834169 True \n", + " 21 1.876128 True \n", + " 22 1.768623 True \n", + " 23 1.827009 True \n", + " 24 1.925073 True \n", + " 25 1.940087 True \n", + " 26 2.039075 True \n", + " 27 1.954845 True \n", + " 28 1.886022 True \n", + " 29 1.934485 True \n", + " 30 1.648818 True \n", + " 31 1.967462 True \n", + " 32 2.049556 True \n", + " 33 1.824283 True \n", + " 34 1.912724 True \n", + " 35 1.947345 True \n", + " 36 1.851388 True \n", + " 37 1.802637 True \n", + " 38 1.985241 True \n", + " 39 2.204562 True \n", + " 40 1.674692 True \n", + " 41 1.903985 True \n", + " 42 1.773384 True \n", + " 43 1.985253 True \n", + " 44 1.969837 True \n", + " 45 1.915818 True \n", + " 46 1.956749 True \n", + " 47 1.897902 True \n", + " 48 1.705099 True \n", + " 49 1.894999 True \n", + " 2 0 1.989579 True \n", + " 1 1.982965 True \n", + " 2 2.114919 True \n", + " 3 1.907508 True \n", + " 4 1.990896 True \n", + " 5 1.861493 True \n", + " 6 2.008146 True \n", + " 7 1.786069 True \n", + " 8 2.172145 True \n", + " 9 2.220969 True \n", + " 10 2.018592 True \n", + " 11 2.442657 True \n", + " 12 2.015810 True \n", + " 13 1.961481 True \n", + " 14 2.081794 True \n", + " 15 1.813645 True \n", + " 16 1.999844 True \n", + " 17 2.159969 True \n", + " 18 2.257566 True \n", + " 19 2.277569 True \n", + " 20 1.902391 True \n", + " 21 2.217394 True \n", + " 22 2.188915 True \n", + " 23 2.021487 True \n", + " 24 2.102162 True \n", + " 25 1.977244 True \n", + " 26 2.307121 True \n", + " 27 2.220272 True \n", + " 28 2.125083 True \n", + " 29 2.078995 True \n", + " 30 2.014643 True \n", + " 31 2.143061 True \n", + " 32 2.173958 True \n", + " 33 2.049757 True \n", + " 34 2.371194 True \n", + " 35 2.135469 True \n", + " 36 2.137147 True \n", + " 37 2.324442 True \n", + " 38 1.998204 True \n", + " 39 1.892044 True \n", + " 40 2.477838 True \n", + " 41 2.314928 True \n", + " 42 2.239384 True \n", + " 43 2.001921 True \n", + " 44 2.229141 True \n", + " 45 1.936316 True \n", + " 46 2.279042 True \n", + " 47 2.094371 True \n", + " 48 2.207686 True \n", + " 49 2.285421 True \n", + " 3 0 2.487546 True \n", + " 1 2.667293 True \n", + " 2 2.132268 True \n", + " 3 2.623142 True \n", + " 4 2.425565 True \n", + " 5 2.488301 True \n", + " 6 2.262800 True \n", + " 7 2.411514 True \n", + " 8 2.789102 True \n", + " 9 2.614549 True \n", + " 10 2.658495 True \n", + " 11 2.351543 True \n", + " 12 2.519751 True \n", + " 13 2.269162 True \n", + " 14 2.319559 True \n", + " 15 2.280113 True \n", + " 16 2.360557 True \n", + " 17 2.512168 True \n", + " 18 2.392422 True \n", + " 19 2.659633 True \n", + " 20 2.653322 True \n", + " 21 2.494442 True \n", + " 22 2.374041 True \n", + " 23 2.286087 True \n", + " 24 2.427750 True \n", + " 25 2.199150 True \n", + " 26 2.848880 True \n", + " 27 2.452523 True \n", + " 28 2.595575 True \n", + " 29 2.655201 True \n", + " 30 2.747372 True \n", + " 31 3.188695 True \n", + " 32 2.470033 True \n", + " 33 2.783777 True \n", + " 34 2.601631 True \n", + " 35 2.493030 True \n", + " 36 2.283166 True \n", + " 37 2.701814 True \n", + " 38 1.833146 True \n", + " 39 2.411508 True \n", + " 40 2.561544 True \n", + " 41 2.637593 True \n", + " 42 3.047893 True \n", + " 43 2.287825 True \n", + " 44 2.321295 True \n", + " 45 2.818349 True \n", + " 46 2.410224 True \n", + " 47 2.362080 True \n", + " 48 2.824875 True \n", + " 49 2.960355 True \n", + "\n", + " relative_affine_slope tanh_affine_rho \\\n", + "method layer neuron \n", + "A-K96 1 0 0.015331 0.089748 \n", + " 1 0.044933 0.062994 \n", + " 2 0.023943 0.087721 \n", + " 3 0.028234 0.068898 \n", + " 4 0.058788 0.076235 \n", + " 5 0.031289 0.071089 \n", + " 6 0.013119 0.092842 \n", + " 7 0.024948 0.078379 \n", + " 8 0.029912 0.079325 \n", + " 9 0.020587 0.090003 \n", + " 10 0.023558 0.080320 \n", + " 11 0.024729 0.086010 \n", + " 12 0.027103 0.068292 \n", + " 13 0.077520 0.058166 \n", + " 14 0.017529 0.070152 \n", + " 15 0.025397 0.071919 \n", + " 16 0.046634 0.058677 \n", + " 17 0.023111 0.085939 \n", + " 18 0.023674 0.084930 \n", + " 19 0.042686 0.067499 \n", + " 20 0.027916 0.067137 \n", + " 21 0.070781 0.070874 \n", + " 22 0.058794 0.061723 \n", + " 23 0.020582 0.066520 \n", + " 24 0.031123 0.075038 \n", + " 25 0.033451 0.076378 \n", + " 26 0.024828 0.085353 \n", + " 27 0.036469 0.077704 \n", + " 28 0.027145 0.071600 \n", + " 29 0.027431 0.075867 \n", + " 30 0.030412 0.052128 \n", + " 31 0.019376 0.078806 \n", + " 32 0.025324 0.086325 \n", + " 33 0.027749 0.066299 \n", + " 34 0.057571 0.074024 \n", + " 35 0.027250 0.077013 \n", + " 36 0.033778 0.068618 \n", + " 37 0.016183 0.064463 \n", + " 38 0.025274 0.080422 \n", + " 39 0.017342 0.101042 \n", + " 40 0.049108 0.054161 \n", + " 41 0.024248 0.073166 \n", + " 42 0.020481 0.062036 \n", + " 43 0.006574 0.080402 \n", + " 44 0.011063 0.079011 \n", + " 45 0.025053 0.074209 \n", + " 46 0.017046 0.077840 \n", + " 47 0.053906 0.072707 \n", + " 48 0.055115 0.056570 \n", + " 49 0.034024 0.072397 \n", + " 2 0 0.026184 0.080818 \n", + " 1 0.009121 0.080196 \n", + " 2 0.080492 0.092721 \n", + " 3 0.004832 0.073458 \n", + " 4 0.071922 0.081105 \n", + " 5 0.064084 0.069578 \n", + " 6 0.054184 0.082595 \n", + " 7 0.020968 0.063086 \n", + " 8 0.008275 0.097897 \n", + " 9 0.076546 0.102929 \n", + " 10 0.032599 0.083482 \n", + " 11 0.020404 0.124758 \n", + " 12 0.049375 0.083279 \n", + " 13 0.078757 0.078477 \n", + " 14 0.065504 0.089488 \n", + " 15 0.063787 0.065493 \n", + " 16 0.014305 0.081735 \n", + " 17 0.006800 0.096726 \n", + " 18 0.075345 0.106517 \n", + " 19 0.056513 0.108352 \n", + " 20 0.042935 0.073066 \n", + " 21 0.068146 0.102514 \n", + " 22 0.059477 0.099685 \n", + " 23 0.023606 0.083729 \n", + " 24 0.044499 0.091307 \n", + " 25 0.026578 0.079700 \n", + " 26 0.024935 0.111137 \n", + " 27 0.040007 0.102636 \n", + " 28 0.036094 0.093450 \n", + " 29 0.042802 0.089120 \n", + " 30 0.065533 0.083245 \n", + " 31 0.024146 0.095131 \n", + " 32 0.042632 0.098154 \n", + " 33 0.034887 0.086366 \n", + " 34 0.012774 0.117515 \n", + " 35 0.007780 0.094383 \n", + " 36 0.023721 0.094566 \n", + " 37 0.020117 0.112849 \n", + " 38 0.039207 0.081634 \n", + " 39 0.059258 0.072213 \n", + " 40 0.027081 0.128366 \n", + " 41 0.043677 0.111988 \n", + " 42 0.050090 0.104553 \n", + " 43 0.054628 0.082028 \n", + " 44 0.022308 0.103445 \n", + " 45 0.057446 0.076116 \n", + " 46 0.058519 0.108511 \n", + " 47 0.073841 0.090726 \n", + " 48 0.010964 0.101336 \n", + " 49 0.009155 0.108956 \n", + " 3 0 0.051000 0.129496 \n", + " 1 0.016954 0.147882 \n", + " 2 0.044297 0.094163 \n", + " 3 0.024914 0.143326 \n", + " 4 0.055567 0.123203 \n", + " 5 0.051929 0.129580 \n", + " 6 0.092201 0.107198 \n", + " 7 0.041768 0.121686 \n", + " 8 0.025836 0.160625 \n", + " 9 0.060445 0.142696 \n", + " 10 0.025526 0.146999 \n", + " 11 0.022339 0.115563 \n", + " 12 0.070426 0.132988 \n", + " 13 0.062926 0.107564 \n", + " 14 0.018826 0.112359 \n", + " 15 0.031734 0.108481 \n", + " 16 0.069965 0.116744 \n", + " 17 0.027309 0.131883 \n", + " 18 0.075850 0.120021 \n", + " 19 0.024618 0.147114 \n", + " 20 0.034587 0.146510 \n", + " 21 0.049405 0.130191 \n", + " 22 0.029645 0.117846 \n", + " 23 0.016805 0.109033 \n", + " 24 0.045643 0.123356 \n", + " 25 0.041784 0.100588 \n", + " 26 0.035713 0.166947 \n", + " 27 0.050355 0.125910 \n", + " 28 0.030466 0.140495 \n", + " 29 0.042849 0.146763 \n", + " 30 0.040000 0.156356 \n", + " 31 0.012091 0.202250 \n", + " 32 0.039671 0.127629 \n", + " 33 0.023951 0.160058 \n", + " 34 0.044006 0.141207 \n", + " 35 0.058555 0.130120 \n", + " 36 0.090893 0.109205 \n", + " 37 0.006464 0.151453 \n", + " 38 0.064926 0.067150 \n", + " 39 0.025422 0.121613 \n", + " 40 0.030068 0.136977 \n", + " 41 0.037070 0.144891 \n", + " 42 0.000070 0.187602 \n", + " 43 0.024745 0.109223 \n", + " 44 0.092672 0.113034 \n", + " 45 0.026830 0.163690 \n", + " 46 0.067156 0.121742 \n", + " 47 0.076053 0.116955 \n", + " 48 0.009215 0.164305 \n", + " 49 0.015517 0.178494 \n", + "B-K96 1 0 0.015331 0.089748 \n", + " 1 0.044933 0.062994 \n", + " 2 0.023943 0.087721 \n", + " 3 0.028234 0.068898 \n", + " 4 0.058788 0.076235 \n", + " 5 0.031289 0.071089 \n", + " 6 0.013119 0.092842 \n", + " 7 0.024948 0.078379 \n", + " 8 0.029912 0.079325 \n", + " 9 0.020587 0.090003 \n", + " 10 0.023558 0.080320 \n", + " 11 0.024729 0.086010 \n", + " 12 0.027103 0.068292 \n", + " 13 0.077520 0.058166 \n", + " 14 0.017529 0.070152 \n", + " 15 0.025397 0.071919 \n", + " 16 0.046634 0.058677 \n", + " 17 0.023111 0.085939 \n", + " 18 0.023674 0.084930 \n", + " 19 0.042686 0.067499 \n", + " 20 0.027916 0.067137 \n", + " 21 0.070781 0.070874 \n", + " 22 0.058794 0.061723 \n", + " 23 0.020582 0.066520 \n", + " 24 0.031123 0.075038 \n", + " 25 0.033451 0.076378 \n", + " 26 0.024828 0.085353 \n", + " 27 0.036469 0.077704 \n", + " 28 0.027145 0.071600 \n", + " 29 0.027431 0.075867 \n", + " 30 0.030412 0.052128 \n", + " 31 0.019376 0.078806 \n", + " 32 0.025324 0.086325 \n", + " 33 0.027749 0.066299 \n", + " 34 0.057571 0.074024 \n", + " 35 0.027250 0.077013 \n", + " 36 0.033778 0.068618 \n", + " 37 0.016183 0.064463 \n", + " 38 0.025274 0.080422 \n", + " 39 0.017342 0.101042 \n", + " 40 0.049108 0.054161 \n", + " 41 0.024248 0.073166 \n", + " 42 0.020481 0.062036 \n", + " 43 0.006574 0.080402 \n", + " 44 0.011063 0.079011 \n", + " 45 0.025053 0.074209 \n", + " 46 0.017046 0.077840 \n", + " 47 0.053906 0.072707 \n", + " 48 0.055115 0.056570 \n", + " 49 0.034024 0.072397 \n", + " 2 0 0.026184 0.080818 \n", + " 1 0.009121 0.080196 \n", + " 2 0.080492 0.092721 \n", + " 3 0.004832 0.073458 \n", + " 4 0.071922 0.081105 \n", + " 5 0.064084 0.069578 \n", + " 6 0.054184 0.082595 \n", + " 7 0.020968 0.063086 \n", + " 8 0.008275 0.097897 \n", + " 9 0.076546 0.102929 \n", + " 10 0.032599 0.083482 \n", + " 11 0.020404 0.124758 \n", + " 12 0.049375 0.083279 \n", + " 13 0.078757 0.078477 \n", + " 14 0.065504 0.089488 \n", + " 15 0.063787 0.065493 \n", + " 16 0.014305 0.081735 \n", + " 17 0.006800 0.096726 \n", + " 18 0.075345 0.106517 \n", + " 19 0.056513 0.108352 \n", + " 20 0.042935 0.073066 \n", + " 21 0.068146 0.102514 \n", + " 22 0.059477 0.099685 \n", + " 23 0.023606 0.083729 \n", + " 24 0.044499 0.091307 \n", + " 25 0.026578 0.079700 \n", + " 26 0.024935 0.111137 \n", + " 27 0.040007 0.102636 \n", + " 28 0.036094 0.093450 \n", + " 29 0.042802 0.089120 \n", + " 30 0.065533 0.083245 \n", + " 31 0.024146 0.095131 \n", + " 32 0.042632 0.098154 \n", + " 33 0.034887 0.086366 \n", + " 34 0.012774 0.117515 \n", + " 35 0.007780 0.094383 \n", + " 36 0.023721 0.094566 \n", + " 37 0.020117 0.112849 \n", + " 38 0.039207 0.081634 \n", + " 39 0.059258 0.072213 \n", + " 40 0.027081 0.128366 \n", + " 41 0.043677 0.111988 \n", + " 42 0.050090 0.104553 \n", + " 43 0.054628 0.082028 \n", + " 44 0.022308 0.103445 \n", + " 45 0.057446 0.076116 \n", + " 46 0.058519 0.108511 \n", + " 47 0.073841 0.090726 \n", + " 48 0.010964 0.101336 \n", + " 49 0.009155 0.108956 \n", + " 3 0 0.051000 0.129496 \n", + " 1 0.016954 0.147882 \n", + " 2 0.044297 0.094163 \n", + " 3 0.024914 0.143326 \n", + " 4 0.055567 0.123203 \n", + " 5 0.051929 0.129580 \n", + " 6 0.092201 0.107198 \n", + " 7 0.041768 0.121686 \n", + " 8 0.025836 0.160625 \n", + " 9 0.060445 0.142696 \n", + " 10 0.025526 0.146999 \n", + " 11 0.022339 0.115563 \n", + " 12 0.070426 0.132988 \n", + " 13 0.062926 0.107564 \n", + " 14 0.018826 0.112359 \n", + " 15 0.031734 0.108481 \n", + " 16 0.069965 0.116744 \n", + " 17 0.027309 0.131883 \n", + " 18 0.075850 0.120021 \n", + " 19 0.024618 0.147114 \n", + " 20 0.034587 0.146510 \n", + " 21 0.049405 0.130191 \n", + " 22 0.029645 0.117846 \n", + " 23 0.016805 0.109033 \n", + " 24 0.045643 0.123356 \n", + " 25 0.041784 0.100588 \n", + " 26 0.035713 0.166947 \n", + " 27 0.050355 0.125910 \n", + " 28 0.030466 0.140495 \n", + " 29 0.042849 0.146763 \n", + " 30 0.040000 0.156356 \n", + " 31 0.012091 0.202250 \n", + " 32 0.039671 0.127629 \n", + " 33 0.023951 0.160058 \n", + " 34 0.044006 0.141207 \n", + " 35 0.058555 0.130120 \n", + " 36 0.090893 0.109205 \n", + " 37 0.006464 0.151453 \n", + " 38 0.064926 0.067150 \n", + " 39 0.025422 0.121613 \n", + " 40 0.030068 0.136977 \n", + " 41 0.037070 0.144891 \n", + " 42 0.000070 0.187602 \n", + " 43 0.024745 0.109223 \n", + " 44 0.092672 0.113034 \n", + " 45 0.026830 0.163690 \n", + " 46 0.067156 0.121742 \n", + " 47 0.076053 0.116955 \n", + " 48 0.009215 0.164305 \n", + " 49 0.015517 0.178494 \n", + "C-K96-G32 1 0 0.015331 0.089748 \n", + " 1 0.044933 0.062994 \n", + " 2 0.023943 0.087721 \n", + " 3 0.028234 0.068898 \n", + " 4 0.058788 0.076235 \n", + " 5 0.031289 0.071089 \n", + " 6 0.013119 0.092842 \n", + " 7 0.024948 0.078379 \n", + " 8 0.029912 0.079325 \n", + " 9 0.020587 0.090003 \n", + " 10 0.023558 0.080320 \n", + " 11 0.024729 0.086010 \n", + " 12 0.027103 0.068292 \n", + " 13 0.077520 0.058166 \n", + " 14 0.017529 0.070152 \n", + " 15 0.025397 0.071919 \n", + " 16 0.046634 0.058677 \n", + " 17 0.023111 0.085939 \n", + " 18 0.023674 0.084930 \n", + " 19 0.042686 0.067499 \n", + " 20 0.027916 0.067137 \n", + " 21 0.070781 0.070874 \n", + " 22 0.058794 0.061723 \n", + " 23 0.020582 0.066520 \n", + " 24 0.031123 0.075038 \n", + " 25 0.033451 0.076378 \n", + " 26 0.024828 0.085353 \n", + " 27 0.036469 0.077704 \n", + " 28 0.027145 0.071600 \n", + " 29 0.027431 0.075867 \n", + " 30 0.030412 0.052128 \n", + " 31 0.019376 0.078806 \n", + " 32 0.025324 0.086325 \n", + " 33 0.027749 0.066299 \n", + " 34 0.057571 0.074024 \n", + " 35 0.027250 0.077013 \n", + " 36 0.033778 0.068618 \n", + " 37 0.016183 0.064463 \n", + " 38 0.025274 0.080422 \n", + " 39 0.017342 0.101042 \n", + " 40 0.049108 0.054161 \n", + " 41 0.024248 0.073166 \n", + " 42 0.020481 0.062036 \n", + " 43 0.006574 0.080402 \n", + " 44 0.011063 0.079011 \n", + " 45 0.025053 0.074209 \n", + " 46 0.017046 0.077840 \n", + " 47 0.053906 0.072707 \n", + " 48 0.055115 0.056570 \n", + " 49 0.034024 0.072397 \n", + " 2 0 0.026184 0.080818 \n", + " 1 0.009121 0.080196 \n", + " 2 0.080492 0.092721 \n", + " 3 0.004832 0.073458 \n", + " 4 0.071922 0.081105 \n", + " 5 0.064084 0.069578 \n", + " 6 0.054184 0.082595 \n", + " 7 0.020968 0.063086 \n", + " 8 0.008275 0.097897 \n", + " 9 0.076546 0.102929 \n", + " 10 0.032599 0.083482 \n", + " 11 0.020404 0.124758 \n", + " 12 0.049375 0.083279 \n", + " 13 0.078757 0.078477 \n", + " 14 0.065504 0.089488 \n", + " 15 0.063787 0.065493 \n", + " 16 0.014305 0.081735 \n", + " 17 0.006800 0.096726 \n", + " 18 0.075345 0.106517 \n", + " 19 0.056513 0.108352 \n", + " 20 0.042935 0.073066 \n", + " 21 0.068146 0.102514 \n", + " 22 0.059477 0.099685 \n", + " 23 0.023606 0.083729 \n", + " 24 0.044499 0.091307 \n", + " 25 0.026578 0.079700 \n", + " 26 0.024935 0.111137 \n", + " 27 0.040007 0.102636 \n", + " 28 0.036094 0.093450 \n", + " 29 0.042802 0.089120 \n", + " 30 0.065533 0.083245 \n", + " 31 0.024146 0.095131 \n", + " 32 0.042632 0.098154 \n", + " 33 0.034887 0.086366 \n", + " 34 0.012774 0.117515 \n", + " 35 0.007780 0.094383 \n", + " 36 0.023721 0.094566 \n", + " 37 0.020117 0.112849 \n", + " 38 0.039207 0.081634 \n", + " 39 0.059258 0.072213 \n", + " 40 0.027081 0.128366 \n", + " 41 0.043677 0.111988 \n", + " 42 0.050090 0.104553 \n", + " 43 0.054628 0.082028 \n", + " 44 0.022308 0.103445 \n", + " 45 0.057446 0.076116 \n", + " 46 0.058519 0.108511 \n", + " 47 0.073841 0.090726 \n", + " 48 0.010964 0.101336 \n", + " 49 0.009155 0.108956 \n", + " 3 0 0.051000 0.129496 \n", + " 1 0.016954 0.147882 \n", + " 2 0.044297 0.094163 \n", + " 3 0.024914 0.143326 \n", + " 4 0.055567 0.123203 \n", + " 5 0.051929 0.129580 \n", + " 6 0.092201 0.107198 \n", + " 7 0.041768 0.121686 \n", + " 8 0.025836 0.160625 \n", + " 9 0.060445 0.142696 \n", + " 10 0.025526 0.146999 \n", + " 11 0.022339 0.115563 \n", + " 12 0.070426 0.132988 \n", + " 13 0.062926 0.107564 \n", + " 14 0.018826 0.112359 \n", + " 15 0.031734 0.108481 \n", + " 16 0.069965 0.116744 \n", + " 17 0.027309 0.131883 \n", + " 18 0.075850 0.120021 \n", + " 19 0.024618 0.147114 \n", + " 20 0.034587 0.146510 \n", + " 21 0.049405 0.130191 \n", + " 22 0.029645 0.117846 \n", + " 23 0.016805 0.109033 \n", + " 24 0.045643 0.123356 \n", + " 25 0.041784 0.100588 \n", + " 26 0.035713 0.166947 \n", + " 27 0.050355 0.125910 \n", + " 28 0.030466 0.140495 \n", + " 29 0.042849 0.146763 \n", + " 30 0.040000 0.156356 \n", + " 31 0.012091 0.202250 \n", + " 32 0.039671 0.127629 \n", + " 33 0.023951 0.160058 \n", + " 34 0.044006 0.141207 \n", + " 35 0.058555 0.130120 \n", + " 36 0.090893 0.109205 \n", + " 37 0.006464 0.151453 \n", + " 38 0.064926 0.067150 \n", + " 39 0.025422 0.121613 \n", + " 40 0.030068 0.136977 \n", + " 41 0.037070 0.144891 \n", + " 42 0.000070 0.187602 \n", + " 43 0.024745 0.109223 \n", + " 44 0.092672 0.113034 \n", + " 45 0.026830 0.163690 \n", + " 46 0.067156 0.121742 \n", + " 47 0.076053 0.116955 \n", + " 48 0.009215 0.164305 \n", + " 49 0.015517 0.178494 \n", + "\n", + " tanh_prime_interval_radius tanh_prime_affine_rho \\\n", + "method layer neuron \n", + "A-K96 1 0 0.305871 0.303512 \n", + " 1 0.259619 0.253699 \n", + " 2 0.303794 0.300122 \n", + " 3 0.269782 0.265936 \n", + " 4 0.288257 0.279597 \n", + " 5 0.274560 0.270216 \n", + " 6 0.310544 0.308496 \n", + " 7 0.287550 0.283930 \n", + " 8 0.289980 0.285594 \n", + " 9 0.307083 0.303895 \n", + " 10 0.290877 0.287420 \n", + " 11 0.301014 0.297255 \n", + " 12 0.268408 0.264737 \n", + " 13 0.252841 0.242795 \n", + " 14 0.270862 0.268474 \n", + " 15 0.275394 0.271865 \n", + " 16 0.250280 0.244357 \n", + " 17 0.300653 0.297147 \n", + " 18 0.299007 0.295435 \n", + " 19 0.268830 0.263008 \n", + " 20 0.266160 0.262410 \n", + " 21 0.279387 0.269247 \n", + " 22 0.258581 0.250831 \n", + " 23 0.263935 0.261200 \n", + " 24 0.282180 0.277739 \n", + " 25 0.285029 0.280203 \n", + " 26 0.299903 0.296143 \n", + " 27 0.287924 0.282602 \n", + " 28 0.275007 0.271238 \n", + " 29 0.283235 0.279311 \n", + " 30 0.233011 0.229436 \n", + " 31 0.287543 0.284737 \n", + " 32 0.301637 0.297779 \n", + " 33 0.264410 0.260707 \n", + " 34 0.283847 0.275503 \n", + " 35 0.285352 0.281425 \n", + " 36 0.269949 0.265336 \n", + " 37 0.259093 0.256985 \n", + " 38 0.291305 0.287588 \n", + " 39 0.323847 0.321017 \n", + " 40 0.240069 0.234085 \n", + " 41 0.277658 0.274262 \n", + " 42 0.254476 0.251852 \n", + " 43 0.288593 0.287642 \n", + " 44 0.286736 0.285144 \n", + " 45 0.279767 0.276231 \n", + " 46 0.285448 0.283000 \n", + " 47 0.280782 0.273066 \n", + " 48 0.246460 0.239549 \n", + " 49 0.277493 0.272715 \n", + " 2 0 0.292145 0.288282 \n", + " 1 0.288591 0.287270 \n", + " 2 0.320469 0.307124 \n", + " 3 0.275560 0.274893 \n", + " 4 0.299138 0.288077 \n", + " 5 0.275883 0.266842 \n", + " 6 0.299321 0.291039 \n", + " 7 0.256790 0.254079 \n", + " 8 0.317673 0.316354 \n", + " 9 0.335912 0.322599 \n", + " 10 0.297790 0.292874 \n", + " 11 0.356439 0.352763 \n", + " 12 0.299849 0.292302 \n", + " 13 0.295228 0.283249 \n", + " 14 0.312899 0.302370 \n", + " 15 0.267361 0.258647 \n", + " 16 0.292070 0.289970 \n", + " 17 0.315649 0.314573 \n", + " 18 0.341030 0.327721 \n", + " 19 0.340725 0.330836 \n", + " 20 0.280004 0.273899 \n", + " 21 0.334003 0.322265 \n", + " 22 0.328378 0.318351 \n", + " 23 0.296916 0.293379 \n", + " 24 0.312814 0.305724 \n", + " 25 0.290188 0.286293 \n", + " 26 0.339515 0.335231 \n", + " 27 0.329805 0.323087 \n", + " 28 0.315024 0.309251 \n", + " 29 0.308938 0.302209 \n", + " 30 0.302093 0.291936 \n", + " 31 0.315866 0.312013 \n", + " 32 0.323429 0.316405 \n", + " 33 0.303115 0.297753 \n", + " 34 0.345991 0.343767 \n", + " 35 0.312161 0.310943 \n", + " 36 0.314909 0.311136 \n", + " 37 0.341050 0.337586 \n", + " 38 0.295467 0.289587 \n", + " 39 0.280528 0.272038 \n", + " 40 0.361971 0.356997 \n", + " 41 0.343750 0.336083 \n", + " 42 0.334220 0.325655 \n", + " 43 0.298368 0.290044 \n", + " 44 0.328186 0.324489 \n", + " 45 0.287846 0.279401 \n", + " 46 0.341268 0.331003 \n", + " 47 0.316191 0.304152 \n", + " 48 0.323268 0.321487 \n", + " 49 0.333901 0.332366 \n", + " 3 0 0.367426 0.357791 \n", + " 1 0.381770 0.378499 \n", + " 2 0.317412 0.310248 \n", + " 3 0.378471 0.373685 \n", + " 4 0.360464 0.350153 \n", + " 5 0.367684 0.357862 \n", + " 6 0.344604 0.328011 \n", + " 7 0.356245 0.348642 \n", + " 8 0.395888 0.390683 \n", + " 9 0.384057 0.372020 \n", + " 10 0.382437 0.377478 \n", + " 11 0.345050 0.341153 \n", + " 12 0.374814 0.361081 \n", + " 13 0.340610 0.329571 \n", + " 14 0.340172 0.336941 \n", + " 15 0.336933 0.331507 \n", + " 16 0.354435 0.341591 \n", + " 17 0.366161 0.361085 \n", + " 18 0.359694 0.345509 \n", + " 19 0.382389 0.377610 \n", + " 20 0.383570 0.376793 \n", + " 21 0.367985 0.358644 \n", + " 22 0.349272 0.344017 \n", + " 23 0.335268 0.332428 \n", + " 24 0.359002 0.350610 \n", + " 25 0.327016 0.320055 \n", + " 26 0.403501 0.396112 \n", + " 27 0.362966 0.353578 \n", + " 28 0.376421 0.370583 \n", + " 29 0.385309 0.376833 \n", + " 30 0.394441 0.386341 \n", + " 31 0.426487 0.423880 \n", + " 32 0.363253 0.355892 \n", + " 33 0.395005 0.390197 \n", + " 34 0.379592 0.371019 \n", + " 35 0.369445 0.358274 \n", + " 36 0.347299 0.330814 \n", + " 37 0.383405 0.382161 \n", + " 38 0.270993 0.261997 \n", + " 39 0.353387 0.348836 \n", + " 40 0.372462 0.366765 \n", + " 41 0.382316 0.375068 \n", + " 42 0.413503 0.413489 \n", + " 43 0.336834 0.332618 \n", + " 44 0.352959 0.335841 \n", + " 45 0.398902 0.393450 \n", + " 46 0.360512 0.347976 \n", + " 47 0.355688 0.341633 \n", + " 48 0.396093 0.394256 \n", + " 49 0.409482 0.406266 \n", + "B-K96 1 0 0.305871 0.303512 \n", + " 1 0.259619 0.253699 \n", + " 2 0.303794 0.300122 \n", + " 3 0.269782 0.265936 \n", + " 4 0.288257 0.279597 \n", + " 5 0.274560 0.270216 \n", + " 6 0.310544 0.308496 \n", + " 7 0.287550 0.283930 \n", + " 8 0.289980 0.285594 \n", + " 9 0.307083 0.303895 \n", + " 10 0.290877 0.287420 \n", + " 11 0.301014 0.297255 \n", + " 12 0.268408 0.264737 \n", + " 13 0.252841 0.242795 \n", + " 14 0.270862 0.268474 \n", + " 15 0.275394 0.271865 \n", + " 16 0.250280 0.244357 \n", + " 17 0.300653 0.297147 \n", + " 18 0.299007 0.295435 \n", + " 19 0.268830 0.263008 \n", + " 20 0.266160 0.262410 \n", + " 21 0.279387 0.269247 \n", + " 22 0.258581 0.250831 \n", + " 23 0.263935 0.261200 \n", + " 24 0.282180 0.277739 \n", + " 25 0.285029 0.280203 \n", + " 26 0.299903 0.296143 \n", + " 27 0.287924 0.282602 \n", + " 28 0.275007 0.271238 \n", + " 29 0.283235 0.279311 \n", + " 30 0.233011 0.229436 \n", + " 31 0.287543 0.284737 \n", + " 32 0.301637 0.297779 \n", + " 33 0.264410 0.260707 \n", + " 34 0.283847 0.275503 \n", + " 35 0.285352 0.281425 \n", + " 36 0.269949 0.265336 \n", + " 37 0.259093 0.256985 \n", + " 38 0.291305 0.287588 \n", + " 39 0.323847 0.321017 \n", + " 40 0.240069 0.234085 \n", + " 41 0.277658 0.274262 \n", + " 42 0.254476 0.251852 \n", + " 43 0.288593 0.287642 \n", + " 44 0.286736 0.285144 \n", + " 45 0.279767 0.276231 \n", + " 46 0.285448 0.283000 \n", + " 47 0.280782 0.273066 \n", + " 48 0.246460 0.239549 \n", + " 49 0.277493 0.272715 \n", + " 2 0 0.292145 0.288282 \n", + " 1 0.288591 0.287270 \n", + " 2 0.320469 0.307124 \n", + " 3 0.275560 0.274893 \n", + " 4 0.299138 0.288077 \n", + " 5 0.275883 0.266842 \n", + " 6 0.299321 0.291039 \n", + " 7 0.256790 0.254079 \n", + " 8 0.317673 0.316354 \n", + " 9 0.335912 0.322599 \n", + " 10 0.297790 0.292874 \n", + " 11 0.356439 0.352763 \n", + " 12 0.299849 0.292302 \n", + " 13 0.295228 0.283249 \n", + " 14 0.312899 0.302370 \n", + " 15 0.267361 0.258647 \n", + " 16 0.292070 0.289970 \n", + " 17 0.315649 0.314573 \n", + " 18 0.341030 0.327721 \n", + " 19 0.340725 0.330836 \n", + " 20 0.280004 0.273899 \n", + " 21 0.334003 0.322265 \n", + " 22 0.328378 0.318351 \n", + " 23 0.296916 0.293379 \n", + " 24 0.312814 0.305724 \n", + " 25 0.290188 0.286293 \n", + " 26 0.339515 0.335231 \n", + " 27 0.329805 0.323087 \n", + " 28 0.315024 0.309251 \n", + " 29 0.308938 0.302209 \n", + " 30 0.302093 0.291936 \n", + " 31 0.315866 0.312013 \n", + " 32 0.323429 0.316405 \n", + " 33 0.303115 0.297753 \n", + " 34 0.345991 0.343767 \n", + " 35 0.312161 0.310943 \n", + " 36 0.314909 0.311136 \n", + " 37 0.341050 0.337586 \n", + " 38 0.295467 0.289587 \n", + " 39 0.280528 0.272038 \n", + " 40 0.361971 0.356997 \n", + " 41 0.343750 0.336083 \n", + " 42 0.334220 0.325655 \n", + " 43 0.298368 0.290044 \n", + " 44 0.328186 0.324489 \n", + " 45 0.287846 0.279401 \n", + " 46 0.341268 0.331003 \n", + " 47 0.316191 0.304152 \n", + " 48 0.323268 0.321487 \n", + " 49 0.333901 0.332366 \n", + " 3 0 0.367426 0.357791 \n", + " 1 0.381770 0.378499 \n", + " 2 0.317412 0.310248 \n", + " 3 0.378471 0.373685 \n", + " 4 0.360464 0.350153 \n", + " 5 0.367684 0.357862 \n", + " 6 0.344604 0.328011 \n", + " 7 0.356245 0.348642 \n", + " 8 0.395888 0.390683 \n", + " 9 0.384057 0.372020 \n", + " 10 0.382437 0.377478 \n", + " 11 0.345050 0.341153 \n", + " 12 0.374814 0.361081 \n", + " 13 0.340610 0.329571 \n", + " 14 0.340172 0.336941 \n", + " 15 0.336933 0.331507 \n", + " 16 0.354435 0.341591 \n", + " 17 0.366161 0.361085 \n", + " 18 0.359694 0.345509 \n", + " 19 0.382389 0.377610 \n", + " 20 0.383570 0.376793 \n", + " 21 0.367985 0.358644 \n", + " 22 0.349272 0.344017 \n", + " 23 0.335268 0.332428 \n", + " 24 0.359002 0.350610 \n", + " 25 0.327016 0.320055 \n", + " 26 0.403501 0.396112 \n", + " 27 0.362966 0.353578 \n", + " 28 0.376421 0.370583 \n", + " 29 0.385309 0.376833 \n", + " 30 0.394441 0.386341 \n", + " 31 0.426487 0.423880 \n", + " 32 0.363253 0.355892 \n", + " 33 0.395005 0.390197 \n", + " 34 0.379592 0.371019 \n", + " 35 0.369445 0.358274 \n", + " 36 0.347299 0.330814 \n", + " 37 0.383405 0.382161 \n", + " 38 0.270993 0.261997 \n", + " 39 0.353387 0.348836 \n", + " 40 0.372462 0.366765 \n", + " 41 0.382316 0.375068 \n", + " 42 0.413503 0.413489 \n", + " 43 0.336834 0.332618 \n", + " 44 0.352959 0.335841 \n", + " 45 0.398902 0.393450 \n", + " 46 0.360512 0.347976 \n", + " 47 0.355688 0.341633 \n", + " 48 0.396093 0.394256 \n", + " 49 0.409482 0.406266 \n", + "C-K96-G32 1 0 0.305871 0.303512 \n", + " 1 0.259619 0.253699 \n", + " 2 0.303794 0.300122 \n", + " 3 0.269782 0.265936 \n", + " 4 0.288257 0.279597 \n", + " 5 0.274560 0.270216 \n", + " 6 0.310544 0.308496 \n", + " 7 0.287550 0.283930 \n", + " 8 0.289980 0.285594 \n", + " 9 0.307083 0.303895 \n", + " 10 0.290877 0.287420 \n", + " 11 0.301014 0.297255 \n", + " 12 0.268408 0.264737 \n", + " 13 0.252841 0.242795 \n", + " 14 0.270862 0.268474 \n", + " 15 0.275394 0.271865 \n", + " 16 0.250280 0.244357 \n", + " 17 0.300653 0.297147 \n", + " 18 0.299007 0.295435 \n", + " 19 0.268830 0.263008 \n", + " 20 0.266160 0.262410 \n", + " 21 0.279387 0.269247 \n", + " 22 0.258581 0.250831 \n", + " 23 0.263935 0.261200 \n", + " 24 0.282180 0.277739 \n", + " 25 0.285029 0.280203 \n", + " 26 0.299903 0.296143 \n", + " 27 0.287924 0.282602 \n", + " 28 0.275007 0.271238 \n", + " 29 0.283235 0.279311 \n", + " 30 0.233011 0.229436 \n", + " 31 0.287543 0.284737 \n", + " 32 0.301637 0.297779 \n", + " 33 0.264410 0.260707 \n", + " 34 0.283847 0.275503 \n", + " 35 0.285352 0.281425 \n", + " 36 0.269949 0.265336 \n", + " 37 0.259093 0.256985 \n", + " 38 0.291305 0.287588 \n", + " 39 0.323847 0.321017 \n", + " 40 0.240069 0.234085 \n", + " 41 0.277658 0.274262 \n", + " 42 0.254476 0.251852 \n", + " 43 0.288593 0.287642 \n", + " 44 0.286736 0.285144 \n", + " 45 0.279767 0.276231 \n", + " 46 0.285448 0.283000 \n", + " 47 0.280782 0.273066 \n", + " 48 0.246460 0.239549 \n", + " 49 0.277493 0.272715 \n", + " 2 0 0.292145 0.288282 \n", + " 1 0.288591 0.287270 \n", + " 2 0.320469 0.307124 \n", + " 3 0.275560 0.274893 \n", + " 4 0.299138 0.288077 \n", + " 5 0.275883 0.266842 \n", + " 6 0.299321 0.291039 \n", + " 7 0.256790 0.254079 \n", + " 8 0.317673 0.316354 \n", + " 9 0.335912 0.322599 \n", + " 10 0.297790 0.292874 \n", + " 11 0.356439 0.352763 \n", + " 12 0.299849 0.292302 \n", + " 13 0.295228 0.283249 \n", + " 14 0.312899 0.302370 \n", + " 15 0.267361 0.258647 \n", + " 16 0.292070 0.289970 \n", + " 17 0.315649 0.314573 \n", + " 18 0.341030 0.327721 \n", + " 19 0.340725 0.330836 \n", + " 20 0.280004 0.273899 \n", + " 21 0.334003 0.322265 \n", + " 22 0.328378 0.318351 \n", + " 23 0.296916 0.293379 \n", + " 24 0.312814 0.305724 \n", + " 25 0.290188 0.286293 \n", + " 26 0.339515 0.335231 \n", + " 27 0.329805 0.323087 \n", + " 28 0.315024 0.309251 \n", + " 29 0.308938 0.302209 \n", + " 30 0.302093 0.291936 \n", + " 31 0.315866 0.312013 \n", + " 32 0.323429 0.316405 \n", + " 33 0.303115 0.297753 \n", + " 34 0.345991 0.343767 \n", + " 35 0.312161 0.310943 \n", + " 36 0.314909 0.311136 \n", + " 37 0.341050 0.337586 \n", + " 38 0.295467 0.289587 \n", + " 39 0.280528 0.272038 \n", + " 40 0.361971 0.356997 \n", + " 41 0.343750 0.336083 \n", + " 42 0.334220 0.325655 \n", + " 43 0.298368 0.290044 \n", + " 44 0.328186 0.324489 \n", + " 45 0.287846 0.279401 \n", + " 46 0.341268 0.331003 \n", + " 47 0.316191 0.304152 \n", + " 48 0.323268 0.321487 \n", + " 49 0.333901 0.332366 \n", + " 3 0 0.367426 0.357791 \n", + " 1 0.381770 0.378499 \n", + " 2 0.317412 0.310248 \n", + " 3 0.378471 0.373685 \n", + " 4 0.360464 0.350153 \n", + " 5 0.367684 0.357862 \n", + " 6 0.344604 0.328011 \n", + " 7 0.356245 0.348642 \n", + " 8 0.395888 0.390683 \n", + " 9 0.384057 0.372020 \n", + " 10 0.382437 0.377478 \n", + " 11 0.345050 0.341153 \n", + " 12 0.374814 0.361081 \n", + " 13 0.340610 0.329571 \n", + " 14 0.340172 0.336941 \n", + " 15 0.336933 0.331507 \n", + " 16 0.354435 0.341591 \n", + " 17 0.366161 0.361085 \n", + " 18 0.359694 0.345509 \n", + " 19 0.382389 0.377610 \n", + " 20 0.383570 0.376793 \n", + " 21 0.367985 0.358644 \n", + " 22 0.349272 0.344017 \n", + " 23 0.335268 0.332428 \n", + " 24 0.359002 0.350610 \n", + " 25 0.327016 0.320055 \n", + " 26 0.403501 0.396112 \n", + " 27 0.362966 0.353578 \n", + " 28 0.376421 0.370583 \n", + " 29 0.385309 0.376833 \n", + " 30 0.394441 0.386341 \n", + " 31 0.426487 0.423880 \n", + " 32 0.363253 0.355892 \n", + " 33 0.395005 0.390197 \n", + " 34 0.379592 0.371019 \n", + " 35 0.369445 0.358274 \n", + " 36 0.347299 0.330814 \n", + " 37 0.383405 0.382161 \n", + " 38 0.270993 0.261997 \n", + " 39 0.353387 0.348836 \n", + " 40 0.372462 0.366765 \n", + " 41 0.382316 0.375068 \n", + " 42 0.413503 0.413489 \n", + " 43 0.336834 0.332618 \n", + " 44 0.352959 0.335841 \n", + " 45 0.398902 0.393450 \n", + " 46 0.360512 0.347976 \n", + " 47 0.355688 0.341633 \n", + " 48 0.396093 0.394256 \n", + " 49 0.409482 0.406266 \n", + "\n", + " tanh_prime_selected_rho quadratic_core_box_radius \\\n", + "method layer neuron \n", + "A-K96 1 0 0.047527 0.595816 \n", + " 1 0.032061 0.506488 \n", + " 2 0.046963 0.589839 \n", + " 3 0.034840 0.528351 \n", + " 4 0.042034 0.562367 \n", + " 5 0.036481 0.533079 \n", + " 6 0.049384 0.607044 \n", + " 7 0.040799 0.558312 \n", + " 8 0.041842 0.565649 \n", + " 9 0.048172 0.599097 \n", + " 10 0.041968 0.563416 \n", + " 11 0.045882 0.586985 \n", + " 12 0.034372 0.522692 \n", + " 13 0.030702 0.498301 \n", + " 14 0.034934 0.525958 \n", + " 15 0.036598 0.533804 \n", + " 16 0.029349 0.484499 \n", + " 17 0.045681 0.586314 \n", + " 18 0.045056 0.584806 \n", + " 19 0.034892 0.518949 \n", + " 20 0.033683 0.520478 \n", + " 21 0.039154 0.545658 \n", + " 22 0.032030 0.505597 \n", + " 23 0.032822 0.517361 \n", + " 24 0.039077 0.554421 \n", + " 25 0.040152 0.562821 \n", + " 26 0.045448 0.579890 \n", + " 27 0.041288 0.563478 \n", + " 28 0.036517 0.531204 \n", + " 29 0.039338 0.551094 \n", + " 30 0.024441 0.457559 \n", + " 31 0.040635 0.562199 \n", + " 32 0.046151 0.590364 \n", + " 33 0.033135 0.514896 \n", + " 34 0.040385 0.555663 \n", + " 35 0.040082 0.558933 \n", + " 36 0.035035 0.525255 \n", + " 37 0.031285 0.504007 \n", + " 38 0.042183 0.566743 \n", + " 39 0.055347 0.630954 \n", + " 40 0.026582 0.467940 \n", + " 41 0.037324 0.546216 \n", + " 42 0.030014 0.502639 \n", + " 43 0.040698 0.554576 \n", + " 44 0.040148 0.559400 \n", + " 45 0.038065 0.549532 \n", + " 46 0.039835 0.560985 \n", + " 47 0.039200 0.538607 \n", + " 48 0.028428 0.481407 \n", + " 49 0.037541 0.534388 \n", + " 2 0 0.042525 0.573800 \n", + " 1 0.040762 0.569545 \n", + " 2 0.056466 0.623782 \n", + " 3 0.036185 0.542694 \n", + " 4 0.046698 0.583127 \n", + " 5 0.037737 0.539142 \n", + " 6 0.046178 0.585102 \n", + " 7 0.030692 0.504042 \n", + " 8 0.052277 0.624268 \n", + " 9 0.064155 0.652623 \n", + " 10 0.044890 0.585813 \n", + " 11 0.072402 0.701459 \n", + " 12 0.046243 0.588793 \n", + " 13 0.045362 0.575782 \n", + " 14 0.052303 0.615495 \n", + " 15 0.034887 0.517690 \n", + " 16 0.042160 0.574848 \n", + " 17 0.051348 0.618683 \n", + " 18 0.066902 0.668712 \n", + " 19 0.065597 0.668256 \n", + " 20 0.038644 0.551924 \n", + " 21 0.062676 0.654889 \n", + " 22 0.059295 0.636738 \n", + " 23 0.044244 0.586840 \n", + " 24 0.051501 0.617788 \n", + " 25 0.041813 0.568469 \n", + " 26 0.063339 0.669444 \n", + " 27 0.059175 0.645000 \n", + " 28 0.052155 0.620622 \n", + " 29 0.049773 0.603407 \n", + " 30 0.047671 0.592138 \n", + " 31 0.052042 0.623294 \n", + " 32 0.056231 0.634857 \n", + " 33 0.047084 0.595866 \n", + " 34 0.066151 0.682921 \n", + " 35 0.049896 0.613017 \n", + " 36 0.051607 0.618034 \n", + " 37 0.063873 0.670104 \n", + " 38 0.044205 0.572482 \n", + " 39 0.039242 0.547681 \n", + " 40 0.076258 0.709562 \n", + " 41 0.066593 0.675611 \n", + " 42 0.061821 0.657020 \n", + " 43 0.045811 0.582483 \n", + " 44 0.057585 0.644201 \n", + " 45 0.041846 0.565834 \n", + " 46 0.066024 0.669298 \n", + " 47 0.054141 0.621499 \n", + " 48 0.054865 0.635279 \n", + " 49 0.059818 0.654803 \n", + " 3 0 0.081405 0.725155 \n", + " 1 0.088718 0.742775 \n", + " 2 0.053530 0.628398 \n", + " 3 0.087087 0.738280 \n", + " 4 0.077217 0.707125 \n", + " 5 0.081658 0.716889 \n", + " 6 0.069858 0.671422 \n", + " 7 0.073715 0.698856 \n", + " 8 0.100548 0.777336 \n", + " 9 0.094295 0.747966 \n", + " 10 0.090013 0.746165 \n", + " 11 0.066112 0.666326 \n", + " 12 0.088167 0.723625 \n", + " 13 0.065936 0.663759 \n", + " 14 0.063347 0.665718 \n", + " 15 0.062359 0.660419 \n", + " 16 0.074459 0.684212 \n", + " 17 0.078936 0.713655 \n", + " 18 0.078172 0.703659 \n", + " 19 0.089895 0.748028 \n", + " 20 0.091645 0.742997 \n", + " 21 0.081672 0.716792 \n", + " 22 0.068867 0.688425 \n", + " 23 0.060783 0.650617 \n", + " 24 0.075644 0.704351 \n", + " 25 0.057898 0.647163 \n", + " 26 0.108119 0.792472 \n", + " 27 0.078436 0.709583 \n", + " 28 0.086105 0.736119 \n", + " 29 0.093635 0.757169 \n", + " 30 0.100693 0.764505 \n", + " 31 0.127976 0.831610 \n", + " 32 0.077945 0.711833 \n", + " 33 0.099624 0.772294 \n", + " 34 0.089433 0.736222 \n", + " 35 0.083412 0.716658 \n", + " 36 0.071372 0.673457 \n", + " 37 0.089115 0.756601 \n", + " 38 0.036103 0.522925 \n", + " 39 0.070959 0.690345 \n", + " 40 0.083330 0.734167 \n", + " 41 0.090911 0.736585 \n", + " 42 0.113335 0.809551 \n", + " 43 0.061956 0.661386 \n", + " 44 0.074962 0.688288 \n", + " 45 0.103184 0.779681 \n", + " 46 0.078107 0.703958 \n", + " 47 0.075633 0.699833 \n", + " 48 0.099044 0.778596 \n", + " 49 0.111237 0.804781 \n", + "B-K96 1 0 0.047527 0.591788 \n", + " 1 0.032061 0.503049 \n", + " 2 0.046963 0.585905 \n", + " 3 0.034840 0.524760 \n", + " 4 0.042034 0.558635 \n", + " 5 0.036481 0.529348 \n", + " 6 0.049384 0.603077 \n", + " 7 0.040799 0.554306 \n", + " 8 0.041842 0.561975 \n", + " 9 0.048172 0.595124 \n", + " 10 0.041968 0.559624 \n", + " 11 0.045882 0.582941 \n", + " 12 0.034372 0.519207 \n", + " 13 0.030702 0.494734 \n", + " 14 0.034934 0.522252 \n", + " 15 0.036598 0.530103 \n", + " 16 0.029349 0.481037 \n", + " 17 0.045681 0.582470 \n", + " 18 0.045056 0.580820 \n", + " 19 0.034892 0.515419 \n", + " 20 0.033683 0.516910 \n", + " 21 0.039154 0.542170 \n", + " 22 0.032030 0.502134 \n", + " 23 0.032822 0.513861 \n", + " 24 0.039077 0.550798 \n", + " 25 0.040152 0.559239 \n", + " 26 0.045448 0.576048 \n", + " 27 0.041288 0.559752 \n", + " 28 0.036517 0.527304 \n", + " 29 0.039338 0.547130 \n", + " 30 0.024441 0.454324 \n", + " 31 0.040635 0.558393 \n", + " 32 0.046151 0.586475 \n", + " 33 0.033135 0.510931 \n", + " 34 0.040385 0.552005 \n", + " 35 0.040082 0.555159 \n", + " 36 0.035035 0.521566 \n", + " 37 0.031285 0.500274 \n", + " 38 0.042183 0.562846 \n", + " 39 0.055347 0.626968 \n", + " 40 0.026582 0.464617 \n", + " 41 0.037324 0.542365 \n", + " 42 0.030014 0.498931 \n", + " 43 0.040698 0.550893 \n", + " 44 0.040148 0.555438 \n", + " 45 0.038065 0.545700 \n", + " 46 0.039835 0.557434 \n", + " 47 0.039200 0.534827 \n", + " 48 0.028428 0.478157 \n", + " 49 0.037541 0.530604 \n", + " 2 0 0.042525 0.570886 \n", + " 1 0.040762 0.566697 \n", + " 2 0.056466 0.620660 \n", + " 3 0.036185 0.539720 \n", + " 4 0.046698 0.580289 \n", + " 5 0.037737 0.536493 \n", + " 6 0.046178 0.582215 \n", + " 7 0.030692 0.501418 \n", + " 8 0.052277 0.621098 \n", + " 9 0.064155 0.649379 \n", + " 10 0.044890 0.582930 \n", + " 11 0.072402 0.698220 \n", + " 12 0.046243 0.585832 \n", + " 13 0.045362 0.572933 \n", + " 14 0.052303 0.612486 \n", + " 15 0.034887 0.515057 \n", + " 16 0.042160 0.571991 \n", + " 17 0.051348 0.615462 \n", + " 18 0.066902 0.665556 \n", + " 19 0.065597 0.665122 \n", + " 20 0.038644 0.549052 \n", + " 21 0.062676 0.651682 \n", + " 22 0.059295 0.633373 \n", + " 23 0.044244 0.583862 \n", + " 24 0.051501 0.614789 \n", + " 25 0.041813 0.565556 \n", + " 26 0.063339 0.666338 \n", + " 27 0.059175 0.641760 \n", + " 28 0.052155 0.617492 \n", + " 29 0.049773 0.600358 \n", + " 30 0.047671 0.589223 \n", + " 31 0.052042 0.620401 \n", + " 32 0.056231 0.631851 \n", + " 33 0.047084 0.592818 \n", + " 34 0.066151 0.679739 \n", + " 35 0.049896 0.609738 \n", + " 36 0.051607 0.614997 \n", + " 37 0.063873 0.666678 \n", + " 38 0.044205 0.569499 \n", + " 39 0.039242 0.544842 \n", + " 40 0.076258 0.706212 \n", + " 41 0.066593 0.672125 \n", + " 42 0.061821 0.653862 \n", + " 43 0.045811 0.579426 \n", + " 44 0.057585 0.641029 \n", + " 45 0.041846 0.562998 \n", + " 46 0.066024 0.666054 \n", + " 47 0.054141 0.618519 \n", + " 48 0.054865 0.632099 \n", + " 49 0.059818 0.651648 \n", + " 3 0 0.081405 0.722185 \n", + " 1 0.088718 0.739533 \n", + " 2 0.053530 0.625716 \n", + " 3 0.087087 0.735103 \n", + " 4 0.077217 0.704270 \n", + " 5 0.081658 0.713797 \n", + " 6 0.069858 0.668515 \n", + " 7 0.073715 0.695848 \n", + " 8 0.100548 0.774081 \n", + " 9 0.094295 0.744955 \n", + " 10 0.090013 0.742948 \n", + " 11 0.066112 0.663297 \n", + " 12 0.088167 0.720592 \n", + " 13 0.065936 0.660770 \n", + " 14 0.063347 0.662695 \n", + " 15 0.062359 0.657643 \n", + " 16 0.074459 0.681141 \n", + " 17 0.078936 0.710501 \n", + " 18 0.078172 0.700575 \n", + " 19 0.089895 0.744860 \n", + " 20 0.091645 0.739564 \n", + " 21 0.081672 0.713643 \n", + " 22 0.068867 0.685154 \n", + " 23 0.060783 0.647430 \n", + " 24 0.075644 0.701299 \n", + " 25 0.057898 0.644319 \n", + " 26 0.108119 0.789309 \n", + " 27 0.078436 0.706458 \n", + " 28 0.086105 0.733038 \n", + " 29 0.093635 0.754064 \n", + " 30 0.100693 0.761212 \n", + " 31 0.127976 0.828240 \n", + " 32 0.077945 0.708868 \n", + " 33 0.099624 0.769019 \n", + " 34 0.089433 0.732879 \n", + " 35 0.083412 0.713577 \n", + " 36 0.071372 0.670409 \n", + " 37 0.089115 0.753236 \n", + " 38 0.036103 0.520482 \n", + " 39 0.070959 0.687329 \n", + " 40 0.083330 0.731049 \n", + " 41 0.090911 0.733339 \n", + " 42 0.113335 0.806114 \n", + " 43 0.061956 0.658316 \n", + " 44 0.074962 0.685339 \n", + " 45 0.103184 0.776412 \n", + " 46 0.078107 0.701021 \n", + " 47 0.075633 0.696907 \n", + " 48 0.099044 0.775198 \n", + " 49 0.111237 0.801633 \n", + "C-K96-G32 1 0 0.047527 0.587342 \n", + " 1 0.032061 0.498685 \n", + " 2 0.046963 0.580904 \n", + " 3 0.034840 0.520568 \n", + " 4 0.042034 0.554628 \n", + " 5 0.036481 0.524109 \n", + " 6 0.049384 0.598174 \n", + " 7 0.040799 0.548199 \n", + " 8 0.041842 0.557634 \n", + " 9 0.048172 0.591039 \n", + " 10 0.041968 0.553294 \n", + " 11 0.045882 0.577989 \n", + " 12 0.034372 0.515455 \n", + " 13 0.030702 0.492291 \n", + " 14 0.034934 0.517931 \n", + " 15 0.036598 0.523416 \n", + " 16 0.029349 0.476571 \n", + " 17 0.045681 0.577785 \n", + " 18 0.045056 0.577072 \n", + " 19 0.034892 0.510837 \n", + " 20 0.033683 0.512798 \n", + " 21 0.039154 0.538250 \n", + " 22 0.032030 0.499115 \n", + " 23 0.032822 0.509654 \n", + " 24 0.039077 0.547019 \n", + " 25 0.040152 0.555240 \n", + " 26 0.045448 0.569544 \n", + " 27 0.041288 0.555384 \n", + " 28 0.036517 0.522250 \n", + " 29 0.039338 0.544593 \n", + " 30 0.024441 0.451611 \n", + " 31 0.040635 0.553419 \n", + " 32 0.046151 0.581637 \n", + " 33 0.033135 0.505647 \n", + " 34 0.040385 0.547786 \n", + " 35 0.040082 0.550650 \n", + " 36 0.035035 0.515698 \n", + " 37 0.031285 0.494699 \n", + " 38 0.042183 0.558216 \n", + " 39 0.055347 0.621877 \n", + " 40 0.026582 0.460433 \n", + " 41 0.037324 0.539615 \n", + " 42 0.030014 0.497380 \n", + " 43 0.040698 0.545100 \n", + " 44 0.040148 0.550698 \n", + " 45 0.038065 0.542368 \n", + " 46 0.039835 0.554827 \n", + " 47 0.039200 0.528216 \n", + " 48 0.028428 0.474334 \n", + " 49 0.037541 0.524317 \n", + " 2 0 0.042525 0.567531 \n", + " 1 0.040762 0.563760 \n", + " 2 0.056466 0.617390 \n", + " 3 0.036185 0.537333 \n", + " 4 0.046698 0.575922 \n", + " 5 0.037737 0.532569 \n", + " 6 0.046178 0.577639 \n", + " 7 0.030692 0.497985 \n", + " 8 0.052277 0.618345 \n", + " 9 0.064155 0.646542 \n", + " 10 0.044890 0.581064 \n", + " 11 0.072402 0.694688 \n", + " 12 0.046243 0.581606 \n", + " 13 0.045362 0.569543 \n", + " 14 0.052303 0.609438 \n", + " 15 0.034887 0.511871 \n", + " 16 0.042160 0.569267 \n", + " 17 0.051348 0.611880 \n", + " 18 0.066902 0.662739 \n", + " 19 0.065597 0.661996 \n", + " 20 0.038644 0.546762 \n", + " 21 0.062676 0.647784 \n", + " 22 0.059295 0.626348 \n", + " 23 0.044244 0.580989 \n", + " 24 0.051501 0.612906 \n", + " 25 0.041813 0.562370 \n", + " 26 0.063339 0.663426 \n", + " 27 0.059175 0.638897 \n", + " 28 0.052155 0.614433 \n", + " 29 0.049773 0.596806 \n", + " 30 0.047671 0.585799 \n", + " 31 0.052042 0.618007 \n", + " 32 0.056231 0.627989 \n", + " 33 0.047084 0.589799 \n", + " 34 0.066151 0.676426 \n", + " 35 0.049896 0.606277 \n", + " 36 0.051607 0.611656 \n", + " 37 0.063873 0.663480 \n", + " 38 0.044205 0.564501 \n", + " 39 0.039242 0.540408 \n", + " 40 0.076258 0.702134 \n", + " 41 0.066593 0.667561 \n", + " 42 0.061821 0.651317 \n", + " 43 0.045811 0.575699 \n", + " 44 0.057585 0.637659 \n", + " 45 0.041846 0.560021 \n", + " 46 0.066024 0.662055 \n", + " 47 0.054141 0.615349 \n", + " 48 0.054865 0.629218 \n", + " 49 0.059818 0.648349 \n", + " 3 0 0.081405 0.718049 \n", + " 1 0.088718 0.734909 \n", + " 2 0.053530 0.622615 \n", + " 3 0.087087 0.730393 \n", + " 4 0.077217 0.699819 \n", + " 5 0.081658 0.709209 \n", + " 6 0.069858 0.664953 \n", + " 7 0.073715 0.691398 \n", + " 8 0.100548 0.770009 \n", + " 9 0.094295 0.739355 \n", + " 10 0.090013 0.736872 \n", + " 11 0.066112 0.658330 \n", + " 12 0.088167 0.715461 \n", + " 13 0.065936 0.655566 \n", + " 14 0.063347 0.657627 \n", + " 15 0.062359 0.654956 \n", + " 16 0.074459 0.676145 \n", + " 17 0.078936 0.704957 \n", + " 18 0.078172 0.695172 \n", + " 19 0.089895 0.739964 \n", + " 20 0.091645 0.732371 \n", + " 21 0.081672 0.707797 \n", + " 22 0.068867 0.681047 \n", + " 23 0.060783 0.640660 \n", + " 24 0.075644 0.697496 \n", + " 25 0.057898 0.640665 \n", + " 26 0.108119 0.784299 \n", + " 27 0.078436 0.703787 \n", + " 28 0.086105 0.728193 \n", + " 29 0.093635 0.749806 \n", + " 30 0.100693 0.754038 \n", + " 31 0.127976 0.822975 \n", + " 32 0.077945 0.705008 \n", + " 33 0.099624 0.764535 \n", + " 34 0.089433 0.728882 \n", + " 35 0.083412 0.707091 \n", + " 36 0.071372 0.663576 \n", + " 37 0.089115 0.750252 \n", + " 38 0.036103 0.516016 \n", + " 39 0.070959 0.683726 \n", + " 40 0.083330 0.727067 \n", + " 41 0.090911 0.727368 \n", + " 42 0.113335 0.801541 \n", + " 43 0.061956 0.654362 \n", + " 44 0.074962 0.680048 \n", + " 45 0.103184 0.770426 \n", + " 46 0.078107 0.696055 \n", + " 47 0.075633 0.692798 \n", + " 48 0.099044 0.770001 \n", + " 49 0.111237 0.796976 \n", + "\n", + " approximation_degree propagated_J_remainder_mean \\\n", + "method layer neuron \n", + "A-K96 1 0 2 0.056034 \n", + " 1 2 0.056034 \n", + " 2 2 0.056034 \n", + " 3 2 0.056034 \n", + " 4 2 0.056034 \n", + " 5 2 0.056034 \n", + " 6 2 0.056034 \n", + " 7 2 0.056034 \n", + " 8 2 0.056034 \n", + " 9 2 0.056034 \n", + " 10 2 0.056034 \n", + " 11 2 0.056034 \n", + " 12 2 0.056034 \n", + " 13 2 0.056034 \n", + " 14 2 0.056034 \n", + " 15 2 0.056034 \n", + " 16 2 0.056034 \n", + " 17 2 0.056034 \n", + " 18 2 0.056034 \n", + " 19 2 0.056034 \n", + " 20 2 0.056034 \n", + " 21 2 0.056034 \n", + " 22 2 0.056034 \n", + " 23 2 0.056034 \n", + " 24 2 0.056034 \n", + " 25 2 0.056034 \n", + " 26 2 0.056034 \n", + " 27 2 0.056034 \n", + " 28 2 0.056034 \n", + " 29 2 0.056034 \n", + " 30 2 0.056034 \n", + " 31 2 0.056034 \n", + " 32 2 0.056034 \n", + " 33 2 0.056034 \n", + " 34 2 0.056034 \n", + " 35 2 0.056034 \n", + " 36 2 0.056034 \n", + " 37 2 0.056034 \n", + " 38 2 0.056034 \n", + " 39 2 0.056034 \n", + " 40 2 0.056034 \n", + " 41 2 0.056034 \n", + " 42 2 0.056034 \n", + " 43 2 0.056034 \n", + " 44 2 0.056034 \n", + " 45 2 0.056034 \n", + " 46 2 0.056034 \n", + " 47 2 0.056034 \n", + " 48 2 0.056034 \n", + " 49 2 0.056034 \n", + " 2 0 2 0.581434 \n", + " 1 2 0.581434 \n", + " 2 2 0.581434 \n", + " 3 2 0.581434 \n", + " 4 2 0.581434 \n", + " 5 2 0.581434 \n", + " 6 2 0.581434 \n", + " 7 2 0.581434 \n", + " 8 2 0.581434 \n", + " 9 2 0.581434 \n", + " 10 2 0.581434 \n", + " 11 2 0.581434 \n", + " 12 2 0.581434 \n", + " 13 2 0.581434 \n", + " 14 2 0.581434 \n", + " 15 2 0.581434 \n", + " 16 2 0.581434 \n", + " 17 2 0.581434 \n", + " 18 2 0.581434 \n", + " 19 2 0.581434 \n", + " 20 2 0.581434 \n", + " 21 2 0.581434 \n", + " 22 2 0.581434 \n", + " 23 2 0.581434 \n", + " 24 2 0.581434 \n", + " 25 2 0.581434 \n", + " 26 2 0.581434 \n", + " 27 2 0.581434 \n", + " 28 2 0.581434 \n", + " 29 2 0.581434 \n", + " 30 2 0.581434 \n", + " 31 2 0.581434 \n", + " 32 2 0.581434 \n", + " 33 2 0.581434 \n", + " 34 2 0.581434 \n", + " 35 2 0.581434 \n", + " 36 2 0.581434 \n", + " 37 2 0.581434 \n", + " 38 2 0.581434 \n", + " 39 2 0.581434 \n", + " 40 2 0.581434 \n", + " 41 2 0.581434 \n", + " 42 2 0.581434 \n", + " 43 2 0.581434 \n", + " 44 2 0.581434 \n", + " 45 2 0.581434 \n", + " 46 2 0.581434 \n", + " 47 2 0.581434 \n", + " 48 2 0.581434 \n", + " 49 2 0.581434 \n", + " 3 0 2 6.093022 \n", + " 1 2 6.093022 \n", + " 2 2 6.093022 \n", + " 3 2 6.093022 \n", + " 4 2 6.093022 \n", + " 5 2 6.093022 \n", + " 6 2 6.093022 \n", + " 7 2 6.093022 \n", + " 8 2 6.093022 \n", + " 9 2 6.093022 \n", + " 10 2 6.093022 \n", + " 11 2 6.093022 \n", + " 12 2 6.093022 \n", + " 13 2 6.093022 \n", + " 14 2 6.093022 \n", + " 15 2 6.093022 \n", + " 16 2 6.093022 \n", + " 17 2 6.093022 \n", + " 18 2 6.093022 \n", + " 19 2 6.093022 \n", + " 20 2 6.093022 \n", + " 21 2 6.093022 \n", + " 22 2 6.093022 \n", + " 23 2 6.093022 \n", + " 24 2 6.093022 \n", + " 25 2 6.093022 \n", + " 26 2 6.093022 \n", + " 27 2 6.093022 \n", + " 28 2 6.093022 \n", + " 29 2 6.093022 \n", + " 30 2 6.093022 \n", + " 31 2 6.093022 \n", + " 32 2 6.093022 \n", + " 33 2 6.093022 \n", + " 34 2 6.093022 \n", + " 35 2 6.093022 \n", + " 36 2 6.093022 \n", + " 37 2 6.093022 \n", + " 38 2 6.093022 \n", + " 39 2 6.093022 \n", + " 40 2 6.093022 \n", + " 41 2 6.093022 \n", + " 42 2 6.093022 \n", + " 43 2 6.093022 \n", + " 44 2 6.093022 \n", + " 45 2 6.093022 \n", + " 46 2 6.093022 \n", + " 47 2 6.093022 \n", + " 48 2 6.093022 \n", + " 49 2 6.093022 \n", + "B-K96 1 0 2 0.055677 \n", + " 1 2 0.055677 \n", + " 2 2 0.055677 \n", + " 3 2 0.055677 \n", + " 4 2 0.055677 \n", + " 5 2 0.055677 \n", + " 6 2 0.055677 \n", + " 7 2 0.055677 \n", + " 8 2 0.055677 \n", + " 9 2 0.055677 \n", + " 10 2 0.055677 \n", + " 11 2 0.055677 \n", + " 12 2 0.055677 \n", + " 13 2 0.055677 \n", + " 14 2 0.055677 \n", + " 15 2 0.055677 \n", + " 16 2 0.055677 \n", + " 17 2 0.055677 \n", + " 18 2 0.055677 \n", + " 19 2 0.055677 \n", + " 20 2 0.055677 \n", + " 21 2 0.055677 \n", + " 22 2 0.055677 \n", + " 23 2 0.055677 \n", + " 24 2 0.055677 \n", + " 25 2 0.055677 \n", + " 26 2 0.055677 \n", + " 27 2 0.055677 \n", + " 28 2 0.055677 \n", + " 29 2 0.055677 \n", + " 30 2 0.055677 \n", + " 31 2 0.055677 \n", + " 32 2 0.055677 \n", + " 33 2 0.055677 \n", + " 34 2 0.055677 \n", + " 35 2 0.055677 \n", + " 36 2 0.055677 \n", + " 37 2 0.055677 \n", + " 38 2 0.055677 \n", + " 39 2 0.055677 \n", + " 40 2 0.055677 \n", + " 41 2 0.055677 \n", + " 42 2 0.055677 \n", + " 43 2 0.055677 \n", + " 44 2 0.055677 \n", + " 45 2 0.055677 \n", + " 46 2 0.055677 \n", + " 47 2 0.055677 \n", + " 48 2 0.055677 \n", + " 49 2 0.055677 \n", + " 2 0 2 0.575656 \n", + " 1 2 0.575656 \n", + " 2 2 0.575656 \n", + " 3 2 0.575656 \n", + " 4 2 0.575656 \n", + " 5 2 0.575656 \n", + " 6 2 0.575656 \n", + " 7 2 0.575656 \n", + " 8 2 0.575656 \n", + " 9 2 0.575656 \n", + " 10 2 0.575656 \n", + " 11 2 0.575656 \n", + " 12 2 0.575656 \n", + " 13 2 0.575656 \n", + " 14 2 0.575656 \n", + " 15 2 0.575656 \n", + " 16 2 0.575656 \n", + " 17 2 0.575656 \n", + " 18 2 0.575656 \n", + " 19 2 0.575656 \n", + " 20 2 0.575656 \n", + " 21 2 0.575656 \n", + " 22 2 0.575656 \n", + " 23 2 0.575656 \n", + " 24 2 0.575656 \n", + " 25 2 0.575656 \n", + " 26 2 0.575656 \n", + " 27 2 0.575656 \n", + " 28 2 0.575656 \n", + " 29 2 0.575656 \n", + " 30 2 0.575656 \n", + " 31 2 0.575656 \n", + " 32 2 0.575656 \n", + " 33 2 0.575656 \n", + " 34 2 0.575656 \n", + " 35 2 0.575656 \n", + " 36 2 0.575656 \n", + " 37 2 0.575656 \n", + " 38 2 0.575656 \n", + " 39 2 0.575656 \n", + " 40 2 0.575656 \n", + " 41 2 0.575656 \n", + " 42 2 0.575656 \n", + " 43 2 0.575656 \n", + " 44 2 0.575656 \n", + " 45 2 0.575656 \n", + " 46 2 0.575656 \n", + " 47 2 0.575656 \n", + " 48 2 0.575656 \n", + " 49 2 0.575656 \n", + " 3 0 2 6.010999 \n", + " 1 2 6.010999 \n", + " 2 2 6.010999 \n", + " 3 2 6.010999 \n", + " 4 2 6.010999 \n", + " 5 2 6.010999 \n", + " 6 2 6.010999 \n", + " 7 2 6.010999 \n", + " 8 2 6.010999 \n", + " 9 2 6.010999 \n", + " 10 2 6.010999 \n", + " 11 2 6.010999 \n", + " 12 2 6.010999 \n", + " 13 2 6.010999 \n", + " 14 2 6.010999 \n", + " 15 2 6.010999 \n", + " 16 2 6.010999 \n", + " 17 2 6.010999 \n", + " 18 2 6.010999 \n", + " 19 2 6.010999 \n", + " 20 2 6.010999 \n", + " 21 2 6.010999 \n", + " 22 2 6.010999 \n", + " 23 2 6.010999 \n", + " 24 2 6.010999 \n", + " 25 2 6.010999 \n", + " 26 2 6.010999 \n", + " 27 2 6.010999 \n", + " 28 2 6.010999 \n", + " 29 2 6.010999 \n", + " 30 2 6.010999 \n", + " 31 2 6.010999 \n", + " 32 2 6.010999 \n", + " 33 2 6.010999 \n", + " 34 2 6.010999 \n", + " 35 2 6.010999 \n", + " 36 2 6.010999 \n", + " 37 2 6.010999 \n", + " 38 2 6.010999 \n", + " 39 2 6.010999 \n", + " 40 2 6.010999 \n", + " 41 2 6.010999 \n", + " 42 2 6.010999 \n", + " 43 2 6.010999 \n", + " 44 2 6.010999 \n", + " 45 2 6.010999 \n", + " 46 2 6.010999 \n", + " 47 2 6.010999 \n", + " 48 2 6.010999 \n", + " 49 2 6.010999 \n", + "C-K96-G32 1 0 2 0.055247 \n", + " 1 2 0.055247 \n", + " 2 2 0.055247 \n", + " 3 2 0.055247 \n", + " 4 2 0.055247 \n", + " 5 2 0.055247 \n", + " 6 2 0.055247 \n", + " 7 2 0.055247 \n", + " 8 2 0.055247 \n", + " 9 2 0.055247 \n", + " 10 2 0.055247 \n", + " 11 2 0.055247 \n", + " 12 2 0.055247 \n", + " 13 2 0.055247 \n", + " 14 2 0.055247 \n", + " 15 2 0.055247 \n", + " 16 2 0.055247 \n", + " 17 2 0.055247 \n", + " 18 2 0.055247 \n", + " 19 2 0.055247 \n", + " 20 2 0.055247 \n", + " 21 2 0.055247 \n", + " 22 2 0.055247 \n", + " 23 2 0.055247 \n", + " 24 2 0.055247 \n", + " 25 2 0.055247 \n", + " 26 2 0.055247 \n", + " 27 2 0.055247 \n", + " 28 2 0.055247 \n", + " 29 2 0.055247 \n", + " 30 2 0.055247 \n", + " 31 2 0.055247 \n", + " 32 2 0.055247 \n", + " 33 2 0.055247 \n", + " 34 2 0.055247 \n", + " 35 2 0.055247 \n", + " 36 2 0.055247 \n", + " 37 2 0.055247 \n", + " 38 2 0.055247 \n", + " 39 2 0.055247 \n", + " 40 2 0.055247 \n", + " 41 2 0.055247 \n", + " 42 2 0.055247 \n", + " 43 2 0.055247 \n", + " 44 2 0.055247 \n", + " 45 2 0.055247 \n", + " 46 2 0.055247 \n", + " 47 2 0.055247 \n", + " 48 2 0.055247 \n", + " 49 2 0.055247 \n", + " 2 0 2 0.571931 \n", + " 1 2 0.571931 \n", + " 2 2 0.571931 \n", + " 3 2 0.571931 \n", + " 4 2 0.571931 \n", + " 5 2 0.571931 \n", + " 6 2 0.571931 \n", + " 7 2 0.571931 \n", + " 8 2 0.571931 \n", + " 9 2 0.571931 \n", + " 10 2 0.571931 \n", + " 11 2 0.571931 \n", + " 12 2 0.571931 \n", + " 13 2 0.571931 \n", + " 14 2 0.571931 \n", + " 15 2 0.571931 \n", + " 16 2 0.571931 \n", + " 17 2 0.571931 \n", + " 18 2 0.571931 \n", + " 19 2 0.571931 \n", + " 20 2 0.571931 \n", + " 21 2 0.571931 \n", + " 22 2 0.571931 \n", + " 23 2 0.571931 \n", + " 24 2 0.571931 \n", + " 25 2 0.571931 \n", + " 26 2 0.571931 \n", + " 27 2 0.571931 \n", + " 28 2 0.571931 \n", + " 29 2 0.571931 \n", + " 30 2 0.571931 \n", + " 31 2 0.571931 \n", + " 32 2 0.571931 \n", + " 33 2 0.571931 \n", + " 34 2 0.571931 \n", + " 35 2 0.571931 \n", + " 36 2 0.571931 \n", + " 37 2 0.571931 \n", + " 38 2 0.571931 \n", + " 39 2 0.571931 \n", + " 40 2 0.571931 \n", + " 41 2 0.571931 \n", + " 42 2 0.571931 \n", + " 43 2 0.571931 \n", + " 44 2 0.571931 \n", + " 45 2 0.571931 \n", + " 46 2 0.571931 \n", + " 47 2 0.571931 \n", + " 48 2 0.571931 \n", + " 49 2 0.571931 \n", + " 3 0 2 5.972857 \n", + " 1 2 5.972857 \n", + " 2 2 5.972857 \n", + " 3 2 5.972857 \n", + " 4 2 5.972857 \n", + " 5 2 5.972857 \n", + " 6 2 5.972857 \n", + " 7 2 5.972857 \n", + " 8 2 5.972857 \n", + " 9 2 5.972857 \n", + " 10 2 5.972857 \n", + " 11 2 5.972857 \n", + " 12 2 5.972857 \n", + " 13 2 5.972857 \n", + " 14 2 5.972857 \n", + " 15 2 5.972857 \n", + " 16 2 5.972857 \n", + " 17 2 5.972857 \n", + " 18 2 5.972857 \n", + " 19 2 5.972857 \n", + " 20 2 5.972857 \n", + " 21 2 5.972857 \n", + " 22 2 5.972857 \n", + " 23 2 5.972857 \n", + " 24 2 5.972857 \n", + " 25 2 5.972857 \n", + " 26 2 5.972857 \n", + " 27 2 5.972857 \n", + " 28 2 5.972857 \n", + " 29 2 5.972857 \n", + " 30 2 5.972857 \n", + " 31 2 5.972857 \n", + " 32 2 5.972857 \n", + " 33 2 5.972857 \n", + " 34 2 5.972857 \n", + " 35 2 5.972857 \n", + " 36 2 5.972857 \n", + " 37 2 5.972857 \n", + " 38 2 5.972857 \n", + " 39 2 5.972857 \n", + " 40 2 5.972857 \n", + " 41 2 5.972857 \n", + " 42 2 5.972857 \n", + " 43 2 5.972857 \n", + " 44 2 5.972857 \n", + " 45 2 5.972857 \n", + " 46 2 5.972857 \n", + " 47 2 5.972857 \n", + " 48 2 5.972857 \n", + " 49 2 5.972857 \n", + "\n", + " J_terms_after_activation J_degree_after_activation \\\n", + "method layer neuron \n", + "A-K96 1 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + " 2 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + " 3 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + "B-K96 1 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + " 2 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + " 3 0 96 2 \n", + " 1 96 2 \n", + " 2 96 2 \n", + " 3 96 2 \n", + " 4 96 2 \n", + " 5 96 2 \n", + " 6 96 2 \n", + " 7 96 2 \n", + " 8 96 2 \n", + " 9 96 2 \n", + " 10 96 2 \n", + " 11 96 2 \n", + " 12 96 2 \n", + " 13 96 2 \n", + " 14 96 2 \n", + " 15 96 2 \n", + " 16 96 2 \n", + " 17 96 2 \n", + " 18 96 2 \n", + " 19 96 2 \n", + " 20 96 2 \n", + " 21 96 2 \n", + " 22 96 2 \n", + " 23 96 2 \n", + " 24 96 2 \n", + " 25 96 2 \n", + " 26 96 2 \n", + " 27 96 2 \n", + " 28 96 2 \n", + " 29 96 2 \n", + " 30 96 2 \n", + " 31 96 2 \n", + " 32 96 2 \n", + " 33 96 2 \n", + " 34 96 2 \n", + " 35 96 2 \n", + " 36 96 2 \n", + " 37 96 2 \n", + " 38 96 2 \n", + " 39 96 2 \n", + " 40 96 2 \n", + " 41 96 2 \n", + " 42 96 2 \n", + " 43 96 2 \n", + " 44 96 2 \n", + " 45 96 2 \n", + " 46 96 2 \n", + " 47 96 2 \n", + " 48 96 2 \n", + " 49 96 2 \n", + "C-K96-G32 1 0 128 2 \n", + " 1 128 2 \n", + " 2 128 2 \n", + " 3 128 2 \n", + " 4 128 2 \n", + " 5 128 2 \n", + " 6 128 2 \n", + " 7 128 2 \n", + " 8 128 2 \n", + " 9 128 2 \n", + " 10 128 2 \n", + " 11 128 2 \n", + " 12 128 2 \n", + " 13 128 2 \n", + " 14 128 2 \n", + " 15 128 2 \n", + " 16 128 2 \n", + " 17 128 2 \n", + " 18 128 2 \n", + " 19 128 2 \n", + " 20 128 2 \n", + " 21 128 2 \n", + " 22 128 2 \n", + " 23 128 2 \n", + " 24 128 2 \n", + " 25 128 2 \n", + " 26 128 2 \n", + " 27 128 2 \n", + " 28 128 2 \n", + " 29 128 2 \n", + " 30 128 2 \n", + " 31 128 2 \n", + " 32 128 2 \n", + " 33 128 2 \n", + " 34 128 2 \n", + " 35 128 2 \n", + " 36 128 2 \n", + " 37 128 2 \n", + " 38 128 2 \n", + " 39 128 2 \n", + " 40 128 2 \n", + " 41 128 2 \n", + " 42 128 2 \n", + " 43 128 2 \n", + " 44 128 2 \n", + " 45 128 2 \n", + " 46 128 2 \n", + " 47 128 2 \n", + " 48 128 2 \n", + " 49 128 2 \n", + " 2 0 128 2 \n", + " 1 128 2 \n", + " 2 128 2 \n", + " 3 128 2 \n", + " 4 128 2 \n", + " 5 128 2 \n", + " 6 128 2 \n", + " 7 128 2 \n", + " 8 128 2 \n", + " 9 128 2 \n", + " 10 128 2 \n", + " 11 128 2 \n", + " 12 128 2 \n", + " 13 128 2 \n", + " 14 128 2 \n", + " 15 128 2 \n", + " 16 128 2 \n", + " 17 128 2 \n", + " 18 128 2 \n", + " 19 128 2 \n", + " 20 128 2 \n", + " 21 128 2 \n", + " 22 128 2 \n", + " 23 128 2 \n", + " 24 128 2 \n", + " 25 128 2 \n", + " 26 128 2 \n", + " 27 128 2 \n", + " 28 128 2 \n", + " 29 128 2 \n", + " 30 128 2 \n", + " 31 128 2 \n", + " 32 128 2 \n", + " 33 128 2 \n", + " 34 128 2 \n", + " 35 128 2 \n", + " 36 128 2 \n", + " 37 128 2 \n", + " 38 128 2 \n", + " 39 128 2 \n", + " 40 128 2 \n", + " 41 128 2 \n", + " 42 128 2 \n", + " 43 128 2 \n", + " 44 128 2 \n", + " 45 128 2 \n", + " 46 128 2 \n", + " 47 128 2 \n", + " 48 128 2 \n", + " 49 128 2 \n", + " 3 0 128 2 \n", + " 1 128 2 \n", + " 2 128 2 \n", + " 3 128 2 \n", + " 4 128 2 \n", + " 5 128 2 \n", + " 6 128 2 \n", + " 7 128 2 \n", + " 8 128 2 \n", + " 9 128 2 \n", + " 10 128 2 \n", + " 11 128 2 \n", + " 12 128 2 \n", + " 13 128 2 \n", + " 14 128 2 \n", + " 15 128 2 \n", + " 16 128 2 \n", + " 17 128 2 \n", + " 18 128 2 \n", + " 19 128 2 \n", + " 20 128 2 \n", + " 21 128 2 \n", + " 22 128 2 \n", + " 23 128 2 \n", + " 24 128 2 \n", + " 25 128 2 \n", + " 26 128 2 \n", + " 27 128 2 \n", + " 28 128 2 \n", + " 29 128 2 \n", + " 30 128 2 \n", + " 31 128 2 \n", + " 32 128 2 \n", + " 33 128 2 \n", + " 34 128 2 \n", + " 35 128 2 \n", + " 36 128 2 \n", + " 37 128 2 \n", + " 38 128 2 \n", + " 39 128 2 \n", + " 40 128 2 \n", + " 41 128 2 \n", + " 42 128 2 \n", + " 43 128 2 \n", + " 44 128 2 \n", + " 45 128 2 \n", + " 46 128 2 \n", + " 47 128 2 \n", + " 48 128 2 \n", + " 49 128 2 \n", + "\n", + " J_noise_after_activation activation_runtime_s \n", + "method layer neuron \n", + "A-K96 1 0 150 0.146239 \n", + " 1 150 0.146239 \n", + " 2 150 0.146239 \n", + " 3 150 0.146239 \n", + " 4 150 0.146239 \n", + " 5 150 0.146239 \n", + " 6 150 0.146239 \n", + " 7 150 0.146239 \n", + " 8 150 0.146239 \n", + " 9 150 0.146239 \n", + " 10 150 0.146239 \n", + " 11 150 0.146239 \n", + " 12 150 0.146239 \n", + " 13 150 0.146239 \n", + " 14 150 0.146239 \n", + " 15 150 0.146239 \n", + " 16 150 0.146239 \n", + " 17 150 0.146239 \n", + " 18 150 0.146239 \n", + " 19 150 0.146239 \n", + " 20 150 0.146239 \n", + " 21 150 0.146239 \n", + " 22 150 0.146239 \n", + " 23 150 0.146239 \n", + " 24 150 0.146239 \n", + " 25 150 0.146239 \n", + " 26 150 0.146239 \n", + " 27 150 0.146239 \n", + " 28 150 0.146239 \n", + " 29 150 0.146239 \n", + " 30 150 0.146239 \n", + " 31 150 0.146239 \n", + " 32 150 0.146239 \n", + " 33 150 0.146239 \n", + " 34 150 0.146239 \n", + " 35 150 0.146239 \n", + " 36 150 0.146239 \n", + " 37 150 0.146239 \n", + " 38 150 0.146239 \n", + " 39 150 0.146239 \n", + " 40 150 0.146239 \n", + " 41 150 0.146239 \n", + " 42 150 0.146239 \n", + " 43 150 0.146239 \n", + " 44 150 0.146239 \n", + " 45 150 0.146239 \n", + " 46 150 0.146239 \n", + " 47 150 0.146239 \n", + " 48 150 0.146239 \n", + " 49 150 0.146239 \n", + " 2 0 200 0.912937 \n", + " 1 200 0.912937 \n", + " 2 200 0.912937 \n", + " 3 200 0.912937 \n", + " 4 200 0.912937 \n", + " 5 200 0.912937 \n", + " 6 200 0.912937 \n", + " 7 200 0.912937 \n", + " 8 200 0.912937 \n", + " 9 200 0.912937 \n", + " 10 200 0.912937 \n", + " 11 200 0.912937 \n", + " 12 200 0.912937 \n", + " 13 200 0.912937 \n", + " 14 200 0.912937 \n", + " 15 200 0.912937 \n", + " 16 200 0.912937 \n", + " 17 200 0.912937 \n", + " 18 200 0.912937 \n", + " 19 200 0.912937 \n", + " 20 200 0.912937 \n", + " 21 200 0.912937 \n", + " 22 200 0.912937 \n", + " 23 200 0.912937 \n", + " 24 200 0.912937 \n", + " 25 200 0.912937 \n", + " 26 200 0.912937 \n", + " 27 200 0.912937 \n", + " 28 200 0.912937 \n", + " 29 200 0.912937 \n", + " 30 200 0.912937 \n", + " 31 200 0.912937 \n", + " 32 200 0.912937 \n", + " 33 200 0.912937 \n", + " 34 200 0.912937 \n", + " 35 200 0.912937 \n", + " 36 200 0.912937 \n", + " 37 200 0.912937 \n", + " 38 200 0.912937 \n", + " 39 200 0.912937 \n", + " 40 200 0.912937 \n", + " 41 200 0.912937 \n", + " 42 200 0.912937 \n", + " 43 200 0.912937 \n", + " 44 200 0.912937 \n", + " 45 200 0.912937 \n", + " 46 200 0.912937 \n", + " 47 200 0.912937 \n", + " 48 200 0.912937 \n", + " 49 200 0.912937 \n", + " 3 0 250 1.090658 \n", + " 1 250 1.090658 \n", + " 2 250 1.090658 \n", + " 3 250 1.090658 \n", + " 4 250 1.090658 \n", + " 5 250 1.090658 \n", + " 6 250 1.090658 \n", + " 7 250 1.090658 \n", + " 8 250 1.090658 \n", + " 9 250 1.090658 \n", + " 10 250 1.090658 \n", + " 11 250 1.090658 \n", + " 12 250 1.090658 \n", + " 13 250 1.090658 \n", + " 14 250 1.090658 \n", + " 15 250 1.090658 \n", + " 16 250 1.090658 \n", + " 17 250 1.090658 \n", + " 18 250 1.090658 \n", + " 19 250 1.090658 \n", + " 20 250 1.090658 \n", + " 21 250 1.090658 \n", + " 22 250 1.090658 \n", + " 23 250 1.090658 \n", + " 24 250 1.090658 \n", + " 25 250 1.090658 \n", + " 26 250 1.090658 \n", + " 27 250 1.090658 \n", + " 28 250 1.090658 \n", + " 29 250 1.090658 \n", + " 30 250 1.090658 \n", + " 31 250 1.090658 \n", + " 32 250 1.090658 \n", + " 33 250 1.090658 \n", + " 34 250 1.090658 \n", + " 35 250 1.090658 \n", + " 36 250 1.090658 \n", + " 37 250 1.090658 \n", + " 38 250 1.090658 \n", + " 39 250 1.090658 \n", + " 40 250 1.090658 \n", + " 41 250 1.090658 \n", + " 42 250 1.090658 \n", + " 43 250 1.090658 \n", + " 44 250 1.090658 \n", + " 45 250 1.090658 \n", + " 46 250 1.090658 \n", + " 47 250 1.090658 \n", + " 48 250 1.090658 \n", + " 49 250 1.090658 \n", + "B-K96 1 0 150 0.326040 \n", + " 1 150 0.326040 \n", + " 2 150 0.326040 \n", + " 3 150 0.326040 \n", + " 4 150 0.326040 \n", + " 5 150 0.326040 \n", + " 6 150 0.326040 \n", + " 7 150 0.326040 \n", + " 8 150 0.326040 \n", + " 9 150 0.326040 \n", + " 10 150 0.326040 \n", + " 11 150 0.326040 \n", + " 12 150 0.326040 \n", + " 13 150 0.326040 \n", + " 14 150 0.326040 \n", + " 15 150 0.326040 \n", + " 16 150 0.326040 \n", + " 17 150 0.326040 \n", + " 18 150 0.326040 \n", + " 19 150 0.326040 \n", + " 20 150 0.326040 \n", + " 21 150 0.326040 \n", + " 22 150 0.326040 \n", + " 23 150 0.326040 \n", + " 24 150 0.326040 \n", + " 25 150 0.326040 \n", + " 26 150 0.326040 \n", + " 27 150 0.326040 \n", + " 28 150 0.326040 \n", + " 29 150 0.326040 \n", + " 30 150 0.326040 \n", + " 31 150 0.326040 \n", + " 32 150 0.326040 \n", + " 33 150 0.326040 \n", + " 34 150 0.326040 \n", + " 35 150 0.326040 \n", + " 36 150 0.326040 \n", + " 37 150 0.326040 \n", + " 38 150 0.326040 \n", + " 39 150 0.326040 \n", + " 40 150 0.326040 \n", + " 41 150 0.326040 \n", + " 42 150 0.326040 \n", + " 43 150 0.326040 \n", + " 44 150 0.326040 \n", + " 45 150 0.326040 \n", + " 46 150 0.326040 \n", + " 47 150 0.326040 \n", + " 48 150 0.326040 \n", + " 49 150 0.326040 \n", + " 2 0 200 0.805258 \n", + " 1 200 0.805258 \n", + " 2 200 0.805258 \n", + " 3 200 0.805258 \n", + " 4 200 0.805258 \n", + " 5 200 0.805258 \n", + " 6 200 0.805258 \n", + " 7 200 0.805258 \n", + " 8 200 0.805258 \n", + " 9 200 0.805258 \n", + " 10 200 0.805258 \n", + " 11 200 0.805258 \n", + " 12 200 0.805258 \n", + " 13 200 0.805258 \n", + " 14 200 0.805258 \n", + " 15 200 0.805258 \n", + " 16 200 0.805258 \n", + " 17 200 0.805258 \n", + " 18 200 0.805258 \n", + " 19 200 0.805258 \n", + " 20 200 0.805258 \n", + " 21 200 0.805258 \n", + " 22 200 0.805258 \n", + " 23 200 0.805258 \n", + " 24 200 0.805258 \n", + " 25 200 0.805258 \n", + " 26 200 0.805258 \n", + " 27 200 0.805258 \n", + " 28 200 0.805258 \n", + " 29 200 0.805258 \n", + " 30 200 0.805258 \n", + " 31 200 0.805258 \n", + " 32 200 0.805258 \n", + " 33 200 0.805258 \n", + " 34 200 0.805258 \n", + " 35 200 0.805258 \n", + " 36 200 0.805258 \n", + " 37 200 0.805258 \n", + " 38 200 0.805258 \n", + " 39 200 0.805258 \n", + " 40 200 0.805258 \n", + " 41 200 0.805258 \n", + " 42 200 0.805258 \n", + " 43 200 0.805258 \n", + " 44 200 0.805258 \n", + " 45 200 0.805258 \n", + " 46 200 0.805258 \n", + " 47 200 0.805258 \n", + " 48 200 0.805258 \n", + " 49 200 0.805258 \n", + " 3 0 250 1.360132 \n", + " 1 250 1.360132 \n", + " 2 250 1.360132 \n", + " 3 250 1.360132 \n", + " 4 250 1.360132 \n", + " 5 250 1.360132 \n", + " 6 250 1.360132 \n", + " 7 250 1.360132 \n", + " 8 250 1.360132 \n", + " 9 250 1.360132 \n", + " 10 250 1.360132 \n", + " 11 250 1.360132 \n", + " 12 250 1.360132 \n", + " 13 250 1.360132 \n", + " 14 250 1.360132 \n", + " 15 250 1.360132 \n", + " 16 250 1.360132 \n", + " 17 250 1.360132 \n", + " 18 250 1.360132 \n", + " 19 250 1.360132 \n", + " 20 250 1.360132 \n", + " 21 250 1.360132 \n", + " 22 250 1.360132 \n", + " 23 250 1.360132 \n", + " 24 250 1.360132 \n", + " 25 250 1.360132 \n", + " 26 250 1.360132 \n", + " 27 250 1.360132 \n", + " 28 250 1.360132 \n", + " 29 250 1.360132 \n", + " 30 250 1.360132 \n", + " 31 250 1.360132 \n", + " 32 250 1.360132 \n", + " 33 250 1.360132 \n", + " 34 250 1.360132 \n", + " 35 250 1.360132 \n", + " 36 250 1.360132 \n", + " 37 250 1.360132 \n", + " 38 250 1.360132 \n", + " 39 250 1.360132 \n", + " 40 250 1.360132 \n", + " 41 250 1.360132 \n", + " 42 250 1.360132 \n", + " 43 250 1.360132 \n", + " 44 250 1.360132 \n", + " 45 250 1.360132 \n", + " 46 250 1.360132 \n", + " 47 250 1.360132 \n", + " 48 250 1.360132 \n", + " 49 250 1.360132 \n", + "C-K96-G32 1 0 182 0.184889 \n", + " 1 182 0.184889 \n", + " 2 182 0.184889 \n", + " 3 182 0.184889 \n", + " 4 182 0.184889 \n", + " 5 182 0.184889 \n", + " 6 182 0.184889 \n", + " 7 182 0.184889 \n", + " 8 182 0.184889 \n", + " 9 182 0.184889 \n", + " 10 182 0.184889 \n", + " 11 182 0.184889 \n", + " 12 182 0.184889 \n", + " 13 182 0.184889 \n", + " 14 182 0.184889 \n", + " 15 182 0.184889 \n", + " 16 182 0.184889 \n", + " 17 182 0.184889 \n", + " 18 182 0.184889 \n", + " 19 182 0.184889 \n", + " 20 182 0.184889 \n", + " 21 182 0.184889 \n", + " 22 182 0.184889 \n", + " 23 182 0.184889 \n", + " 24 182 0.184889 \n", + " 25 182 0.184889 \n", + " 26 182 0.184889 \n", + " 27 182 0.184889 \n", + " 28 182 0.184889 \n", + " 29 182 0.184889 \n", + " 30 182 0.184889 \n", + " 31 182 0.184889 \n", + " 32 182 0.184889 \n", + " 33 182 0.184889 \n", + " 34 182 0.184889 \n", + " 35 182 0.184889 \n", + " 36 182 0.184889 \n", + " 37 182 0.184889 \n", + " 38 182 0.184889 \n", + " 39 182 0.184889 \n", + " 40 182 0.184889 \n", + " 41 182 0.184889 \n", + " 42 182 0.184889 \n", + " 43 182 0.184889 \n", + " 44 182 0.184889 \n", + " 45 182 0.184889 \n", + " 46 182 0.184889 \n", + " 47 182 0.184889 \n", + " 48 182 0.184889 \n", + " 49 182 0.184889 \n", + " 2 0 264 1.464546 \n", + " 1 264 1.464546 \n", + " 2 264 1.464546 \n", + " 3 264 1.464546 \n", + " 4 264 1.464546 \n", + " 5 264 1.464546 \n", + " 6 264 1.464546 \n", + " 7 264 1.464546 \n", + " 8 264 1.464546 \n", + " 9 264 1.464546 \n", + " 10 264 1.464546 \n", + " 11 264 1.464546 \n", + " 12 264 1.464546 \n", + " 13 264 1.464546 \n", + " 14 264 1.464546 \n", + " 15 264 1.464546 \n", + " 16 264 1.464546 \n", + " 17 264 1.464546 \n", + " 18 264 1.464546 \n", + " 19 264 1.464546 \n", + " 20 264 1.464546 \n", + " 21 264 1.464546 \n", + " 22 264 1.464546 \n", + " 23 264 1.464546 \n", + " 24 264 1.464546 \n", + " 25 264 1.464546 \n", + " 26 264 1.464546 \n", + " 27 264 1.464546 \n", + " 28 264 1.464546 \n", + " 29 264 1.464546 \n", + " 30 264 1.464546 \n", + " 31 264 1.464546 \n", + " 32 264 1.464546 \n", + " 33 264 1.464546 \n", + " 34 264 1.464546 \n", + " 35 264 1.464546 \n", + " 36 264 1.464546 \n", + " 37 264 1.464546 \n", + " 38 264 1.464546 \n", + " 39 264 1.464546 \n", + " 40 264 1.464546 \n", + " 41 264 1.464546 \n", + " 42 264 1.464546 \n", + " 43 264 1.464546 \n", + " 44 264 1.464546 \n", + " 45 264 1.464546 \n", + " 46 264 1.464546 \n", + " 47 264 1.464546 \n", + " 48 264 1.464546 \n", + " 49 264 1.464546 \n", + " 3 0 346 1.695728 \n", + " 1 346 1.695728 \n", + " 2 346 1.695728 \n", + " 3 346 1.695728 \n", + " 4 346 1.695728 \n", + " 5 346 1.695728 \n", + " 6 346 1.695728 \n", + " 7 346 1.695728 \n", + " 8 346 1.695728 \n", + " 9 346 1.695728 \n", + " 10 346 1.695728 \n", + " 11 346 1.695728 \n", + " 12 346 1.695728 \n", + " 13 346 1.695728 \n", + " 14 346 1.695728 \n", + " 15 346 1.695728 \n", + " 16 346 1.695728 \n", + " 17 346 1.695728 \n", + " 18 346 1.695728 \n", + " 19 346 1.695728 \n", + " 20 346 1.695728 \n", + " 21 346 1.695728 \n", + " 22 346 1.695728 \n", + " 23 346 1.695728 \n", + " 24 346 1.695728 \n", + " 25 346 1.695728 \n", + " 26 346 1.695728 \n", + " 27 346 1.695728 \n", + " 28 346 1.695728 \n", + " 29 346 1.695728 \n", + " 30 346 1.695728 \n", + " 31 346 1.695728 \n", + " 32 346 1.695728 \n", + " 33 346 1.695728 \n", + " 34 346 1.695728 \n", + " 35 346 1.695728 \n", + " 36 346 1.695728 \n", + " 37 346 1.695728 \n", + " 38 346 1.695728 \n", + " 39 346 1.695728 \n", + " 40 346 1.695728 \n", + " 41 346 1.695728 \n", + " 42 346 1.695728 \n", + " 43 346 1.695728 \n", + " 44 346 1.695728 \n", + " 45 346 1.695728 \n", + " 46 346 1.695728 \n", + " 47 346 1.695728 \n", + " 48 346 1.695728 \n", + " 49 346 1.695728 " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "all_activation_rows = []\n", + "for method, trace in traces.items():\n", + " all_activation_rows.extend(activation_rows(method, trace))\n", + "per_neuron_table = pd.DataFrame(all_activation_rows).set_index([\"method\", \"layer\", \"neuron\"])\n", + "\n", + "layer_summary = (\n", + " per_neuron_table.reset_index()\n", + " .groupby([\"method\", \"layer\"], sort=False)\n", + " .agg(\n", + " neurons=(\"neuron\", \"count\"),\n", + " zero_crossing_fraction=(\"crosses_zero\", \"mean\"),\n", + " mean_tanh_affine_rho=(\"tanh_affine_rho\", \"mean\"),\n", + " mean_tanh_prime_interval_radius=(\"tanh_prime_interval_radius\", \"mean\"),\n", + " mean_tanh_prime_affine_rho=(\"tanh_prime_affine_rho\", \"mean\"),\n", + " mean_tanh_prime_selected_rho=(\"tanh_prime_selected_rho\", \"mean\"),\n", + " max_tanh_prime_selected_rho=(\"tanh_prime_selected_rho\", \"max\"),\n", + " mean_quadratic_core_box_radius=(\"quadratic_core_box_radius\", \"mean\"),\n", + " max_quadratic_core_box_radius=(\"quadratic_core_box_radius\", \"max\"),\n", + " mean_propagated_J_remainder=(\"propagated_J_remainder_mean\", \"first\"),\n", + " quadratic_neurons=(\"approximation_degree\", lambda values: int(np.sum(np.asarray(values) == 2))),\n", + " J_terms_after_activation=(\"J_terms_after_activation\", \"first\"),\n", + " J_degree_after_activation=(\"J_degree_after_activation\", \"first\"),\n", + " J_noise_after_activation=(\"J_noise_after_activation\", \"first\"),\n", + " activation_runtime_s=(\"activation_runtime_s\", \"first\"),\n", + " )\n", + ")\n", + "display(layer_summary)\n", + "\n", + "# Full per-neuron tables for the three primary K=96 variants.\n", + "primary_neuron_table = per_neuron_table.loc[[\"A-K96\", \"B-K96\", \"C-K96-G32\"]]\n", + "display(primary_neuron_table)" + ] + }, + { + "cell_type": "markdown", + "id": "83abbc1b", + "metadata": {}, + "source": [ + "## Plots" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "7bfd3365", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAACJQAAAJiCAYAAACbqwq8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAT/gAAE/4BB5Q5hAABAABJREFUeJzs3Xd8jef/x/HXyV4SK0RixqYUtWcQRc1Sq1p7Fh1qlQpaVJWiVUqrjZr1LYoWNWPF3rVri9iyI/P8/sgvpz2SIBKO8X4+HufBue7rvu7PfSXizn0+9+cyGI1GIyIiIiIiIiIiIiIiIiIiIiIi/8/K0gGIiIiIiIiIiIiIiIiIiIiIyLNFCSUiIiIiIiIiIiIiIiIiIiIiYkYJJSIiIiIiIiIiIiIiIiIiIiJiRgklIiIiIiIiIiIiIiIiIiIiImJGCSUiIiIiIiIiIiIiIiIiIiIiYkYJJSIiIiIiIiIiIiIiIiIiIiJiRgklIiIiIiIiIiIiIiIiIiIiImJGCSUiIiIiIiIiIiIiIiIiIiIiYkYJJSIiIiIiIiIiIiIiIiIiIiJiRgklIiIiIiIiIiIiIiIiIiIiImJGCSUiIiIiIiIiIiIiIiIiIiIiYkYJJSIiIiIiIiIiIiIiIiIiIiJiRgklIiIiIiIiIiIiIiIiIiIiImJGCSUiIiIiIpIhBoMh1ZeLi0uKvqdOnaJly5Zky5YNZ2dnatWqxaZNm1IdNzQ0lAEDBuDl5YWDgwOlS5dm5syZGI3GJ31KIiIiIiIiIiIiIi89G0sHICIiIiIiz79atWrRq1cvszZbW1uz92fPnqV69erY2NgwZMgQ3Nzc+OGHH2jYsCFr1qzB19fX1Dc2NpYGDRpw8OBBBgwYQMmSJVmzZg3vvfce169fZ/To0U/jtEREREREREREREReWgajHu8TEREREZEMMBgMdO7cGX9//wf2a9u2LUuXLmX//v2UK1cOgIiICEqXLo2DgwMnT57EYDAAMGPGDPr168c333zDgAEDTGO0bt2aVatWcebMGQoUKPCkTklERERERERERETkpaclb0REREREJFPExsYSERGR6rbIyEhWrlyJj4+PKZkEwMXFhR49enD69Gn27t1ral+4cCFOTk707NnTbJwPP/yQuLg4fv311ydyDiIiIiIiIiIiIiKSREveZJKrV6/yxx9/4O3tjbOzs6XDEREREXmuRUZGcu7cOZo2bYqnp6elw5FH8NtvvzF//nwSEhJwd3enXbt2jB07Fjc3NwCOHDlCTEwM1apVS7Fv1apVAdi7dy+VK1cmMTGRAwcOUKFCBRwcHMz6Vq5cGYPBYJZ8kh66bhcRERHJXLp2lydF1+4iIiIimedxr9uVUJJJ/vjjD3r37m3pMEREREReKLNmzaJXr16WDkMeonLlyrRp04YiRYoQFhbG6tWrmT59Olu2bCEwMBAXFxeuXr0KgJeXV4r9k9uCgoIAuHv3LtHR0an2tbe3J2fOnKa+D3L58mWuXLli1vb7778zceLEdJ+jiIiIiDyYrt0ls+meu4iIiEjmS+91uxJKMom3tzeQ9AUoU6aMhaMREREReb4dPXqU3r17m66x5Nm2e/dus/edOnWibNmyjBgxgmnTpjFixAiioqKApISQ+yVXIUnu86C+yf2T+zzInDlzGDNmTKrbJk+eTMmSJR86xssgISGB6OhoHB0dsba2tnQ4LxXNveVo7i1Hc285mnvLeBnm/cSJE3z88ce6dpdM96j33OPj44mIiMDFxQUbG33k8V+am9RpXtKmuUmb5iZtmpvUaV7SprlJ25Ocm8e9566vUAbExsYSFxcHgJWVFQBlypRJtYy3iIiIiKSfyho/vwYPHsyYMWP4888/GTFiBE5OTgDExMSk6Hvv3j0AU58H9U3un9znQbp3707Dhg3N2pJ/capUqZJpqZ2XXVxcHGFhYbi6umJra2vpcF4qmnvL0dxbjubecjT3lvEyzLuLiwuga3fJfMnfUw+75x4XF0doaChubm4v7L+zx6W5SZ3mJW2am7RpbtKmuUmd5iVtmpu0PY25Se91uxJKMmD8+PFpPvEoIiIiIvIys7W1xdPTk1u3bgGY1uVMbama5LbkJW6yZcuGo6Njqn1jYmK4desWderUeWgM+fLlI1++fKlus7Gx0S+s/5E8H5qTp09zbzmae8vR3FuO5t4yXvR511OlIiIiIiIvLitLB/A8Gz58OBEREURERLBx40ZLhyMiIiIi8sy4d+8eV65cIXfu3EDSU4X29vbs3LkzRd9du3YBULFiRSCp+l+FChU4ePBgiiole/bswWg0mvqKiIiIiIiIiIiIyJOhhJIMsLOzw9nZGWdnZxwdHS0djoiIiIjIU3f79u1U20eOHEl8fDzNmjUDkkqhN2vWjICAAA4fPmzqFxERwY8//kjRokWpXLmyqb1Dhw5ERUUxe/Zss3GnTp2KjY0N7dq1ewJnIyIiIiIiIiIiIiLJVI/wKYmPj+fatWvExMSQmJho6XBEXkhWVlbY29vj4eGhcqsiIiJPydixY9m1axd169Ylf/78REREsHr1ajZv3kyVKlUYMGCAqe8XX3zBxo0bef311/noo49wdXXlhx9+ICgoiD///BODwWDq27NnT37++WcGDhzIhQsXKFmyJKtXr2b58uV8+umnFCxY0AJnKyIiIiIiIiIiIvLy0CeuT0F8fDyXLl0iJiYGg8GAlZWV2c1yEck4o9FIXFwcsbGxxMbGkj9/fiWViIiIPAU+Pj4cP36cuXPncvv2baytrSlatCjjxo1j4MCBODg4mPoWKVKEHTt2MGzYMCZMmEBsbCwVKlRg7dq1+Pr6mo1rZ2fHhg0b+PTTT1m0aBG3b9+mcOHCfPvtt/Tr1+9pn6aIiIiIiIiIiIjIS0eftj4FyZVJXF1d8fT0VDKJyBNiNBq5evUqYWFhXLt2jbx581o6JBERkRdeixYtaNGixSP3L1myJCtWrHikvlmzZmX69OlMnz79ccMTEREREZEXXFxcHKGhody9e9fSoTxzEhMTiY+P5/bt21hZWVk6nExhMBiwt7cnS5YsuLm56fMWERGRJ+zFuIJ4xiVXJlEyiciT9d9/ZzExMZYOR0RERERERERERJ6g5AfM7t27R0JCgqXDeeYYDAZsbGxeqM8lEhISiIiIIDg4mKtXr2I0Gi0dkoiIyAtNFUqegsTERC1zI/KUJC8rlZiYaOlQRERERERERERE5Am6e/cusbGxuLi44OXlpSWw75OYmEhCQgLW1tYvTIUSo9HIvXv3CA4OJiwsDBcXF9zc3CwdloiIyAvrxbiCeA4omUTk6dG/NxERERF5mb333nsYDAZ27tz5wH4+Pj5MmjTJ9P7SpUuUKFGC7t27k5CQQGRkJP3798fT0xM3NzfatGnD7du3zcY4efIkb7zxBq6urmTLlo02bdo8kXMSERERSU14eDgAOXPmfGESJuTBDAYDjo6O5MmTB4CwsDALRyQiIvJi0xWWiIiIiIiIyAsiOjqahQsXkj17dn788cdH3u/EiRPUqFGD5s2bM2fOHKytrRk0aBDHjh3jyJEjXL16FXt7e9555x3TPteuXcPHx4cWLVoQHBzM9evXGTp06JM4LREREZFUxcXFYWVlha2traVDkafMwcEBKysrLX0uIiLyhCmhREREREREROQF8b///Q8rKyumTp3KkiVLiIiIeOg+u3fvplatWgwYMICJEyea2pcuXcqwYcPImTMnzs7OjB07lrVr13Lx4kUApkyZgo+PD71798bZ2Rk7OzsqVqz4xM5NRERE5H5Go1HLzb+kDAYDBoMBo9Fo6VBEREReaEookYfq0qULPj4+lg4j0xQsWJDRo0dbOgwREREREZEMiYiJYPXxlYz961OG/zGQsX99ysRvvqR1m9a0bdsWW1tbFi9e/MAxtmzZQsOGDZk4cSJDhgwx22Y0Gs1u0Cf//fDhwwAEBATg6upKrVq1yJEjB1WrVmXz5s2ZfJYiIiIiIqlTIpGIiDyvIuMi+evCX3y550tGB47myz1fsu7COqLioiwdWgpKKJGHmjZtGsuWLUvXPv7+/s/txVz+/Pn5+OOPzdrWr1+PwWCgd+/eZu07d+7EYDAQGBgIwPLly2ncuDEeHh4YDAb8/f1TjP/TTz9Rt25dcubMSZYsWXjttddYsGDBEzsfERERERF58QSc2UC/37oxf99PHL/+NxfunGPX4V0c23+cIM9z7Ly0jfbt2zNnzpwHjrNlyxayZ89Os2bNUmxr2rQpX3zxBdevXycsLIzhw4djMBhM69Tfvn2bhQsXMm7cOK5du0bv3r1p1qwZV65ceSLnLCIiIiIiIiLyvNt2ZRsfbPqAGYdmsPbCWjZd2sTaC2v57tB3DN06lCM3j1g6RDNKKMmA2NhYIiMjiYyMJDo62tLhPDFubm5kz57dYsePj49/qmXr6tevz4YNG8zaNm7cSP78+VNtd3V1pXLlygBERERQuXJlvv/++zTH37RpEy1atGDNmjUcPHiQ9u3b8+677/Lrr79m/smIiIiIiMgLJ+DMBmbvnE5cQqxZ+9ktF3HN40LWQlmYvXM6ZeqVYteuXRw7dgwXFxfT678J7Z988gnlypXDx8eH69evm403depUihcvTsWKFSlZsiSVK1fGxcWFHDlyAJAlSxZatmxJ7dq1sbW1pWvXrhQsWJBNmzY9+UkQEREREREREXnObLuyjZmHZ3Im5AyJxkS8XLwonLUwXi5eJBoTORt6lhX/rGDn1Z2WDtVECSUZMH78eNMNufr161s6nCfm/iVvRo8eTcGCBVmxYgUlSpTA2dkZHx8fzpw5AySVPe7atSvw7zqG/91/8eLFlC9fHgcHBwoUKMBHH31EZGRkiuPNmDEDb29v7O3tWbp0KVZWVly4cMEstj///BNra2uCgoIA+OabbyhfvjxZsmQhV65cNG/enH/++Sdd5+vr68vRo0e5ceOGqW3jxo0MHTqUK1eumMWwceNG6tSpg42NDQDvvvsuY8aMoWXLlmmOP3/+fD788EMqVapEkSJFGDx4MI0aNWLJkiXpilNERERERF4+ETER/Lxndor2xPhELuy4TOStKJa/v5bl769lYI9BAMyZM4eIiAjTq2PHjqb9bG1tWbJkCaVKlaJOnTpcvXrVtM3NzY3Zs2dz+fJlgoKCeP3114mLi6Nq1aoAvPrqq89tZUoRERGR51FAQECalbFFRETk2RYZF8ncY3MJigjC08UTdyd37KztMBgM2Fnb4e7kjoezB7fu3WLhiYVExkU+fNCnQAklGTB8+HDTDbmNGzc+1hjffPMNRYsWfejr66+/NtvvwoULj7TffxM5kmXGmtY3btxg5syZLFiwgMDAQEJDQ+nWrRsA1atXZ+rUqQAEBwcTHBxsWjLH39+fvn378tFHH3Hs2DHmzZvHxo0b6dGjh9n4+/fvZ/Xq1SxdupQjR47QrFkz8ufPz7x588z6zZ07lwYNGuDl5QVAYmIiEydO5PDhw6xevZq4uDiaNm1KXFzcI59b/fr1MRqNpqfqQkJCOHDgAM2aNaNq1aqmKiXR0dHs3LkTX1/f9E/gfUJCQnB2ds7wOCIiIiIi8vh8fHywt7c3PThQokSJB96sHz16NE2bNjW9j4mJ4c0336RixYrcvHkTgOnTp1OkSBFcXFyoVq0aBw8eNBsjLCyM9957D3d3d7JkyUL58uVN+wIEBgbyxhtvkC1bNtzc3HjlldIcXHaU+Jh4s3FOrDnDvdAYEuISiY9JwNbRloI18lGhRnmmTp1qWqZz06ZNlChRAoPBwJ49ezh48CA2NjZUqFCBCxcuUKVKFUaMGIGnpydWVlZYWVnh5eXFpEmT6NatG4MGDSJbtmwYjUacnZ1ZsGABjo6OeHh4UKBAAY4dO8aAAQMoWbIkgwYNMktQ+eOPP/D29sbGxgYrKyscHBxS7Xfq1Clq166Nk5MT3t7eWiJURERE5AW1ZMkSunbtyquvvoqtrS0GgyHFQ6UiIiIvih1BO7gZfRNXO1ccbRxT7eNo44izjTO37t0i8GrgU44wdUooyQA7OzucnZ1xdnbG0TH1L/rD3Llzh3/++eehr9u3b5vtFxcX90j7pXbxFRUV9Vix/te9e/eYO3cur732Gq+++iqDBg1ix44d3Lt3Dzs7O9zc3ADw8PDAw8PDtGTO6NGjGTduHJ06daJw4cLUrl2b6dOns3jxYm7dumV2jAULFlC+fHlKly6Nvb09nTp14pdffjFtDwkJYdWqVXTu3NnU9uGHH9KgQQO8vb2pWLEic+fO5dSpU+zdu/eRz83Dw4PSpUubEkcCAgIoXLgw+fLlo27duqb2HTt2EBMTk+GEkrlz57Jv3z4+/PDDDI0jIiIiIiIZN27cOCIiIggPD2fGjBn07t2bv//++6H7hYWF0ahRI0JDQ9m8eTPu7u78+uuvjBs3jt9//53Q0FDeeecdGjVqRFhYGJCUEP/OO++QmJjIiRMnCA0N5ZdffsHJyQmA1atXU69ePSpVqsTx48cJDQ2l1YimRIfcI/z6v0+pXD18nSNLT+Ls7oS1nTVtZjWhzsAqXDt6g1PHTuPi4sKVK1c4f/48zZs3p0OHDjg5OVG8eHGWLl3K22+/zcyZM9m/fz9Zs2Zl0qRJ3L17l0GDBpE7d25u3rzJkCFDqFSpEp999hkAfn5+HDhwgC+//BJnZ2euX79OfHy86Vw3bNhArly52LFjB5D0UMKbb75JUFAQX3/9Ndu3bydr1qz069fPrF98fDzNmzenZs2a3Llzh1mzZtGrVy/27duXqV9nEREREbG8GTNmsHjxYhwdHSlcuLClwxEREXmiDt44SFhsGG72bg/s52zrTHhsOAdvHHxgv6dFCSUWlj17dooUKfLQV/Ia1clsbW0fab+CBQumOGbyzcmMyJMnD7lz5za99/T0xGg0mi0Tc7+bN29y8eJFPv74Y7P1uxs3bgxgWjIHoESJEqaklGSdOnXin3/+MT1Zt2TJEuzt7XnzzTdNfbZu3cobb7xBgQIFyOLignehQgD8s3s3sWfPEnv2LMTHk3Dnjul9aq+6lSqZqs5s3LiRevXqAVCvXj02bdqE0Whk48aNeHp6UqpUqceexxUrVtCnTx/mzJlDhQoVHnscERERERFJv4iYCFYfX8nYvz5l+B8DuXjnPCevHyMiJgKDwUC9evXInz8/x48ff+A4N27coG7dumTPnp01a9aQJUsWAJYuXUqXLl145ZVXsLa2pl+/fjg6OrJ8+XIA1q1bx8WLF/n222/JmTMnVlZWlClTBmdnZ4xGI/3796dTp06MGTOGPHnyAGCb3YZKnV8lW/6k35eMRiP75h2hcO0CVOleHoMBEhONHFt5msT4RLp83YGwsDDatm3LmjVrKFOmDHXq1MFgMLBv3z7y5s3L6tWr2bFjB56enpw5c4ahQ4dSrlw5Jk6cSHBwMLGxsXTq1InIyEgMBgN3797lq6++4ueff6ZPnz7ExsYyYcIEgoKCaNGiBQBeXl4MGTKENm3aAElVR+Lj4xk9ejTvv/8+1atXp1atWly5csWs39atW7l27RqjRo3CwcGBBg0a0Lx5c37++ec05z9lFZdX+Pzzz1N9mCIgIAAXFxezti+//BJ3d3fTgwibNm2iUqVKZMmShaJFi7Jo0SKz/kajkYkTJ+Lt7Y2zszOFCxd+pKqlf/zxB3Xr1sXV1ZWsWbOmqM5y8eJFqlWrRo4cOXBzc6NcuXKm7xWAXbt20ahRI9zd3cmePTv16tXjwIEDDz3uy+69997DYDCwc+eD15/28fFh0qRJpveXLl2iRIkSdO/enYSEBP7++28aN25Mrly5Un16OjIykv79++Pp6Ymbmxtt2rQxezho0qRJvPrqq7i6uuLl5UX//v3Nlv8VEZFnT0RMPH8eCeazVcf5ZNkRPlt1nNVHg4m4r1KcpC4hIeGhD7f+8ssvhIeHs2vXrkypBC4iIvIsi46PJiExAVsr2wf2s7GyISExgai4jBeJyAw2lg7gZff+++/z/vvvp3u/ggULmiVgpEfdunUfa7//srU1/0ZPXjc7MTExzX2St3399dc0aNAgxfbkZWuAVJd/KVKkCDVq1GDu3LlUr16dX375hbZt2+Lg4AAk3exp3Lgx7dq145NPPsEtOprEhATKv/EGselY8gagbrVqTPf35+zZs2zcuJExY8YAULVqVSIiIjhy5AgbN26kfv366Rr3vxYvXkzXrl2ZM2cOb7/99mOP8zCxZ89meAy7FyQ7PKh9uwyP4bX410yIRJ4V+p74V0bnQvPwrxdlLkREXnQBZzbw857ZxCXEmtqi4iLZf3kv/X7rRueKPYi/mLSMZ6VKldIcJzg4mJo1a1KnTh2+//57rK2tTduMRiNGo9Gsv9Fo5PDhwwBs2bIFb29vunfvzrp168jpnpN6rX3wqu3O5QtBnD9/nkJV8xMRE4GLvQsBZzZwJeSS2Xjh1yOJvBlF/qpe/z8+bP9mD9Eh9/AdURMPL49U40lMTKRVq1aEhISQP39+8uXLx5o1a7C3t+fAgQPs37+fAgUK0Lx5cyZMmIDRaOTIkSNAUmKDvb09S5YsYdq0aYSHh3Ps2DEiIyPTXMozIiICSPrdMSEhgZMnT7Jjxw4GDBhg1u/IkSOULFkSe3t7U1uFChVYuXJlquOuXr2aVq1aMXToUObMmUOePHk4d+4ckyZN4syZM7z66qtpfOWS5mPo0KH89ttvbNu2jRIlSpiquMydO5c333zTlMRRsGBBqlWrBiRVZ9mwYQNr1qyhWLFiBAUFPfB3YYCZM2cyaNAgJkyYwK+//kquXLkICgpiwYIF7NixgzZt2pAjRw7mzp1LkSJFsLKyIjAwkAYNGnDkyBEKFy7M3bt36dSpE4sWLSJLlixMmTKFRo0acf78eS2hmobo6GgWLlxI9uzZ+fHHH01fw4c5ceIEr7/+Oh06dGDixIlA0n2Q1q1bM3To0FTvqwwaNIiTJ09y5MgRHB0d6d27N++88w5r1qwBkqrv/PDDD5QrV447d+7Qrl07Bg8ezIwZMzLvhEVEJNNsPnWDH7ee43p4DKHRcSQkGLG2NrD1zE0W7LpIj9re1C2ey9JhAknXdV988QV//fUXp0+f5s6dO3h4eNCkSRM+++wzsmbNCiQlQefNm5c2bdqkuqRgv379mDlzJufOnTM9qBoaGsr48eNZunQply9fxtXVFV9fX8aNG4e3t7dpX39/f7p27cr69evZuXMn/v7+XLp0iR9++IEuXbqkGXv+/PkzcypERESeaY42jlhbWROXGIedtV2a/eIT47G2ssbJNuNFIjKDEkrkibCzS/pHkJCQYLqhmjt3bvLly8epU6fo27fvY43buXNnhgwZwgcffMCOHTtMN3YA9u7dS1RUFNOmTSNLlizEnj3L1t27H+s4dapUwcbGhnnz5nHq1CnTzSI7OzuqV6/O0qVLOXDgAP3793+s8X/44QcGDBjAvHnzTE/iiTwt+sBcREREXnYBZzYwe+f0VLcdXXaCY6tOsSBuKYnxRvz8/Cj0/5UPU3PmzBmio6P55ZdfzJJJAJo2bcrQoUNp3749pUuXZsaMGVy+fNm05M2dO3fYvn07kydPptunnZi6dDI/TfyZyuHlcHBNSqjYcWMzJ387QrWCNdl6dlOK48eExQDglNWB6NAYEuMTuXbsJmXeLIF9Fntey1fZ1LdBgwZ8/PHHbNq0iZiYGP766y+MRqPpydHbt2+bYjMYDAwbNoyffvqJzp07s3btWjw8PMz6nThxgi+//JKePXty/Phxhg8fzrRp0xgyZAizZ88mPj6eDh068MMPP3Dnzh0gqUrDp59+SkJCAlWrVqV58+Zm/cLDw00feiTLmjUr4eHhKc79/iouyby9vR/6IX1CQgLvv/8+Bw8eZMeOHeTLlw/AVMWldevWAFSvXp1WrVoxe/ZsqlWrZqrOcujQIYoXLw5A3rx5H3is8PBwhg4dip+fn1kCTXIVl2QuLi4UK1bMdG7JyTfnzp2jcOHCpuqeyQYNGsSIESM4deqUKl6SVHFo69lNHLi8h6i4KJxsnQg5EIGVlRVTp07lvffeY9q0aSkq1Nxv9+7dNGnShCFDhph9fYoXL07x4sUJCQlJdb+lS5cyb948cubMCcDYsWMpVKgQFy9epECBAgwbNszU18PDg169evHll19m/MRF5LmUvFx5o0aNTG27d+9m7Nix3Llzh86dO9OrVy8LRvhy23zqBlM3nObKnWjcHG3Jl80RW2sDcQlGQqLiOHktnKkbTgM8E0klsbGxfPXVV7Ru3ZoWLVrg7OzM3r17mTNnDtu3b2fXrl04OjqSK1cumjdvzrJlywgJCTG75rp37x4LFy7E19fXLJmkevXqXLp0iW7dulG6dGmCg4OZMWMGVapUYd++fRQoUMAslkGDBhEXF0fPnj1xdXU1XS+JiIgIlM9VnsCrgYTGhOLu5J5mv8i4SLLYZaF8rvJPMbq0KaFEnojk9Q6XL19OnTp1sLOzw83NjXHjxtGtWzdy5MjBm2++ia2tLSdPnuSPP/7ghx9+eOi4bdu25f3336djx44ULVqU6tWrm7YVK1YMg8HA5MmT6dy5M39v2YLf5MmPFX8WFxcqVarE119/TdmyZU03hCBp2ZsJEyaQkJCQogzfnTt3uHTp3ycGL126xKFDh3BxcaFIkSIATJkyxfQUUq1atbh27RoA1tbWuLun/cNDRETkaVLymYi8qCJiIvh5z+w0t5dpVZKSbxTBaDQSeTWamdNmkjt3blauXMm2bdsAeOedd/j+++8BqF27NnXq1KFhw4asXr2aGjVqmMbq1KkT165do127dty6dYuWLVvi6+trWtI0S5Ys5MmTh/JvlOGHPd/hVtCFAlXzEnToOqWaJP3+EBVyD1fP2BTJJMdWneb4qtMkF0CJCrmHwWDA2taK2h9VZevUXdjZ2bL74j7afNkBgAIFCrB06VIGDhxIYmIixYsX58SJE8THx5viAZg6dSqnTp1i1KhRnDx5kr///ptevXqxaNEiXFxcSEhIAODzzz/n7P9XROzatSuTJ09m2rRphIWFER8fT0xMDKtXrwbgypUrprHffvttLl++bKrQcPPmTdOH9FmyZCE0NNTsXENCQsiSJUuKhIGo69GcP3+eFq1aPNLXPlliYiJt2rQhKCiIzZs3m5YTgrSryqRWnWXWrFnY2NiYqrikVSUkMDCQ8PDwR36YoGzZspw8eZK4uDjq1auHj49Pqv22b9+OjY0NRYsWfaRxX2SpVRwC2OC/HfcK2cldIQe2trYsXryYHj16pDnOli1bGDt2LF9//TXdunVLVwz3f+8k//3w4cMpPmyDpKWVypYtm65jiMiLY+jQody5c8eUUHLr1i0aN25MREQEjo6O9O3bl1y5ctGyZUvLBvoSioiJ58et57hyJ5q82RxxtPs3YdjOxkAuV3uyONhw5U40P249R6WC2XGxt+zHLPb29gQHB+Po6Ghq69OnD9WrV6dHjx6sWLGC9u3bA9CrVy+WLl3KwoULee+990z9ly5dSkhIiNn/k35+fpw7d45du3aZVX3r0qULZcqUYdSoUfj7+5vFEh0dzcGDB3FyejaeqBYREXmW1PCqwZJTSzgTcgYXOxccbRxT9ImOjyYxPpGcWXJSw7NGKqM8fVaWDkBeTFWqVOHjjz+mf//+5M6d27SO9rvvvstvv/3G6tWrqVy5MhUrVmTUqFFmy908iJubGy1btuTQoUN07tzZbFuZMmWYOXMmP//8MyVLlmT0lClMGTXqsc/B19eX8PBw6tWrZ9Zet25dwsPDKVmyJJ6enmbbVq5cSfny5SlfPiljbNSoUZQvX97sQnzatGkkJCTQu3dv8uTJY3o9qIy2iIiIiIhkjm1nN6X40Dk1BoMBFy8nir1ahNWrV7NmzRoiIiKIiIgwJZMkGzx4MGPGjKFRo0Zs3brVbIyhQ4dy6tQpbt++zcyZMzl27JgpQSD5w+T5+39OcfwsHi44uztxaXdQqvGVblaMNrOb0mZ2kxT9cpfMSZ2B1fh72Wlcnd1McR87dowmTZowa9YsnJ2dOXz4MDVq1CA4OJivv/7a9EGBwWCgSZMm7Nu3D39/f/LkycPVq1fp2rUrERERnDhxwtSvWrVquLi4sGvXLtPxv//+eyIiIujXr59pudO4uDisrKwIDg7GysqKAgUK0LFjR1atWmV2XmXLluX48ePExv77NTp48CDZ82ej32/dmL/vJ45f/5sLd85x7PzfAPxw9FsCzmx4wFfTXGxsLJs2baJFixZmDw9AUhWXQ4cO8euvvxIfH8/WrVtZunSpqXLLf6uznDx5ksDAQHbv3s3w4cPTPN6tW7cAzH5/HDJkCFmzZsXFxYWePXua9T9y5AgRERGsWrWKxo0bp6h8A0lLLb377ruMGzfOlAj0skquOHT/v+uwaxHcPH2bAjU88d8/m1qNazJnzpwHjrVlyxayZ89Os2bN0h1H06ZN+eKLL7h+/TphYWEMHz4cg8Fg+t75rwULFrB8+XI+++yzdB9HRF4M+/btM3tQbdGiRYSFhXHgwAFu3rxJlSpVmDZtmgUjfHltPX2T6+ExuDnamiWT/JejnTVujjZcD49h2+mbTznClAwGgymZJCEhgZCQEG7dumW6r713715T3wYNGlCoUKEU/yfOmTOHHDlymJKYjEYjCxYsoHbt2nh5eXHr1i3Ty9nZmapVq7Ju3boUsfTt21fJJCIiImlwtnWmc+nOeLl4cTXiKjejbhKbEIvRaCQ2IZabUTe5FnmNnA45ebvk28/MkjdKKJGH8vf3JyAgwPR+9OjRXLhwwayPj48PRqPRVA4PkkoZX7t2jcTERLP9W7Rowc6dO4mKiiI8PJxDhw4xevToNI93v0WLFmE0GhkxYkSKbb179+bixYvcu3ePnb//TvXXXiPmn3/o9P/ligFOb9nCyA8+eOh5f/bZZxiNRibfV+WkWrVqGI1Gjh8/nmKfLl26mJ5K+u/rv+dz4cKFVPvcP6ciIiIiIpL59l/e88h9Q4PCOLT3MGXKlHlo3w8//JAvvviCN954g02bkqqJhISEcPLkSYxGIzdv3qRPnz4UKVKEhg0bAtCyZUuiY6L4+68TJCYauXMxhIu7rpDvtTwYDAYqvluW8zsuc3T5SaJD7wEQeSuK/fOPcPdSUhWP//Y7t+0SRiMYMNCjQR/eaPwGo0aNMltWY+/evcTHx2M0GpkxYwYnTpzgt99+Y8yYMfz66680bNiQ0aNHs3XrVq5cucJXX31FoUKF2LZtGx999BEABQsWNPWztrZm2LBhLFy4kEKFCpmSJ4KDgzl16pTpuHXr1sXW1hY/Pz+mT5/O0aNHWbhwISVKlDDrV7t2bXLnzs1nn31GTEwMGzduZPmK5dwrFpoiYcA+S9JSq2G3w5m9c3qKpJLx48fj4uKCi4sLpUuXNrU7ODiwYsUKvvrqK6ZPN1/6qFixYixdupSvvvqKXLlyMXLkSLp3725WVQaSqrNkyZLFtGxNcmJMnz59TMdMXqImOWnl6tWrpuNMnDjR9BRwXFxciu8nOzs7mjZtyubNm5k3b57ZtmvXrlGvXj06dOhg+pq8rB5Ucejslou45nEhZ+HsAMQXj2LXrl0cO3bM9DVycXFhwYIFpn0++eQTypUrh4+PD9evX09XLFOnTqV48eJUrFiRkiVLUrlyZVxcXEzfO8l+++03PvjgA9auXYu3t3c6z1hEXhQ3b940SzRcu3YtNWrU4JVXXsHOzo727duneu9Rnrx9F+4SGh1HVifbB/bL6mRHWHQc+y7efUqRPdiSJUuoUqUKjo6OZMuWDXd3d9P/M3fv/hujwWCgR48eHDhwgEOHDgFw7tw5AgICePfdd01L2d+8eZPbt2+zbt063N3dU7zWr1+f6v+Vycv3iYiISOpq5a1F31f7UjRrUawMVgRFBHE25CxBEUFYGawo7FaYFkVaUM2zmqVDNclQLbYbN27w+++/ExAQwLFjx7hx4wYGgwF3d3deeeUVfHx8aNGiBblyWX4dQREREREREZGouKgHbj+y9AR//34SoxHsnG0pVaMEfn5+jzR2//79sbGxoVmzZixfvpxixYrRunVrLly4gJOTE61atWLVqlVYWSU92+Hq6kpXv3f55ZsF7P/1KI7ZHHi1TSm8ynsA4PlqbuoNrcGxlac4te4sGMEpuyMFqnqRJfe/y6sk9zuw8CgJsQn81nc1hwudoUOHDgwYMIAWLVoQGxvLyJEjGT58OIGBgURFRfHHH38QEBBAiRIlWL9+PQ0bNqR37978888/1KtXj8TERGxsbKhbty4BAQF4eHiYjjl//nz69OmDh4cHWbJkoXHjxoSFhVGoUCGsra3x9PTkjTfe4MMPPwSgRo0azJgxg1GjRvHhhx+axr5z5w7NmjUz9bOxsWHlypX07NmTyZMnkzt3bip2LkuOQtlSzPd/q7h4lHLHf89sKuaviou9CwDDhw9Ps3JInTp1WLRoER07dsRoNDJw4EDTtiZNmtCkSRPT+zZt2piqyvy3iktqvv/+e7MKNj4+PgQGBgJQokQJvL29GTZsGF26dEl1/9GjR7Nv3z7++OMPAGJiYhg/fjzffvsta9asISoqijJlymBlZcWECRMoXrx4irGmT5/O1KlTuXbtGmXKlGHGjBmmKpp//vknEydO5MiRI1hbW1O9enWmTJliWrb2eZNWxaHE+EQu7LhMXFQcy99fa7Ztzpw5REREpDqera0tS5YsoUOHDtSpU4dNmzalqEyaFjc3N2bP/je55dixYwwfPpyqVaua2v73v//Rt29f/vzzTypXrvxI44rIi8nZ2dm03FtCQgLbt2/n/fffN213dHRMtcKRPHnRcfEkJBixtU79//pkttYG4hONRMXGP6XI0rZs2TLatWtH5cqVmTZtGvny5cPBwYGEhAQaNWpEYmKiWf9u3boxatQo5syZw7fffstPP/2E0Wg0q7KdvHSbr68vQ4cOfeRYVJ1ERETk4WrlrUWF3BXYEbSDQzcPERUXhZOtE+Vzlaeye2XiolI+eGJJj5VQcuTIEcaNG8eKFSuIjY3F3t6evHnzUqBAAYxGI3fu3GHp0qUsWrSIAQMG0LJlS0aMGPFIT3WJiIiIiIiIPCkPKhda/5OaKdpKeZTBwcEh1f7/rbSYrE+fPvTp08f0/tixYw+Mx907J2+Mqk88qd8scC+aHZ+PH/5UinvR7DQcVccU86evf27a9t+nUtevX5/q/pUrVzbr9zA5c+bkt99+e+T+kPThRbdu3R7ar0SJEmzbtg2ANcdXMm/fT6n2S67Osu3bPThmdaBIvYJsO7eZUk5lmTx5Mj169DAtK5SaatWq8ccff9CsWTPi4uJMH5bs3buXcuXKERsby9y5c9m2bRvffvstYF6dJXlpn8mTJ/Pmm2+meZzx48djb2/PJ598wuuvv06vXr2oWLEiOXLk4NSpU+TOnRuAzZs3c+XKFRITE4mJiWHOnDls3LiRcuXKsXnzZsLDw2nQoIFp6di+ffumONavv/7KuHHjWL9+PSVLluT777+nUaNGnDlzBldXV0JDQxk0aBA+Pj7Y2toydOhQmjdv/tDv02dVWhWHgg5dIzYyjkaf1cHO2c7UHn00nnnz5jFhwgTTE9j3s7GxYdGiRbz77rv4+PiwadMm8ubNi9FoJCYmhpiYGCAp2efevXvY29tjMBg4f/48jo6O5M6dm5MnT9KtWzcGDRpEtmxJyVC//vor7733Hn/++SdVqlTJ5JkQkedN6dKl+eWXX+jUqRP/+9//iIiIMC0TB3Dx4kXc3d0tGOHLy9HWBmtrA3EJRuxs0k4qiUswYmNlwMkuQ8/sZop58+bh4ODA5s2bzRI6Tp48mWp/Dw8PmjVrxoIFC5gwYQL+/v5UqVLFrKKbu7s7WbNmJSwszGx5JhEREckczrbOvF7wdV4v+LpZe1xcHKGEWiiq1KV7yZtu3bpRoUIF9u/fz7Bhw9i5cydhYWGcOXOGXbt2sXv3bs6cOUNYWBg7d+5kyJAh7N27lwoVKphluIqIiIiIiIg8ba/lS19VgPT2Ty8H69STVTLiScf8ND1siaLk6iy3z93lz2EbebNiG5o0aULu3LkpUqTIQ8evVq0a69evZ8KECXz+eVISzvDhw8mRIwceHh6mKi73V2eJjo7Gw8OD8uXLU7lyZcaOHfvA4wwYMIAFCxZw5MgR4uPjqVKlCvXr16d06dKmfSMjI1m1ahXr1q0jd+7cDB48mMqVK7Nr1y6yZMnCDz/8wJkzZ9izZw9NmjTh1KlT9OnTx2zJlqVLl9KlSxdeeeUVrK2t6devH46OjixfvhyAt99+m2bNmpElSxYcHBwYMmQIx48f586dOw+dq2dRWhWHzm65SIGqXrh5ueKY1cH0euX1EsTHx/P7778/cFwbGxvmz59PlSpVqFOnDhcvXuTixYs4OjqavhdKlCiBo6MjFy9eBODw4cNUrlwZZ2dnmjRpwltvvcVnn31mGnPYsGGmD+X+u+SOiLycBg8ezNGjR8mVKxf9+vWjfPny1KpVy7R93bp1VKhQwYIRvrwqFsyGm6MtIQ95MjgkKhZXR1sqFkhZRe1ps7a2xmAwmFUiMRqND7w+6dmzJ3fv3qVPnz4EBQWl+OzGysqKjh07smfPnjSTiG/cuJE5JyAiIiLPtHSnzx4+fJjly5fTrFmzB/azs7OjSpUqVKlShc8++4wVK1YwZsyYxw5UREREREREJKNqFa7H4oPzU10m43521nbULlz3icZTLFcJDt7al2njPY2Yn6aHLVEE5lVcCmb3ZnzTr9Ps6+PjQ0REBHFx/35IdH91lrSquCR7UHWWiJgItp7dxIHLe4iKi+LinfOcvH6MiJgImjVrhqOjI/v27ePo0aMUKlTIbN+mTZvSt29fVq1aRXh4OHXq1OH777/H2toagFGjRjFq1ChT/3LlyvHhhx/SsWNHU5vRaDSVqP9v2+HDh1ONd9OmTXh5eZE9e/YHnvOzKq2KQ2lV9cnqli3NSjwBAQFm762trZk3b55Z2/1z+18tW7akZcuWaW4/f/58mttEnrTAwEDGjh3Lzp07SUxMJF++fLRr146PP/44xfIUAQEBNG3a1GxpqC+//JJJkyaxevVqKlWqhJ+fHytWrOD48eN07NiR3r17m42xadMmhg4dysmTJ/Hw8OCzzz6jQ4cOAJw+fZphw4YRGBhIdHQ0JUqUYNy4cY9UAeGPP/5g8uTJ7N+/HysrK/LkyUOTJk0YOHAgnp6eXLx4kfbt23P69Gni4+MpVKgQo0aNMlWR2rVrF6NHj2b//v0kJCRQrlw5Jk2aZJHEjSZNmrBp0yZWrFiBm5sb/fv3Ny2ndvv2bfLmzUunTp2eelwCtYu5s2DXRU5eCyeLgw2OdtYp+kTHJhAaHU8JjyzULmb5SjJvvfUWS5cupV69enTq1Im4uDh+//13oqLSvo5q2LAhBQoUYP78+bi4uNC+ffsUfcaNG8eOHTto27Ytbdu2pWrVqtjZ2XHx4kVWr17Na6+9hr+/f4Zi37p1K1u3bgVg376ka+Lp06eTNWtWAD799NMMjS8iIiIZl+4KJfv3739oMklqWrRowYEDB9K9n4iIiIiIiEhmcbF3oWvlXo/Ut0vlXjjbPdkKAmU9K2BrbZtp4z2NmJ+mBy1RlGp/O+cnFMnDBZzZQL/fujF/308cv/43F+6cIyouEv9pv5A9e3YcnRxp1KgRgwYNSpFM8l9nzpzh/PnzdO/e3ZRM8qiaNm2Kv78/hw4dIi4ujmnTpnH58mXCwsJS9D1+/DgffPAB06ZNS/e5PiuetYpDIs+i1atXU69ePSpVqsTx48cJDQ1l5cqVBAcHc+bMmQfuazQaGTRoEN999x3btm2jUqVKAHh7ezN+/HjefvvtFPucP3+e5s2bM2zYMEJDQ5k7dy69e/dm586dAISEhNCwYUOOHDnC3bt3+fDDD2nZsqWp2k9aZs6cSbt27WjVqhX//PMPISEhbNiwgVy5crFjxw4AcuTIwdy5c7l58yahoaHMmDGDd955h7NnzwJJS8B16tSJ06dPc+PGDRo3bkyjRo2IjIxM97xmhtq1azN58mT8/PzMEvty5MjBsmXLHpikJk+Oi70NPWp7kze7I1fuRnMj7B6x8YkYjUZi4xO5EXaPK3ejyZvdkR61vXG2t/ySN+3bt2f27NlEREQwaNAgJk6cSPHixfnrr7/S3MfKyoru3bsD0LZt21SrZrm5ubFjxw7GjBnDsWPH+OSTTxg6dCgrV66katWqqS6/l16bNm1i5MiRjBw5kt27dwMwefJkU5uIiIhYnuWvdkRERERERESeIp+iSU9B/7xndqqVSuys7ehSuZep35PkaOfIO6915Yc93z20b+3C9dh5YbvFY36aXstXmePX/05Xf0sIOLOB2Tunp7qtTKuSlHyjCEajEd/sTRjR04/cuXOzcuVKtm3bBsA777zD999/DyR9wFinTh0aNmzI6tWrqVGjxiPH0alTJ65du0a7du24desWLVu2xNfXlxw5cpj1O3nyJL6+vowfP57WrVs/5llb3rNWcUjE0u6vkuRo48iM3nPo0LGDWeVob29vZsyY8cCxEhIS6NGjB7t27WLHjh3ky5fPtK1Lly5AUkWnkJAQs/3WrFlDmTJlTD9bqlevTqtWrZg9ezbVqlWjcuXKVK7878/qDh064Ofnx/79+ylQoECqsYSHhzN06FD8/PwYMGCAqd3Ly4shQ4aY3ru4uFCsWDEgKRnGYDCQkJDAuXPnKFy4MI0bNzYbd9CgQYwYMYJTp05peRkxU7d4LgB+3HqO6+ExXLkbTXyiERsrA66OtpTwyEKP2t6mfk+Tj49PqhWzevbsSc+ePVO0JyQkkJCQkOpY9vb2ACmWu/kvJyenR0ru6NKli+lnQ3qMHj2a0aNHp3s/EREReXoyLaHkwoULbNiwgevXr9OxY0cKFixIbGws165dw8PDAzs7u8w6lIiIiIiIiEiG+BT1pWL+qmw7u4n9V/YSFRuJk50zr+WrTC3vurjYP70qHzUL+2CwNjxSgss7Fbs9EzE/Lc9DwkBETAQ/75n90H4Gg4FtYRuoVqMaq1evZs2aNWn2HTx4MLa2tjRq1Ig///yT2rVrP1IsBoOBoUOHMnToUABiY2MpVKgQH330kanPiRMnqF+/PiNHjqRPnz6PNO6zKrniUFrJPP/1olXvEblfwJkNKf4fCbsWwbUr1wnKfZ6AMxseOekwMTGRVq1aERwczLZt28iZM+cjx5HW0ltHjhxJtf+5c+e4dOkSZcqUSXPMwMBAwsPDadOmzSPFULZsWU6ePElcXBz16tXDx8cn1X7bt2/HxsaGokWLPtK4malevXoP7WMwGNi4ceNTiEZSU7d4LioVzM7W0zfZf/EuUbHxONnZULFANmoVc8flGahMkhHx8fHMmjWLMmXKUK1a6kvFiYiIiEAmJZQMHTqUr7/+moSEBAwGA9WqVaNgwYLcu3ePUqVKMXbsWD788MPMONQzJTY21rTucnR0tIWjERERERERkfRwsXehcanmNC7V3NKhPHKCy7MU89PwPCQMbDu76ZESXgBuXrrFzi0H6dPr4SXiP/zwQ2xsbHjjjTdYuXKl6cPHmJgY0we2cXFx3Lt3D1tbW6ytrQkJCeHatWsUL16cW7duMXToUIoUKULDhg2BpGVukpNJMqNM/bPgWao4JGIpaVVJigmLAcDO1dq0/VH+LcTGxrJhwwbGjBmTrmQSgAYNGvDxxx/z66+/0rp1awIDA1m6dCkeHh4p+iYnifTp0+eBSR23bt0CwNPT09Q2ZMgQZs+eTXx8PB06dOCHH34wbTty5AixsbGsW7eOkydPprp8WHBwMO+++y7jxo0jS5Ys6TrHzHDu3DkMBoNZW3x8PMHBwSQmJpIzZ06cnS23jJskcbG34Y0yeXijTB5Lh5Jpzp8/z86dO1mxYgXnzp1j0aJFlg5JREREnnFWGR1g1qxZfPXVV/Tr149169aZZaC7urrSvHlzVq1aldHDPJPGjx+Pi4sLLi4u1K9f39LhiIiIiIiIyHMsOVnk09c/Z3zTr/n09c9pXLLZC1l5JD18ivrSq1p/bK1Tr3xqZ21Hr2r9LZYwsP/yngduP7L0BP/r9QdLev7B5q92UrxmUfz8/B5p7P79+zNp0iSaNWvGunXrAChevDiOjo4cOXKEXr164ejoyLx58wAICQmhdevWuLi4UKpUKWxtbVm1ahVWVkm3f7766iuuX7/OkCFDTPczXFxcTEvvPK98ivry3Vs/8W7FbpTyKEPB7N6U8ijDu5W6M/2tn5RMIi+0B1VJss+S9HMzKuQeAP57ZhMRE2Ha/t97m6VLlza1Ozg48OeffzJmzBi++eabdMVTrFgxli5dyldffUWuXLkYOXIk3bt3T7H0VkREBE2aNKFkyZJMmTLF1N6nTx9TTMlL1CQntVy9etXUb+LEiYSEhNCjRw/TA3//ZWdnR9OmTdm8ebPpZ2Sya9euUa9ePTp06GBWwelpunDhAufPnzd7Xb58mcjISMaNG0fWrFkJDAy0SGzyYtuyZQsdO3Zk06ZN+Pn50b59e0uHJCIiIs+4DFcomTFjBm+++SZTp07l9u3bKbaXLVuW6dMf/iTR82j48OEMHjwYgN27dz9WUkmzT37N7LAeatUX7Z76MV9kBoOBn3/+2bRGZMGCBenSpcsTXfvxwoULFCpUiM2bN6dZtjMjitWpw7utWjHygw/S7NOlSxcuXLhAQEDAA8fy8fGhYMGC+Pv7Z26QIiIiIiLy0niWlii6X1RcVJrb6n9SM0VbwezeODg4pNo/td8j+/TpY7Y0zYULF9I8XsGCBTl27Fia23/++Wd+/vnnNLc/z1626j0iyR5UJSmLhwvO7k5c2h2ERyl3YhNi2XZuM41LNgOS7m0OHz481X19fHxYvXo1TZo0IT4+noEDBz5yTE2aNKFJkyam923atDG7fxUeHs4bb7xB/vz5mTt3rinpDeD777/n+++/NxuvWrVquLi4sHTpUtO92EcVFxfH6dOnTe+Dg4OpV68eLVq04IsvvkjXWE+Dvb09n3zyCcePH2fgwIGqHiGZrkuXLqb72CIiIiKPIsMJJadPn35gmVR3d3dTWcIXjZ2dHXZ2SZn+jo6OFo7myXpQ8sBXX33FsmXLOHnyJEajkVdeeYVPP/2URo0aPf1AnwF79+597r8fdixbhmMaNzjT8qgJJiIiIiIiIo/jWU0YcLJ1Sl9/Oy1hICKZ50FVkgwGAxXfLcu2b/fgmNWBIvUKsv/yHko5lWXy5Mn06NGDsmXLprl/rVq1WLt2LY0bNyYuLo6hQ4cCSUkaCQkJpldMTIzZfnv37qVcuXLExsYyd+5ctm3bxrfffgskJZM0atSIAgUK8Msvv6S6HM39XF1dGT9+PJ988glOTk60a9eOnDlzEhwczKlTp8idOzcAmzdvxtHRkQoVKmA0Glm4cCGbNm0yJc1cvXqVunXr0qJFCyZMmPDQ41pSzZo1+eSTTywdhoiIiIhIxpe8cXBwIDIyMs3tFy9eJGvWrBk9jDzDNm3aRLdu3di8eTO7d++mSpUqNG3alB07dlg6NGJjH20d68zk7u6Oi8vzXZLaPUcOXLROq4iIiIiIyEO9lq/yE+0vIvIgD6qSBOD5am7qDa3B7XN3+XPYRsa+OZEmTZqQO3duihQp8tDxq1evzvr165kwYQKff/45AD179sTR0ZHp06czf/586tata7bP8OHDyZEjBx4eHvzxxx8EBATg4eEBwLJlywgMDGTFihW4ubmZlrcZP378A+MYMGAACxYs4H//+x+FChUia9as1K9fn9KlSzN27FgAIiMj6dmzJ9mzZ8fDw4Pvv/+eRYsWmaqj/PDDD5w+fZoZM2aYLfu1YMGCh87D03b+/HmL3NcUEREREblfhhNKKleuzPLly1Pddu/ePebNm0eNGjUyehh5hq1Zs4aePXtSrlw5ihcvzuTJkylWrBjLli17pP0vXLiAwWBg4cKFNGvWDCcnJ7y9vVMskRIcHEz79u3JmjUrjo6O+Pj4sHfv3hTjzJs3j6ZNm5KtTBkGjx/Pll27sC9ShNWbN1OvfXvcSpemXKNGBO7fT9C1a7zZsyfZy5alVP36/Llpk9kxP/30U0qXLo2zszNeXl688847XL9+/YHnU7BgQVOZYn9/fwwGQ4pXwYIFTf3/+ecfWrduTbZs2ciaNSu+vr4cPnzYbMwlS5ZQpEgRHBwcqF69OkeOHHmkuQX4eckS8let+u883riBfZEitOjRw9T2x8aNZH3lFdMTJcXq1OHzadNM2+/cuUO7du1wdnYmd+7cfPrppxiNRtP20aNHM3fuXLZs2WI6x/tLNY8dOxYPDw+yZ89Op06diIiIQERERERE5HlXq3A9bK3tHqmvnbUdtQvXfXhHEZFH9ChVktyLZsfn42q8NbMJny4fwt9//82IESNwckq5r4+PT4p7NpUrV+bu3buMHDkSSLrfZTQaTa/AwECz/uvXrycsLIzw8HBWr15NiRIlTNs6d+6M0WgkMjKSiIgI0yutpXf+q0WLFgQEBBAeHk5ISAjHjx9n0qRJ5M2bF4CmTZty9OhRIiIiuHv3Lrt376ZNmzam/UeNGoXRaDQ7bkREBB07dnzosTPbpUuXUn0dOnSISZMm8c0331C7du2nHpeIiIiIyP0ynFAyePBgdu7cybvvvmv6kPvatWv89ddf+Pj4cOXKFQYNGpThQOX5kZCQQHh4OM7prHAxYsQI0/dR27Zt6dGjh2mNU6PRSMuWLTl58iR//PEHe/bswcPDA19fX27cuGE2ztChQ3nrrbfY/+effNCtm6n906++Yth777Fn1Srye3nR8f336TZ4MO+2asXulSupVakSnT/6iJCwMNM+9vb2zJw5k2PHjrF48WJOnz5Nhw4dHvmc2rVrR3BwsOl1+PBhcuXKRb169QC4fv06NWvWJFeuXGzZsoXdu3dTsmRJfHx8TIkrBw8epEOHDrRq1YrDhw8zaNAgPvjgg0eOoW716ly/dYtj/z+XATt34p49Ozv27iU+Ph6AzTt3UrVCBezt7VMdo3v37uzdu5cVK1awadMmLly4YJZINmjQINq2bUu1atVM5/rff/fLly/n9u3bBAQEsHDhQlauXMmXX375yOcgIiIiIiLyrHKxd6Fr5V6P1LdL5V442z3fFS1F5NmiKknPp4IFC1KoUKEUr9dee40hQ4ZQoEABvvnmG0uHKSIiIiKCTUYH8PX1ZebMmXzwwQcsXLgQgHfffRcAOzs7fvjhB6pVq5bRw8hzZOzYsYSHh9Or16PdUEv23nvv0bZtW9MY33zzDZs3b6ZYsWJs2rSJPXv2cPToUV555RUA5s6dS8GCBZk+fTqfffaZaZwePXrQpUsXYs+eBeDy1asADO/Xj9f/P7P/k3798Gnblv5dutCqcWMARn30Ef6//caeQ4dM/ZKfvABMx6pSpQpXr17F09Pzoefk6OiIo6MjANHR0bRo0YLSpUvz/fffAzBz5kzy58/PzJkzTft88803rF69mnnz5jFo0CAmT55MxYoVmThxIgDFixfn6tWrDBgw4JHmtWDevBTMl49NgYGULlaMgJ076fTWW8xbupS9R45QrUIFNgcG0r5Zs1T3/+fCBX7//XdWr16Nr68vAD/99BOFChUy9XFxccHR0RE7OztTCdP/ypcvH1OmTAGgRIkStGnThg0bNphKpYqIiIiIiDzPfIom/a70857ZxCWkXKLAztqOLpV7mfqJiGSWWoXrsfjg/FR/9txPVZKeHX5+fhgMBrM2g8FA9uzZKVasGL6+vlhZZfhZUBERERGRDMtwQglAr169aN68Of/73/84efIkRqORokWL0rZtW7y8vDLjEPKcmDFjBhMnTmTlypWmcpOPqly5cqa/29jY4O7ubqrScezYMbJmzWpKJoGk6iFVqlTh2LFjZuNUqVIl1fFfLV3a9HcPd/ektpIlTW15cuUC4Mbt26a2FStWMGXKFM6cOUNYWBiJiYlA0vI6j5JQksxoNPLOO+8QERHBX3/9hZ1dUjnkvXv3cvDgQVxczJ9Qi46O5syZMwAcP36cOnXqmG2vWbPmIx8boF61amwODGRAly5s3rWLGWPHcvnqVQJ27qRwgQIcP3MGnzQSv0788w+QtGZuMjs7OypVqkRISMgjHf/VV181e+/p6cnGjRvTdQ4iIiIiIiLPMp+ivlTMX5VtZzex/8peomIjcbJz5rV8lanlXRcXe1UmEZHMl1wlafbO6Q/tqypJz477l4oWEREREXlWZUpCCYCHh8cjV0yQF9OkSZMYPXo0q1atMi3pkh62trZm7w0GgymBIz3SWmrH1ubfb/fkJwBSa0s+5u7du2nVqhWDBw9mwoQJZMuWjQsXLtCoUSNiYx/+1Md/JS8NtWvXLrJmzWpqT0xMxMfHx6xCSTJXV9d0HeNBfKpVo//IkZy5cIFrN25Qo2JFLl65wq+rVlG4QAFcXVx4rUyZTDve/TLraysiIiIiIvIsc7F3oXGp5jQu1dzSoYjIS0RVkkRERERE5EnJtIQSebn5+fkxdepU1qxZQ61atTJ9/NKlSxMSEsLff/9tqlISExPD7t270720zqPavn07WbNmZcKECaa2Xbt2pXuc77//nlmzZrFlyxby589vtq1ixYr89NNPeHp64uTklOr+pUqVIjAw0Kxtx44d6YrBp1o1wiIimDxrFlXLl8fRwYG61avz0Wef4eXhQc1KlbC2tk5135JFigAQGBhI4/9fHig2Npa9e/dStGhRUz87OzsSEhLSFZeIiIiIiIiIiGScqiQ927Zu3QpA7f9fZjv5/cMk9xcRERERsZR0J5R069Yt3QcxGAzMmTMn3fvJsyUiIoJDhw6ZtdnZ2TF79mxmzZrFokWLKFq0KNeuXTNty549e6Ycu169elSuXJl33nmH7777Djc3N8aOHUtUVBT9+vXLlGPcr0SJEty5c4fZs2fToEED9u3bx9ixY9M1xsaNGxkwYACzZs3C09PTNDfW1ta4u7vTv39/5syZQ4sWLfDz8yNfvnwEBQWxdu1aXn/9dWrVqsVHH31EpUqVGDZsGF27duXEiRNMnjw5XXHkzpmTUkWLMm/5cj79/0pC3vnz4+HuzpI//2TCsGFp7lukYEGaN2/OgAEDmD17Nh4eHkyYMIHw8HCzfoULF2bZsmUcPXqUPHny4OTklGaSjIiIiLy4oqKieOWVVzh//jz9+vVj+vR/y6+PHj2aMWPGpLrfV199xaBBg8zaEhMTmTZtGrNmzeLChQu4u7vTtm1bPvvsszSr0omIiIi8rFQl6dnl4+ODwWAgOjoaOzs70/u0GI1GDAbDYz+8ldHr6C+++IIDBw6wf/9+zp8/T4ECBbhw4UKKfvfu3WPevHn88ccfHD58mOvXr5MnTx6qVKmCn58fJf+z3LhkjoCAAOrWrcvPP/9Mly5dLB2OiIiIvATSnVDi7++f7oMooSRtq75oZ+kQHtn+/fspX768WZuXlxdBQUEAvPnmm2bb6tSpQ0BAQKYc22Aw8Pvvv/PRRx/RpEkTYmJiqFy5Mhs2bCBXrlyZcoz7NWnShFGjRjFq1Cg++OADqlSpwrRp02jSpMkjj7Ft2zbi4+Pp3r27WXvyL2G5c+dm586djBgxgtatWxMaGoqHhwc1a9bE09MTgNdee42FCxcyYsQIpkyZQrly5ZgyZQotW7ZM1/nUrV6d42fOULdatX/bqlXD/7ffzNpS89NPP9G3b1+aNWuGk5MTPXr04M033+TixYumPr169WL79u3Url2bkJAQRo0apfVgRUREXkJ+fn7cvHnzgX2mTJlCzpw5zdpee+21FP0++ugjvvnmG958800+/vhjTpw4wTfffMPBgwfZsGEDVlZWmRq7iIiIiMiT8NNPP2EwGEzLQv/8889P9HgZvY4ePnw42bNnp0KFCoSEhKTZ78KFC/Tq1YuaNWvSvXt3PD09OXfuHDNnzmTZsmWsXbuWunXrZvLZyYvi7t27/PLLL/z555+cOHGCW7dukT9/furUqcPIkSPJly+fpUMUERERHiOhJDEx8UnEIc84f3//x0omehQFCxbEaDSmaL8/6z1PnjwsXrw43ePUqVqVmH/+Me+bN2+KNiBF2+jRo1MkRdx/jPvf/zfu1Pa/X4ECBZg/f/4D+7Rv35727ds/8LgP8/XIkXw9cqRZ26wJE5j1nyV9kp3essXsfY4cOViyZMkDx8+WLRurVq1K0Z5aUtGjzIuIiIg8fw4cOMDUqVOZOHEiH3/8cZr9WrZsScGCBR841rFjx/j2229p1aoVS5cuNbUXKlSI999/n8WLF/P2229nVugiIiIiIk/M/ZUkOnfu/MSOlRnX0WfPnsXb2xuAV155hYiIiFT7ubu7c/DgQcqVK2fW3rFjR8qXL8/gwYPZt29fxk5IXli7d+/m448/pn79+vTv35+cOXPy999/M2vWLJYsWUJgYCClSpWydJgiIiIvPT3SJyIiIiIiGZaQkEDPnj1p1KgRrVq1emj/sLAw4uPj09y+aNEijEYjH374oVl7z549cXJyemhCroiIiIjIyygzrqOTk0keJkeOHCmSSQBKlSrFK6+8wt9///1I4zyzYsLh2HJY+wmsfD/pz2O/J7XLQyUkJBAVFZXm9hIlSnDq1Cn++usvhg4dSvfu3ZkyZQorV64kNDQUPz+/pxitiIiIpEUJJfLElS5dGhcXl1Rfffr0sXR4z73x48enOb8uLi6WDk9EREReElOmTOHkyZNMnz79oX3Lli2Lm5sbDg4OVK9enTVr1qTos3fvXqysrKhcubJZu4ODA+XKlWPv3r2ZFruIiIiIyJO0devWx3o9jmfhOjoxMZHg4GBy5879xI/1xJxZD4s7QsAX8PdSOPlH0p8B45Paz6y3dIQmiYmJjBs3jtq1a+Ph4YGdnR358+enb9++3L5929Tvxo0b2NnZ0bFjx1TH6devH1ZWVmYVuENDQxk6dChFihTB3t4ed3d3OnTowLlz58z29ff3x2AwsGHDBj7//HMKFy6Mg4PDA6teFyxYkMKFC6do9/X1JXv27M9/QpKIiMgLIt1L3qQmPj6e33//nd27d3P37t0Uy+IYDAbmzJmTGYeS59Dq1auJi4tLdZurq+tTjubF06dPH9q2bWvpMEREROQldv78eUaNGoWfnx8FCxZMsXRhsqxZs9KrVy+qV69OtmzZOHXqFFOnTqVJkyb89NNPZqXAr169Ss6cObG3t08xjpeXF4GBgcTGxmJnZ5dmXJcvX+bKlStmbUePHgWSfodJ6xr1ZRMXF6f5sBDNveVo7i1Hc285mnvLeBnm/UFV5ySJj48PBoMh3fslJCSke5/MuI7OqO+//57g4GBG3rf89oM87rV7YmKiaW6NRmOKzyYey5n1GLZ8CaGXwN4N3AqAtS0kxEH0HbhxHDZ/gTExEYo2yPjx0iH5/BITE01/v3fvHl999RWtWrWiefPmODk5sW/fPubMmcP27dvZtWsXDg4O5MyZk2bNmrFs2TLu3LlD1qxZTePeu3ePhQsXUr9+ffLnz09iYiKhoaHUrFmTS5cu0bVrV0qVKsW1a9eYOXMmVapUYc+ePRQoUMAsrkGDBhEXF0ePHj1wdXWlaNGi6f6ahIaGEh4eziuvvPLQfY1GI0ajMd0/Y1+Gn82PS3OTNs1N2jQ3qdO8pE1zk7YnOTePe92e4YSSO3fuULduXf7++2+MRiMGgwGj0Qhg+rsSSl5uyReV8mRkz56d7Nmzp7k99uzZpxiNiIiIvIz69OmDt7c3AwcOfGC/+8tuA3Tr1o1XXnmFjz76iLfeestUYS0qKirVm+CQ9HRlcp8H3QifM2cOY8aMSXVbREQEoaGhD4z3ZREfH09kZCQANjaZ8syBPCLNveVo7i1Hc285mnvLeBnmPSIiwtIhPPN+/vlns/dGo5Fvv/2W06dP07FjR0qVKgXAsWPHWLRoEcWKFaN///6PdazMuI7OiMDAQAYOHMirr77K8OHDH3m/x712j4+Px8bG5rGSb1IVE471zukQcglj1rxg6/TvNhtbyJIbHLJgCLkEO6eT4FUJ7LNkzrEfQXKChdFoNJ2zjY0Nly5dwtHR0dSvZ8+eVK1ald69e7N8+XLatWsHQPfu3Vm2bBkLFiwwqx7+v//9j5CQELp27Woa18/Pj3PnzrFt2zZeffVVU9933nmHChUqMGrUKNPnPsmfCUVFRbF3716cnP6dt/R+bcaOHUtcXBzvvPPOQ/c1Go3Ex8en+3e7l+Fn8+PS3KRNc5M2zU3qNC9p09yk7UnOzeNet2c4ik8//ZSTJ0/y448/4uPjQ+HChfnrr7/Inz8/n3/+OWfOnOGvv/7K6GFEREREROQZNH/+fNavX8/WrVuxtbVN9/45cuSgT58+jB49msDAQF5//XUAnJycuHHjRqr73Lt3z9TnQbp3707Dhg3N2o4ePUrv3r1xcXHBzc0t3fG+iJKfeHB1dX2sr6E8Ps295WjuLUdzbzmae8t4GeZdSy4/XOfOnc3eT5s2jZs3b3Lq1Ck8PT3Nto0cOZJq1aoRHh7+WMfKjOvox7V//36aNGmCp6cnf/75pymB5VE87rX77du3MRgMWFtbY2Vl9ViVYMxc2IIh4jo4uGGwdU69j61z0vaI61hf3Aolm2fsmOlgZWUFYDrnZMn/DhMSEggPDyc+Pp769esDSV+XDh06YDAYaNiwIYUKFcLf359+/fqZ9p87dy45cuSgVatWWFtbYzQaWbRoEbVq1SJ//vzcvXvX1NfV1ZUqVaqwYcMGUwzJ8963b1+yZHn8BJvffvuNKVOm0LBhQ7p16/bQr6fBYMDGxibdv9u9DD+bH5fmJm2am7RpblKneUmb5iZtT3JuHve6PcMJJX/++SedOnWia9eupvX4rK2tKV68OPPnz8fHx4dPPvmEmTNnZvRQIiIiIiLyDImJiWHgwIG88cYbeHh48M8//wAQFBQEJJUq/ueff8iZM6dZOeX7FSxYEIBbt26Z2jw9PTl+/DgxMTEpnrAMCgoiZ86cD32qMl++fOTLly/VbTY2NvqF9T+S50Nz8vRp7i1Hc285mnvL0dxbxos+73qqNP2mT59O7969UySTAOTNm5fevXvz7bffMmDAgHSPnRnX0Y/jwIEDNGjQADc3NzZv3oyXl1e69n/ca3crKyuziunJCReP7coeuBcCWQvCg5IZnHJAyAUMl/dA6ZYZO2Y6JJ+flZWV2bkuWbKEyZMnc/DgwRQl8u/evWs2Nz169GDEiBEcOXKEcuXKce7cOQICAvjggw9MSUA3btzg9u3brF+/nty5c6cZy3/jAShevPhjfw1Wr17Nu+++y2uvvcaSJUvMEmbSYjAYMBgMj/Xz9UX/2ZwRmpu0aW7SprlJneYlbZqbtD2puXnc6/YMXl3BtWvXqFSpklkQyZnOAC1btmTlypUZPYyIiIiIiDxjoqOjuXnzJn/++SdFixY1vXx8fICk6iVFixblxx9/fOA4Z86cATC7UVmpUiUSExPZs2ePWd979+5x6NAhKlasmLknIyIiIiLylFy+fPmBVUKcnZ25fPnyY41tievoAwcO4OvrS5YsWdi8efPzvQR6bCQkxoP1Qz7AsbZN6hdr+SWfli1bZlrSZtq0aaxatYr169ezdu1a4N9lcpJ169YNGxsb03I1P/30E0ajkR49epj6JCfp+Pr6sn79+lRfqVWmf9zqN2vXrqVVq1aULl2adevW4erq+ljjiIiISObLcPp49uzZTev4ZMmSBVtbW7OLXVtbW7NyaC+S2NhYU7ZvdHS0haMREREREXm6nJ2d+d///pei/ebNm7z33ns0atSI7t27U7ZsWdP6n/eXIr58+TIzZ84kR44cVK9e3dTerl07xo8fz9SpU6lVq5ap/YcffiAqKoqOHTs+uRMTEREREXmCChUqxPz58+nbt2+KZWHu3bvHL7/8Yqril17puY4+e/YscXFxlChR4rGOBXDw4EEaNGiAi4sLmzdvplChQo891jPBzhmsbCAhDmweUMklIS6pn53ll3yaN28eDg4ObN682Syh4+TJk6n29/DwoFmzZixYsIAJEybg7+9PlSpVKF26tKmPu7s7WbNmJSwsDF9f3yca/9q1a2nZsiUlSpRgw4YNZMuW7YkeT0RERNInwwklxYoV4/jx40BSWbPy5cvj7+9Ply5dSEhI4JdffsHb2zvDgT6Lxo8fz5gxYywdhoiIiIiIRdja2vLWW2+laL9w4QIAhQsXNm0PCQmhUKFCtGzZkpIlS5ItWzZOnTrFjz/+SEREBIsWLcLR0dE0RpkyZejXrx/Tp0+nVatWvPHGG5w4cYJvvvmGOnXq8Pbbbz+VcxQRERERyWwDBw6kd+/eVKpUiX79+lG8eHEgKQHgu+++48SJE3z//fePNXZ6rqPr16/PxYsXTdUoks2bN4+LFy8CScnisbGxjB07FoACBQrw7rvvAnDx4kUaNGjA3bt3ef/99wkMDCQwMNBsrDfffBNnZ+fHOheLyFcFzm6E6DuQxSPtftF3wDEr5K/y1EJLi7W1NQaDwawSidFoNH3NUtOzZ0+WL19Onz59CAoKYvTo0Wbbrays6NixI9999x2//fZbqr/33bhxg1y5cmUo9nXr1vHmm29SvHhxNm7cSPbs2TM0noiIiGS+DCeUvP7660yaNInp06djb2/PwIEDad++PdmzZ8dgMBAdHc3s2bMzI9ZnzvDhwxk8eDAAu3fvpn79+haOSJ43/v7+dO3a1fRLW0BAAHXr1uX8+fOP/RTCo/h82jTmLVvG6S1bMn3sRz0Hg8HAzz//TJcuXdLsc+HCBQoVKsTmzZtNpfNFRETk+eTo6Ejr1q3ZvXs3v//+OxEREeTMmRNfX1+GDBlC5cqVU+wzdepUChYsyOzZs/nzzz/JmTMnAwYM4LPPPsv42ugiIiIiIhbSs2dPIiMj+fTTT3nvvfcwGAxAUhKAo6MjX331FT179nzs8TN6HT1nzhy23HffcOTIkQDUqVPHlFBy/vx5bt++DZAiISHZ+fPnn6+EkiL1Yd9PcOM42LuCXSpLuMRGwb0QyFUKClv+M4G33nqLpUuXUq9ePTp16kRcXBy///47UVFRae7TsGFDChQowPz583FxcaF9+/Yp+owbN44dO3bQtm1b2rZtS9WqVbGzs+PixYusXr2a1157DX9//8eOe9++fbRo0QKj0UjXrl1Zs2ZNij7vvPPOY48vIiIimSPDCSXDhw9n0KBB2NvbA9C2bVtsbGyYP38+1tbWvPXWW6b1+140dnZ22Nkllb3779OU6RHU/unPjdfiX9PV/6effqJXr17cunWLrFmzmtpr1KhBYGAgV69eJU+ePKb2hg0bYjQaWblyJX379uXQoUMcO3YMT09P09OqyS5dusS4cePYtGkTV65cIUeOHDRo0ICxY8fi5eWVkdN8LlWvXp3g4GDc3d0tHcpje9xzeJQEExEREXk+FCxYMMVTjvb29vz444/pGsfa2pqPP/6Yjz/+ODPDExERERGxuA8//JCuXbuybt06zp07B4C3tzcNGjQwuwf7OB71Ovr+e7XJAgICHuk4Pj4+Ka77n3v2WaD6AAj4AkIugUNWcMwO1rZJy9xE30lKJsmaP6mfveWXvGnfvj3h4eFMmTKFQYMGkS1bNpo1a8aECRPIkSNHqvtYWVnRvXt3/Pz8aNu2LS4uKc/Dzc2NHTt2MHnyZJYsWcKKFSuwsbEhb9681KxZkx49emQo7r///pt79+4B8NFHH6XaRwklIiIilpfhhBKDwWBKJknWqlUrWrVqldGh5RlRv359EhIS2Lx5M2+++SYA4eHh7Nmzh/z587NhwwZTVnpsbCzbt29n1KhRJCQkYGdnR69evdi5cydbt25NMfapU6eIjIxk6tSplChRguDgYD788EMaNWrEoUOHsLa2fqrn+l+JiYkkJiZiY5PhfyaPzM7ODg+PB5RSfA68COcgIiIiIiIiIiLypLm5udGmTRtLhyH3K9og6c/AbyHiGoRcgMR4sLJJWuYmV6mkZJLkfk9RWkk8PXv2TLWqTUJCAgkJCamOlfy5zoMSQ5ycnBg5cqSpQk1aunTpku4HBR9nHxEREXn6MlwnOj4+nrCwsDS3h4WFER8fn9HDiAUVKFCAIkWKsGHDBlPb1q1b8fLyolOnTmbtO3fuJCoqCl9fX5ydnZk1axZ9+/bF29s71bEbNGjA/PnzadKkCYULF6ZmzZrMmDGDv//+m+PHjz9SfKNHj6ZgwYKsWLGCEiVK4OzsjI+PD2fuy7BfExBA1RYtyFKyJHkrV6a/nx8RkZGm7Z9Pm0axOnVYtmYNrzZsiJ2dHYcOHaJLly74+Pjw/fffU6BAAZydnWnTpg0RERH8/vvvlCpVChcXFxo3bsz169dN450/f5633noLLy8vHB0dKVmyJN98880Ds/YDAgIwGAympwN8fHwwGAwpXv8tIbl48WLKly+Pg4MDBQoU4KOPPiLyP+d1LyaG/n5+uJcrR+4KFejv50dMbOwjzS1AvfbtGTp0qOn9F198gcFg4H//+5+prXXr1nTr1i3VcwDYvHkzZcuWxcHBgbJly7J582azYyQvjdO1a1fTOf53/6tXr9KsWTOcnJzw9vbOUClFERERERERERERkQcq2gDaLwCf4fDKW1CiadKfdUcktVsgmSQzxcfHM2vWLMqUKUO1atUsHY6IiIg8wzKcUPLxxx9TsWLFNLdXqlTJ7MNoeT7Vr1+fjRs3mt5v3LiR+vXrU69evRTtOXLkoHz58o99rJCQECAp+/lR3bhxg5kzZ7JgwQICAwMJDQ2l97Bhpu1HT56kde/e1KxUib1//MGcr75i7ebN9B0xwmyc67du8c3PPzPriy84ceIERYsWBeDAgQMEBASwevVqVq5cyYYNG2jdurXpmJs2beLkyZMMGjTINFZERAQ+Pj78+eefHDt2jCFDhjBs2DB++eWXRz6vZcuWERwcbHpNnToVGxsbateuDYC/vz99+/blo48+4tixY8ybN4+NGzeaZZV/+tVXLFuzhh8nTmTrb7/h7OjI9/PnP3IMdatVM/sab9q0CXd3dzZt2gQkVXLZvHkz9erVS3X/q1ev0rRpU8qVK8f+/fv5+uuv+eCDD8z67N27F0ha3zX5XPPly2faPmLECN59912OHDlC27Zt6dGjB6dPn37kcxAREREREREREXmWnD17lv79+1OpUiWKFCmCt7e32atw4cKWDlHss0DpltBoPDT/JunPUi2S2p9T58+fZ+HChXTs2JFz584xfPhwS4ckIiIiz7gMJ5T89ddftG7dOs3trVu3Zs2aNRk9jFiYr68vp06d4sqVK0BSUkH9+vWpVq0at2/f5uTJk6b2evXqYTAYHus4ERERDBo0iNatW6frl6Z79+4xd+5cXnvtNV599VUGDRpE4P793IuJAeDrH37g1ZIlmfTpp5QoXJiGderwtZ8fS/74g3OXLv07TkwMc776iqoVKlC0aFHc3NwAsLW1xd/fn9KlS1O/fn3atGnD+vXr+eWXXyhfvjyVK1emZ8+eZtVaypQpQ//+/SlXrhze3t507dqVLl26sGDBgkc+r+zZs+Ph4YGHhwfnzp1j2LBhzJgxw5S8MXr0aMaNG0enTp0oXLgwtWvXZvr06SxevJhbt24RGRnJ7IULGfXhh7R4/XWKe3vz5SefULRQoUeOwadaNQ4ePMjdu3eJjY1lx44dDBs2zJRkkrwtrYSSGTNmkC1bNubMmUPp0qXx9fVl/PjxZn3c3d2BpDKfyef73+WO3nvvPdq2bUuRIkUYO3YsdnZ2KaqciIiIiIiIiIiIPA+OHj1KhQoV+PHHH4mNjeXcuXM4Oztz7949Lly4gLW1Nfnz57d0mPIC2rJlCx07dmTTpk34+fnRvn17S4ckIiIiz7gMJ5Rcvnz5gR/8e3t7c/ny5YweRiysXr16WFlZsWHDBm7dusXRo0epV68eDg4OVKtWjQ0bNhAREcGePXvw9fV9rGNERkbSvHlz7O3tmTNnTrr2zZMnD7lz5za99/T0xGg0cuP2bQCOnzlDjfsq6dSuUsW0LVnObNkoXKBAivFLlCiBg4OD2fHuP2aePHnMlryJjo5mxIgRlClThhw5cuDi4sKPP/5otpTLozp79iwtWrRg4MCBprUwb968ycWLF/n4449xcXExvRo3bgzAmTNnOHv2LDGxsVSrUMFsvOoPqCp0vyrlyuHg4MDmzZvZuXMn2bJlo1evXly4cIErV66wadMmSpQogaenZ6r7Hz9+nEqVKmFra2tqq1mzZrrOv1y5cqa/29jY4O7ubjbXIiIiIiIiIiIizws/Pz/s7Ow4fPiw6aGtadOmcfXqVWbNmkVISAjfffedhaOUF1GXLl0wGo3cvHmTMWPGWDocEREReQ7YZHQAOzs7goOD09x+7do1rKwynLciFpY9e3bKlSvHxo0bcXJyokSJEnh4eABQt25dNm7ciLe3N3FxcY+VUBIaGkqTJk1ISEhgw4YNpsogj+q/yQqAqUJKYmJiusZxTmOZndTGT63NaDSa3g8ePJilS5cyefJkSpUqhYuLC1999RV//fVXumK6c+cOb7zxBg0bNmTs2LGm9uRz+/rrr2nQIOWanV5eXpz5T7LM47Kzs6NmzZps2LABd3d36tevj4uLC5UqVWLjxo2m5Y+epNTmOr1fWxERERERERERkWfB9u3b6dWrF8WLF+f2/z8Ql3xfsWfPnmzbto1hw4axcuVKS4YpIiIiIpLxCiXlypVjyZIlxMbGptgWFxfHr7/+StmyZTN6GHkG+Pr6pppAULduXQICAli3bh2FChXC29s7XePeunWLunXrYmVlxbp169KdTPIoShUtSuD+/WZtW3fvNm17ErZs2cLbb7/N22+/Tbly5ShSpAinT59O1xixsbG0bNkST09PfvrpJ7OlhHLnzk2+fPk4deoURYoUSfFydHSkcOHC2NnasvPAAbNxd943Fw9Tv359Nm7caFrqKLlt7dq1bN++Pc3lbgBKlSrFvn37iIuLM7Xt2LEjRT9bW1sSEhLSFZeIiIiIiIiIiMjzJjw83FT1287ODkiq3pysRo0abN++3SKxiYiIiIj8V4YTSvr378+xY8do0qQJ+/btIzY2lri4OPbt20eTJk04fvw4/fv3z4xYxcLq169PcHAwixYtMksoqVKlCnFxcfz0008pqpMcP36cQ4cOce3aNWJjYzl06BCHDh0iIiICgODgYGrXro3BYMDf35/IyEiuXbvGtWvXiI6OzrTYB/bsyaHjxxk0diwnz55l3datDPzsM9o2bYr3E1qPtESJEqxatYpdu3Zx8uRJBg8ezIH7Ejsepnfv3ly8eJGZM2dy584d09wkz9+4ceP47rvv+Pzzz/n77785deoUK1asMC2L4+zsTM+33+bzadNYsX49p86d45Mvv+T0uXPpiqNevXqcPn2aXbt2mb729erV47fffiM6Opq6deumuW/fvn25ffs2PXv25Pjx42zcuJERI0ak6Fe4cGE2bNhAUFAQt27dUgUSERERERERERF5IeXOnZtr164BkCVLFpydnc0eRLt7964evBIRERGRZ0KGE0pat27NJ598wsaNG6lSpQpOTk44OTlRpUoVNmzYwJAhQ2jXrl1mxCoWVqtWLezt7YmKiqJOnTqmdltbW2rUqEF4eHiKhJI33niD8uXLM2vWLIKDgylfvjzly5dn3759APz111+cOHGCAwcOULhwYfLkyWN6/frrr5kWe5kSJVg6axbb9+6lUtOmdP34YxrVrcvMceMy7Rj3mzp1KkWKFMHX15datWoRGxtLv3790jXG5s2buXTpEiVLljSbm0mTJgHw7rvv8ttvv7F69WoqV65MxYoVGTVqFF5eXqYxxg0eTPPXX6fH4MHUbNWKkLAw+rzzTrriqFChAtmyZcPb25u8efMCUK1aNWxtbSlXrhzZsmVLc18vLy9WrVrF/v37KV++PB988AFff/11in7Tpk3j6NGjFC5cGHd3dy5dupSuGEVERERERERERJ4H5cqVM90fBahTpw7Tpk1j69atBAQEMH36dF599VULRigiIiIiksRgTF6cMYP27t3L/Pnz+eeffwAoVqwYb7/9NpUqVcqM4Z95O3fupHr16gQGBlKtWjWzbWfOnAGg6BNaWkVSF3v2bIb2t/v/spPPu4zOAzx/c5HWv7mg9hlPbvNanHmJTpaiefiX5uJfGZ0LzcO/NBf/yshcPOjaSiQj9L2VUlxcHKGhobi5uWFra2vpcF4qmnvL0dxbjubecjT3lvEyzLuur9Lv119/5bvvvuOvv/7C0dGRgwcPUqdOHdOyN46Ojqxdu5aaNWtaOFLLepTvrTNnzmA0GilUqBDW1tZYWWX4GdoXSmJiIgkJCS/s3DzuZy8vw8/mx6W5SZvmJm2am9RpXtKmuUnbk5ybx71ut8msACpVqvTSJI+IiIiIiIiIiIiIiDyOdu3amVX1Ll++PMeOHWP58uVYW1vTuHFjvL29LRihiIiIiEiSTEsoud/+/fu5c+cOtWrVwsHB4UkdRl5w27Zto3HjxmluX7NmDbVq1XqKEb14yjVqxKWrV1Pd1qFFC777/POnHJGIiIiIiIiIiMiLKSYmht27d5MnTx6zqgr58uXj/ffft2BkIiIiIs++yLhItgdt59CNQ0THR+No40j5XOWp4VUDZ1tnS4f3QspwQsmkSZPYsmULq1atMrW9/fbb/PprUolzb29vtm/fTu7cuTN6KHkJVaxYkUOHDqW53cvL6+kF84JaMWcOcXFxqW5zzZLlKUcjIiIiIiIiIiLy4rK2tqZ+/fpMnjxZS6SLiIiIpMO2K9uYe2wuN6NvEhYbRkJiAtZW1gReDWTJqSV0Lt2ZWnlViCCzZTihZPHixVSpUsX0ftOmTSxevJgOHTpQpkwZxo4dy8SJE5k8eXJGDyUvIUdHR4oUKWLpMF5oBZSUIyIiIiIiIiIi8lTY2Njg4eGB0Wi0dCjyHAoICKBu3br8/PPPdOnSxdLhiIiIPDXbrmxj5uGZBEUE4WrnipeLF7ZWtsQlxhEaE8qZkDPMPDwTQEklmcwqowNcuHCBkiVLmt7//vvv5MmTh/nz5zNs2DD69OljVr3kZaVfEESeHv17ExERERERERGRZ1WbNm1YsmQJiYmJlg5FxGLi4uLo06cPr732Gjlz5sTe3p5ChQrRrl07Dh48aOnwRETkGRIZF8ncY3MJigjC08UTdyd37KztMBgM2Fnb4e7kjqeLJ0ERQcw9NpfIuEhLh/xCyXBCSWRkJI6Ojqb3mzZtwtfXF4PBAECpUqUICgrK6GGea1ZWViQmJupDbpGnwGg0kpiYiJVVhn+8iYiIiIiIiIiIZLoePXoQFRVFgwYNWLVqFSdPnuTSpUspXiIvstjYWPbt20eNGjX49NNPmTFjBp06dWLnzp1UqVKFTZs2WTpEERF5RuwI2sHN6Ju42rniaOOYah9HG0dc7Vy5GX2TwKuBTznCF1uGl7zx8vLi6NGjAFy8eJHjx48zcOBA0/a7d+9ib2+f0cM81+zt7YmNjeXq1at4enqakm1EJHMZjUauXr2K0Wh86X/uiIiIiIiIiIjIs+mVV17BYDBgNBoJCAhIs19CQsLTC0pSiIyLZHvQdg7dOER0fDSONo6Uz1WeGl41cLZ1tnR4z7yEhARiYmJwcnJKdbuzszP79u1L0d6nTx/y58/PpEmTqFev3pMOU0REngMHbxwkLDYMLxevB/Zzs3cjKCKIgzcO0qBAg6cU3YsvwwklzZo1Y8aMGcTHx7N7927s7e1p0qSJafvff/9NwYIFM3qY55qHhwexsbGEhYURHh6OlZWVkkqegsSoqAztb3XmTCZFYlkZnQd4PuYiuTJJcjKJh4eHpUMSERERERERERFJwc/PT/eHn3Hbrmxj7rG53Iy+SVhsGAmJCVhbWRN4NZAlp5bQuXRnauWtZekwAUhMTOSLL77gr7/+4vTp09y5cwcPDw+aNGnCZ599RtasWQG4ceMGefPmpU2bNixYsCDFOP369WPmzJmcO3fO9JlOaGgo48ePZ+nSpVy+fBlXV1d8fX0ZN24c3t7epn39/f3p2rUr69evZ+fOnfj7+3Pp0iV++OEHunTpkq7zyZUrFw4ODty9e/dxp0RERF4w0fHRJCQmYGtl+8B+tla2JCQmEBWX8c9G5V8ZTijx8/PjyJEjzJgxA3t7e6ZOnUru3LkBiI6OZvny5XTv3j3DgT7PbGxsyJ8/P9euXSMmJkZrYz4lsRlMgnB49dVMisSyMjoP8HzMhcFgwNbW1pRMYmOT4R9vIiIiIiIiIiIimW706NGWDkEeYNuVbcw8PJOgiCBc7VzxcvHC1sqWuMQ4QmNCORNyhpmHZwI8E0klsbGxfPXVV7Ru3ZoWLVrg7OzM3r17mTNnDtu3b2fXrl04OjqSK1cumjdvzrJlywgJCTElmgDcu3ePhQsX4uvra5ZMUr16dS5dukS3bt0oXbo0wcHBzJgxgypVqrBv3z4KFChgFsugQYOIi4ujZ8+euLq6Urx48YfGn5CQwN27d4mPj+fy5ctMmjSJiIgI3njjjcycJhEReY452jhibWVNXGIcdtZ2afaLS4zD2soaJ9vUq2PJ48nwJ67ZsmVj48aNhIWF4ejoiK2teWbQli1byJcvX0YP89yzsbEhb968lg7jpRI08tMM7e/11luZFIllZXQe4MWZCxEREREREREREZG0RMZFMvfYXIIigvB08cTRxtG0zc7aDncnd1zsXAiKCGLusblUyF3B4svf2NvbExwcjKPjv7H26dOH6tWr06NHD1asWEH79u0B6NWrF0uXLmXhwoW89957pv5Lly4lJCSEHj16mNr8/Pw4d+4cu3bt4tX/PHDYpUsXypQpw6hRo/D39zeLJTo6moMHD6a5zE1qTpw4QZkyZUzv3dzc+OSTT/jkk08eeQwREXm6nvaycOVzlSfwaiChMaG4O7mn2S80JhRXO1fK5yqf6TG8zDLtEX5XV9cUbY6OjmYXGiIiIiIiIiIiIiIiIs+iHUE7uBl9E1c7V7Nkkv9ytHHE1c6Vm9E3CbwaSIMCDZ5ylOYMBoMpmSQhIYHw8HDi4+OpV68eAHv37jUllDRo0IBChQoxZ84cs4SSOXPmkCNHDlq2bAkkLS++YMECateujZeXF7du3TL1dXZ2pmrVqqxbty5FLH379k1XMglAoUKFWL9+PbGxsfzzzz/Mnz+f0NBQYmJiVIVaROQZZIll4Wp41WDJqSWcCTmDi51Lqv9HR8dHExYbRtGsRanhWSNTj/+y0//GIiIiIiIiIiIiIiLy0jt44yBhsWF4uXg9sJ+bvRtBEUEcvHHQ4gklAEuWLGHy5MkcPHiQuLg4s2137941/d1gMNCjRw9GjBjBoUOHKFeuHOfOnSMgIIAPPvgAO7ukZQRu3rzJ7du3WbduHe7uqT8JbmVllaKtWLFi6Y7d2dkZX19f0/tu3bpRoUIFWrduzdq1a9M9noiIPDmWWhbO2daZzqU7mx3bzd7N7NjJ/393Lt1ZS95kMiWUiIiIiIiIiIiIiIjISy86PpqExARsrWwf2M/WypaExASi4qKeUmRpW7ZsGe3ataNy5cpMmzaNfPny4eDgQEJCAo0aNSIxMdGsf7du3Rg1ahRz5szh22+/5aeffsJoNJotd2M0GgHw9fVl6NChjxxLequTpMbFxYVWrVrx5ZdfcvbsWQoXLpzhMUVEJOMsvSxccoJKcnWUoIggU3UUVztXimYt+kSqo4gSSkRERERERERERERERHC0ccTaypq4xDjsrO3S7BeXGIe1lfUz8QT0vHnzcHBwYPPmzWYJHSdPnky1v4eHB82aNWPBggVMmDABf39/qlSpQunSpU193N3dyZo1K2FhYWbVQ56W6OhoAO7cuaOEEhGRZ8SzsCxcrby1qJC7AjuCdnDo5iGi4qJwsnWifK7yVPesnqkJLPKvlDXJREREREREREREREREXjLlc5XH1c6V0JjQB/YLjQnF1c6V8rnKP6XI0mZtbY3BYDCrRGI0Ghk7dmya+/Ts2ZO7d+/Sp08fgoKCzKqTQNJyNh07dmTPnj389ttvqY5x48aNDMV98+bNFNVTAK5du8b//vc/XFxczJJcRETEspKXhXOzd3tgPzd7N8Jiwzh44+ATicPZ1pnXC77OkEpDGF19NEMqDaFBgQZKJnmCVKFEREREREREREREREReejW8arDk1BLOhJzBxc4l1Sewo+OjCYsNo2jWotTwrGGBKM299dZbLF26lHr16tGpUyfi4uL4/fffiYpKezmehg0bUqBAAebPn4+Liwvt27dP0WfcuHHs2LGDtm3b0rZtW6pWrYqdnR0XL15k9erVvPbaa/j7+z923AsWLGDq1Km8+eabFCpUCDs7O06fPs3cuXO5e/cuP/74Y6YsoSMiIpnjeVwWTjKHEkpERERERERERERERJ6iyMhIFi5cyJkzZ7h9+zZGo9Fsu8FgYM6cORaK7uXlbOtM59KdmXl4JkERQbjaueJm74atlS1xiXGExoQSFhuGl4sXnUt3fiaWvGnfvj3h4eFMmTKFQYMGkS1bNpo1a8aECRPIkSNHqvtYWVnRvXt3/Pz8aNu2LS4uLin6uLm5sWPHDiZPnsySJUtYsWIFNjY25M2bl5o1a6aoapJetWrVYt++ffzxxx8EBwcTGxtL7ty58fX15YMPPqB69eoZGl9ERDLX87gsnGSODCeUfPbZZw/cbjAYcHR0JH/+/Pj4+JArV66MHvKZERsbS1xcHPDvmn4iIiIiIiIiIiIiImnZs2cPTZs25datW2n2UUKJ5dTKWwuAucfmcjP6JkERQSQkJmBtZY2rnStFsxalc+nOpn5Pk4+PT4rkI0hawqZnz54p2hMSEkhISEh1LHt7e4AHJoY4OTkxcuRIRo4c+cC4unTpQpcuXR7Y536vvfYa8+fPT9c+IiJiOeVzlSfwaiChMaG4O7mn2e9ZWhZOMkeGE0pGjx6NwWAASDWL+r/ttra2DBo0iHHjxmX0sM+E8ePHM2bMGEuHISIiIiIiIiIiIiLPiYEDBxIbG8uSJUuoV68e2bNnt3RIcp9aeWtRIXcFdgTt4NDNQ0TFReFk60T5XOWp7lkdZ1tnS4eYIfHx8cyaNYsyZcpQrVo1S4cjIiLPgedxWTjJHBlOKPn777/p3Lkz9vb2fPDBBxQvXhyAkydPMm3aNOLj4/n222+5fPkyX3/9NRMmTCB//vz07t07w8Fb2vDhwxk8eDAAu3fvpn79+haOSERERERERERERESeZfv372f48OG89dZblg5FHsDZ1pnXC77O6wVft3Qomeb8+fPs3LmTFStWcO7cORYtWmTpkERE5DnxPC4LJ5kjwwklP/zwAw4ODgQEBGBtbW1qL1u2LK1atcLHx4fFixfz9ddf06JFCypWrMisWbNeiIQSOzs77OyS1ohydEyZhSUiIiIiIiIiIiIi8l+urq7kyJHD0mHIS2jLli107dqVnDlz4ufnR/v27S0dkoiIPEee5WXh5MnJcELJ4sWLGT58uFkyiWlwGxvatm3LhAkT+Prrr03vX5Qlb0RERERERERERERE0qNVq1b89ddfvPfee5YORV4yXbp0oUuXLpYOQ0REHkFkXCTbg7Zz6MYhouOjcbRxpHyu8tTwqmHRpdde9GXhJKUMJ5SEhoYSGhr6wO0hISGm9zlz5sRgMGT0sCIiIiIiIiIiIiIiz50vv/yShg0bMmDAAD788EO8vb11z1xERERMtl3ZZqoCEhYbZqoCEng1kCWnlli8CsiLuCycpM0qowO8+uqrzJgxg4sXL6bYduHCBWbMmEG5cuVMbadOnSJPnjwZPayIiIiIiIiIiIiIyHMna9as7NmzhxkzZlCsWDFsbGywtrY2e9nYZPhZUJEXntFotHQIIiKZbtuVbcw8PJMzIWdINCbi5eJF4ayF8XLxItGYyJmQM8w8PJNtV7ZZOlR5SWT4qnTChAk0bNiQkiVL0rJlS4oVKwYkJY6sWLGCxMREFi1aBEBMTAwLFiygadOmGT2siIiIiIiIiIiIiMhzp1OnTqpIkkkMBgMJCQlKLHgJGY1GjEYjVlYZfm5aROSZERkXydxjcwmKCMLTxRNHG0fTNjtrO9yd3HGxcyEoIoi5x+ZSIXcFLTEjT1yGE0rq1KnDhg0bGDhwIIsXLzbbVrFiRSZNmkTt2rUBsLe35+LFi9ja2mb0sCIiIiIiIiIiIiIizx1/f39Lh/DCsLW1JTY2lri4OFV1ecncu3ePxMREnJycLB2KiEim2RG0g5vRN3G1czVLJvkvRxtHXO1cuRl9k8CrgTQo0OApRykvm0xJ3axZsyZ79uzh2rVr7Ny5k507dxIcHMyePXtMySTJ7O3tlTEqIiIiIvICi4qKMq0D379//xTbT506RcuWLcmWLRvOzs7UqlWLTZs2pTpWaGgoAwYMwMvLCwcHB0qXLs3MmTP1BKKIiIiIiJAlSxYAbt26RWJiooWjkafBaDQSHR1NcHAwAK6urhaOSEQk8xy8cZCw2DDc7N0e2M/N3o2w2DAO3jj4lCKTl1mmpuzmypWLXLlyZeaQIiIiIiLynPHz8+PmzZupbjt79izVq1fHxsaGIUOG4Obmxg8//EDDhg1Zs2YNvr6+pr6xsbE0aNCAgwcPMmDAAEqWLMmaNWt47733uH79OqNHj35KZyQiIiIikvk2b97M8uXLOXfuHADe3t68+eab1K1b18KRPT+yZcvGnTt3iIiI4J9//sHKykrLCf1H8rIwBoPhhZiX5PNJTh5ydXVVQomIvFCi46NJSEzA1urBq33YWtmSkJhAVFzUU4pMXmaZllCii18RERERETlw4ABTp05l4sSJfPzxxym2f/LJJ4SEhLB//37KlSsHJK0hX7p0afr168fJkydNNzp//PFH9u7dyzfffMOAAQMA6NmzJ61bt2b8+PF07dqVAgUKPLVzExERERHJDImJiXTu3JmFCxdiNBpNFb0TExP57rvv6NixI3Pnzn0hEgCeNIPBgKenJ8HBwapQkgqj0Uh8fDw2NjYvxPeTwWDAysoKJycnUzLJi3BeIiLJHG0csbayJi4xDjtruzT7xSXGYW1ljZOtlv2SJy/DCSW6+BUREREREYCEhAR69uxJo0aNaNWqVYqEksjISFauXImPj48pmQTAxcWFHj164Ofnx969e6lcuTIACxcuxMnJiZ49e5qN8+GHH7Js2TJ+/fVXhgwZ8sTPS0REREQkM02ePJkFCxbQpk0bRowYQcmSJQE4ceIEX3zxBQsWLODVV19NNUFbUrK1tcXNzQ03NzdsbR/8RPfLJi4ujtDQUM2NiMhzonyu8gReDSQ0JhR3J/c0+4XGhOJq50r5XOWfYnTysspwQokufkVEREREBGDKlCmcPHmSpUuXprr9yJEjxMTEUK1atRTbqlatCmBKKElMTOTAgQNUqFABBwcHs76VK1fGYDCwd+/eB8Zz+fJlrly5YtZ29OhRAOLj44mLi3vkc3uRxcXFaT4sRHNvOZp7y9HcW47m3jJehnmPj4+3dAjPHX9/f15//XV+/fVXs/ayZcuyaNEi7t69y08//aR76iIiIi+ZGl41WHJqCWdCzuBi54KjjWOKPtHx0YTFhlE0a1FqeNawQJTysslwQokufkVERERE5Pz584waNQo/Pz8KFizIhQsXUvS5evUqAF5eXim2JbcFBQUBcPfuXaKjo1Pta29vT86cOU190zJnzhzGjBmT6raIiAhCQ0MfuP/LIj4+nsjISABsbDJtVVR5BJp7y9HcW47m3nI095bxMsx7RESEpUN47pw7d4733nsvze3NmjVj0KBBTzEiEREReRY42zrTuXRnZh6eSVBEEK52rrjZu2FrZUtcYhyhMaGExYbh5eJF59KdteSNPBUZ/i1GF78iIiIiItKnTx+8vb0ZOHBgmn2ioqL4P/buPDzK8urj+PeZLZlkmARIABP2RUUWCQqyGFYBtXVXVKjGYt1FWZSq1QoUsWpVFCtWXIjiUlrxta3WqiwaEgWqgAioyCoTkLAkIckks75/pBkZshAySQbI7+PFpc+9PWdukzCZOXNuKE8IOVJFFZKKMTWNrRhfMaY6N954I2PGjAlrW79+PbfccgsOh4OEhIQa5zcVFZ+YdjqdKoPdyLT30aO9jx7tffRo76OjKey7w+GIdggnnPj4eH766adq+/fs2UN8fHwjRiQiIiLHi/S26QBkbsgkz52Hq8iFP+DHbDLjtDnpltiNjB4ZoXEiDS3ihBI9+RURERERadoWLlzIxx9/zGeffVbjGyVxceWfmigrK6vUV1paGjamprEV4yvGVKddu3a0a9euyj6LxXLSvqlTFxX7oT1pfNr76NHeR4/2Pnq099Fxsu/7yVp5pSGlp6fz3HPPcfXVV9OjR4+wvo0bN/LnP/+ZYcOGRSc4ERERibr0tun0bd2XbFc2a/PWUuItIc4aR1qrNAalDCLeqvfepfFE/GxfT35FRERERJqusrIypkyZwoUXXkibNm344YcfgJ+PrikoKOCHH34gKSmJlJSUsL7DVbRVHHHTvHlz7HZ7lWPLysrYt28fQ4cObZDHJCIiIiLSkGbOnMmAAQNIS0vjkksu4YwzzgBgw4YN/POf/8Rms1V7dKOIiIg0DfHWeEZ3HM3ojqOjHYo0cREnlOjJr4iIiIhI0+V2u8nLy+P999/n/fffr9S/cOFCFi5cyBNPPMGtt95KTEwMn3/+eaVxX3zxBQBnn302ACaTib59+7JmzRrKysrCjr5ZtWoVwWAwNFZERERE5ETSq1cvPv30U+6++27eeecd3nnnnVDfoEGDeOaZZ+jVq1cUIxQRERERKRdxQome/IqIiIiINF3x8fH87W9/q9Sel5fH7bffzvnnn8+NN95I7969cTgcXHTRRSxevJh169Zx5plnAlBUVMRLL71Et27d6N+/f2iNa6+9luzsbF588UUmTpwYap8zZw4Wi4Wrr7664R+giIiIiEgDOPvss8nOziYvL49t27YB0KlTJ5KTk6McmYiIiIjIz+rlgEs9+RURERERaZqsVitXXnllpfbt27cD0KVLl7D+Rx99lCVLljB69GgmT56M0+lk/vz5uFwu3n//fQzDCI296aabePXVV5kyZQrbt2+ne/fufPDBB7z77rs8+OCDdOzYsaEfnoiIiIhIg0pOTtbr6CIiIiJy3KqXhJIKevIrIiIiIiI16dq1K9nZ2dx333388Y9/xOPx0LdvXz788EPOO++8sLE2m41PPvmEBx98kLfeeov9+/fTpUsX5s6dyx133BGlRyAiIiIiIiIiIiLSNNRrQomIiIiIiAhAx44dCQaDVfZ1796d9957r1brJCYm8txzz/Hcc8/VZ3giIiIiIo3GZDJhMpkoKSnBZrNhMpnCKvNVxTAMfD5fI0UoIiIijaHYW8wK1wrW7l2L2+fGbrGT1iqNwamDibfGRzs8kSodc0JJbZ7sHklPfkVERERERERERESkKbr++usxDAOz2Rx2LSIiIk1H1q4sMjdkkufOo9BTiD/gx2wyk5Obw6LvFpHRI4P0tunRDlOkkmNOKNGTXRERERERERERERGR2lmwYEGN1yIiInJyy9qVxbx183AVuXDanKQ6UrGarHgDXgrKCticv5l56+YBKKlEjjvHnFCiJ7siIiIiIiIiIiIiInXz2Wef0b17d5KTk6vs37dvHxs3bmTIkCGNHJmIiIjUtxJvCZkbMnEVuUhxpGC32EN9NrON5LhkHDYHriIXmRsy6du6r46/keOKKdoBiIiIiIiIiIiIiIg0FcOHD+fjjz+utn/JkiUMHz68ESMSERGRhvLF7i/Ic+fhtDnDkkkOZ7fYcdqc5LnzyMnNaeQIRWp2zAklPp+vzjeLZK6IiIiIiIiIiIiIyIkuGAzW2O/3+zGZ9FlQERGRk8HXeV9T6CkkISahxnEJMQkUegpZs3dNI0UmUjvH/Kz0tNNO47XXXsPv99d6js/n45VXXuHUU0891tuJiIiIiIiIiIiIiJxUDMOoti8nJ4ekpKRGjEZEREQaitvnxh/wYzVZaxxnNVnxB/yUeEsaKTKR2rEc64Qrr7ySW265hWnTpjFu3DguuOAC+vXrR2JiYti4AwcO8MUXX/DBBx/w17/+leLiYu6+++76iltERERERERERERE5ITwzDPP8Mwzz4SuJ02axO9+97tK4w4ePEhhYSETJkxozPBERESkgdgtdswmM96AF5vZVu04b8CL2WQmzhrXiNGJHN0xJ5Q89thj3Hrrrfzxj39k/vz5oSfBzZs3p0WLFgSDQQ4cOEB+fj4ADoeDX/3qV0ybNo0OHTrUa/AiIiIiIiIiIiIiIse7xMTE0Ovj27dvp2XLlrRu3TpsjGEY9OzZkwEDBjB58uRohCkiIiL1rHdyb1bsWUFBWQHJccnVjisoK8Bpc5LWKq0RoxM5umNOKAHo1KkTf/nLX/jTn/7E+++/z2effcbGjRvJy8vDMAx69+5Nz549GTZsGOeffz7x8fH1HbeIiIiIiNRSMBjkk08+YfPmzezfv7/Sme2GYfDQQw9FKToRERERkZNfRkYGGRkZQPnr63/84x+5+OKLoxyViIiINLQBpwzgbz/8jc35m3HYHNgt9kpj3D43hZ5CuiV2Y3DK4ChEKVK9OiWUVGjWrBnXXHMN11xzTX3FIyIiIiIi9Wjz5s1ceumlfPvtt5USSSoooUREREREpPFs27Yt2iGIiIjIEYq9xaxwrWDt3rW4fW7sFjtprdIYnDqYeGvNxROqmtunZR/OiD+DBGsCGT0ymLduHq4iF06bk4SYBKwmK96Al4KyAgo9haQ6UsnokaEjb+S4E1FCiYiIiIiIHN8mTpzIli1beOyxxxgxYgQtW7aMdkgiIiIiIiIiIiLHjaxdWWRuyCTPnUehpxB/wI/ZZCYnN4dF3y0io0cG6W3Tj2nu6tzVdLF14Rc9fsGQDkMAQuNcRa7QOKfNSbfEbjXeQySalFAiIiIiInISy8rKYtKkSdxzzz3RDkVERERERP4nOzubRx99lJUrV3Lw4MEqj6X0+XxRik5ERKTpyNqVFVY9JNWRGlY9ZHP+ZuatmwdQKeGjprklZSXsKt7FS+tfwjAbpLdNp2/rvmS7slmbt5YSbwlx1jjSWqUxKGXQUaugiESLKdoBiIiIiIhIw4mJiaFTp07RDkNERERERP7ns88+Y/jw4axcuZJzzjmHQCDA8OHD6devH8FgkJ49e3LddddFdI9AIMDTTz/N6aefTmxsLO3atWPq1KkUFxfXav6jjz7KVVddRefOnTEMg44dO9Y4fuXKlZx33nk0a9YMp9PJ+eefz9q1ayN6DCIiIg2t2FtM5oZMXEUuUhwpJMclYzPbMAwDm9lGclwyKY4UXEUuMjdkUuwtrvXcJHsSSbFJ7C7eHZobb41ndMfRTOs3jemDpjOt3zRGdRilZBI5rimhRERERETkJDZmzBiys7OjHYaIiIiIiPzPI488wimnnMLGjRtZsGABAA888ABffPEFH374Idu2beM3v/lNRPeYPHkyU6ZM4YwzzmDu3LlcddVVPPvss1x00UUEAoGjzn/ggQdYunQpXbp0oXnz5jWO/eKLLxg6dCjbtm1j5syZzJgxg82bN5Oens769esjehwiIiINKduVTZ47D6fNid1ir3KM3WLHaXOS584jJzcnbO5PJT9hNszkleSxvWA7Owt3cqD0AP6AHwCb2UYzW7NKc0VOJEooERERERE5iT311FN8/vnnPPnkk3g8nmiHIyIiIiLS5K1atYrf/OY3JCcnYzKVv0RfkeQxevRorrvuOh566KE6r79hwwbmzp3L5ZdfzuLFi7npppt46qmneOqpp1i2bBlvv/32UdfYsmUL+/fv5+OPPyYlJaXGsXfddRc2m43PPvuMyZMnM3nyZD777DMMw2Dq1Kl1fhwiIiINbc3eNRR6CkmISahxXEJMAoWeQtbsXRNq+9fWf+EqclHoKeRA6QEOlh7kQOkBXIdc/JD/AwVlBQA4bc5Kc0VOJJZoByAiIiIiIvWnc+fOldqKioqYNm0a9913HykpKZjN5rB+wzDYsmVLY4UoIiIiItKklZWVkZqaCpQfUQlw6NChUH+fPn1YuHBhndd/6623CAaDTJo0Kaz9pptu4r777mPhwoWMGzeuxjWq+r2iKj/88AOrV69mwoQJoccEkJqaylVXXcWrr77Knj17aNOmzTE/DhERkYbm9rnxB/xYTdYax1lNVvwBPyXeEgCydmWxdu9afAEfNrOt/KgbDIIE8QV8lHhL2F28mxbxLbDawudC+XE5K1wrWLt3LW6fG7vFTlqrNAanDtbxN3LcUUKJiIiIiMhJpH379hiGEe0wRERERESkGqeccgq7du0CID4+nsTERL755hsuu+wyAHbt2oXFUveX7levXo3JZKJ///5h7bGxsfTp04fVq1fXPfgq7gUwcODASn0DBgzglVde4csvv+QXv/hFjev8+OOPoT2pUHFcjs/nw+v1VjvX6/UedUxTpb2pmvaletqb6mlvqnci7028KR67yU7QH8Rqrj6pxOP3YDfZcZgdFJQU8MY3b+D3+Ykz4rCZbJiMww4FMYPf5AcfHCo9hNWwhuZ6vV4+z/2cNze9yb7SfRzyHMIf8GM2mVmdu5rF3y5mXPdxDEyp/PfqyeRE/pppaA25Nz6fr07z6i2hZPPmzWzevJn9+/cTDAYr9V9//fX1dSsREREREanG8uXLox2CiIiIiIjUoF+/fmRnZ4euR48ezdNPP02HDh0IBAI899xznHPOOXVePzc3l6SkpFD1k8OlpqaSk5ODx+PBZrPV+R6H36ti3aruBeByuY66zssvv8yMGTOq7CsqKqKgoKDauT6fj+LiYoCIEnFORtqbqmlfqqe9qZ72pnon8t70dPTkB9sPBD3BGo+9KfAU0NHWkZ7xPcnZmkOwNEjnmM64fW4AzCZz5UkGJJCAu8wdmpv1Qxbv/fAeh0oP0cLSgnZx7bCYLPgCPoq9xRw6dIh3vn4HX4mP3sm9G+phR92J/DXT0Bpyb4qKiuo0L+IofvrpJzIyMvj4448BqkwmMQxDCSUiIiIiIlHw2Wef0b17d5KTk6vs37dvHxs3bmTIkCGNHJmIiIiISNN04403smDBAtxuN3a7ndmzZ5OVlcUNN9wAQJs2bXj88cfrvH5JSUmVySRQXqWkYkx9JJSUlJSX76/qfoff62huvPFGxowZE9a2fv16brnlFhwOBwkJ1b/JV/EJXqfTidVa85EFTY32pmral+ppb6qnvaneibw3g+IG8d6u99hasJU21jbYLfZKY9w+N3s8e+iS0IVBnQfxl6//wnbPdlrbW7OvaB+lvlJs5iOqlACWoIWgL8je4F5Ob3Y6Z7Y/kxk5M9hSsoU28W3AAsWUJw5gAixg8pnYWLyRsh1lnNXhLOKscY2wC43vRP6aaWgNuTcOh6NO8yJOKLnzzjv5+OOPue222xgxYgQtW7aMdEkREREREaknw4cP5/XXX6/2jPQlS5Ywbtw4/H5/I0cmIiIiItI0jRo1ilGjRoWuO3fuzPfff8+SJUswm82ce+65NSZQHE1cXBx79+6tsq+0tDQ0pj5UrFNWVhbRvdq1a0e7du2q7LNYLEd9Q6VijN6Uqkx7UzXtS/W0N9XT3lTvRN2bBGsC43uOZ966eews2onT5iQhJgGryYo34KWgrIBCTyGpjlTG9xyPM85JcaAYd8CN2WqmRVwLcotzOeQ/hMWwYDaZMWEiQACT30ScKQ6LxcL4nuNZf2A9u0t3E2uLxWq14qPy8SNWq5VYWyy7S3ezKm8VozqMqiLqk8OJ+jXTGBpqb+pa8STihJKPP/6YW2+9leeeey7SpUREREREpJ5VVUHwcH6/H5PJVOMYERERERFpWPHx8Vx88cX1slZKSgobN26krKysUuUQl8tFUlJSvVQnqbhXxbpHqmir6jgcERGR40V623QAMjdkkufOw1Xkwh/wYzaZcdqcdEvsRkaPjNA4u8WO2WTGG/CSGJsIwJ6SPXj9Xjx+D8FgEMMwiCUWs2Gmd3Jv0tum89iqx0LJKTVJiEnAVeRizd41J3VCiZw4In7lOBAIcOaZZ9ZHLI2qrKyMm266ic6dO9OsWTNOPfVU5s6dG+2wRERERETqnWEY1fbl5OSQlJTUiNGIiIiIiEhD6tevH4FAgFWrVoW1l5aWsnbtWs4+++x6vRfA559/Xqnviy++wDAMzjrrrHq7n4iISENIb5vOMyOe4Y4+d3BBpwsY0X4EF3S6gDvT7uSZEc+EkkkA0lql4bQ5KSgrACAxNpFuid1IdaTSIrYFzWOb0yK2Bc4YJ8n2ZM7vdD5QfnSOP+DHaqq56oTVZMUf8FPiPfqRcSKNIeKEkvT0dNatW1cfsTQqn89HmzZt+OijjygoKGDRokXMmjWLRYsWRTs0EREREZGIPPPMM3Tu3JnOnTsDMGnSpND14X+aN2/OvHnz+OUvfxnliEVEREREmpY333yTwYMH06pVK8xmc6U/dS1JDnD11VdjGAZz5swJa58/fz4lJSWMHz8+1LZlyxa+/fbbOt+ra9eunH322fztb38jNzc31J6bm8vf/vY3RowYQZs2beq8voiISGOJt8YzuuNopvWbxvRB05nWbxqjOowi3hofNm5w6mCS7ckUegpx+9wAmE1mWthb0N7Zno4JHUmOS8Yf9NM8pjkD2gwAwiub1MQb8GI2mYmz1s/xdCKRivjIm6eeeorhw4czYsQIrrjiivqIqVHEx8fzhz/8IXTdp08fLr74YlasWMHYsWOjGJmIiIiISGQSExPp0KEDANu3b6dly5a0bt06bIxhGPTs2ZMBAwYwefLkaIQpIiIiItIkzZo1i4cffpjWrVszaNAgmjdvXq/r9+rVizvuuIPnnnuOyy+/nAsvvJBNmzbx7LPPMnToUMaNGxcaO3LkSHbs2FHpqMzXX3+dHTt2AJCXl4fH42HWrFkAdOjQgeuuuy409plnnmH48OGkp6czceJEAObOnUsgEODJJ5+s18cmIiISbfHWeDJ6ZDBv3TxcRS6cNicJMQlYTVa8AS8FZQUUegrpEN+BMZ3GYLfagfLKJjm5ORSUFZAcl1zt+gVlBThtTtJapTXWQxKp0TEnlIwYMaJSm8PhYOzYsaSkpNC5c2fMZnNYv2EYLFmypM5BPvroo3z11Vd8+eWXbNu2jQ4dOrB9+/YqxwYCAZ555hn+8pe/sH37dpKTkxk7diwzZ84kPj6+yjkAXq+XrKws7rnnnjrHKSIiIiJyPMjIyCAjIwOATp068cc//rHezmMXEREREZHIPP/88wwbNowPP/wQq7Xmsvd1NWfOHDp27MiLL77I+++/T1JSEhMnTmTmzJmYTEcvXP7yyy/z6aefhrU99NBDAAwdOjQsoWTQoEEsX76cBx98kAcffBDDMBg0aBB/+9vfOPPMM+v3gYmIiBwHKo7AydyQSZ47D1eRC3/Aj9lkxmlz0i2xG9effj1nxJ8RmjM4dTCLvlvE5vzNOGwO7BZ7pXXdPjeFnkK6JXZjcMrgRns8IjU55oSSrVu3VnkGe/v27QHYuXNn5FEd4YEHHqBFixb07duX/Pz8GsdOnjyZZ599lssuu4ypU6eGMq/XrFnDJ598Uu2T5TvvvJNmzZpx/fXX13v8IiIiIiLRsm3btmiHICIiIiIihyksLGTs2LENlkwCYDabmTp1KlOnTq1xXHUf3Fy+fPkx3W/gwIERfahURETkRJPeNp2+rfuS7cpmbd5aSrwlxFnjSGuVxqCUQdiwUVBQEBpf28omqY5UMnpk6MgbOW4cc0JJdU8wG9KWLVtC57/37NmToqKiKsdt2LCBuXPncvnll/POO++E2jt16sRdd93F22+/HVbOr8KUKVP4/PPPWbp0KTabrWEehIiIiIiIiIiIiIg0eWlpafz444/RDkNEREQiFG+NZ3TH0YzuOLpSn9frrdRWm8omGT0yQuNEjgfHnFBypJ07d5KcnIzdXrksD4Db7SYvLy9UwaQuKpJJjuatt94iGAwyadKksPabbrqJ++67j4ULF1ZKKJk0aRJLlixh6dKlJCUl1TlGEREREZHjgclkqrKiYE0Mw8Dn8zVQRCIiIiIicrhZs2ZxxRVXcMUVV5CWlhbtcERERKQRHa2ySbw1PtohioSJOKGkU6dOvP7661VW/gD4xz/+wbhx4/D7/ZHe6qhWr16NyWSif//+Ye2xsbH06dOH1atXh7XfddddLF26lGXLlpGcnNzg8YmIiIiINLTrr7++UkLJl19+yTfffMNpp51G9+7dAdi4cSPff/89PXv25KyzzopGqCIiIiIiTdLQoUN5+eWXGTBgAAMGDKBjx46YzeawMYZh8PLLL0cpQhEREWlINVU2ETneRJxQEgwGa+wPBALH/AnJusrNzSUpKYmYmJhKfampqeTk5ODxeLDZbOzYsYO5c+cSExNDp06dQuPS09P597//XeN9fvzxR3bt2hXWtn79+vp5ECIiIiIiEViwYEHY9ccff8zf//53/u///o+LL744rO///u//uO6663jqqacaMUIRERERkaZt5cqVZGRk4PV6ycrKIisrq9IYJZSIiIiIyPEg4oQSoMaEkU2bNpGYmFgftzmqkpKSKpNJoLxKScUYm81Ghw4djpoMU52XX36ZGTNm1DlOEREREZHG8tBDD3HLLbdUSiYBuPTSS7n55pt58MEHOe+886IQnYiIiIhI03P33Xdjs9l47733SE9Pb7TXz0VEREREjlWdEkoyMzPJzMwMXc+aNYv58+dXGnfgwAG++eYbLrvssrpHeAzi4uLYu3dvlX2lpaWhMZG68cYbGTNmTFjb+vXrueWWWyJeW0RERESkPn399ddkZGRU29+1a1deeOGFRoxIRERERKRp+/rrr5k+fToXXXRRtEMREREREalRnRJK8vPz2bZtG1BenSQvL4+SkpKwMYZh4HA4mDBhAo888kjkkdZCSkoKGzdupKysrFKlEpfLRVJSEjabLeL7tGvXjnbt2kW8joiIiIhIQ2vevDkfffQRt912W5X9H374IQkJCY0clYiIiIhI09WqVat6eZ1aRERERKShmeoy6e6772bbtm1s27aNYDDInDlzQtcVf7Zu3crXX3/Niy++SHJycn3HXaV+/foRCARYtWpVWHtpaSlr167l7LPPbpQ4RERERESOF+PGjeO9997jxhtvZNOmTfj9fvx+P5s2bWLChAn861//Yvz48dEOU0RERESkyZgwYQILFy7E5/NFOxQREZF6Vewt5j/b/8Njqx5jes50Hlv1GB9t/4hib3G0QxOROqpThZLDBQKB+oijXlx99dXMnj2bOXPmkJ6eHmqfP38+JSUleqFcRERERJqcWbNm8cMPP/Dqq6+yYMECTKbynPJAIEAwGOSiiy5i1qxZUY5SRERERKTpOPfcc/nXv/7FgAEDuP322+nUqRNms7nSuCFDhkQhOhERkbr5PPdzXvv2NfLceRR6CvEH/JhNZnJyc1j03SIyemSQ3jb96AuJyHEl4oSSxvD666+zY8cOAPLy8vB4PKEXvTt06MB1110HQK9evbjjjjt47rnnuPzyy7nwwgvZtGkTzz77LEOHDmXcuHFRewwiIiIiItEQExPDu+++y0cffcR7773H1q1bAejcuTOXXHIJo0ePjmj97777jpkzZ/LVV1+Rm5uL1+ulffv2XHjhhdx7772ccsopobHTp09nxowZVa7zxBNPcM8994S1BQIBnnnmGf7yl7+wfft2kpOTGTt2LDNnziQ+Pj6iuEVEREREouW8884L/fdvfvMbDMMI6w8GgxiGgd/vb+zQRERE6uTrvK9ZsGUBO4p34LQ5SXWkYjVZ8Qa8FJQVsDl/M/PWzQOoMqmk2FvMCtcK1u5di9vnxm6xk9YqjcGpg4m36jUgkWg65oSSzp07H/NNDMNgy5Ytxzyvwssvv8ynn34a1vbQQw8BMHTo0FBCCcCcOXPo2LEjL774Iu+//z5JSUlMnDiRmTNnhj6NKSIiIiLS1IwePTri5JGq7Nq1i927d3PZZZfRtm1bLBYL69ev58UXX+Ttt99m7dq1tGrVKmzO008/TVJSUljbWWedVWntyZMn8+yzz3LZZZcxderUULL4mjVr+OSTT/T8XkREREROSK+++mq0QxAREak3Jd4S/rPtP+wu3k2KIwW7xR7qs5ltJMcl47A5cBW5yNyQSd/WfcOSRLJ2ZZG5IVOVTUSOU8ecUNK+fftKGdO7du1iy5YtOJ3OUMLJ1q1bKSwspEuXLrRt2zaiIJcvX17rsWazmalTpzJ16tSI7lkbHo8Hr9cLgNvtbvD7iYiIiIgcb0aOHMnIkSMrtQ8ZMoSxY8eyYMECpk2bFtZ36aWX0rFjxxrX3bBhA3PnzuXyyy/nnXfeCbV36tSJu+66i7ffflsVCEVERETkhJSRkRHtEEREROrNF7u/IN+TTzNbs7BkksPZLXacNid57jxycnMY1WEUUJ5MMm/dPFxFrjpVNhGRhnfMCSVHJnd89dVXnHfeecyZM4dbb70Vm80GlCdbPP/88/zhD3/gr3/9a70Ee7yZPXt2pZLdV111FXZ71T8sK9x2221MmTIldL19+3ZGjRp11PulpqZW2v/LL7+c9evXH3Xu22+/HfapzwULFvDII48cdd748eOZPn166PrAgQOcc845R53XrFkzvvrqq7C2G264gezs7KPOffHFFxk+fHjo+p133uG+++476ryLLrqIp556KnTtDQQY/u8PjzoPYMUvLgy7fvDLr1jRrdtR5z355JNcfPHFoeuPP/6Y22+//ajzhg0bxvz588PaevfuXavEpC+//BKn0xm6/t3vfseiRYuqHe/bsweAyT16cEXHDqH2r/bv564vVh71fr2aN+cfR7Sde+65/PTTT0edu2zZsrCEsscee4yXXnrpqPMmT54cto8bN27kkksuOeq8rl278u9//zus7YILLuCHH34Aft6Lqrxy7mBOTUgIXWdu/oH5339faZzliK+L3/zmN/z2t78NXe/atSvs67c6rVu3ZsWKFWFtV199daXvm6osXLgw7Ptw4cKF1R4fcLixY8eGfd8f8nq54KOPjzov1mzmk/PHhLXddNNNtUr2e/7558N+vv3jH/+oVcLf+eefz9y5c8PautXiexLKv16sVmvoesqUKfzzn/+scuzhXxP39+7NL9r9/PWa/dNefvvf/x71fkNvuIEFCxaEtfXt25dDhw4dde7KlStp0aJF6Hr69Om88cYbR533u9/9jhtuuCF0/eWXX3LNNdccdV6vXr1YvHhxWNuwYcNwuVw1fn8AvDVsKO0OO17ixe++47Uffq4+duT3RoWJEydy1113ha43b97MhRdeWOXYw3Xo0IFPPvkkrO3iiy9m06ZNR537zjvv0Lt379D1/Pnzefzxx486LyMjg18fdr3X7ebypcuOOi/BZuP9UeeFtY0fP55Vq1Ydde6rr77KueeeG7r+61//yoMPPnjUeZdffjmPPfZY6LqkpIQzzzzzqPMsFkulPbztttsq7TVU/pk5s28aIw47umTp7t38/qs1Nd+vWzfOO+885s2bF9bevXt3fD5fjXOVrFt3M2fOxDAMfve732EymZg5c+ZR5xiGEaoAWF86dCh/3nHw4MEq+wsLC4mLi8NiqfpXkrfeeotgMMikSZPC2m+66Sbuu+8+Fi5cqIQSEREREREREZEo+zrva0q8JTjjnDWOS4hJwFXkYs3eNYzqMIpibzGZGzJxFbnqVNlERBrHMSeUHOmee+5h7NixYW8WAdhsNiZNmsSmTZu49957WbJkSaS3Ou488MAD3HvvvUD5m4IjR47E5XIddd7+/fvDrr1eb+gN75pUVEM53I8//liruaWlpWHX+fn5tZqXl5cXdu33+2s1L+GwN+YruFyuWs0tKSkJuz506FCt5lWV4LC9qOio86qSV1rKD7t2HXXckW8YFxcX1yrWU089tVLbli1bKj32qgQCgbDrvXv31uqeh7yesOtSv79W+5MUE1Opbdu2beTm5h517pFvFu7bt69WsR75xpPH46nVvIqEtsPt3Lmzdt9fR+xrgddT9f4csda+ffvCrn0+X52+zqE8GaU2c498k7WgoKBW8/bu3Rt2HQgGa/U1YDebK7Xl5ubW6p7FxcVh17X9ft5TRXJDbeZV5aeffqpdrL7wn7Glfl+t9qdrFT/3t27dSkFBwVHnHnkWcl5eXq1izc/PD7suLS2t1bzExMRKbdu3b2fHjh1HnXvk98jBsiO+R6q5/4EDB8Kua/v9XJXafj+XlZWFXdf277wjv5/9tfweaVHFz8na/p135PdzYWFhnf7OCwaDtZpX1Rv3u3fvrt3PrSN+ppf4avE98sMP9OjRo4rmH46aUCJ1N336dAzD4Le//S02my0sObg69ZFQUlpaSlFREaWlpWzcuDGUcFlVElnv3r05dOgQZrOZ/v3789BDD3HBBReEjVm9ejUmk4n+/fuHtcfGxtKnTx9Wr14dUbwiIiIiIo3ls88+A8qr+B1+fTQV40VERI5nbp8bf9BPrCkWP/5qx1lNVvwBPyXe8vdHsl3Z5LnzcNqcx1zZREQaT8QJJatWrWLs2LHV9qelpfHmm29Gepvjks1mC72BXVGVJDU19agVSlq2bBl2bbVa6dq161Hvl5qaWqmtXbt2ld5YrEpsbGzYdWJiYq3umZycHHZtNptrNa9Zs2aV2lJTU2s1Ny4urtJatZnXunXrSm0dHY6jzqtKcmxsnR5nfHx8realpKRUauvSpUutPo1tMpnCrlu1alXjPSs+Yd7MGp5sEWs212p/Tjni/weUl5o/8v9TVY584zIpKalW+9O8efOwa5vNVqt57du3r7LN4ylPpqmpAoP1iH1NsNqq3B9LmzZh10lJSeH9Fkudv17btm1bq7lH/oxJSEio1bxWrVqFXZsMo1ZfA7FVJJSkpKTU6p7x8eHZwrX9fm5zxD4DtZpXldatW1c79/CviXiLNawv1myp1f5U9bO5c+fOtapQYj5ib5OTk2v1OI9MDImt5c+sdu3aVWrr2LEjVqv1qBVKjvweaR4T/j1y5PdGhcMrsEDtv58rKhscrn379pWSlKoSc0SCR23/zktKSoI9u0PX5lp+jyRUkcxW27/zjvx+djqddfoZYhhGreZVlVByyimnVDn3yK+JuCPmxlmO/j1iadOGUw6ralKha9eutapQUptEXals27ZtwM+JlhXXDe2ll15i4sSJoeuOHTuycOFC0tN/LkeamJjIzTffzKBBg2jevDnfffcdc+bM4Re/+AWvvPJKWPWl3NxckpKSKn1PQ/n3WE5ODh6Pp8qE0go//vgju45IEq6o7ufz+apM2G6KvF6v9iNKtPfRo72PHu199Gjvo6Mp7LuSxY9u2LBhGIaB2+3GZrOFrqsTDAYxDKPSB1FERESOR3aLHbNhxhvwVnoP63DegBezyUyctfz9pTV711DoKSTVUfk19sMdWdlERBqXEQwGg5EskJyczC9/+UteffXVKvszMjL44IMPKlW6ONl8/vnnDBo0iJycHAYOHBjtcARwXXN1RPNT3z45jmqKdB9Ae3G4k2EvtA8/0178TD8zy+lr4mfR3gs9tzrx7Nq1i2+//ZaioiLWrFnDP/7xD2644QbuvvvuGuft37+fnj17Ulpayo8//ojjf4lKXbp0wev1snPnzkpzrr/+el5//XUOHjxYZfWlCtOnT6/2WLgPPviAfv361f4BnsR8Ph/FxcXEx8dXewSRNAztffRo76NHex892vvoaAr7vnr1ai688EI9d6/BggULMAyD66+/HsMwyMzMrNW8jIyMBo7s+Fbb3wu9Xi8FBQUkJCSEHYMs2pvqaF+qp72pnvamev/Z8h8Wr19MXjCPxLjEasflleRhMkzcmXYnozqMYnrOdJbuXEqXxC5HTbTckr+FEe1HMH3Q9Pp/AA1EXzPV095UryH3pq6vuUf8W8yll17KK6+8QqdOnZgyZUroBeCioiKefPJJFi5cyIQJEyK9jdRStN/8EREREZHjy+rVqzn77LNr/MW8PrRt25a2bdsC5b8jXHHFFfTr14+SkhLuv//+aue1bNmSW2+9lenTp5OTk8Po0aOB8qp1Rx7XVqHiOMejVUy78cYbGTNmTFjb+vXrueWWW3A4HFUe09gUVXxi2ul06pf4Rqa9jx7tffRo76NHex8dTWHfHXWsENyUHF6JD5QoIiIiJ5cBpwxg6XdL2XpoKzG2mCqPr3H73BR6CumW2I3BKYOB/1U2MZVXNrGZq69Ae2RlExFpXBEnlDzxxBOsW7eO6dOnM2vWrFBZ8927d+Pz+ejbty9PPPFExIGKiIiIiMixO+ecc0hISGDIkCGMGDGCESNG0KtXrwa/b+/evUlLS+P555+vMaEEyo/HAdi3b1+oLSUlhY0bN1JWVlbp2BuXy0VSUlKNx91A+VFfVR33BeVHQJ2sb+rURcV+aE8an/Y+erT30aO9jx7tfXSc7Pt+slZeibaKT6aKiIgc7+KscYzpNIbdW3azo2gHTpuThJgErCYr3oCXgrKC0NE2GT0yQokhaa3SyMnNoaCsgBaxLSgoK6DIW0QgGMBkmHBYHSTEJFBQVoDT5iStVVqUH6lI01T9QVa1lJiYSE5ODi+88AKjRo0iLi6OuLg4Ro0axQsvvMDnn39eYxlqERERERFpOH/+858ZNWoUn3/+OZMnT6ZPnz60bt2aq6++mr/85S9s3ry5we7tdrs5cODAUcdVxNC6detQW79+/QgEAqxatSpsbGlpKWvXruXss8+u32BFRERERBrJyJEj2bNnT7X92dnZ9OnTp/ECEhERiVDv5N78ptdv6JbYDZNhwlXkYkv+FlxFLkyGiW6J3bjtzNtIb5semjM4dTDJ9mT2u/fz7cFvcRW5OFB6gIOlBzlQegBXkYtvD37Lfvd+ku3JocomItK46iV93GKxcPPNN3PzzTfXx3IiIiIiIlJPbrvtNm677TYAvv76a5YuXcrSpUv56KOP+Pvf/w6UH1ezY8eOOq2/Z88e2rRpU6l92bJlfPPNNwwbNgwAn89HcXFxpU9Z/vjjj8ybN4+WLVsyaNCgUPvVV1/N7NmzmTNnDunpP7/YMH/+fEpKShg/fnyd4hURERERibacnBzOPPNMXn31VS688MJQezAYZNasWfzhD38IVQIXERE5UQxMGchZKWeR7cpmbd5aSrwlxFnjSGuVxqCUQcRb48PGx1vjObv12Xy972tKfaWYDTNWsxUzZvz48fq9+P1+Yi2xnN36bB15IxIlqkcYAY/HEzoH1e12RzkaEREREZGa9e7dm969e3PllVfyn//8h6effpqNGzeya9euOq952223sXv3bkaMGEGHDh0oLS3lyy+/5O2336ZZs2Y8+eSTABQVFdGpUycuvfRSunfvTvPmzfnuu+946aWXKCoq4q233sJu//mM3V69enHHHXfw3HPPcfnll3PhhReyadMmnn32WYYOHcq4ceMi3g8RERERkWhYuXIlV199NRdddBF33303jz32GHv37uVXv/oVn376KZdccgmvvPJKtMMUERE5ZvHWeEZ3HM3ojqOr7C/2FrPCtYK1e9dyyHOIlbtXEgwGsVvsBIIBfAEf3qAXwzCwmCzEmGIwMPjvT/+l2FtcKSlFRBreMSeUvPbaawBcd911GIYRuj6a66+//lhvddybPXs2M2bMiHYYIiIiIiI1OnDgAEuXLmXJkiUsXbqUH374AbPZTFpaGr/97W8ZOXJknde+9tpree2113j99dfJy8vDMAw6dOjALbfcwr333kv79u0BsNvtXHHFFaxcuZL/+7//o6ioiKSkJM477zymTZtG//79K609Z84cOnbsyIsvvsj7779PUlISEydOZObMmZhMEZ/eKSIiIiISFb179+arr77izjvvZM6cOSxZsoTc3FyKi4t57rnnuP3226MdooiISL3L2pVF5oZM8tx5FHoKcfvclPpKMRkmrCYrzhgnwWCQQDCAyTDhsDlIsCVwoPQAee48cnJzGNVhVLQfhkiTc8wJJTfccAOGYXDNNddgs9lC18FgsNo5hmGclAklDzzwAPfeey9QnlUeyQvxIiIiIiINIS0tjfXr1xMMBunRowcXXHABI0aMYNiwYTidzojXHzt2LGPHjj3quJiYGF566aVjWttsNjN16lSmTp1a1/BERERERI5LdrudF154ge+//57s7GwMw2Du3LlKJhERkZNS1q4s5q2bh6vIhdPmJNWRyu6i3ZT5yjAMA7fPTYAAKY4UEmMSw+YmxCTgKnKxZu8aJZSIRMExJ5QsW7YMAJvNFnbdFNlsttA+HF6eW0RERETkeLFu3TpMJhO/+tWvuPbaaxkyZAhxcTpzVkREREQkmrZs2cI111zDV199xfjx41mxYgWTJk1i//79PPjggxiGEe0QRURE6kWxt5jMDZm4ilykOFKwW8rfUw1SXqwgxhSD3+SnzFfGj4U/8pPpJ7xBLwA2k43msc3xBXyUeEui9hhEmrJjTigZOnRojdciIiIiInL8ePPNN0NH3bz++utYrVb69evHyJEjGTFiBIMGDcJqtUY7TBERERGRJuONN97g9ttvx2Qy8fbbb3PVVVdRUFDAhAkTePjhh1m2bBlvvPEGp5xySrRDFRERiVi2K5s8dx5OmzOUTAJgMkwYhkGAAMFgEH/Qj8/vo9RfGhrjxk2hpxCTYeJg6cFohC/S5EV88HheXl59xCEiIiIiIg3gmmuuYf78+WzZsoVt27Yxb948OnXqxEsvvcSIESNITExk9OjR0Q5TRERERKTJuO666zjjjDNYs2YNV111FQAJCQm88847PP/886xcuZIzzzwzylGKiIjUrNhbzH+2/4c5X85hwTcLmPPlHD7a/hHF3uKwcWv2rqHQU0hCTEJYu8PqwGyY8fg9lPnKQhVLoDzZxGSYMDAIUp5ssn7ferJ2ZTXKYxORnx1zhZIjtW7dmh49ejBixIjQWewJCQlHnygiIiIiIo2qQ4cOTJgwgYEDB9K/f39eeOEFNm3axJIlS6IdmoiIiIhIk3HvvffyyCOPYLFUfnn+1ltv5dxzz+Xaa6+NQmQiIiK1k7Uri8wNmeS58yj1lNKSluxnPyv2rGDRd4vI6JFBett0ANw+N/6AH6spvEJuQkwCe0v2UuYNTyYx/vdPhYq+A+4DvPrNq/Rt3Zd4a3ylmIq9xaxwrWDt3rW4fW7sFjtprdIYnDq4yvEiUjsRJ5Tce++9LFu2jOeff565c+diNpvp06dPKMEkPT1dZ7SLiIiIiETRtm3bWLp0KUuXLmXZsmX89NNPACQmJnLppZcyYsSIKEcoIiIiItJ0PPbYYzX29+zZk9WrVzdSNCIiIscma1cW89bNw1Xkwmlzckr8KSQbydiCNvZ79rM5fzPz1s0DIL1tOnaLHbPJjDfgxWa2hdYxm8w4bA6KvEVh6wcJEgwGQ4kkJsOE2TATDAbZeWgnObk5jOowqlJMFQkuhZ5C/AE/ZpOZnNycSgkuInJsIk4oqXjyW1hYyPLly0MvUv/pT3/iT3/6E1arlf79+/PZZ59FHKyIiIiIiBybTp06sXPnToLBIA6Hg3PPPTeU/J2WloZhGEdfRERERERE6t2yZct499132bp1KwCdO3fmsssuY/jw4cTGxkY5OhERkcqKvcVkbsjEVeQixZGC3WLHErRAAGxmG8lxyThsDlxFLjI3ZNK3dV/SWqWRk5tDQVkByXHJYesFgoEq73NkxZKKsQVlBazZuyYsoeTIBJdURypWkxVvwEtBWUGlBBcROTYRJ5RUcDqdXHzxxVx88cV4vV7ee+89pk+fzsaNG8nOzq6v24iIiIiIyDHo2LEjN954IyNGjKB///5VltUWEREREZHGEwgEyMjI4M033yQYDGIymULtf/7znxk/fjyZmZlK/hYRkeNOtiubPHceTpsTu8Ve5Ri7xY7T5iTPnUdObg6DUwez6LtFbM7fjMPmCJt3ZEJJxXE3FQklQYIEggECwQBBgpT5y9iavzU0vqoElwrVJbjo+BuRY1MvryYHg0G+/PJLlixZwtKlS8nOzsbtdpOcnMzVV1/NyJEj6+M2xx2Px4PX6wXA7XZHORoRERERkcqWLVsW7RBEREREROQwTz75JG+88QZXXXUVv/vd7+jevTsAmzZt4tFHH+WNN97gzDPPZOrUqVGOVEREThTF3mJWuFawdu9a3D43doudtFZpDE4dXK8JFGv2rqHQU0iqI7XGcQkxCbiKXKFqIhk9MsKqiCTEJGA1WcMqkZgMExbDgi/og+D/kkuMnxNMKo7B+fbAt2TtyiK9bXqdElyOPC5HRGoWcULJZZddxqeffkpBQQFOp5OhQ4cye/ZsRo4cSY8ePeojxuPW7NmzmTFjRrTDEBEREREREREREZETxIIFCxg9ejR//etfw9p79+7NW2+9xcGDB3nllVeUUCIiIrWStSuLzA2Z5LnzKPQU4g/4MZvM5OTmsOi7RWT0yKi3o17cPjf+gB+ryVrjOKvJij/gp8RbAvx81ExFnK4iF/6AH1/Qh8kwEQgGMBkm/EE/wWAwlEhSIRgMYmBgNpkp85eFqo3UNcFFRGov4oSS9957D7PZzK9+9SumTZt20ieRHO6BBx7g3nvvBWDlypUnbSUWEREREREREREREakfW7du5fbbb6+2/6KLLuKee+5pxIhERORElbUrK6zyR6ojFavJijfgpaCsgM35m5m3bh5AvSSV2C12zCYz3oAXm9lW7ThvwIvZZCbOGhdqS2+bTt/Wfcl2ZbM2by0l3hKsJiurd69me+F2AoHyY20qjr2pEPzfPwYGdrOd5rHNQ9VG6prgIiK1F3FCySOPPMLSpUv5+9//zsKFC2nTpg3Dhw9n5MiRjBgxgg4dOtRHnMclm82GzVb+w9Jur7qMkoiIiIiIiIiIiIhIhfj4eH766adq+/fs2UN8fP0dTyAiIienYm8xmRsycRW5SHGkhB35YjPbSI5LxmFz4CpyhSp6RHr8TVqrNHJycygoKyA5LrnacQVlBThtTtJapYW1x1vjGd1xNKM7jg61Ze3K4vHVj7OjcMfPR+AEf04kOfwxtXG0Ic4SF6o2EkmCi4jUjinSBe6//34+/vhjDh48yJIlS7jxxhvZsWMHt956K507d6ZLly7cfPPN9RGriIiIiIiIiIiIiMgJLT09neeee44NGzZU6tu4cSN//vOfGTJkSBQiExGRE0m2K5s8dx5OmzMsmeRwdosdp80ZqugRqcGpg0m2J1PoKcTtc1c5xu1zU+gppGVsS9w+N4+teozpOdN5bNVjfLT9I4q9xWHj09umM63fNJrZmgHliSQBAqFkEhMm7BY77Z3tSYxJDKs2ktYqDafNSUFZQY1xV5fgIiJHF3GFkgo2m41hw4YxbNgwDh06xL/+9S9mzZrFpk2bePnll3nxxRfr61YiIiIiIiIiIiIiIiekmTNnMmDAANLS0rjkkks444wzANiwYQP//Oc/sdlszJgxI8pRiojI8W7N3jUUegpJdaTWOC4hJiFU0WNUh1ER3TPeGk9Gj4ywY3Za2lqCAR6/h/2e/RR6CnHanBR6Cnl5/csUegrxB/yYTWZycnNY9N0iMnpkhB3Bk942nQs6XsD7297HMAx8fh8ANouNFjEtaB7bHLPJDIRXGxmcOphF3y1ic/5mHDZHlYk1FQku3RK7MThlcESPX6QpqpeEkrKyMlasWMHSpUtZunQpX375JX6/H4A+ffowYsSI+riNiIiIiIiIiIiIiMgJrVevXnz66afcfffdvPPOO7zzzjuhvkGDBvHMM8/Qq1evKEYoIiInArfPjT/gx2qy1jju8Ioe9aEiESRzQyZ57jx2F+/Gg4f97CfWFkuyPZkyf1moekqqIxWryYo34KWgrIDN+ZuZt25e2FoA/U/pz+qfVhMIBmp9nE5VCS4JMQlh96tIusnokaEjb0TqIOKEkuHDh/PFF1/g8XgIBoOcfvrp3HzzzYwYMYLhw4fTvHnz+ohTREREREREREREROSkcPbZZ5OdnU1eXh7btm0DoFOnTiQnV/8GmoiIyOHsFjtmkxlvwIvNbKt23OEVPepLett0+rbuS7Yrm69/+hrcgB1OSzqNxd8vJs+dR4ojJaxiiM1sIzkuGYfNgavIReaGTPq27ku8NR6gztVGjkxwcRW5QhVRnDYn3RK7VaqIIiK1F3FCyY4dOxg/fjwjRoxgxIgRtGnTpj7iEhERERGRCOzdu5c//elPfPPNN7Rp04aMjAyGDh1aadx7773H5MmT2bp1axSiFBERERFpWoqKikhISGD69Ok89NBDJCcnK4lERETqJK1VGjm5ORSUFdS6okd9KPYWs8K1grV71+L2uYk3xdMzqSeDOg9i5d6VHCg7gNPmrDIhBMoTYZw2J3nuPHJyc0LH8ERSbeTwBJe1eWsp8ZYQZ40jrVUag1IGhZJWROTYRZxQoheeRURERESOLwcOHKBfv378+OOPobbMzExuueUW5s6di9lsDrUXFRWxY8eOaIQpIiIiItLkOBwOEhMTadWqVbRDERGRE1xdK3pEImtXVqgSSKGnEH/Aj91k5wfbD7y36z1sNlso6aMmCTEJuIpcrNm7JpRQApFVG4m3xjO642hGdxwd8eMUkZ9FnFAiIiIiIiLHlz/+8Y/s3r2b+fPnc/XVV/Pjjz/y8MMP88ILL+Byufj73/+O1Vrz+boiIiIiItIwhg8fzqeffsott9wS7VBEROQEFklFj7rI2pUVdq9URypWk5WgP0jQE2RrwVbKgmV4/V6spppfd7KarPgDfkq8JZX6VG1E5PiihJIIeDwevF4vAG63O8rRiIiIiIiU++CDD/j1r3/NjTfeCED37t1ZtGgRTz75JNOmTePSSy/l3XffxWar/nxdERERERFpGE888QRDhw7l4YcfZurUqTidzmiHJCIiJ6hIKnoci2JvMZkbMnEVuUhxpIRVQ7GarSTEJNDG2oZv87/FH/RT6ivFbq36yBsAb8CL2WSuNslF1UZEjh9KKInA7NmzmTFjRrTDEBEREREJs337diZPnlypferUqSQkJHDLLbdwySWX8H//93+NH5yIiIiISBM3cuRISktLmTVrFrNmzSI5OZm4uPA31AzDYMuWLVGKUERETiSNUdEj25VNnjsPp81Z5dE6AHaLnWa2ZuSX5fNTyU90TOhY7XoFZQU4bU7SWqVFHJuINCwllETggQce4N577wVg5cqVjBw5MsoRiYiIiIiA0+nk0KFDVfb95je/wWQycdNNN3HxxRdz1VVXNXJ0IiIiIiJNW/v27TEMI9phiIjISaShK3qs2bsmdHxOTdrEtSG/LJ9CTyFun7vK5BO3z02hp5Buid0YnDK4QeIVkfqjhJII2Gy2UJlwu736sk0iIiIiIo3p1FNPZcWKFUyaNKnK/gkTJgBw880389///rcRIxMRERERkeXLl0c7BBERkWPi9rnxB/xYTdYax8VYYogxx2A1WcktysVpc5IQk4DVZMUb8FJQVhBKTMnokVHtkTcicvwwRTsAERERERGpX2PGjOH9999n79691Y6ZMGECL730EgUFBY0YmYiIiIiIiIiInGjsFjtmkxlvwFvjOG/AS6wllrRWaXRL7IbJMOEqcrElfwuuIhcmw0S3xG7cduZtpLdNb6ToRSQSqlAiIiIiInKSycjIoEWLFuzfv59WrVpVO+6GG26gRYsWrFmzphGjExERERERgNzcXP75z3+ydetWADp37swvf/lLUlNrPk5ARESksaW1SiMnN4eCsgKS45KrHVdQVoDT5uSXXX7JoJRBZLuyWZu3lhJvCXHWONJapTEoZRDx1vhGjF5EInHMCSUjRow45psYhsGSJUuOeZ6IiIiIiBy7lJQUbrnlllqNvfjii7n44osbOCIRERERETncH/7wB2bNmoXP5yMYDIbaJ06cyO9+9zsefvjhKEYnIiISbnDqYBZ9t4jN+Ztx2BzYLfZKY9w+N4WeQroldmNwymDirHGM7jia0R1HRyFiEakvx5xQsnXrVgzDaIhYREREREREREREREROas899xwPP/ww/fr1Y/LkyZxxxhkAbNiwgaeffpqZM2fSsmVL7rzzzihHKiIix4NibzErXCtYu3ctbp8bu8VOWqs0BqcOrpdKH7VZP94aT0aPDOatm4eryIXT5iQhJgGryYrH76HAU8Aezx5SHalk9MggzhoXcVwicnw45oSS7du3N0AYIiIiIiISDVlZWSxbtozf//730Q5FRERERKRJmDt3Lv3792fFihVYLD+/RN+7d2+uvPJKBg8ezNy5c5VQIiIiZO3KInNDJnnuPAo9hfgDfswmMzm5OSz6bhEZPTJIb5veKOtX/LtivKvIhT/gx26y09HWkS4JXRjfc3xE8YjI8eeYE0pEREREROTk8dlnnzFjxgwllIiIiIiINJKdO3dy++23hyWTVLBarYwfP5777rsvCpGJiMjxJGtXVlhFkFRHKlaTFW/AS0FZAZvzNzNv3TyAOiVx1GX99Lbp9G3dl2xXNmvz1lLiLcFhdtAzvicDOw8kIS6h/jZARI4LSigREREREREREREREWkk7du359ChQ9X2Hzp0iPbt2zdiRCIicrwp9haTuSETV5GLFEcKdos91Gcz20iOS8Zhc+AqcpG5IZO+rfse0/E3kawfb41ndMfRjO44GgCv10tBQYGOuRE5SdVLQsnBgwd5+eWXWblyJQcPHiQQCIT1G4bBkiVL6uNWIiIiIiJyFBMmTKj12HXr1jVgJCIiIiIicqQ777yTxx9/nBtvvJFTTjklrM/lcvHCCy+oQomISBOX7comz52H0+YMS/Y4nN1ix2lzkufOIyc3h1EdRh0364vIySPihJIdO3YwePBgcnNzSUhIoLCwkBYtWoQSS5KSkoiPr31GnIiIiIiIRGbBggUYhkEwGKzVeMMwGjgiERERERGpkJCQQOvWrTn99NP51a9+xemnnw7Apk2beOONNzj11FNxOp289tprYfOuv/76aIQrIiJRsGbvGgo9haQ6UmsclxCTgKvIxZq9a44p4aOh1xeRk0fECSUPPvgg+fn5LFmyhF69etGqVSv++te/MmDAAB555BHefvttPv300/qIVUREREREaiEpKYn+/fuTmZl51LFPPvkkjz32WCNEJSIiIiIiADfccEPov+fNm1ep/8svvwwbA+VJ4EooERFpOtw+N76Aj0NlhyjyFuH2u/EH/FhMFmLNsTSzNSMhJgGryYo/4KfEW3LM6/sDfqwma43j6rq+iJw8Ik4oWbJkCTfddBPDhw9n//79AASDQeLi4njkkUfYsGEDv/3tb3njjTciDvZ44/F48Hq9ALjd7ihHIyIiIiJS7qyzzuKbb76hZcuWRx2raoIiIiIiIo1r2bJl0Q5BRESOcwdLD1LsLeZQ2SH8+MP6DnGI/LJ89rr3khSbhNlkJs4ad0zr2y12zCYz3oAXm9lW7ThvwFun9UXk5BFxQsn+/fvp2bMnAFZreRbb4ckVo0aNYsaMGZHe5rg0e/bsk/axiYiIiMiJq2/fvvznP/9h//79R00qCQaDtT4aR0REREREIjd06NBohyAiIsexrF1ZrN+3Hm/AW2V/kCDegBe/14/L76JFTAvSWqUd0z3SWqWRk5tDQVkByXHJ1Y4rKCvAaXMe8/oicvIwRbpAcnIyBw4cAKBZs2bExsayffv2UL/H4zlpq3c88MADFBUVUVRUxJIlS6IdjoiIiIgIABMnTmTZsmXExsYedeyDDz5IIBBohKhERERERKQqbrf7pH0NXUREjk2xt5hnvnqGfe59NY4LEsQf9OP1eynxldAnuc8x3Wdw6mCS7ckUegpx+6r+O8jtc1PoKSTZnszglMHHtL6InDwiTijp0aMH69atA8rPcezfvz/PP/88O3fuZPv27bz44oucfvrpEQd6PLLZbMTHxxMfH4/dbo92OCIiIiIiALRp04ahQ4c2ynE23333HePHj6d79+4kJCQQFxfH6aefzpQpU9i9e3eV4y+99FKaN29OfHw86enpLF26tMq1CwoKmDhxIqmpqcTGxtKjRw/mzZuniioiIiIicsLbu3cvt99+OykpKTgcDhwOB6eccgq33347P/30U7TDExGRKFm6cyk7C3fWerzZMBNniWNt3tpjuk+8NZ6MHhmkOlLJLcolryQPj99DMBjE4/eQV5JHblEuqY5UMnpk6MgbkSYs4iNvLrnkEp588kncbjd2u53f//73jBkzhk6dOgHlSSaLFy+OOFAREREREamdAwcO0KJFi0a5165du9i9ezeXXXYZbdu2xWKxsH79el588UXefvtt1q5dS6tWrQDYsmULgwYNwmKxMG3aNBISEpg/fz5jxozh3//+N+edd15oXY/Hw6hRo1izZg0TJ06ke/fu/Pvf/w69wD59+vRGeXwiIiIiIvVt27ZtnHvuuezevZvTTjuNAQMGALBp0yZeeOEF3nvvPbKysujcuXOUIxURkcb24bYP8QQ8mDARoLyibJDqP1hjNVvxBX2s2buGUR1GHdO90tumA5C5IZM8dx6uIhf+gB+zyYzT5qRbYjcyemSExolI0xRxQsntt9/O7bffHroeMWIEn3/+OW+++SZms5nLLruMQYMGRXobERERERGppeTkZHr37s3QoUMZPnw4Q4cOJTExsUHuNXLkSEaOHFmpfciQIYwdO5YFCxYwbdo0AO6//37y8/P58ssv6dOnDwDXX389PXr04I477uDbb7/FMAwAXnrpJVavXs2zzz7LxIkTAbjpppu44oormD17Nr/+9a/p0KFDgzwmEREREZGGNHXqVPbv38/ixYu59NJLw/reffddrr32Wu655x59UFNEpAnKLc4lEAxgMkwQBON//0DlxJIgQYLBIP6AnxJvSZ3ul942nb6t+5LtymZt3lpKvCXEWeNIa5XGoJRBxFsbvvqtiBzfIj7ypipnn302Tz31FE888YSSSUREREREGtmoUaP44YcfePbZZ7n88stJSkqib9++TJ06lX/+858UFBQ0eAwVyR4HDx4EoLi4mH/84x8MGzYslEwC4HA4+M1vfsP333/P6tWrQ+1vvvkmcXFx3HTTTWHrTpo0Ca/Xy1//+tcGfwwiIiIiIg1hyZIl3HHHHZWSSQAuu+wybrvtNpYsWRLRPQKBAE8//TSnn346sbGxtGvXjqlTp1JcXFzv84PBIG+++SaDBg0iKSmJZs2a0aNHD2bOnElhYWFEj0NEpKmqSCKB8tMgDMPAZJhCfyo+kANgNpkjOpIm3hrP6I6jmdZvGtMHTWdav2mM6jBKySQiAtRDQsmUKVP4+uuv6yMWERERERGpBx9++CH5+fnk5OTwyCOPMHLkSDZv3szTTz/NpZdeSlJSEv369ePee+/lgw8+4NChQxHfs7S0lH379rFr1y4++ugjbrnlFgAuvPBCAL7++mvKysoYOHBgpbkVJb4rEkoCgQBfffUVaWlpxMbGho3t378/hmGEJZ+IiIiIiJxIDMOgW7du1fafeuqpYW8U1sXkyZOZMmUKZ5xxBnPnzuWqq67i2Wef5aKLLiIQCNTr/AcffJDx48djt9t5+OGHeeKJJ+jVqxcPP/wwo0ePJhis/qgGEREJd0r8KeXVSYzypJLqjrup+NlqGAZOm5O0VmmNGaaINCERH3kzZ84cnnnmGXr16sUNN9zAuHHjQmeki4iIiIhIdJjNZgYMGMCAAQO477778Pv9rFq1iuXLl7N8+XJycnL46quveOqpp7BYLJSVlUV0v5deeil0NA1Ax44dWbhwIenp5efs5ubmApCamlppbkWby+UCyquauN3uKsfGxMSQlJQUGluTH3/8kV27doW1rV+/HgCfz4fX663NQzvpeb1e7UeUaO+jR3sfPdr76NHeR0dT2HefzxftEE44Q4cOZdmyZdx6661V9i9fvpxhw4bVef0NGzYwd+5cLr/8ct55551Qe6dOnbjrrrt4++23GTduXL3M9/l8zJkzh759+/Lxxx9jMpV/hvXWW2/FYrHwxhtvsG7durAqhSIiUr3zO57P6j2rKfOXlScXBsuTRw5PNAz+7x+D8qolyfZkBqcMjmLUInIyizih5NtvvyUzM5M33niDKVOmMG3aNMaMGUNGRgYXX3wxNputPuIUEREREZEImM1mBg4cyMCBA7n77rtZtmwZTzzxBJ999lm9vAlw6aWXcvrpp1NUVMSaNWv4xz/+wb59+0L9JSXlZ/nGxMRUmltRhaRiTE1jK8ZXjKnJyy+/zIwZM6rsKyoqapSjf04EPp8vVLrcYon4V0Q5Btr76NHeR4/2Pnq099HRFPa9qKgo2iGccObMmcOwYcOYOnUqv/3tb0Mf0Ny7dy9//OMfWblyJcuXL6/z+m+99RbBYJBJkyaFtd90003cd999LFy4sMaEkmOZ7/V6cbvdtGnTJpRMUiElJQWA+HgdmSAiUlsjO4zktY2v8UP+D6EqJAECGMHyhJLDK5aYDBMdnR3J6JER0ZE3IiI1ifi3mFNPPZVHHnmERx55hGXLlvHaa6+xePFi3n//fRITE7nmmmu47rrrqixtLSIiIiIiDc/tdpOdnc3y5ctZtmwZ//3vf/H5fLRo0YJLL72UoUOHRnyPtm3b0rZtW6A8ueSKK66gX79+lJSUcP/99xMXV/7CRlWVUEpLSwFCY2oaWzG+YkxNbrzxRsaMGRPWtn79em655RYcDgcJCQm1fHQnt4pPTDudTqxWa5SjaVq099GjvY8e7X30aO+joynsu8PhiHYIJ5yRI0dSWlrKnDlzmDNnDomJiQDk5+cDkJSUxIgRI8LmGIbBli1barX+6tWrMZlM9O/fP6w9NjaWPn36HPX4yGOZb7fbGTJkCB9++CGPPfYYV1xxBRaLheXLl/P888/zq1/9qsbjfUREJFy8NZ67+97N46sfZ9ehXQQJ4g/6Kx19YzVZ6ZzQmRt73UiJr4THVj2G2+fGbrGT1iqNwamDibcqoU9EIlevafHDhw9n+PDhPP/88yxevJjXX3+d+fPn8+KLL6r0oYiIiIhIIyktLSUnJyeUQLJ69Wo8Hg+tWrViyJAhjBs3jqFDh9KzZ88Gi6F3796kpaXx/PPPc//994c+nVjVUTUVbRVH3DRv3hy73V7l2LKyMvbt21erJJh27drRrl27KvssFstJ+6ZOXVTsh/ak8Wnvo0d7Hz3a++jR3kfHyb7vJ2vllYbUvn37sKML6ltubi5JSUlVVvxLTU0lJycHj8dTbXXxY53/xhtvcMMNN3Dfffdx3333AeUJML/73e+YOXNmrWKO5LjKpnC0VF1pb6qmfame9qZ6jbk3A1oP4N6+9/L6htfZVbyLgtICAsEAQYJYTBacNicXdr6Q05qfxjub3mFf6T4OeQ7hD/gxm8yszl3N4m8XM677OAamNPwH/vV1UzXtS/W0N9VryL2pa75Ggzzbt9vtpKamcsopp9S6HLWIiIiIiNSP5s2b4/F4aNOmDUOGDGH8+PEMGzaM008/vVHjcLvdHDhwAIBevXoRExPD559/XmncF198AcDZZ58NgMlkom/fvqxZs4aysrKwF7JXrVpFMBgMjRUREREROdFEcpxNbZSUlNR4fGTFmOoSSo51fkxMDJ06deL666/nggsuAOCdd95h1qxZxMbG8rvf/e6oMUdyXGVTOFqqrrQ3VdO+VE97U71I98btc7N+33q2HNxCmb+MGHMMXZt3pWdST+wWe6XxZ8Sfwe/7/p5v9n3DD/k/UOYrI8YSQ7fEbvRI6sHmg5t5b+N7HCo9RAtLC9rFtcNisuAL+Cj2FnPo0CHe+fodfCU+eif3jvjx10RfN1XTvlRPe1O9htybuh5VWa9RfP/997z22mu88cYb7Ny5E7PZzAUXXEBGRkZ93kZERERERGpQVlaGxWIhLS2Nvn370rdvX0499dQGudeePXto06ZNpfZly5bxzTffMGzYMKC8FPpFF13E4sWLWbduHWeeeSZQ/ovMSy+9RLdu3cJKal977bVkZ2fz4osvMnHixFD7nDlzsFgsXH311Q3yeERERERETnRxcXHs3bu3yr4jj5uMdH5JSQmDBg2ib9++vP3226Fx11xzDddccw2///3vufLKKznttNNqjDmS4yqbwtFSdaW9qZr2pXram3Il3hI+3/056/PWh46R6dWiF6fHnY41zsrqvNVhfb2TezPglAHEWav+2fp57ue8uenNSpVEmu1vRtKuJC7vdjkBAlWueV7L8ziP8yrFt2jtIraUbKFNfBuwQDHlb0BjAixg8pnYWLyRsh1lnNXhrGpjqw/6uqma9qV62pvqNeTe1PWoyogTSg4ePMhbb73Fa6+9xurVqwkGg/Tp04dJkyYxbtw4kpOTI72FiIiIiIgcg88//5xly5axfPlyZs6cSUlJCQ6Hg0GDBjF06FCGDh1Kv3796iXL/bbbbmP37t2MGDGCDh06UFpaypdffsnbb79Ns2bNePLJJ0NjH330UZYsWcLo0aOZPHkyTqeT+fPn43K5eP/998PKft900028+uqrTJkyhe3bt9O9e3c++OAD3n33XR588EE6duwYcewiIiIiIiejlJQUNm7cWKnaH5QfN5mUlFRtdZJjnf/3v/+dzZs38+ijj1Za56qrruKvf/0rK1asOGpCSaTHVZ7sR0tFQntTNe1L9Zr63mTtyiJzQyZ57jwKPYWh5I+Ve1bS2mjNAQ5QEiwJ61uxZwV/++FvZPTIIL1teqX1XvjmBVxFLpw2J8mOZKwmK96Al4KyAtYdWMfalWtpZm2GL+ir1ZorXSvZXbqbWFssVqsVH5WPsbBarcTaYtlduptVeasY1WFUg+5bU/+6qY72pXram+o11N7U9bXgiF9BbtOmDV6vlzZt2jBlyhQyMjIa9Cx2ERERERGp2TnnnMM555zDfffdh9/vZ9WqVSxfvpxPP/2URx55hAceeIC4uDgGDhwYSjBJT08/+sJVuPbaa3nttdd4/fXXycvLwzAMOnTowC233MK9995L+/btQ2O7du1KdnY29913H3/84x/xeDz07duXDz/8kPPOC/+0jc1m45NPPuHBBx/krbfeYv/+/XTp0oW5c+dyxx13RLQ/IiIiIiLRlp2dzaOPPsrKlSs5ePAgwWAwrN8wjDqfc9+vXz8++ugjVq1aFfY8v7S0lLVr1zJkyJB6m+9yuQDw+/2V1qmIv66PQ0SksWXtymLeunmh5I9UR2oo+WN/0X5cXhe7/btxxDrC+grKCticv5l56+YBhBJAir3FZG7IxFXkIsWREna0jc1sw2q24g/6KfWV4gv4ODXxVGIsMTWuCbBm7xoKPYWkOlJrfDwJMQm4ilys2bumwRNKROTkFXFCyWWXXUZGRgZjxozBZDLVR0wiIiIiIlJPzGYzAwcOZODAgdx///34/X5Wr17N8uXL+de//sXDDz8c0YvVY8eOZezYsbUe3717d957771ajU1MTOS5557jueeeq1NsIiIiIiLHo88++4zzzjuPhIQEzjnnHD744ANGjBhBUVERq1atolevXvTt27fO61999dXMnj2bOXPmhCWEzJ8/n5KSEsaPHx9q27JlC16vl9NPP71O88844wwAMjMzK/1ekJmZCZQnqIiIHO9qSv4wG2ZK/aXEUF61ye13YzbMGIaBzWyjRWwL/AE/mw9u5uGchxnRbgT9T+lPmb+MPHceTpszbD0Af8DPT8U/4fV7sZqsGBiU+EuItcZiM9tIjkvGYXPgKnKRuSGTvq37Em+NL7+/z40/4Mdqqrl6gdVkxR/wU+ItqefdEpGmJKKEkuLiYs444wxMJlOTTCbxeDyhc4zcbneUoxERERERqZnb7WbFihUsX76cZcuW8eWXX1b6JKSIiIiIiDSsRx55hFNOOYX//ve/GIZBq1ateOCBBxgxYgQfffQRV155Jc8//3yd1+/Vqxd33HEHzz33HJdffjkXXnghmzZt4tlnn2Xo0KGMGzcuNHbkyJHs2LEj7PeCY5n/y1/+kv79+/PBBx8wZMgQLr/8cgAWL15MVlYWV111VUTJMSIijSXblV1t8keBpwCP34PZbMYStODxeyjwFNAitgX5pfnsKdmD1+8tr2RSup8Ptn/A6p9WU+ItocRXQqeETpXuF1rTZMZiKl+zyFNEi9gWoTF2ix2nzUmeO4+c3JxQlRG7xY7ZZMYb8GIzV3+EmTfgxWwyE2eNq6ddEpGmKKKEkvj4eGbPnt1kPzE4e/ZsZsyYEe0wRERERESqVFpaSnZ2diiB5L///S9er5dgMEhsbCznnnsuw4cPZ/jw4dEOVURERESkyVi1ahVTpkwhOTmZAwcOABAIBAAYPXo01113HQ899BBLly6t8z3mzJlDx44defHFF3n//fdJSkpi4sSJzJw5s1YfDq3tfLPZzCeffMKjjz7K4sWL+e1vf4thGHTr1o3HHnuMKVOm1PkxiIg0ppqOkSnyFOEP+jEbZswmM36/nyJPESZM5BbnUuYvw2JYsJlt+AI+YswxBIIBDpYdxBfwUewpxma3VbmmzWzDhIlgMEggGKh076qOrUlrlUZObg4FZQUkxyVX+5gKygpw2pyktUqLcHdEpCmL+MibLl26sGfPnvqI5YTzwAMPcO+99wKwcuVKRo4cGeWIRERERETg97//PcuXL2f16tV4PB6CwSA2m41zzjknlEAycOBAbLbqP8UiIiIiIiINo6ysjNTU8jcsY2LKj084dOhQqL9Pnz4sXLgwonuYzWamTp3K1KlTaxy3ffv2iOYDNGvWjNmzZzN79uy6hCoiclyo6RiZQDAAQTAwQskf/oCfPSV7KPOXEWOOwWSUt3uDXgwMkuOSOeQ5xIHSA+wu2Y0zxonZZA5bMxgMYmAQIIBhGJiMygl/VR1bMzh1MIu+W8Tm/M04bI5KFVUqHk+hp5Buid0YnDK4nnZJRJqiiBNKbr/9dh5//HFuu+02WrZsWR8xnTBsNlvoRXi7vfIPaxERERGRaJg1axYWi4V+/fqFEkgGDx5MbGxstEMTEREREWnyTjnlFHbt2gWUVwFPTEzkm2++4bLLLgNg165dWCwRv3QvIiLHoKZjZEyGCQwIEgwlf3gDXrx+LxbDEkoECRIMSwxJjE2k0FNIma8sdETO4WsahkGQ8uQUs2HGYYmD4jwoPQRBPxhmvLY4zIYp7NiaeGs8GT0ymLduHq4iF06bk4SYBKwmK96Al4KyglC1lYweGTryRkQiEvGz0mbNmtGiRQtOO+00MjIy6NatG3FxlX8wXX/99ZHeSkREREREauHf//435557LvHx8dEORUREREREjtCvXz+ys7ND16NHj+bpp5+mQ4cOBAIBnnvuOc4555woRigi0vTUdIyMw+bAXebGH/SHkj+A0JE1FXwBX3liiM0BQIItgRhzDMXeYvJL88MSShw2BwVlBXgDXgLBAHbDSkJhLvi8EPABQcCgoNSM02whzR9evSS9bToAmRsyyXPn4SpylcdmMuO0OemW2I2MHhmhcSIidRVxQskNN9wQ+u+nn366yjGGYSihRERERESkkYwZMybaIYiIiIiISDVuvPFGFixYgNvtxm63M3v2bLKyskKvtbdp04bHH388ukGKiJxkir3FrHCtYO3etbh9buwWO2mt0hicOph4a3yNx8gk2BIoMBfgD/jxBXxYrVasJivBYBAT5YkeFX0mw0RBaQFFniIcNget7K3Y4dtBkbeIvJK8UCURu7l8fa/fS6zJQhuvB7PPAyYLWGLBMHAHAxTipZunlMHr34eWZ0K3UaG40tum07d1X7Jd2azNW0uJt4Q4axxprdIYlDKIeKs+aCQikYs4oWTZsmX1EYeIiIiIiIiIiIiIyElv1KhRjBr18xuCnTt35vvvv2fJkiWYzWbOPfdcEhISohihiMjJJWtXVqiSR6GnMFTJIyc3h0XfLQpV8qjuGBl/0E+sORYC5evZzXYC/7vwBX34/D58AV/ofvll+RiGQUFZARaTBZvZRoItAZNhCqskkhiTyCHPIUy+ErwBPx5LDFbDhBcowE+hESAVGxlGLHEHf4ScudB+AMQ0C90r3hrP6I6jGd1xdGNuqYg0IREnlAwdOrQ+4hARERERERERERERaZLi4+O5+OKLox2GiMhJJ2tXVliSSKojFavJijfgpaCsgM35m5m3bh5Q8zEyLW0taR3TGjt2SoIl5LnzCBCgzFdG8H//WEwWYswxGBgECeIL+CjxlWA1WRnTcQy9k3tXqiTCzlX89ftF5BlmXEYQPz7MGDgx0Q0bGThJt9ghtgyK9sCWpXDGJdHcUhFpYiJOKDlcWVkZ+/btIzk5GZvNdvQJIiIiIiIiIiIiIiIiIiK1cLSja44cm7khE1eRixRHStgxNjazjeS4ZBw2B64iF5kbMunbum+1x8j0admH7nHdcTqdrNy7ktV7VvPJjk/YX7ofA4NYSywW089vuxoYWEwWfAEf/oCfjfs3cluf2ypXEtm0lEH5brITW7MWPyUEicMgjRgGYSf+f0fqYG8B+dth50ollIhIo6qXhJKvvvqKe+65hxUrVuD3+/n4448ZMWIEe/fu5dprr+X+++/nvPPOq49biYiIiIiIiIiIiIiIiEgTU9ujaypku7LJc+fhtDnDkkkOZ7fYcdqc5LnzyMnNYVSHUVUeI+P1eikoKCDOGhfqc1gdvL7pdbwBL8FgkEAwgAkTAQL4A358QR+xlljiLfHsL90fWj+Mp5h4v4/RJiejMap/8GYrBHzgKYpoD0VEjpUp0gXWrl1Leno6W7Zs4frrrw/ra9WqFW63m8zMzEhvIyIiIiIiIiIiIiIiIiJNUMXRNZvzNxMIBkh1pNIlsQupjlQCwUDo6JqsXVmhOWv2rqHQU0hCTAL+gJ8D7gPsLNzJ9oLt7CzcyQH3AfwBPwkxCRR6Clmzd80xxVTqLyXGHBNKVvH4Pbh9bjx+D1CerJISn8IpjlOqX98WDyYL+L0138zvLR9ncxxTjCIikYq4Qsnvf/97UlJSWLNmDaWlpbzyyith/SNHjmTRokWR3kZEREREREREREREREREmpi6HF0Tb43H7XPjD/gp9hTzk/snvH4v/qCfYDCIYRgUlBWw172X1vbW+AN+SrwlxxSX2+fGbJjpktCFQk8hRd6i8iolhgmHzUGCLQGzyUwwGKx+/XbnwJYl4D4AzdrUcLMDYE+E9uccU4wiIpGKOKEkKyuL+++/H4fDQVlZWaX+9u3bk5ubG+ltRERERESkFiZMmHDMcwzD4OWXX26AaEREREREREREIlPXo2vsFjv+oB9XsQtvwIvFsGAz28KOpXH73LiKXVhNVuKscUeNpcRbwheuL1i7dy3r8tbh9rk5UHqAlvaWtLC3qHKON+DFbDJXvX7XkfDfV2DvRohxgq2KMZ4SKM2HVmdAl5FHjVFEpD5FnFBSWlpKQkJCtf2FhYWR3kJERERERGppwYIFldoMo/wM3mAwWKm94lM5SigRERERERERkeNRxdE1qY7UGsclxCTgKnKxZu8aRnUYxRktz+Cd79+hLFCG3WLHZJhCY02YMJlNmINm3D43BKF7i+41rv913tcs2rGI3aW7KfQU4va68Qa8uIpcHCg7QJu4NiTGJlaaV1BWgNPmJK1VWuVFY5rBoImw/FHI3wmxiWBvAWZr+TE37gPlySSJ7cvHxejIGxFpXKajD6lZly5d+PLLL6vtX7p0KWeccUaktxERERERkVoIBAJhf3766Sf69OnDJZdcQk5ODvn5+eTn55Odnc3FF19M3759+emnn6IdtoiIiIhIk+L3+3nttdf41a9+xahRo1izZg0ABw8e5LXXXsPlckU5QhGR40fF0TVWk7XGcVaTNexomWAwSJDyD9dU/PtIh/cbGNWu/Xnu57z3w3tsKdhCIBgg1ZHKqc1PxW6xEyBAibeE3OJc8kvzK8Ve6Ckk2Z7M4JTBVS/ebRQMu7+8AonJBPnbIW9T+b9NpvL2YfeXjxMRaWQRVygZN24cf/jDHxg7dixpaeWZdRWfgHzyySf58MMPeeaZZyK9jYiIiIiI1MGUKVNo1aoVixcvDmsfOHAg7777Lueffz5Tpkzhtddei1KEIiIiIiJNS0lJCaNHjyYnJ4f4+HhKSko4ePAgAE6nk/vuu48JEyYwa9asKEcqInJ8sFvsmE1mvAEvNrOt2nFHHi2z6cAmbGYbhmHg8Xkwm8xYTBYMDIIE8QV8+AN+YswxWE1WNh7YyC+7/LLSusXeYt7c9CaHSg/RJr4NVuvPiS2nOE4htyiXUl8pbp+b3OJcHFYHAQIUlBWEKqtk9Mio+UidbqOg/QD4YQn8uAo8RWBzQPtzoMuI8komIiJREHGFknvuuYcBAwYwZswYhgwZgmEYTJ48mdTUVKZNm8aoUaO4/fbb6yNWERERERE5Rh988AEXXXRRtf0XXXQRH3zwQSNGJCIiIiLStE2fPp3//ve/vPvuu2zdujXsaEqz2czll1/Of/7znyhGKCJyfElrlYbT5qSgrKDGcUceLeP2uTEbZto62hJnjcPAwOP3UOorxeP3YGAQZ42jraMtZsMcqmxypGxXNvtK9xFvicdusYf1JcYkkuJIId4aj9kwU+or5fv873EVuTAZJrolduO2M28jvW360R9oTDPocSmcPxsufrb832dcomQSEYmqiBNKbDYbH3/8MX/605+w2+3Exsby/fffk5SUxOOPP86//vUvTKaIbyMiIiIiInVQVlbGrl27qu3ftWsXZWVljRiRiIiIiEjT9re//Y2bb76ZSy65pMrXzrt27cr27dsbPzARkePU4NTBJNuTKfQU4va5qxxT1dEyFZVN4qxxdE3sSmqzVFrEtqB5bHNaxLYgtVkqXRO7EmeNC6tscqQ1e9dwyHOIeGt8lf2JMYl0TexKiiMFq8nKKfGncEGnC7iz5408c8p5pH/zAfzjLvjwftjwf1B2qF72RUSkMUR85A2AxWJh8uTJTJ48uT6WExERERGRenLuuecyd+5czj//fIYMGRLW9+mnnzJ37lzOPffcKEUnIiIiItL05ObmcuaZZ1bbHxcXx6FDerNRRKRCvDWejB4ZzFs3D1eRC6fNSUJMAlaTFW/AW+3RMmmt0sjJzaGgrIDkuGRaxLagRWyLSusfKDkQVtnkSG6fG3/Aj8VU/duqZpO5POmlrJAzk89kWmIaZM+Foj3gzoeAD0wW2LIE/vsKDJpYfsyNiMhxLuLSIRMmTGDlypXV9q9atYoJEyZEehsREREREamDp556CqvVyvDhwznnnHPIyMggIyODc845hxEjRmC1WnnyySejHaaIiIiISJPRsmVLXC5Xtf0bNmwgJSWlESMSETn+pbdN57Yzb6NbYjdMhglXkYst+VtqPFqmrpVNjlRR6cQX8NUYozfgLa90UrQPlj8KezdCIACJHSG5e/m/A4Hy9uWPwuaP67odIiKNJuKEkgULFrBly5Zq+7dt20ZmZmaktxERERERkTo444wz+Oqrr7j66qvZtGkTr7/+Oq+//jqbNm3i6quv5ssvv6RHjx7RDlNEREREpMkYOXIkr776KiUlJZX6tm3bxiuvvML5558fhchERI5v6W3TeWbEM9zR5w4u6HQBI9qPKD9aJu1OnhnxTFgyCfxc2STVkUpuUS55JXl4/B6CwSAev4e8kjxyi3IrVTY5UlqrNJrZmlHsLa4xvoKyApxWB2m7N0H+TkhoD83agMUGhlH+72Ztytvzd0LOXB1/IyLHvXo58qYmxcXFWK3Whr5NVHg8HrxeLwBud9WZjSIiIiIi0daxY0fefPNNgsEge/fuBSA5ObnK89pFRERERKRhPfzww5x99tn069ePa6+9FsMw+PDDD/n444954YUXiImJ4f777492mCIiDa7YW8wK1wrW7l2L2+fGbrGT1iqNwamDibfGVzkn3hrP6I6jGd1xdK3uUZFkkrkhkzx3Hq4iF/6AH7PJjNPmpFtiNzJ6ZFRKRjnc4NTBLP52MYcOHcLkM1X5vmdFpZNu1kQGFxVCbCLYqk5QwRZX3l+0B7YshTMuqdVjERGJhjollOzcuZPt27eHrr/99ls+++yzSuMOHDjAvHnz6Nq1a50DPJ7Nnj2bGTNmRDsMEREREZFaMQyD1q1bRzsMEREREZEmrWvXrixZsoQJEybw+9//HoA//elPAPTs2ZPXX3+ddu3aRTNEEZEGl7UrK5TkUegpDCV55OTmsOi7RUdN8jgW6W3T6du6L9mubNbmraXEW0KcNY60VmkMShlUbfJKhXhrPOO6j+Odr99hY/FGYm2xJMQkYDVZ8Qa8FJQVUOgpLK904rMT595UfrxNTewtIH877FyphBIROa7VKaHk1VdfZcaMGRiGgWEYPPLIIzzyyCOVxgWDQUwmE6+++mrEgR6PHnjgAe69914AVq5cyciRI6MckYiIiIhIZYcOHeLpp5/mo48+4qeffuK1115j4MCB7Nu3j+eff56xY8dy+umnRztMEREREZEm46yzzmLdunV88803bNq0iWAwSLdu3UhLS4t2aCIiDS5rVxbz1s3DVeTCaXOS6kgNS87YnL+ZeevmAdRbUsmxVjY50sCUgfhKfJRsK2Fr8Vb2FO8hEAxgMkw4Y5x0dnZmQq8JpH/1Nwj4wHyU0xvM1vJxnqI6xSMi0ljqlFBy6aWX0rFjR4LBIBMmTODmm29m4MCBYWMMw8DhcNCvX7+TNpvaZrNhs9kAsNvtUY5GRERERKSyvLw8zj33XLZu3UrXrl3ZunVr6LjGpKQkMjMzyc/P56mnnopypCIiIiIiTU/Pnj3p2bNntMMQEWk0xd5iMjdk4ipykeJIwW75+f01m9lGclwyDpsDV5GLzA2Z9G3d96gVRBqTYRgYGGAAQSj/z/IP4ANgiweTBfxesNiqX8jvLR9nczRG2CIidVanhJIzzzyTM888E4AdO3ZwxRVX6EmviIiIiMhx6MEHH2TPnj2sXLmS9u3b06pVq7D+Sy65hCVLlkQpOhERERGRpq2kpIT9+/cTDAYr9bVv3z4KEYmINKxsVzZ57jycNmdYMsnh7BY7TpuTPHceObk5jOowqpGjrOzz3M9574f32FayDbvNTpv4NmFVVbYUbCmvqtK8D+n2RHAfgGZtql/QfQDsidD+nMZ6CCIidWKKdIGHH35YySQiIiIiIsepf/3rX9x+++307dv350/LHKZz5878+OOPUYhMRERERKRp8vv9zJ49m9TUVJo1a0bHjh3p1KlTpT8iIiejNXvXUOgpJCEmocZxCTEJFHoKWbN3TSNFVr1ibzFvbnqTfaX7aBPfhuS4ZGxmG4ZhhKqqpDhSyquqFG6i2NEKSvPBU1L1gp6S8n5HG+gysjEfiojIMatThZIjBYNBPvnkEzZv3lxlNrVhGDz00EP1cSsRERERETkG+/bto2vXrtX2m0wmSktLGzEiEREREZGmbcqUKcydO5e+ffty1VVX0bx582iHJCLSIIq9xaxwrWDt3rW4fW7sFjs/5P+AL+DDarLWONdqsuIP+CnxVpOU0YiyXdnsK91HC0sLsIAPX6UxoaoqpQfIOW0YozwlkL8TYhPB3gLM1vJjbtwHypNJEtvDoIkQoyNvROT4FnFCyebNm7n00kv59ttvqyzLB0ooERERERGJljZt2rBly5Zq+9esWaNS2iIiIiIijeiNN97g8ssv5+9//3u0QxERaTBZu7LI3JBJnjuPQk8h/oAfs8mMx+/B7XOz372fpLikaud7A17MJjNx1rhGjLpqa/au4ZDnEO3i2lFMcbXjEmIScBW5WGMOMGrY/ZAzF4r2QP52CPjAZCk/5qbVGeXJJN2if5SPiMjRRJxQMnHiRLZs2cJjjz3GiBEjaNmyZX3EJSIiIiIi9eDCCy/k5ZdfZuLEidhstrC+lStX8tprrzFp0qToBCciIiIi0gR5vV5Gjx4d7TBERBpM1q4s5q2bh6vIhdPmJNWRitVkxRvwkluUS5G3CFexC4vJQmJsYpVrFJQV4LQ5SWuV1rjBV8Htc+MP+LGYan5bNayqSrdR0H4A/LAEflwFniKwOaD9OdBlBMQ0a6ToRUQiE3FCSVZWFpMmTeKee+6pj3hERERERKQePfzww/zjH/8gLS2Niy++GMMwyMzMZP78+SxevJiUlBR++9vfRjtMEREREZEmY9CgQWzcuDHaYYiINIhibzGZGzJxFblIcaRgt9hDfTazjXbN2lHiK6HEW4Kr2EUzWzPMJnPYGm6fm0JPId0SuzE4ZXBjP4RK7BY7ZpMZX8AHpurHVaqqEtMMelxa/kdE5AQVcUJJTEwMnTp1qo9YRERERESknrVp04YvvviCO++8k1deeYVgMMjrr7+OYRhceOGFzJs3jxYtWkQ7TBERERGRJuPxxx9n5MiRDB8+nEsuuSTa4YiI1KtsVzZ57jycNmdYMkkFs8lMiiOFHQU7KPWV8uOhH0lxpIQqmBSUFVDoKSTVkUpGj4wGPfKm2FvMCtcK1u5di9vnxm6xk9YqjcGpg4m3xofGpbVKY3Xuaoq9xTW+s3o8VVUREakvESeUjBkzhuzsbG655Zb6iEdEREREROpZu3bteO+99ygsLOS7774jGAzStWtXJZKIiIiIiERBr169mD9/PldccQUpKSl06tQJszn80/mGYbBkyZIoRSgiUndr9q4JJYRUJzEmEZ/Dx66iXXj8HlxFLvwBP2aTGafNSbfEbmT0yCC9bXqDxZm1K4vMDZnkufMo9BSG7p+Tm8Oi7xaF3X9w6mAWf7uYQ4cOYfKZsFqtldY73qqqiIjUl4gTSp566imGDBnCk08+WeW57CIiIiIicnxwOp3069cv2mGIiIiIiDRp77//PmPHjiUQCFBYWMjOnTujHZKISL1x+9z4A36spspJF4draW/JwdKDnNbiNE5tcSol3hLirHGktUpjUMqgsAoh9S1rVxbz1s3DVeTCaXOS6kgNq5CyOX8z89bNAyC9bTrx1njGdR/HO1+/w8bijcTaYkmISYhKVRURkcYWcULJ4MGDKS4uZtq0adx3332kpKRUmU29ZcuWSG8lIiIiIiLHyGQy0aZNG957770qk0neeOMNrr/+evx+fxSiExERERFpeu6//37atWvHu+++S69evaIdjohIvbJb7JhNZrwBLzZz9R9C9wa8WMwWTm1xKtP6TWu0+Iq9xWRuyMRV5CLFkRJ2LI/NbCM5LhmHzYGryEXmhkz6tu5LvDWegSkD8ZX4KNtRxu7S3VGpqiIiEg0RJ5S0b98ewzDqIxYREREREWkA+/fvZ9iwYWRmZnLllVdGOxwRERERkSZt8+bNPPbYY0omEZGTUlqrNHJycygoKyA5LrnacQVlBThtTtJapTVidJDtyibPnYfT5gxLJjmc3WLHaXOS584jJzeHUR1GAdA7uTdndTiLlXtXsjZvbaNWVRERiZaIE0qWL19eD2GIiIiIiEhDeeKJJ/j73//ONddcw8yZM3nggQfqbe3vv/+ehQsX8tFHH7FlyxZKS0vp0qULV111FZMmTSI+/ucXU6ZPn86MGTOqjfGee+4JawsEAjzzzDP85S9/Yfv27SQnJzN27FhmzpwZtq6IiIiIyImkQ4cOlJaWRjsMEZEGMTh1MIu+W8Tm/M04bI4qkzbcPjeFnkK6JXZjcMrgRo1vzd41oeNpapIQk4CryMWavWtCCSUAcdY4RncczeiOoxs6VBGR40LECSUiIiIiInJ8S0pKYsmSJUyYMIEHH3yQzZs3M3/+fCyWyH8deOWVV/jzn//MxRdfzPjx47FarSxbtowHH3yQRYsW8cUXX2C3h7949PTTT5OUlBTWdtZZZ1Vae/LkyTz77LNcdtllTJ06lU2bNvHss8+yZs0aPvnkE0wmU8Txi4iIiIg0trvuuos5c+Zw++2343A4oh2OiEi9irfGk9Ejg3nr5uEqcuG0OUmIScBqsuINeCkoKwgldGT0yCDOGteo8bl9bvwBP1aTtcZxVpMVf8BPibekkSITETk+KaFERERERKQJsFqtvP7663Tt2pUZM2awfft2Fi9eHPG6V155Jffffz8JCQmhtltvvZVu3brxyCOP8PLLL3PnnXeGzbn00kvp2LFjjetu2LCBuXPncvnll/POO++E2jt16sRdd93F22+/zbhx4yKOX0RERESksTkcDhITE+nevTu//vWv6dSpE2azudK466+/PgrRiYhELj25DzQ/k8yCXPKK9uIq2o3fZMZsseOMSaBbYjcyemSQ3ja90WOzW+yYTWa8AS82s63acd6AF7PJ3OgJLyIix5tjTijp1KkTJpOJb7/9FqvVSufOnY86xzAMtmzZUqcARURERESk/jz88MN07dqV3/zmNwwYMIBrrrkmovXOPvvsKtuvvvpqHnnkEb755psq+wsLC4mLi6u2Sspbb71FMBhk0qRJYe033XQT9913HwsXLlRCiYiIiIickG644YbQf8+aNavKMYZhKKFERE5Mmz+GnLmkF+2hb2k+2ZYga20WSqwxxFktpHVIZ9BZtxFvPcajbMsOwQ+fwI+rwFMMtnhodw50HQkxzWq9TFqrNHJycygoKyA5LrnacQVlBThtTtJapR1bnCIiJ5ljTijp0KEDhmFgGAYA7du3D/23iIiIiIgc/8aPH0+HDh247LLLqn0BO1K7du0CoHXr1pX6evfuzaFDhzCbzfTv35+HHnqICy64IGzM6tWrMZlM9O/fP6w9NjaWPn36sHr16qPG8OOPP4biqLB+/XoAfD4fXq/3mB7Tycrr9Wo/okR7Hz3a++jR3keP9j46msK++3y+aIdwwlm2bFm0QxARaRibP4blj0L+TohNJD6hI6PNVkb7veA+AAd/As8H0PJM6Dbq2NbNmQtFe8CdDwEfmCywZQn89xUYNLHW6w1OHcyi7xaxOX8zDpsDu8VeaYzb56bQU0i3xG4MThlc+zhFRE5Cx5xQsnz58hqvRURERETk+JGRkUGXLl0qtZ977rl88cUXjBs3jr1799brPf1+P3/4wx+wWCxhVUQSExO5+eabGTRoEM2bN+e7775jzpw5/OIXv+CVV14J+6Rmbm4uSUlJxMTEVFo/NTWVnJwcPB4PNlv15WlffvllZsyYUWVfUVERBQUFdX+QJxGfz0dxcTFAtRVjpGFo76NHex892vvo0d5HR1PY96KiomiHcMIZOnRotEMQEal/ZYfKkz7yd0JCe7AddlSMxQbN2kCMs7x/xdNQvA/2fH30aiNHJKmQ2BHMVqhIUtm7sbwfKiWVFHuLWeFawdq9a3H73NgtdtJapXH16Vfz6tcv4crfhjMICZixGha8MfEUmAwKvUWkOlLJ6JGhI29EpMk7OX+LERERERERAF599dVq+7p06cLKlSvr/Z6TJk3i888/Z/bs2Zx22mlh7UeaMGECPXv2ZPLkyVx55ZU4HA4ASkpKqkwmgfIqJRVjakooufHGGxkzZkxY2/r167nllltwOBwkJCQc60M7KVV8YtrpdGK1WqMcTdOivY8e7X30aO+jR3sfHU1h3yuev4mISBP3w5LyCiKxieHJJP9TTIAVtiBrm8XgLtmM/YtZpHn8DC7zEG9UU23kWJJUcuZC+wGhhJSsXVlkbsgkz51HoacQf8CP2WQmJzeHZMwMKcjjvz43efhwGeA3DMyeAzhNVroldCTjzNtIb5veCBsnInJ8q/eEksLCQiZNmsS0adM4/fTT63t5ERERERE5jj300EM899xz3Hzzzdx///1HHd+yZUtuvfVWpk+fTk5ODqNHjwYgLi6u2soppaWloTE1adeuHe3atauyz2KxnLRv6tRFxX5oTxqf9j56tPfRo72PHu19dJzs+36yVl6pT6+99hoA1113HYZhhK6P5vrrr2/IsERE6tePK8uPo0nsWKkrCzeZFJIX8FBoC+LHipkgOVYzi+x2Mtx+0t0FULw/vNrIUZJUgPL22MTycVuWwhmXkLUri3nr5uEqcuG0OUl1pGI1WfEGvBQcymWzex8lPh+/9toI2lqw1uSnJOgjzushrbSIQZ4DxLtLG2ijREROLPX+bN/tdpOZmcmvfvUrJZSIiIiIiDSyCRMmYBgGL774ImazmQkTJhx1jmEYvPzyyxHfe/r06cyaNYtf//rXvPDCC7We17FjRwD27dsXaktJSWHjxo2UlZVVqlTicrlISkqqsTqJiIiIiMjx4oYbbsAwDK655hpsNlvoOhgMVjvHMAwllIjIicVTDAFf+XE0h8nCzTzyceHDGfCS6vNhJYjXMCgwm9hsNjEvNgheN+k+P+z74edqIzUkqYSxt4D87bBzJcXdziNzQyauIhcpjhTsFntomM0wk1x6CIfXh8tq4a8WK8/QjNGYwABsACVVVjwREWmqGiR9vKYnwiIiIiIi0nAWLFiAYRjMmzcPs9nMggULjjqnPhJKpk+fzowZM8jIyOCll17CMIxaz928eTMArVu3DrX169ePjz76iFWrVpGe/nOJ2dLSUtauXcuQIUMiildEREREpLEsW7YMIJQQXXEtInJSscWDyQJ+b/lxNJQfc5NJIS58pPiD2P1+oPw9RBuQHABHMIjLYiHTYadvyQHiMWD/lvJqI9UkqVRitpaP8xSR7com7//Zu/f4Luv6/+OP67o+h322zz7bYCDHcT4IqWAqiAIhiKVFYZqlv8I8ZGF4wDCPJYRomYWHokwUPJSZmvnN8qvw9UwOVDBDBETOCMyx02ef43X4/XGNwdoGwgbj8Lzfbovtut7Xdb0+79DN7bnXK1lGLBRrECYBIFkBdoaIGSCGRRkOi0hyJnm7vY7GHU9ERI5mByRQsi/fPBYRERERkdbjuu4ePz4QZsyYwfTp0/n2t7/NQw89hGmajdbYtk1tbS0FBQUNjm/cuJE5c+bQvn17RowYUX/8ggsuYNasWcyePbtBoOQPf/gDiUSCiy666MC9IBERERGRVjR69Og9fiwickToPgzWLITkDsjvBMCbJCnDIYZJxM2At/v3KAzAIOJBzPUosywW5eVxZk0N1G6Hta83GVJpkpP114WiLN2+lOpMNV2jXRuvS1X7wZNADgXAZhyWkm4YKIEGHU8UKBGRo506lIiIiIiIyH77zW9+w09/+lNKSkoYN24cf/zjHxucP+aYYzjzzDOJx+P06tWLr33taxx77LEUFRWxcuVKHnzwQeLxOH/605+IRHb95tBxxx3HlVdeyf3338+5557L2WefzYoVK7j33nsZPXo0F1544cF+qSIiIiIiLRaPxznhhBOYMmUK11xzTVuXIyLSevqOhbcfgu0fQDgGoVyWkqYah64EGoZJDMN/q1PgemwOmCwNhzizNgBuFspXw1BmzzMAAQAASURBVOe/2yik0qTkDogUQskwkpVLcVyHoNlEVxOvrkOKYRDEw8EjQRM/09yt44mIyNGu1QMlHTp0YO3atXTqtId/sYuIiIiIyBFhyZIlAGzYsIFJkyY1Oj969GjOPPNMIpEIX//61yktLeXZZ58lHo9TXFzMuHHjuP766znllFMaXTt79mx69uzJAw88wPPPP09xcTFTpkxhxowZTXZBERERERE51EWjUcrLy4lGo21diojIZ5eugY8WwMbF/hiaUJ7fkaTvWAjn+2vC+TBiCrxyB1RugJxCklETx4Cg5zXZnWSnIOAACdMAywI3A4mKJkMqjWQSkKqEjoOgz1gi732IZVpk3Swh67+6mhiW/1zPI2uAhUEuTUxd2K3jiYjI0a7VAyWmadKjR4/Wvq2IiIiIiHwGM2bM2OdrDMPg1ltv3a/nzZs3j3nz5u11XTgc5sEHH9yne1uWxXXXXcd11123X7WJiIiIiByKhg8fzttvv81ll13W1qWIiOzd6pdg0X0Q3wrJSr9zhxnwO4e8/ZAfIul3pr9255916yOpGqywSdaxqY92GAYYDX9JJAtYQK7rgesCBkSKmgypEGnn36P2U0iUgZ32wyZdPw94DO04lEVbFlGVrqJDboeGryUnBskKcLNUWRYxTIYSbvyad+t4IiJytGu1QMlLL73E6tWrKS8vbzTypiXfoBYRERERkc/utttu2+dr9PW6iIiIiMjBc+edd3LGGWcwbNgwLr74Ygyjid+OFxE5FKx+qWGYo7CnPw7Gyfqhi+0f+OehYaikZDh8tJChq59lUc0KqkIGHRzX7yQCgMfuHUqqTIOY6zE0nfHH3ZgmFPdreN+doZZPV/pdUnDrbmOAk4GVz8Pmdzht2OU8GenA6srVRENRIoFd43WJFEHNJyQztVRb0I8wp7HbeWjU8URE5GjX4kDJhx9+yMSJE1m1alWjIMlO+ga1iIiIiMjBsXbt2rYuQURERERE9mDq1KkUFRVx2WWXcf3119OnTx9ycxuOcTAMg4ULF7ZRhSIi+GNuFt3nh0kKShqOmwmEIL+T3xmkcoO/rmR4w/E3g7/Gaf3P5Mn/u9oPd4QjRLIJP4zi1XUhMQySBlSbJv2yNqfV1oJpQW4x9BrZcNROrCu16SreCFssy4uRDASJWBGGBmKc5obIS1bC9g/Ie/3XTDrha8yxE2yObyYWilEQLiBoBsl6DlU5+VR7abraNpOyFrlBu2FIJlUJhSV+Z5SwRt6IiLQ4UHLFFVewceNGZs+ezciRIykqKmqNug4LmUyGbDYLQDKZbONqRERERETQ+EkRERERkUPcxx9/jGEYlJSUALBt27Y2rkhEpAkfLfQ7guQUNgyT7C6U65+Pb4U1/weDvtrgdF4wj0mDJzHnvTlsrtlELBikwLUJYpD1PKpMP0zS1baZFE+Sa4X9jiPt+wIGPHFR/aid1y2X+RGTsmiIasvCMSwsw2MRcZ40LSblFzCyLuAycsUCGPVD5q96krJkGZvjm3FcB8u0iIVj9AsXMqmqmpHpaqhct2uMT6TQ70yy+xgfEZGjXIsDJYsXL+aGG25gypQprVHPYWXWrFlMnz69rcsQERERERERERERkcPEunXr2roEEZG921gKyUp/zM2eRNr5oYwNpY0CJQAju40EYP7y+ZRVrWOzV4bj2lgYxDyPfg5MShmM9CIQyEJRT+hxGrw5u37UzutFnZhDFZsNm5jr0DWbJWg4ZANhqkyP1WSYQyWEChlZF3AZmUxy4hn38ObmN1lWtoxENkFuMJehHYcyossI8lzXD81sXAyZOISiUDIM+pyxq9OKiIi0PFDSvn17iouLW6OWw85NN93EtGnTACgtLWXsWM1SExEREZFD09tvv01paSkVFRW4rtvgnEZUioiIiIiIiEgDmVp/DEyqCjLV4Lpgmv6Ym0iR39ED/HExru2HMpoxsttITjzmRD/cseo5Ep+8S24mwdBkihHpDHlGXXeQaCc4+TJY8mD9qJ3aUA7zKWOz59AlaxPBAMMEzyVkZ+gQyiWKwWZs5lPNiZFC8io3wIZS8gZ9lfE9xzO+5/imCxv8Nf9NRESa1eJAyTe/+U2effZZrrzyytao57ASCoUIhUIARCKRNq5GRERERKSxZDLJueeey4svvojneRiGged5APXvK1AiIiIiInLwpdNpXnnlFT7++GMAevfuzejRo8nJyWnjykREgNpPIVMD2SR4NnieP44mWQE1n0CsG+S280MnZsDv8LEHecG8+nBHbXwrbyx9kHe3LuZNO0UkkMPQzsM4bcil5K1/q8GonTeppQyHmOsR2VkDdW+eC45NxAoSw6MMh0WWw5l7CbiIiMhn1+JAye233855553HxIkTueqqq+jZsyeWZTVat3MepIiIiIiIHDwzZszgxRdf5Oabb2bs2LGMGTOG+fPn07FjR+644w6SySSPPPJIW5cpIiIiInJUeeSRR5g6dSoVFRUNAt+FhYXcfffdXHzxxW1boIgc3da8DGUr/K4khg3BHPwQhweODZkEVK731zoZv7tIybDPdOvXN73uj79JllHt1eAYDpaXZdGni3nyzbVMSnmM3G3UzlLSVOPQ1fuvGxl1gRLPAYIUYLIZh6VekjM/Q8BFREQ+mxYHSoLBIIMHD+auu+7iueeea3ad4zgtfZSIiIiIiOyjp556ivPPP58ZM2ZQXl4OQNeuXTnjjDMYO3YsJ598MvPmzeOOO+5o40pFRERERI4Of/7zn7n44ospKSnhRz/6EYMGDQJg+fLl/O53v+PSSy8lEolwwQUXtHGlInJUyiSg9PeQqoZgBOy0353ENAEDAiFwLf94xXo/Z5JTAKv+Fza8Bd2HQd+xEM5vdOvXN73OnPfmsDm+mVgoRtdoV0xMdqR38GnyUzbHN/OR6zIxYnCZZZEHJPFwgKBhAbt1SsHwb1oXygsCDh4JJ7lPARcREdmzFgdKrr/+en79618zdOhQTj/9dIqKilqjLhERERERaQUbN25k6tSpAPWdBDOZDACBQIBvfetbzJkzR4ESEREREZGDZNasWQwcOJC33nqLWCxWf3zChAlMnjyZYcOGMWvWLAVKRKRtbH4XardBpMgPj1Ss98MjZgCsAGDu6g6SjYMZhEwtrHrBX7NmIbz9EIyYAv3OrL9tbbaW+cvnszm+mS7RLkQCESrTlWyr3UbGyeB4Dq7nssNzeDw3yHts47sUEcHAArJWgJCTretIslu7EsMPlmQBC8i1MxDtAX3GHsRNExE5crU4UPLoo48yceJEnnrqqdaoR0REREREWlF+fj62bde/b5omW7ZsqT9fUFDA1q1b26o8EREREZGjzsqVK/nZz37WIEyyU0FBAd/97ne57bbbDn5hIiLgj7pJVUGssx8oAaja5I+2sVN1HUE8P1AC/priAWAFwclCcgds/wBeqfvFlbpQyZub36QsWUYsFKsPk2yJbyFtp7FMi5AVwsAgm02RNWxWeFnmUMkwcohhUYVHh0C4roa6Z2OAYYHnUeVliHkuQ4Pt/DBLWCNvRERag9nSGyQSCcaPH98atYiIiIiISCvr06cPq1atAvwOJYMHD64Pg3uexzPPPEP37t3bskQRERERkaNKp06d9njeMAyOOeaYg1SNiMh/ySbBtf2ACECkHXQcBIU9IK+j/7ERAMP0wxyhfD9UYtSNw8nvBAUlULkBFt0H6RoAlm5fSnWmmoJwAY7rsK12G2k7TSgQImSFMA0TwzAIBMIYBoQ9l81keY807TCpxiVpmhDI8Z/tAXjgZEjaKaoN6BDI5bTTb2zQGUVERFqmxYGS4cOHs3z58taoRUREREREWtm4ceN4+umncRwHgCuuuIIXXniBPn360K9fPxYsWMCll17axlWKiIiIiBw9Lr74Yh5++GHi8Xijc9XV1Tz88MN897vfbYPKRESAYMQfXeNkdx0zA5DXAdr1htx2YJr++BvTBNNqfI9QLuQUQnwrrPk/AJJ2Esd1CJpBqjJVZJwMlmlhYpJ1s6TsFCk7RcbN4GCAaRFzHMrdDEPdAF0JsAWbMhwyhokXDJPJKaQsks+WcA5d87sxadTt5A788sHZJxGRo0SLR97cfffdjB8/ntGjR3Puuee2Rk0iIiIiItJKbrjhBr797W/jef584cmTJ5NKpXjsscewLIvLL7+c66+/vo2rFBERERE5eowcOZK///3vHHfccUyePJmBAwcCsGLFCubMmUNxcTEjR47ktddea3DdqFGj2qJcETnadDgW1hX4o2vym+iolK72O5hg+UGTcOPxXYDfyaRyHWwohUFfJRKIYJkWWTdLPBPH8RwswyJpJ/E8D89vOQKAh0eNGSBmWFS7NqlUFT9I28zPC1FmWWwOhXACOVjBCLFQjH6RDkwaPImR3UYekC0RETmatThQcs0115Cfn8/5559P165d6dWrF5bVMI1oGAYLFy5s6aNERERERGQfRaNRBgwY0ODY1KlTmTp1ahtVJCIiIiJydDvzzF2jGH784x9jGAZAfQh8/fr1DdZ4nodhGPVdB0VEDqiuJ8IHx8D29/ywSCi34XnXBc8Dz4ZgHkQKm76PFfSDJxm/G9PQjkNZtGURVekqXM/F8Rxs165fbhgGBgau52Jg4OCxzQphWhaJvBJG5vThxEAOb8YKWBbwSLg2ucFchnYcyoguI8gL5h2gDRERObq1OFDy8ccfYxgGJSUlAGzYsKHFRYmIiIiIiIiIiIiIHIkefvjhA/4M13W55557+P3vf8+6devo0KED3/jGN5gxYwZ5eXv/oeu+Xm/bNr/97W+ZN28eK1euJBAI0KdPH6644gquuOKKA/ESReRACeXCsCvg9TuhcoM/uibSzg+IOFmwU+C5EMiBgm5+l5KmOFn/XCgKwGldT+PJlU+yunI1eNR3JTENE4O6YF1dlxLTMAlbYdJOGtMwCZaMgOE3kweMr3sTEZGDo8WBknXr1rVCGSIiIiIiciC99NJLrF69mvLy8vrffNzJMAxuvfXWNqpMREREROToMmnSpAP+jGuvvZZ7772XiRMnct1117FixQruvfdeli5dyoIFCzBNs9Wuz2QyTJgwgZdffpmLLrqI73//+9i2zerVq1m/fv2BfqkiciD0GQOWAYvug/hWf3SNa9cFRHLByfidSSLtmr9Hcoe/pmQYAHnBPCYNnsSc9+awumJ1fXjEqB934/+vgUHIDGKZFobjB03CgfCBfLUiIrIHLQqUxONxJkyYwEUXXcSll17aWjWJiIiIiEgr+fDDD5k4cSKrVq1qFCTZSYESEREREZG243keGzdupFOnToRCoRbfb/ny5dx3332ce+65PP300/XHe/XqxVVXXcUTTzzBhRde2GrX/+xnP2PBggW89NJLjBkzpsX1i8ghot+ZUDIcPloIGxf7o2tCUeh8PLz7KHy6EjKJxiNxwD+eqoSOg6DP2PrDI7uNBOC2RbeRSqYAcDwPAw8DMIEQEHAyOJ6Lh4eBQdpJH/CXKyIiTdtzDHkvotEoS5Ysaa1aRERERESklV1xxRVs3LiR2bNn8+6777J27dpGbx9//HFblykiIiIictTavn07vXr14o033miV+/3pT3/C8zyuueaaBscvv/xycnNzeeyxx1rt+traWu655x6++tWvMmbMGDzPo6amplVeh4gcQOkaWP5XeOFGeO4q/8/lz/qhkd2F86HvWOh+MoTy/POfvAfdT4FYV6jaADVbwc6A5/l/1mz1jxeWwIgpEI42uOXIbiM5NdqDsAcBb2eQxCCIQQ4GpgcZ1yHjZMgxAoTNMFkne9C2RkREGmrxyJshQ4awYsWK1qhFRERERERa2eLFi7nhhhuYMmVKW5ciIiIiIiLNaK6b4P5YsmQJpmlyyimnNDiek5PDkCFD9vpLovty/euvv05NTQ2f//znufrqq3nooYeIx+MUFxdz+eWXM2PGDAKBFv8YQkRa0+qXdo2ySVbuGmWzZiFEH4UTLoeCsXteGykEMwj5nfzxN7uPxIkU+p1JRkzxu5z8t3QN+WUryXdd8o0A5YZHBnDwSAOGARYGuZ5He9ehJhQlN9hEFxQRETkoWvyV3PTp05k4cSLnnHOO2tmJiIiIiBxi2rdvT3FxcVuXISIiIiIiB8mWLVsoLi4mHA43Ote1a1cWLVpEJpNpdrzOvly/cuVKAGbPnk0oFOIXv/gF7du35/HHH+eOO+5g8+bNzJ8/f681b9y4kU2bNjU49v777wNg2zbZbPPdCbLZ7F7XHK20N007qvdlzcvwxq+gaiOEC6Conx8McbOQrCBbvhZ72Z/JBg2wjD2uJVEJ+V2g7xchm941EqfbSdB7lP9+U3u8aiFDUi5LQrm4hsVADKrxiOPVj7iJYhDzbCpcg/bkMKT9kDb//+uo/nuzF9qbpmlfmqe9ad6B3BvbtvfruhYHSh577DFKSkoYN24cJ5xwAv379yc3t2FS0DAM5s6d29JHiYiIiIjIPvrmN7/Js88+y5VXXtnWpYiIiIiIyEGQSCSaDIOA32Vk55rmAiX7cv3O8TY7duxg+fLlDBgwAIBvfOMbjBkzhkceeYQbbriBY489do81z507l+nTpzd5Lh6PU1VV1ey1tm1TW1sLoG4o/0V707Sjdl8yCXjnKUh50O5ECOY0PB/ugm071GY9ePcZAjh7XEs2BbXbYdsGGHMLhHb72WDSgWQz/9xuXMEgp4A+gVw2GQ5RTNphNi7Xc0lh080rZFDuoD3+e+BgOGr/3nwG2pumaV+ap71p3oHcm3g8vvdFTWhxFfPmzat/f9myZSxbtqzRGgVKRERERETaxu233855553HxIkTueqqq+jZsyeWZTVaV1JS0gbViYiIiIhIJBJh0qRJdOnSpVXul5uby/bt25s8l0ql6te0xvWRSASA4cOH14dJdvrOd77DK6+8wiuvvLLXQMmll17KWWed1eDY+++/zxVXXEE0GqWgoKDZa3f+Bm8sFiMYDO7xOUcb7U3Tjtp9+fANqPoAcMHIBbvxDxazBAGTWPkHBL00BHObXYsBUOPfs3wpDDz7s9WR3U5BYj3nRHrwoFHFBzjkYxLDIAhkgWo8anDpbGc5J7cdx7Q/Zr9fdms5av/efAbam6ZpX5qnvWnegdybaDS6X9e1OFDium5LbyEiIiIiIgdIMBhk8ODB3HXXXTz33HPNrnMc5yBWJSIiIiIiO8ViMR5++OFWu1+XLl344IMPSKfTjTqNbN68meLi4ma7k+zr9d26dQOgU6dOje7TuXNnACoqKvZac/fu3enevXuT5wKBwF5/oLJzjX4o1Zj2pmlH5b5sLoXEdijsiR/b2KUWlzdI8m88jJxivEycE9JxTsspJo89jF3IiULlOv/ex331s9URzgEcRjkuRiCX+VRTRob1uDh4WBjEMOnhGUxKJBnZrSscIv8/HZV/bz4j7U3TtC/N094070Dtzf52PFEPGRERERGRI9j111/Pr3/9a4YOHcrpp59OUVFRW5ckIiIiIiIH0Mknn8yLL77I4sWLGTlyZP3xVCrFsmXLGDVqVKtdf8oppwCwadOmRvfZeaxjx44tej0i0koyteDaYDX8AeXrJOtCHQ4pArQ3spRHQrwRivBkoJJJFDASvxsRrg3JCkhXg+uCYfr3Te49OFav+zBYsxCSOxiZ34kTCfMmSZaRJoFHLgZDCTMiXk2eFYOSYa24CSIisq9aLVDieR5Lly7l448/BqB3794MHToUwzBa6xEiIiIiIrKPHn30USZOnMhTTz3V1qWIiIiIiEidJ554gvvuu4/Vq1dTXl7e6LxhGNi2vV/3vuCCC5g1axazZ89uEAj5wx/+QCKR4KKLLqo/tmbNGrLZLAMHDtyv63v16sVpp53GokWLePfddznxxBMBvwPiH/7wBwKBAOPHj9+v1yEirSyUB2YAnCwE/C5Dr5NkDpVsxiaGSWcsOngmIduh3DRYTYY5VAIwMpGE6k3gZPxgiecBBngurH0NVr8E/c7cex19x8LbD8H2DyAcIy+Uy3jyGE/erjWZBKSqoOMg6DO29fdCREQ+s1YJlLzwwgtMnjyZ9evXNzjes2dPfvvb3zaafSgiIiIiIgdHIpHQN3BFRERERA4hd911FzfccAPt27dn+PDhtG/fvlXvf9xxx3HllVdy//33c+6553L22WezYsUK7r33XkaPHs2FF15Yv3bs2LGsX78ez/P263qA++67j5EjRzJu3Diuuuoq2rdvz5///GcWL17MT37yE0pKSlr19YnIftqtMwj5najFZT7VbMamCxYRTAIY4DmErAAdPIeo47HZspnvlHNiVRl52bQfSgnkAAbYGTDwO5a8cof/nL2FSsL5MGKKv75yA+QUQqSd3znFyfr1pSqhsMRfF44e2H0REZE9anGg5M0332TChAnk5eVx9dVXM3jwYACWL1/OvHnzmDBhAi+//DIjRoxocbEiIiIiIrJvhg8fzvLly9u6DBERERERqfOb3/yGYcOGsXDhQiKRyAF5xuzZs+nZsycPPPAAzz//PMXFxUyZMoUZM2ZgmmarXj906FAWLVrELbfcwuzZs0mlUhx77LE8/PDDXHzxxQfk9YnIfvivziBvhjzKcIhhEqHun2vPBdcBKwcwiWQTxEyTMi/DoqDBmV4YTMtf6zrgORDKhaJefjhk0X1QMtwPjezJztDJovsgvhUq1/ldT8wARAr9ziQjpny2jiciInJAtThQMmPGDDp16kRpaSmdO3ducG7atGkMGzaMGTNm8MILL7T0USIiIiIiso/uvvtuxo8fz+jRozn33HPbuhwRERERkaPe1q1buf766w9YmATAsiyuu+46rrvuuj2uW7duXYuu3+n444/nueee29cyReS/pWvgowWwcTFkav0xNd2H+WGQvYU09ua/OoMsLYhSHfLoSsAfX+Nm/YBIOATRrkAWKjdQkE2w2TJZmhPizKzth04c2w+ABMJQ0M2/d06hHw5Z838w6Kt7r6ffmX745KOFda83DqEolAyDPme0/PWKiEiraHGgpLS0lB/96EeNwiQAnTt35vLLL+fuu+9u6WNERERERGQ/XHPNNeTn53P++efTtWtXevXqhWVZDdYYhsHChQvbqEIRERERkaNL3759qaysbOsyRORQs/qlXR07kpW7OnasWeh3Ftnfjh3/HVLJ7wTZJEkjgeM5BO0sYPjPCuZC9BiIuGDYAATLP8IxXBIA2Vow6taGcv0wSaSd/5xIO7/TyIbSzxYoAT80Mvhr/puIiBySWhwoyWQy5Oc3nxKMxWJkMpmWPkZERERERPbDxx9/jGEY9XPLN2zY0MYViYiIiIgc3a677jpmzpzJVVddRTQabetyRORQsPql+s4h5BRCYU+wguBkIbnDH1Pzyh3+2n0JlTQXUsmJETEsLNMkG8klZAYhJwY5HYB8oMq/Prcd2XgMK1tFbiACuUF/5E045o+mMXf7MaMV9O+fibfGjoiIyCGixYGSY489lieeeIIrr7ySQKDh7Wzb5s9//jPHHntsSx8jIiIiIiL7obkW1iIiIiIicnA88sgjDT62LIuOHTsycOBALrnkkia7CAJ85zvfOVglikhbStf4oY/KDVBQ4nf+2CkQ8juKhGP++UX3+WNiPss4mL2EVIYm4izKz6UqJ0yHwp7+NZ4FbsPbVBkuMQ+GBttBcUHzz3OydZ1LFJQTETmStDhQ8oMf/IDvfe97jB07luuvv55BgwYBsHz5cu666y5KS0t54IEHWlyoiIiIiIjsm3g8zoQJE7jooou49NJL27ocEREREZGj0sUXX4xhGHie1+jczJkzm7zGMAwFSkSOFh8t9DuI5BQ2DJPsLpTrn49vhTX/t/eRMp8hpHJaJsqT9nZWp8qJZjoQCeU1uk3STlJtQD8CnJZIQP4eAiXJHX7XkpJhe3vFIiJyGGlxoOSyyy5j9erV/PKXv+SNN95odH7atGn65rWIiIiISBuIRqMsWbKEiy66qK1LERERERE5ar388sttXYKIHMo2lvrjaHZ2CWlOpB1UroMNpXsPlHyGkEpeKMqkZBVzLIfN1euJ5XagKNAOMlnWJquocXdgk6RjThf+X6aK3NR6CBc0fb9MAlKV0HEQ9Bm799csIiKHjRYHSgB+/vOfc+mll/K3v/2NtWvXAtC7d28mTJhA//79W+MRIiIiIiKyH4YMGcKKFSvaugwRERERkaPW6NGj27oEETmUZWrBtf1xNHtiBf11mfje7/kZQyojgx0gvoH5Oe1Zn/VYVbWRY6wktdnN5DhpChyTU8scKr2+VIWSFFTVjc+JtGswPodUJRSWwIgpENbIGxGRI0mrBEoA+vfvz7Rp01rrdiIiIiIi0gqmT5/OxIkTOeeccxgzZkxblyMiIiIiIiIiuwvlgRnwwxmBUPPrnKy/LvQZAhv7EFIZmcrQ2e7NFdUn0NF+i+NzUpipNENSlQxPpAl526n0PqLcCmLmtyff9PxOKa7t1xMp9DuTjJgC/c7cl1cuIiKHgVYLlIiIiIiIyKHnscceo6SkhHHjxnHCCSfQv39/cnMbtqc1DIO5c+fu1/1XrVrFY489xosvvsiaNWtIpVL06dOH888/n2uuuYa8vIYzmFeuXMmPf/xjXn31VTKZDCeeeCLTp0/njDPOaHTvqqoqbrnlFp555hnKy8vp06cPP/zhD/n+97+PYRj7Va+IiIiIyKGgtraWP/7xj6xevZry8nI8z2twviVfo4vIYab7MFiz0O/0kd+p+XXJHX54o2TY3u+5DyEV1wiwapvLoGSSyYH1RNI5mCmXJMXssAIEsMlzq4k4Oyir7UjopC8T9jJ+p5RQ1K+nzxkQzt/nly4iIoe+VgmU/Otf/+L+++/f4xe/a9asaY1HiYiIiIjIPpg3b179+8uWLWPZsmWN1rTkm9UPPfQQv/nNb5gwYQIXXXQRwWCQl19+mVtuuYUnn3ySt956i0gkAsCaNWsYMWIEgUCA66+/noKCAv7whz9w1lln8c9//pNx48bV3zeTyXDmmWeydOlSpkyZwrHHHss///lPJk+ezLZt27jtttv2q14RERERkba2ePFivvzlL/Ppp582u0aBEpEjWLoGPloAGxf7nUSsIBgWJD+FcAxCuY2vyST8sTIdB0GfsXt/xj6EVGrNKO9le3Cx8Xc6uJ+yyToOywLTzQBgE6TKak+1l0snezs1H71F+LK/KEAiInKUaHGg5JFHHuG73/0uwWCQ/v37U1JS0hp1iYiIiIhIK3Bd94De/7zzzuPGG2+koKCg/tj3v/99+vXrx+23387cuXP54Q9/CMCNN95IZWUl77zzDkOGDAHgO9/5DoMHD+bKK6/kww8/rO888uCDD7JkyRLuvfdepkyZAsDll1/O17/+dWbNmsV3v/tdevTocUBfm4iIiIjIgTB16lQymQxPPvkkZ5xxBu3atWvrkkTkYFn9Eiy6D+JbIVm5a2yMGQQnA+WrIXoMRNr5QRMn64dCUpVQWOKPlQl/hpE3fcfC2w/B9g/2GlIpM3uQyDh0MCpJGHlkCWE1cUvHilCdzSOv5hNY838w6Kst3AwRETkctDhQcvvttzNgwAAWLFhAly5dWqMmERERERE5TJx00klNHr/gggu4/fbb+c9//gP4Lb2fe+45vvCFL9SHSQCi0SiXXXYZP/nJT1iyZAmnnHIKAH/84x/Jzc3l8ssvb3Dfa665hmeeeYY///nPXH/99QfmRYmIiIiIHEDvvPMON910E+edd15blyIiB9Pql+CVO6ByA+QUQmHPhqGRdA14LmQTkK7eFTaJFFLbcSBv9B/Nsop3SS56k0ggwtCOQzmt62nkBfMaPyuc74dPdn9eMyGVV4MXMLDmFfLNGj4xm/+lcdOASqK0sz+FDaUKlIiIHCVaHChZv349d911l8IkIiIiIiKHMM/zWLp0KR9//DEAvXv3ZujQofUdQVrbpk2bADjmmGMA+Pe//006nebUU09ttHb48OEA9YES13V59913OfHEE8nJyWmw9pRTTsEwDJYsWXJA6hYREREROdBisRjt27dv6zJE5GBK1/idSSo3QEFJw44hgZA/liYcg8r1kFMAPU7zO5aEorweK2J+9QeUbXuD6kw1jutgmRaLtiziyZVPMmnwJEZ2G9n4mf3O9P/c2RGlcl2DkAodB8GIKWz8sCvHrf9fTM/B2cOPDV0PbAJYuJCJt+r2iIjIoavFgZJu3bqRTqdboxYRERERETkAXnjhBSZPnsz69esbHO/Zsye//e1vOeuss1r1eY7j8LOf/YxAIMCFF14IwJYtWwDo2rVro/U7j23evBmAiooKkslkk2vD4TDFxcX1a/dk48aN9cGWnd5//30AbNsmm83uw6s6cmWzWe1HG9Hetx3tfdvR3rcd7X3bOBr23bbtti7hsHPuuefyv//7v0yePLmtSxGRg+WjhX6oI6ew6fEz4B+PFIHnQK+RMOirvL7pdea8N4fN8c3EQjG6RrsSNINk3SxV6SpWV65mzntzAJoPlZQM95+/cbEfBAlFoWQY9DkDwvmclPqE5Pu52FkLi+b/nW67LjmmixUI+vcQEZGjQosDJd///vd5/PHHufbaa7GspqaqiYiIiIhIW3nzzTeZMGECeXl5XH311QwePBiA5cuXM2/ePCZMmMDLL7/MiBEjWu2Z11xzDf/617+YNWsWAwYMACCRSAB+IOS/7exCsnPNntbuXL9zzZ7MnTuX6dOnN3kuHo9TVVW113scDWzbpra2FoBAoMX/iSj7QHvfdrT3bUd733a0923jaNj3eFy/pb6vfv7zn3PWWWcxZcoUrrnmGnr37n3AOgeKyCFiYykkK/0xN3sSaed3EtlQSm2/ccxfPp/N8c10iXYhEojULwtZITrkdiAairI5vpn5y+dz4jEnNj/+ZvDX/LcmjOrfgbnR46nc8S55bg21TaxxPQ/H8TjGqiEYbecHUkRE5KjQ4v+K+fznP8/TTz/NKaecwpVXXkmvXr2aDJaMGjWqpY8SEREREZF9NGPGDDp16kRpaSmdO3ducG7atGkMGzaMGTNm8MILL7TK82699Vbuv/9+vve973HjjTfWH8/N9X8Dq6nuhqlUqsGaPa3duX7nmj259NJLG3Vfef/997niiiuIRqMUFBR8hld05Nv5G9OxWIxgMNjG1RxdtPdtR3vfdrT3bUd73zaOhn2PRvVb6vuqsLAQwzBYvHgxv/3tb5tcYxiGur+IHEkytf64GWsvnwusoL8uE+fNzW9SliwjFoo1CJPsLhKIEAvFKEuWsWjLIs7sceY+lxYNBxh6xtep+dv/0NnZiuGkcT0wqRtz47o4jkcskKHYSmHl94U+Y/f5OSIicnhqcaBk7NhdnzQuu+yyRklqz/MwDAPHcVr6KBERERER2UelpaX86Ec/ahQmAejcuTOXX345d999d6s867bbbmPmzJl897vf5Xe/+12Dc126dAFoclTNzmM7R9wUFRURiUSaXJtOp/n0008ZPXr0Xuvp3r073bt3b/JcIBA4Yn+osz927of25ODT3rcd7X3b0d63He192zjS9/1I7bxyIH3nO99RRxKRo0W6Bj5aAFuWQiYB5av9sTaRIjCb+Penk/WPh6Is3b6U6kw1XaONx8HuriBcwOb4ZpZuX9o4ULLz+RsX+6GWUB50HwZ9x/qdS+qM+lxv3iubQuVb99COCrzsDmrcIDYBckyXY6waiq0UoeKeMGIKhBUmFBE5WrT4q/2HH364NeoQEREREZEDIJPJkJ+f3+z5WCxGJpNp8XNuu+02pk+fzqRJk3jwwQcbfYP8uOOOIxwO869//avRtW+99RYAJ510EgCmaXLiiSeydOlS0ul0g9E3ixcvxvO8+rUiIiIiIoebefPmtXUJInIwrH4JFt0H8a1QsxWcDNR+CulqqPkECrr5I252l9wBkUIoGUaycimOaxNMVkI6Dp4DhgU5MQjHIF0FqRqCro3jJklsX+EHSHYGRXZ/frLS73xiBmDNQnj7IT8Y0m9XAOWEMedTUxxle+lfyHXidMiUYeFiBYIEo+38ziT/dY2IiBz5WhwomTRpUmvUISIiIiIiB8Cxxx7LE088wZVXXtnot0dt2+bPf/4zxx57bIueMWPGDKZPn863v/1tHnroIUzTbLQmGo3yla98hWeeeYb33nuPE044AYB4PM6DDz5Iv379OOWUU+rXf+tb3+LNN9/kgQceYMqUKfXHZ8+eTSAQ4IILLmhRzSIiIiIiIiIHzOqX4JU7oHID5BRCu36wY3Xd6BsXnARUrPfX7gyVZBKQqoSOg6DPWCKvvoiVriFbW0XIdQAPMCDxKbgOmBZ4Hlk8LMskd8tSeOIiP/QBDZ9f2NMfp+Nk/dDK9g/889AgIJIzcBwF7QdTUP4uwc2LIROHUBRKhkGfMxp0NRERkaPDUd2P8Mknn+Tee+9l2bJlFBcXs27durYuSURERESkVf3gBz/ge9/7HmPHjuX6669n0KBBACxfvpy77rqL0tJSHnjggf2+/29+8xt++tOfUlJSwrhx4/jjH//Y4PwxxxzDmWf635y64447WLhwIePHj+faa68lFovxhz/8gc2bN/P888836Gpy+eWX8/DDDzN16lTWrVvHscceyz/+8Q/++te/csstt9CzZ8/9rllERERE5FARj8eprKzEdd1G50pKStqgIhFpsXSN3xmkcgMUlEAo1z8e6waV68FOg2FCNgUVGyCbhES5fzwcg66fhzULGbphGYuMLFWGSYdADhiG3+XEToHn1XccqQoEiRkmQ20DKj+A/5vpn4tva/h8gEAI8jv5z6nc4NdZMrxhUCSUCwPPgeO+dlC3TUREDk1HdaCkqKiIH/7wh2zbto1f//rXbV2OiIiIiEiru+yyy1i9ejW//OUveeONNxqdnzZtGpdeeul+33/JkiUAbNiwocnuhaNHj64PlPTt25c333yTG264gTvvvJNMJsOJJ57ICy+8wLhx4xpcFwqFWLBgAbfccgt/+tOfKC8vp0+fPtx3331ceeWV+12viIiIiMih4IknnmDmzJmsWLGi2TWO4xzEikSk1Xy00B8zk1PYMMyRW9eJpHqTHwxxs5CpgYpawNgVGFnxP/D2Q5yGx5PHFLPacIjiEXFsP4SCV3/LpOdQjUm/bIbTvCKIxaBspX/vQATin/jhkUiRP+5mp1CuX198K6z5Pxj01YOwMSIicjg6qgMlO7+x/eyzz7ZtISIiIiIiB9DPf/5zLr30Uv72t7+xdu1aAHr37s2ECRPo379/i+49b968fZoBf+yxx/K3v/3tM60tLCzk/vvv5/7779/P6kREREREDj3PPvssF154If379+eKK67gd7/7HRdeeCG2bfPss89y/PHHc84557R1mSKyvzaWQrLSHzPz33LbQU4MKjf6YQ7PBTPodxLJa+93HqlcD+ka8swAkzIB5oQNNntZYl6WAjyCQBaosiyqTZOuts2kyji5mcq6UErav2+2FpwUJCug5hO/Q8rOUAv4o3Yq18GGUgVKRESkWYdFoOSOO+7g3Xff5Z133mHt2rX06NGj2fE0rutyzz338Pvf/55169bRoUMHvvGNbzBjxgzy8vIObuEiIiIiIoeI/v37M23atLYuQ0RERETkqPfLX/6SY489lnfeeYd4PM7vfvc7LrnkEs444wz+85//cNppp3HzzTe3dZkisr8ytf7IGSu4hzU1+F1JLL97SKzTrnOm6QdDXIeRFVuhQ0/mO9spM2BzMIBjGFgexFyXftksk2oSjMzY/igcdo2SxQhAIAyODZmEH1SBXaESK+jXmYm39g6IiMgR5LAIlNx00020a9eOE088kcrKyj2uvfbaa7n33nuZOHEi1113HStWrODee+9l6dKlLFiwANM0D07RIiIiIiIiIiIiIiL/5d///je33HILOTk5JBIJYNd4m8997nN873vf44477uCrX1XHAJHDUijPHy/jZCEQanw+WeGPtjFNf3qNaTU877r+n2YAnAwjqz7lxFQVbwYNloVDJEyTXM9jaCrNiESSPNPyO5LsZJjgOYDnvx8IgWuBnfbH7eTEdtVnBiAUPVA7ISIiR4DDIlCyZs0aevfuDfhfUMfjTaclly9fzn333ce5557L008/XX+8V69eXHXVVTzxxBNceOGFB6VmEREREZFDxb/+9S/uv/9+Vq9eTXl5OZ7nNThvGAZr1qxpo+pERERERI4ujuPQvn17ACKRCABVVVX15wcMGMCcOXPapDYRaQXdh8GahZDcAfmdGp9PV/udQbD8QEc41vD8zg4lluUHT1KV5DlZxtswvjbhh0QM/DCK5+162707CTQMmZhWfUCFZCXkFfv1RQqhZFhrvnoRETnCHBbtOnaGSfbmT3/6E57ncc011zQ4fvnll5Obm8tjjz12AKoTERERETl0PfLII5x++uk8/fTTpFIpSkpK6NGjR4O3kpKSti5TREREROSo0a1bN9av90dPRCIROnbsyDvvvFN/fuXKlRrfLnI46zsWop0gVemPmvlvrusHQFwbrJAf6thduK6DSP26uj+Nuk4mnlN3zNl1zX8HSoy6H/+5u62xAv4z09V+XalKv84+Y1v+mkVE5Ii1zx1KLrnkkn1+iGEYzJ07d5+v21dLlizBNE1OOeWUBsdzcnIYMmQIS5YsaXDccRyy2SzZbBbP80ilUhiGQTgcPuC1ioiIiIgcDLfffjsDBgxgwYIFdOnSpa3LERERERE56o0YMYIFCxYwY8YMACZMmMDs2bOJRCK4rstvfvMbvvKVr7RxlSKy38L5MGIK/N9MKF/thzusIJhBCOZCNul3DwnkQEE3Pzyyu0gR1HwCmVr/Y9MEF7+7CDs7ju72p1s33mZnoMTzIOh3P8JO+/e3AuAZ/rl0jX+vwhK/zrBG3oiISPP2OVAyb968RscMw/8k1VTrbM/zDlqgZMuWLRQXFzcZCOnatSuLFi0ik8kQCvkz6x599FG++93v1q+JRCL06NGDdevW7fE5GzduZNOmTQ2Ovf/++y1/ASIiIiIirWz9+vXcddddCpOIiIiIiBwiJk+ezF//+leSySSRSITbb7+dxYsXc9tttwEwePBgfvnLX7ZtkSLScp7jdwRx0n5HEfDDJablh0sihRBp1/g6M4CT3xXKP8JwbTJZl5DnYNSFRgx2/1ncztk34KdOgEAYinr6p6o2+eERO1VXg+c/v+MgP0zS78wD89pFROSIsc+BEtd1G3xcVlbGF7/4RXr06MH111/PoEGDAFi+fDm/+MUv2LhxIy+88ELrVLsXiUSi2e4iOTk59Wt2BkouvvhiLr744n1+zty5c5k+ffp+1ykiIiIicrB069aNdDrd1mWIiIiIiEidk08+mZNPPrn+4w4dOrBs2TL+/e9/Y1kWxx57LKZ5WEyrF5GmrH4JXrkDarZCfifAhGwCnCy4WT9o4nmQjvujZ0K5DS6vSGQor7Q5xg3gEiDkZetDJF59qKQZhgWFPSG3LqgSjkGywu9Kkq7xwyRD/x+M/rHfSUVERGQvWvxV6dSpU+nYsSPPPPMMw4cPJxaLEYvFOPXUU/nrX/9KcXExU6dObY1a9yo3N7fZb5anUqn6NS116aWXsmjRogZvv//971t8XxERERGR1vb973+fxx9/HMdx9r5YREREREQOuNdee42ysrJGx48//ngGDx7Mjh07eO2119qgMhFpsXQNLLoPKjdAQQnEukKsM7TvAx0HQqfjoH1/P/iBB5Xr/OCJnQHPo7ImTm35Joqy29hkduXlyFkkzAKS5GBj4WFgY2EHIv7IHDNQd686ue0h2mHXx2YA8jpAtJPfuaTT8TD6BoVJRETkM9vnDiX/7R//+Ac/+9nPmj3/la98hZ/+9Kctfcxn0qVLFz744APS6XSjTiWbN2+muLi4vjtJS3Tv3p3u3bu3+D4iIiIiIgfa5z//eZ5++mlOOeUUrrzySnr16oVlWY3WjRo1qg2qExERERE5+owZM4ZHH32UCy+8sMnzCxcu5MILL1QoXORw9NFCiG+FnMJGnUfqhXIh2hGySciJ+R1LKtfhOjZe2iPt5vJpsCcv5J/H8el3MXHYGOxFjpeknVNGwLMJ2Bk8EwzDAKvu51522g+01Gz1R+lYQb8rSnIHpCqhsMQfcxOOHqzdEBGRI0CLAyXpdJpNmzY1e37Tpk0HrcX2ySefzIsvvsjixYsZOXJk/fFUKsWyZcv0TXIREREROeqMHTu2/v3LLrvM/2bTbjzPwzAMfbNaREREROQg8Txvj+cdx9HIG5HD1cZSSFb6Y2f2JNIO0uugx2nQ83TYuJhNW7ezeEuG982BrC8cRsrM5eTUIiwcbALEzRgJI48cp5o8L0EsZJITCvljbXIKYPsHEMoD0/Q7n7i236EkUggdB/lhkn5nHvAtEBGRI0uLAyWnn3469913H1/84hcbBTZeffVV7rvvPk4//fSWPuYzueCCC5g1axazZ89uECj5wx/+QCKR4KKLLjoodYiIiIiIHCoefvjhti5BRERERET+y38HvXe3aNEiiouLD2I1ItJqMrV+kMMKNjplux5ViQw1KRvHdemUSVNeVkm7cV8mOvhrzPufD/ifzVvoHosQqguVpY0cHCwC2NgEcQ2LuFVEuR2jOBimZ7u8uptnIBiBgV+uD6iQiUMoCiXDoM8ZGnMjIiL7pcWBkl/96lecfvrpjBkzhpNOOomBAwcC8OGHH/L2228Ti8W4++67W/SMRx99lPXr1wNQVlZGJpNh5syZAPTo0YNvf/vbABx33HFceeWV3H///Zx77rmcffbZrFixgnvvvZfRo0c320JQRERERORINWnSpLYuQURERETkqHfPPfdwzz331H98zTXXcPPNNzdaV1FRQXV1NZdccsnBLE9EWksoz+8K4mQhEKo/XJHI8EllkozjYbsuAS9LDFi8JcNfH3mby0b1Jpm1cRyPoLUrcLYyNIjj0u8QdWuotNoBYBrgeeC4u3U7Su7wO5H0GgmDvgqDv3ZwXq+IiBzxWhwoGTRoEO+++y433XQTf//731myZAkA0WiUCy64gJkzZ9K7d+8WPWPu3Lm8+uqrDY7deuutAIwePbo+UAIwe/ZsevbsyQMPPMDzzz9PcXExU6ZMYcaMGWoTKCIiIiIiIiIiIiIHXWFhIT169ABg3bp1tG/fnmOOOabBGsMw+NznPsfw4cO59tpr26JMkaNTugY+WlDX1aPWD4V0HwZ9x+57V4/uw2DNQj/gkd8J8MMkG3ckSGVdMMAEColTa+Tzb2MAH26tYfaCVXTMz8GyDLKORyjgh0reC3+ecbXP091eR8LLJWPk4HpgGGCZdcGTTAJSlf5Ymz5jm65LRERkP7U4UALQs2dP/vjHP+J5Htu3bwegQ4cOrRbgeOWVVz7zWsuyuO6667juuuta5dl7kslkyGazACSTyQP+PBERERERERERERE5/EyaNKm+e2CvXr248847mTBhQhtXJSKsfgkW3QfxrZCs9MfVmAE/FPL2QzBiCvQ787Pfr+9Y/7rtH0A4hh2IsGFHgmTGwcDvLBIiTdSIs8or4X/TnyMWDbBpR5JU1iU/HKAykaVjLAxAyszl+ejX+XrNY3RwtlFr5LHDixIwgsSCLtRs9cMkhSV+reHogdglERE5irVKoGQnwzAapaqPZLNmzWL69OltXYaIiIiIiIiIiIiIHCbWrl3b1iWICMCal+H1O6FyA+QUQmFPsIL+uJrkDj8U8sod/trdQyVNdDRJdj6ZV+3jWLwlS3fvy5zpldGhfB015OFkwngECBk2RdSSb9SyxevIw+45lDtB4jVp8sMBso5L0DL4NJ4hPydAJGQBsCznZADOiT9NgVNOJ3cbYdMjms6B3EK/M8m+Bl9EREQ+o1YJlNTU1PDrX/+aF198kW3btvHII49w6qmn8umnn/Lb3/6Wb3zjGwwcOLA1HnVIuemmm5g2bRoApaWljB2rVmIiIiIiIiIiIiIiIiKHhKbG2XQdBrFjofT3fpikoARCubuuCYT8cTXhmH9+0X1QMtwff9NER5O0a1L+9t/p6BVRzpd51hvKMmMi3+F/iGXL6UoZQcPBMSyqifIR3XnE+Aql1lDCpkc662JgY7seJ/YoIidksWlHkoJIgMLcEEHLYHHg87we7Mtg+21Oi6zh5C5hYh2KoWQY9Dlj30fziIiIfEYtDpSUlZVx+umn8/HHH9O3b18+/vjj+vEvxcXFzJ8/n8rKSn71q1+1uNhDTSgUIhQKARCJRNq4GhEREREREREREREREQGaH2fz8esQ6go1a/3OJLuHSXYXyvXPx7fCmv+DYK7fsWS3jiYVaY+tFdVE7Gp6so4rA3+hUyzMkuBpXFdxLIPsJZxoriJmpkkZEZYxgEWcQAL/Z0qWYRCwDGzHI4VL+7wQ3zqlhAdf+5htNWk2VSSxXY+AaRCL5LG583hio3rTZUDHg7SJIiJytGtxoOSWW25h69atlJaWUlJSQseODT+JffWrX2XhwoUtfYyIiIiIiIiIiIiIiIjI3q1+qVH4o36cTSoONVv8oEgoCKkqcLP+dVbQD52EYxApgkg7qFwHa1+HT1dB5QbsWHeqsgGqKjNU1GawHZNUoB0ZK0pHZztfrn2Gle0+xxYjl3+6w/iHOwzTgLxwAKOJUgOmSSprYxgGuaEAYwZ05OSe7XhtVRnvrK8gkbHJDQU4qUcRI/t3IBpuleEDIiIin0mLP+v8/e9/Z/LkyZx44omUl5c3Ot+7d2/mzZvX0seIiIiIiIiIiIiIiIiI7Fm6xu9M0tw4GysE6Sx4DiRrAQPw6hYYYFqQrICaTyDW1e9s8ukqiG8lbkRZsz1LxknjuB6e51+ScVyybpCImUehu4Pj0+/ygTcUo+7Wnge24xG0GkdKDANcD4KWwUk9igCIhgOcfVxnzj6u8wHeLBERkT0zW3qDTz/9lL59+zb/ANMklUq19DEiIiIiIiIiIiIiIiIie/bRQr/7SFPjbOJlULEWPHu3gzvDJHU/MvNccB3IJKBinf9+spJ45aesqg6QyDjYTl2YpO5y1wPX9fjUySXXqWFAZjmmYWAafmDEA2zHabJcx/Hw8EMko/p3aK1dEBERaRUt7lDSqVMn1qxZ0+z5pUuXUlJS0tLHiIiIiIiIiIiIiIiIiPhdSD5aABsXQ6YWQnnQfRj0HQsbSyFZ6Y+52V1tGZSvBiMIoSbuaQCYfqDEcyAQgWwSD1hZCR0yGdJeMz9W8/zQSJoAhmcTdpPkRwJUJS2yjoPrge1BxnYJWEZ9TxTb8cg4LiHLZOLQruRpnI2IiBxiWtyh5Oyzz2bu3Ll88sknjc6VlpbyyCOP8NWvfrWljxEREREREREREREROez95je/Ydy4cc2eHz9+PL///e8PYkUih5nVL8ETF8Erd8B/noYP/+7/+cos/3jZSn9MjRXcdU1ih99txHPwkyONR8/guYBX11LEA9fG9TxSWYettR62ZxHEbnzdbgLY2J5FpROmMBIkaBl4HpgGBE0Dw4C07ZLMuqRtFw8wDYM+xXlMHtP8NAAREZG20uJAyU9/+lMCgQBDhw7lxhtvxDAM5s+fz7e+9S1GjRpFly5d+PGPf9watR5yMpkMtbW11NbWkkwm27ocERERERERERERETnEzZs3j379+jV7vn///jz00EMHsSKRw8jql/wgyfYPwHX9LiQdjvX/dF3/+Nb3/TE1Tta/xrWhehM4Gf9jYw8/GvPc+jfPTpP0gqTcADYmVeRRZMT3WF4Rcaq8PN5x+2GZBl0KI1h1HUlikQCdCnJonxeiIBIkN2gRtEwGds5n2pcGqjuJiIgcklocKOnUqRNvvfUWw4YN46GHHsLzPB599FGefPJJxo8fz+uvv067du1ao9ZDzqxZs4hGo0SjUcaOHdvW5YiIiIiIiIiIiIjIIW716tUcd9xxzZ4fPHgwq1evPogViRwm0jWw6D6o3AAFJZDfCQIhv6NIIOR/XFACdsoPjyQ+9a9LVtSFSZrpTNIED/A8Dw8Py3DZ4eWz3SskZtSSQ7rJa3JIEzNq2UYhixgCQDhgkhOwiOYEiOWEqE07ZBwPyzToEMvhhG4FXDd+AGMGdGzx9oiIiBwIrRJ37N69O3/729+orq5m5cqVeJ5H3759j9ggyU433XQT06ZNA/zxPgqViIiIiIiIiIiIiMieZLNZUqlUs+dTqdQez4sctT5aCPGtkFMIodym14RyIa8YarZCbZm/Nl3tdykxzLqxNnvm7fZnmCweNjEjyYPOOVxtPE03s4xqL48KL0qWAEFsiow4MaOWTV4H5jrnkDQibK9OUZW06dUhj++P7oPnwTvrK0hkbHJDAU7qUcTI/h2IqjOJiIgcwlr1s1QsFuPkk09uzVse0kKhEKFQCIBIJNLG1YiIiIiIiIiIiIjIoa5///689NJLTJ06tcnzL774In369DnIVYkcBjaWQrLSH2+zJ7kdILEDrBBUbQA7DZ4HRgCwwXPYFRtpngFYfq8SBrGOJe4A7rG/zmXW83Q0KululBEwHGzPooo8VrrdedA5h1fdIeRnHWKRIAM75XPZqN71HUjOPq5zy/ZARETkIGvxyBvTNOnSpQtLlixp8vzjjz+OZVktfYyIiIiIiIiIiIiIyGHvW9/6Fi+++CK33normUym/ng2m+WnP/0pL774IhdeeGEbVihyiMrU+p1GrOCe11lBMC3ofDx0HOS/7zrgpPgsQRLw8yfubuNxOphVfN96jlfcIVyRncqv7fP4H/dU/tc5if9xT+XX9vlckZ3KK+4QYpEAXxvalevO7M/vv3OSxtmIiMhhrVU6lJSXl/OFL3yB+fPnc95557XGLUVEREREREREREREjjjXXnst//znP7n99tuZM2cOAwcOBODDDz9kx44djBw5kuuuu66NqxQ5BIXywAyAk4VAqPl1TtZfVzwAxtwIf/0+rPynP+7GMPB7jxjNXw+AgYmHi0kWixA251pv8DtnArVE+Kc7jH+6wxpdlRMw+fnXj+eLn1MnEhEROTK0uEMJwF133cXnP/95vvnNbzJr1qzWuKWIiIiIiIiIiIiIyBEnGAzy4osvcuedd9KtWzeWLl3K0qVL6d69O7/4xS9YsGBB/ah1EdlN92EQKYTkjj2vS+7w15XUBT6SFWCYfqeSYC4EI3XBkuZ5BriYpAmSIYgHRI0kI833m73GMuDSkb0UJhERkSNKqwRKiouLWbhwId/61re45ZZb+O53v4tt261xaxERERERERERERGRI0owGOT6669n2bJl1NbWUltby9KlS/nRj35EMLiXcR4iR6u+YyHaCVKVkEk0vSaT8M9HO0GfsfDRQkh8CnkdIJADdtqfZ2OFwbCauIGJawbJECJBGBurvlNJEIeTzJVNPjYcMPn+F/ow7ayBrfVqRUREDgmtMvIG/C+AH330Ufr27cv06dNZt24dzzzzTGvdXkRERERERERERERERI5W4XwYMQVeuQMqN0BOIUTagRX0x9wkd/hhksISf104ChtLIVkJhT0hE4fqTeABbrbpZ1gBjEAEN2OD52EAARxsTBxM8kg1uqRLQQ63fmUQX1JnEhEROQK1WqBkp5/+9Kf07duXyy67jOHDh/PNb36ztR8hIiIiIiIiIiIiInJYeO211wAYNWpUg4/3Zud6EdlNvzP9PxfdB/GtULkOXBvMgD/mpuMgP0yyc12qEjK1ULXB70wSjIAZgXABeO0gngTPrbu5AXiYBoQCJmnbBdfFwiFFiDQhEuSwc1hOwDI4a9Ax3HneCUTDrf7jNhERkUPCAfkMd9FFF9GjRw8mTpzIzJkzD8QjDgmZTIZs1k+xJpPJNq5GRERERERERERERA41X/jCFzAMg2QySSgUqv+4OZ7nYRgGjuMcxCpFDiP9zoSS4f44m42L/c4joSiUDIM+Z0A4n3ja5oNXn2bA8gVEs0m8TBLTMDDwwIqAFwXPhkiR39nE8/Bbl/h/BE0wTQfXs+vG30So8SK8ZwwkFgnSKRbm6jP7c7a6koiIyBGuxYGSSZMm0adPn0bHTz/9dN566y0uvPBCtm/f3tLHHJJmzZrF9OnT27oMERERERERERERETlEPfTQQxiGQTAYbPCxiDQtnrZ5dWUZ76yvIJm1iQQDnNSziFH9O+zqBBLOh8Ff89/+y8srt1P6whN8ufIRDLcGBwPPn3ODiYfn2v7Im0QlGA47O5OA4XcrydaCYWCZAYxAHl6giGi6inhOJ7r0+zJ39u7KyN1rEREROYK1+LPdww8/3Oy5Pn36UFpa2tJHHLJuuukmpk2bBkBpaSljx45t44pERERERERERERE5FBy8cUX7/FjEdnl5ZXbeeyV/9Cj8l/0TX9AjpsiZeaw5IPBPL1oOP/vC59jzICOja6Lp23e+M9a1pU+R8H2t7jYW0IBtWzzCuhgVBMhDR54GDiYuJj+ha5d35gEw4BgFKwQWEEI5mLiEk3XwDG96fuFG7mh30kHbzNEREQOAWZbF3A4C4VC5OXlkZeXRyQSaetyRERERETaxB133MH5559P7969MQyDnj17Nrv24osvxjCMJt+eeuqpRuvT6TQ/+clP6NWrF+FwmD59+jBz5sz60ZMiIiIiIoebGTNm8J///KfZ88uXL2fGjBkHsSKRQ8PLK7fz2vN/5Huf/ITvpP7EF3mTceZivsibTEr5x197/o+8vHJ7o+t++8AcOvzjEsZuncvZ3hsUU0UQm45GFWEyDR/keXh1PUvwPL9BiWFCIAL5HcE0wUlDpsYPlnQcBF+40R+1IyIicpTZ5w4ll1xyCYZh8MADD2BZFpdccslerzEMg7lz5+5XgSIiIiIicmi76aabaNeuHSeeeCKVlZWf6ZpHH3200bFTTjml0bELLriAv/3tb1xyySWceuqp/Otf/+LWW2/lo48+Yt68eS2sXERERETk4Lvtttvo27cvn/vc55o8/5///Ifp06fzk5/85CBXJtJ24mmbJS/+mXOrH6W7UUatmU+Z2QmbAAFsom4N/d0NRKoe5eG/ZHl+wHhiOUFyQia1//kn59U8RhfKqCSXpBciYqRJEySE7Y+5Ab8DiQcmHoa3sy2JB2YIcttDKA/6neWHTDJxCEWhZBj0OcMfsSMiInIU2udAybx58zAMgzlz5mBZ1mf6Jq4CJSIiIiIiR641a9bQu3dvAD73uc8Rj8f3es3/+3//b69r/vGPf/C3v/2NqVOncvfddwNw2WWXUVhYyK9+9Su+973vMWLEiJYVLyIiIiJyiEmlUgQCLZ5WL3JYeXP5WsZV/oWulFEWOIaMkVN/zibIp0YRhpdDF7ZzXuZZrnm/LykzFzMT5zfWX+hslrGJYhJemN5GEgN2jbWBuo8hYwTxCOBgYmMRMkzIKYCC7lC5zg+TfHHWwX75IiIih6x9Hnnjui6O4xAKheo/3tub4zitXriIiIiIiBwadoZJ9oXneVRXV+O6brNr/vjHPwJwzTXXNDi+8+PHHntsn58rIiIiItIWqqur2bBhAxs2bACgvLy8/uPd35YtW8bjjz9O9+7d27hikYOr+j//S6FTTsLKbxAmAbBdj3TWIeGGqPby6EgFw5xlpDIOI3iPjkYlVV4uCTcMgIeBgUeEDAEcjLr7GB7gQZoQaUJkCYFp+WNtrCC4tt+ZREREROop5iwiIiIiIgddQUEBNTU1hEIhRo0axcyZMxk2bFiDNUuWLKFr166NvpnevXt3unTpwpIlSw5mySIiIiIi++3Xv/41M2bMAPyO3tdcc02j4PROnufxi1/84iBWJ3LwxNM2r64s4531FSSzNpFggJN6FnFM5TLyibPD7NxgvQekbRfXA9OASqJ09coYYqzkH+4pnGStooBaNnodACg0asg3Eph4mNi7wiSAgUcQh1TdMQsbzACEY+Bk/fdD0YO1FSIiIocFBUpEREREROSg6dSpE9deey2f//znycvL47333mP27NmMHDmSf/zjH4wbN65+7ZYtWxg0aFCT9+natSubNm3a47M2btzYaM37778PgG3bZLPZFr6aI0M2m9V+tBHtfdvR3rcd7X3b0d63jaNh323bbusSDgtf+MIXAD8sMmPGDCZOnMjxxx/fYI1hGESjUYYPH67RjnJEennldh587WO21aSpSmZxHA/LMnhtdRk/jFdi4ZDxApiGHySxHY+s4+K4Hnj+yJq0FyBgOuQZKVwPIqQIGA5ZL0ChUUM3o4wgNh47x9wYmHj1NVg4WDiYuFi4YIUgUgi1n/p/lgxrsnYREZGj1T4HSnamqPeFYRjceuut+3ydiIiIiIgcWe68884GH3/ta1/jwgsvZMiQIfzgBz9g9erV9ecSiQThcLjJ++Tk5JBIJPb4rLlz5zJ9+vQmz8Xjcaqqqvax+iOTbdvU1tYCEAjodw4OJu1929Hetx3tfdvR3reNo2Hf43GNh/gsRo8ezejRowFYv3493//+9xt16GtNrutyzz338Pvf/55169bRoUMHvvGNbzBjxgzy8vIO6PUXXHABTz75JIMHD+Y///lPa70kOcy9vHI7sxesYtOOJAWRIN2LIgQtg6zjUZnIUuMEsbFw7TRuIEzadvE8j92npHoeBLGxPYtazx+LkyQH27PIIUNno5wwWdIEMfEIk8Wk4ZhVAwiTIYiNa4WhoBPYGUhVQsdB0GfswdsUERGRw8A+/1fMbbfdts8PUaBERERERESa069fP77xjW8wb948Vq1aRf/+/QHIzc0lnU43eU0qlSI3N3eP97300ks566yzGhx7//33ueKKK4hGoxQUFLTOCzjM7fyN6VgsRjAYbONqji7a+7ajvW872vu2o71vG0fDvkejGg+xrx5++OED/oxrr72We++9l4kTJ3LdddexYsUK7r33XpYuXcqCBQswTfOAXP/3v/+dp556ikgkciBelhym4mmbB1/7mE07knQrihAJWfXnQgGDjrEwmziequr3iHo1fJoJ7NZTpKEiI04VebztDgDgbbc/I81/09nYQQgbBwt3txhJiGzd+Bv/jn7nEg8bi3SkI3l2BuLbobAERkyBsP6dJiIisrt9DpSsXbv2QNRxWMpkMvX/UZhMJtu4GhERERGRw1fPnj0B+PTTT+sDJV26dGHz5s1Nrt+8eTNdu3bd4z27d+9O9+7dmzwXCASO2B/q7I+d+6E9Ofi0921He992tPdtR3vfNo70fT9SO68cDI7j8OGHH1JRUYHruo3Ojxo1ar/uu3z5cu677z7OPfdcnn766frjvXr14qqrruKJJ57gwgsvbPXr4/E4kydP5sorr+S5557br9rlCJGugY8WkFn3FlvLyvlwh0On6p6s4QQSmRChgIllGg0uWRk9hYra5+nlrqfazSVF426VOaSJGbWsdLvzmuuPjHrNPZ6LvAV0M8sI4JAiVL/erhtuE8QmjP+zHAf/uZbhYaSrIJzvdyYZMQX6nXmgdkREROSwtc9f7ffo0eNA1HFYmjVrVrMttEVERERE5LPbOermmGOOqT928skn8/jjj7Nx48YGwZCNGzeyZcsWJkyYcNDrFBERERFpDT//+c+58847qa6ubnaN4zj7de8//elPeJ7HNddc0+D45Zdfzg033MBjjz22x0DJ/l5/88034zgOM2fOVKDkKBJP27y6sox31leQzNocl1jC+Kq/EE5tx67dQcxzGOKa9PTyOJcXmV9xDm9Vf57OhTkU5frhD8f12JwI8KB9Nj8wnqKbWUa1l0eFFyVLgCA2RUacmFHLJq8DDzrnkMAfeVNLhAedc/ilMYd2RpwADjZWXScSCNT1LKklhy1ee0zToLNRTTmFVMSOZ9SYL0KfM/xgiYiIiDSi+HgL3HTTTUybNg2A0tJSxo7VbD0RERERkebU1tZiWRY5OTkNji9dupS//OUvHHvssfTp06f++Le+9S0ef/xxZs+ezd13311/fPbs2QBcdNFFB6VuEREREZHWNHfuXG688UZGjx7N+PHjufnmm7n22msJBoPMnTuX3r17M3ny5P2+/5IlSzBNk1NOOaXB8ZycHIYMGcKSJUta/frFixdz//3386c//YlYLLbPNW/cuJFNmzY1OPb+++8DYNt2fafwpmSz2b2uOVod6L15fXUZ8xetY3tNhupUlpPsZXyOZ3C9crZ6uVQbXcEKYhpZ8tw4fYwyfuA9B47J21UnEDQ8DAy2ViVJZV02uMfjmgbfMf6XDmYVnakhYDjYnkU1uazwOvGIdxb/Mo4jbO0aivMvTuBt4zhGmv/GxcTCxcDDw8AmSIIA22lHKlBA2LSp9iyWmwNZ2/07nNpv0M7NOiB7dLjRP0/N0940T3vTNO1L87Q3zTuQe2Pb9n5d12qBkrfffpvS0tIm2/MZhsGtt97aWo86ZIRCIUIhP0GrmZAiIiIicrR69NFHWb9+PQBlZWVkMhlmzpwJ+B0Ov/3tbwN+F5IvfelLfO1rX6Nfv37k5eXx3nvv8dBDD2FZFg888ECD+55zzjl8+ctf5le/+hVVVVWceuqp/Otf/2Lu3Ln8v//3/zj99NMP7gsVEREREWkFc+bMYfjw4bz88suUl5dz8803c84553DGGWdw9dVXM2TIkP3uTgKwZcsWiouLCYcbjwzp2rUrixYtIpPJ1H9vu6XX27bNZZddxvjx4/nGN76xXzXPnTu32W7g8XicqqqqZq+1bZva2lpAI5j+24Hcm6UbKnjq3U3UVKcpDgfoH3W4qHYZMSeXj91OpL0QhgGhgInnOuAkMI0ExxhprvMW8mfH4730QFKEaBf0cIMengfr+By/oD8nGGvob24iTIY0IVa63fm315sUIbrjNarndWMMnSyDXNK4mJi4uJgkCFNLDqZlEbIsTLeKMorZEe3PcR1De/y7dTTSP0/N0940T3vTNO1L87Q3zTuQexOPx/fruhZXkUwmOffcc3nxxRfxPA/DMPA8/5P5zveP1ECJiIiIiIj43/x99dVXGxzb+fX/6NGj6wMlnTp1Yty4cbz88ss8/vjjJJNJOnfuzAUXXMCNN97IwIEDG937L3/5CzNnzuSxxx7j0UcfpWvXrsyYMYMbbrjhwL8wEREREZEDYMWKFfUBbMMwgF3jbTp37sz3vvc97rnnHi655JL9un8ikWgyDALUdwtMJBLNBkr29fq77rqLjz76iGeffXa/6gW49NJLOeussxoce//997niiiuIRqMUFBQ0e+3O3+CNxWIEg8H9ruFIdKD2pjZj89i7q1i1LUvXwly8kEWv5L/omFiJ5zh4rk2wLvNRYNbSiR1YXhYLF9NwKMLgEm8ja71OPOKcRalxAlnXxa27xiTMRwzCY9BnrmkHvTkr8A+OMTezxW1Pygjjz73JYFFD0DLID2Qptj9lpdGLDUW9uODYbsRyc/Z676OJ/nlqnvamedqbpmlfmqe9ad6B3JtoNLpf17U4UDJjxgxefPFFbr75ZsaOHcuYMWOYP38+HTt25I477iCZTPLII4+09DEiIiIiInKIeuWVVz7Tuk6dOvHoo4/u071zcnKYOXNm/TfcRUREREQOd5ZlkZeXB1D/Z3l5ef35nj17snr16v2+f25uLtu3b2/yXCqVql/TGtd/9NFHzJgxg1tuuYXevXvvd83du3ene/fuTZ4LBAJ7/YHKzjX6oVRjB2JvFn34KZurs0TCIQLBILYH/dP/ocD+lA1uBww3g2VAjDjdnDLCRgYbC9uzcIGwkSbqVTOQWq40/oLtuLzsDKm/vwEYBvUBk88iTYSHvDO5OvA0JcZmqt08KohiewFChk2xW0s0XctmOvB/BV9i7Am9iOXm6O9ME/TPU/O0N83T3jRN+9I87U3zDtTe7G/HE7OlD37qqac4//zzmTFjBp/73OcAv+3dWWedxYIFC8hkMsybN6+ljxEREREREREREREROeyVlJSwdu1aAMLhMN27d+f111+vP79kyRLatWu33/fv0qULn376Kel0utG5zZs3U1xc3Gx3kn29/rrrrqNdu3ZMnDiRjz76qP7Ntm0ymQwfffQRn3zyyX6/FjmEpGtg+V/hhRs55tUfc0n8Ab5klpLjJgAIeyksHLJ1v8ds4dKZckJ1I2scI4iHgQsYHqS9IJvcDnQ1yrjUfJ48kvWP8gBvL2ESy/CDJ7t7xR3CPfbXWeWV4GJSYpTRz9hEN8qwPZN1Vk/+2f5iTjvrfIaWFLXe3oiIiBzBWtyhZOPGjUydOhXwk9UAmUzGv3kgwLe+9S3mzJnDHXfc0dJHiYiIiIiIiIiIiIgc1kaNGsXzzz9f/z3z888/n9mzZ5NMJnFdl8cee2y/x90AnHzyybz44ossXryYkSNH1h9PpVIsW7aMUaNGtdr169evZ8uWLQwePLjJe/Xr149zzjmHv//97/v9euQQsPolWHQfxLdCspIByRQlrkEq8W8qU//g+ejXSRs5OFgEsbEJECNOkCwOFq5nYOBhGAaG5+Fh4GCSIky1l0dHo5KR5vv8r3sKO3MkHn5gpKlciQFYpgEueLut8DxYZAxlW3QIJ9lL6Zf+gHwzTSQao6bjSRQMHs/kwb0Imx5VVVUHft9ERESOAC0OlOTn52Pbdv37pmmyZcuW+vMFBQVs3bq1pY8RERERERERERERETnsXX311Zxwwgkkk0kikQjTp09n1apVzJ8/H4Dx48dz55137vf9L7jgAmbNmsXs2bMbBEL+8Ic/kEgkuOiii+qPrVmzhmw2y8CBA/fr+l/+8pdUVlY2qmHy5Mnk5OTwq1/9is6dO+/3a5FDwOqX4JU7oHID5BRCYU8+9TJU1dbS3kvQ3V7H12se4z/hIcTNfIq8WpIUkOclCBguKYK7dRzxCOJgY1Lj+WOTKonS3SjjZHMlL3qnYPDZRt2YhkE4ZBDNCVJRmyHruJimQShgUZ61+FdkJB93GMdlo3ozZkDHBtdms9lW3iQREZEjV4sDJX369GHVqlWA36Fk8ODBPPXUU1xyySV4nsczzzzT7OxDEREREREREREREZGjyYABAxgwYED9x3l5eTz33HNUVVVhWRbRaLRF9z/uuOO48soruf/++zn33HM5++yzWbFiBffeey+jR4/mwgsvrF87duxY1q9fj7fbfJF9uX7cuHFN1vCjH/2IaDTKeeed16LXIm0sXeN3JqncAAUlEPJDIPm5HpWpMGWESFq5dHC20T/9AdVmAZ2dDYTJwcLFwO9GspOJSwCHBGEq8f+eZwkQMBzyjDSeB4ZR9wYETBPH8/A8j1DA9EMjhkFuKIDneVimgeN6dCmKELRMerTLpX00RG4owEk9ihjZvwPRcIt/DCYiInJUa/Fn0nHjxvHQQw8xe/ZsLMviiiuu4Ic//CF9+vTBMAzWrl3LrFmzWqNWEREREREREREREZHDVjwe56qrruJLX/oS559/foNzBQUFrfac2bNn07NnTx544AGef/55iouLmTJlCjNmzMA0zQN+vRwhPlroj7nJKawPkwAUREJss1IkMi4pM0ytkUfMq2Jx6FScTJJuxnaCnl03tsYFDAI4BHBIE+QTtz0u/t+jgGdjexYpM4Jh7Oxk4o+6cT0XPAhZJnmhAN3aRbhidB/w4J31FSQytsIjIiIiB1iLP7vecMMNfPvb365PME+ePJlUKsVjjz2GZVlcfvnlXH/99S0uVERERERERERERETkcBaNRnniiSc47bTTDuhzLMviuuuu47rrrtvjunXr1rXo+n29rxxmNpZCshIKezY4HDANOhdG2FSRIJ112WFE6cJ2ampq+JV9LpdZz9Pb+IQoKXLJ4GJiY5IgzCdueyrJB/xOJEXEqfTyeNfrX9eFxMN1PTzAdv1ntcsL0as4r8H4mrOP0yglERGRg6HFgZJoNNqgPR/A1KlTmTp1aktvLSIiIiIiIiIiIiJyRBk0aJACF3LIqM3W8sbmN1i2fRlJO0kkEGFox6Gc1vU08jK14NpgBRtdV5QbAmDDjgSJrAnYhNwkr7hDWOIOYLz5Nj8KPEk7o5oaL5cKL58aIsSMBD3Y6o/EMTxyyLDc68lr7gk4dQNyDAPwIBK0OL1/MROHdFUHEhERkTaiz74iIiIiIiIiIiIiIgfJ9ddfz+TJk/n2t79N//7927ocOYq9vul15i+fT1myjOpMNY7rYJkWi7Ys4smVTzLJ8xhpBsDJQiDU5D0MIOjZ2FgkyAGglgh/dUdSYedzdeBpuhllxKilk1FOEJsALmbd0JsMAWIkOC20indDJ+G4HmnbIRIM8OMvDWDi0G4Hb0NERESkkVYLlLz00kusXr2a8vLy+vE3OxmGwa233tpajzpkZDIZstksAMlkso2rEREREREREREREZFD3Ycffkj37t057rjj+PKXv0y/fv3Izc1tsOZI/Z66HDpe3/Q6c96bw+b4ZmKhGF2jXQmaQbJulqp0FasrVzPHikA0n5HJHZDfqcH1tuvxSWWStO3SnjhV5PG227Cb/SvuELBhWuDP9Da2EMLGw8DFIEMQG4taI8IxVPADnuTveRH+ZZ7IpookfTrkMX5Qw2eKiIjIwdfiQMmHH37IxIkTWbVqVaMgyU5H6he/s2bNYvr06W1dhoiIiIiIiIiIiIgcJm677bb69//61782ueZI/Z66HBpqs7XMXz6fzfHNdIl2IRKI1J8LWSE65HYgGoqyuWYz8yMmJ8YryAvHILQr+FSVzJBxPCKkiRm1fOh25w33+EbPWuIOoNqLkDEC1Hi5eBg4mNR4uVQZeXhYhL003b0yzqz6C897PenWvh2XjepNnkbciIiItLkWfza+4oor2LhxI7Nnz2bkyJEUFRW1Rl2HhZtuuolp06YBUFpaytixY9u4IhERERERERERERE5lK1du7atS5Cj3Jub36QsWUYsFGsQJtldJBAhGoqxwbH4nyCcs+1j0oF8zLx2xPLySNQmKXTKyaeWjV4HHnTOqR95s7tR5r9pb9TwqVfIdm/Xz48Mwx+X43mQJEw1ebT3Kvh64UpOGDeJMQM6HqiXLyIiIvugxYGSxYsXc8MNNzBlypTWqOewEgqFCIX8uYGRSNNfdImIiIiIiIiIiIiI7NSjR4+2LkGONpk4rHoFNi6GTC1L0+uozpbRNdarfonjelQmstSksrieR9bxSGZNbDPDPKMnJdkQHe0KCtLriFe45GKxw8tlJd15wD6bV9whmAaYgLtbM/uTzFUU1IVOdjIMyAlaeJ5HxnYJWgZWpB0d3W1M7lNOSGESERGRQ0aLAyXt27enuLi4NWoRERERERERERERERGR1rLpbXjvDxDfCMlKcG2SsRycsEWwfBXEulFBlE8qU2QcF8fxcDwP1/PAAyPssN6L8QP7Ak43/s1QYxW5pEiQw9vuAN5wjyOO/wu3rocfKjF2hUoipAgYDllv14+jgpaBh4fjeuSGLboV5VIUCULZJ+Ak2mCTREREpDktDpR885vf5Nlnn+XKK69sjXpERERERERERERERI4Yl1xyyT5fYxgGc+fOPQDVyFFlzcuw7HEo/xBCESjsCVaQiFeORS3ZTAJzx3qqvA4knFwCpkkoYJLKOngeYDgYnonrhEgYufyfeSr/tIc16EBiUje6pu7j3UMlAElysD2LIDZZglC31sAgN2TSuTBCUW4I7AyYAQhFD9r2iIiIyN61OFBy++23c9555zFx4kSuuuoqevbsiWVZjdaVlJS09FEiIiIiIiIiIiIiIoeVefPmNTpmGP5P2z3Pa3Tc8zwFSqTl0jVQ+nuIxyHWHYK7fhw01IiwiAyVAYt2mRTtvO3UBnuCaZDOujh1iRHDSuC5EZxkD2zHH4MDflikroEJ7m4f77QzVALwttufkea/KTLilFFELBIgNxQgPydAQSREoD55sgMihVAy7IBvjYiIiHx2ZktvEAwGGTx4MM899xzjxo2jb9++9OrVq9GbiIiIiIiIiIiIiMjRxnXdBm/btm1jyJAhfPWrX2XRokVUVlZSWVnJm2++yYQJEzjxxBPZtm1bW5cth7uPFkLtNr/jRzDS4NRpROiARTUucSxC2OQ5NSQzDmnb9buNGBkMK4ln52PH+ze6fV0myuf918f4oRLXg9fc4ymjkAIjQY+YwaDOBfRsn0f7vPCuMEkmAalKiHaCPmNbcxdERESkhVrcoeT666/n17/+NUOHDuX000+nqKioNeoSERERERERERERETniTJ06lY4dO/LMM880OH7qqafy17/+lS9+8YtMnTqVRx55pI0qlCPCxlJIVUH7vuBVNTiVh8kkYsx2drA14BFzHYJODVk3zx9zYyUwrCRuth2Z8lHghRtcv3sHEgMIWP7vLmcdl9177pgGEIryj+h5DA79leLMJ1CTgUg7sILgZP3OJKlKKCyBEVMgrJE3IiIih5IWB0oeffRRJk6cyFNPPdUa9YiIiIiIiIiIiIiIHLH+8Y9/8LOf/azZ81/5ylf46U9/ehArkiNSphZcG8wAOA1P2a7HoITJlxNB/pHvkgzYVAQzmGwHz8RzI7jpTmTKR+HUDmjy9p7nh0kACiIBCvNCVCezJDMOtWmb4vwwZw3uxKm92zOy/ziiG46DRfdBfCtUrttVW6QQOg7ywyT9zjyQOyIiIiL7ocWBkkQiwfjx41ujFhERERERERERERGRI1o6nWbTpk3Nnt+0aRPpdPogViRHpFCeH9hw7QaHKxIZPqlMknZcBmYt+sYDbM2L8z/hYyilD7ghnGQP7Nr+4IabuTl4+GNuTMPvUNI+L0T7vBAZ22VTRZIvH9+FW788aNcF/c6EkuH+KJ6NiyET98fxlAyDPmdAOP8AbYSIiIi0RIsDJcOHD2f58uWtUYuIiIiIiIiIiIiIyBHt9NNP57777uOLX/wio0aNanDu1Vdf5b777uP0009vo+rkiNF9GHz8OqRrIOQfqkhk2FSRIJ11wfA7jHShlmNqwyysGk3aPWWfHxM0TfJzdv2oqTKRIRYJclKPosaLw/kw+Gv+m4iIiBwWWhwoufvuuxk/fjyjR4/m3HPPbY2aRERERERERERERESOSL/61a84/fTTGTNmDCeddBIDBw4E4MMPP+Ttt98mFotx9913t3GVctjrOxbefhRq4mAY2JbldybJuliWQTrrEvbSxMxaVrrdec09fp9ub9S9hQImBZEgAMmMQ1XSZmCnfEb179D6r0lEREQOuhYHSq655hry8/M5//zz6dq1K7169cKyrAZrDMNg4cKFLX2UiIiIiIiIiIiIiMhhbdCgQbz77rvcdNNN/P3vf2fJkiUARKNRLrjgAmbOnEnv3r3buEo5HMTTNq+uLOOd9RUkszaRYICTehYxqn8HouF8GHYFvPUolL9Lygvg2jkYmGBn6ECcmFnLZq8DDzrnkCBnn57tATlBk84FOTiuR3k8TVXSplu7CJeN6k1euMU/fhIREZFDQIs/o3/88ccYhkFJSQkAGzZsaHFRIiIiIiIiIiIiIiJHqp49e/LHP/4Rz/PYvn07AB06dMA0zTauTA5FTQVHckIm76yroLw2Q1Uyi+N4WJbBa6vLePyt9Vw2qjen9xkDKRfeS5D45COOcbdieQ42FlXksdLtzkPul3nNO+Ez1WHgB0kM/DBJyLIoi2eoSGSJRYIM7JTPZaN6M2ZAxwO5HSIiInIQtThQsm7dulYoQ0RERERERERERETk6GIYBsccc0xblyGHsJdXbufB1z5mW026PjjieB6prIPjehRHQ3QvihC0DLKOR2Uiy4dba5i9YBXemN4c3+0k6DeMl/78CPb6UiJekgQ5vOsN5GV7MLVEMAwDywDX8/C8Xc8OWgamAa7nv++4EMsJ8LUhXTm2S4zlW6pJZGxyQwFO6lHEyP4diKoziYiIyBGlRZ/Z4/E4EyZM4KKLLuLSSy9trZpERERERERERERERI5ojuPw4YcfUlFRgeu6jc6PGjWqDaqSQ8nLK7cze8EqNu1IUhAJ0r0ogmnAh1tryDouhmFQk7apzdgU5YYIBQw6xsLk5wTYWJ7gVy+uYlTPPKrdIEtrTmS13R/HdYmEAhgmJBwHz/UAD8MwMA3DD5XUPd92/PcClkGnWITOBTkNOpCce2Lb7IuIiIgcPC0KlESjUZYsWcJFF13UWvUcVjKZDNlsFoBkMtnG1YiIiIiIiIiIiIjI4eDnP/85d955J9XV1c2ucRznIFYkh5p42ubB1z5m044k3YoiREIWAOW1GRwXQgETyzBIZ10+qUwRywlimQYAKdshkXVYV16LaSfYngqQsD1s18X1wHFdgpZJOGCSyjp1XUk8DAwwwPD80TamAQW5IU7qUcTEoV3VgUREROQo1OKBjEOGDGHFihWtUcthZ9asWUSjUaLRKGPHjm3rckRERERERERERETkEDd37lxuvPFGhgwZwsyZM/E8j2uuuYZp06bRrl07TjrpJB566KG2LlPa2GurythWk6YgEqwPkwDUJG1s1yVgmlimQcA0yDoulQn/l18rEhk2VSSxHQ/X8wgFLPp0zKVXcZ4fGAHStkvWcQmYBjlBC7MuiOLUjbzZGSY5JpbD3d84gQe+cxJfOq6zwiQiIiJHoRZ/9p8+fToTJ07knHPOYcyYMa1R02HjpptuYtq0aQCUlpYqVCIiIiIiIiIiIiIiezRnzhyGDx/Oyy+/THl5OTfffDPnnHMOZ5xxBldffTVDhgxRd5IjUDxt8+rKMt5ZX0EyaxMJBjipZxGjmun68fa6CqqSWboXRRocd+tCH4afASFmpRjmLGVE7Ro6ZG0+SZqU2v15KzCEpBcCwDAMIiGTTgVhNlemcD1IZV1MAyzDICdokrFdPNfDMgxCAYNYTohpXxxQP95GREREjk4tDpQ89thjlJSUMG7cOE444QT69+9Pbm5ugzWGYTB37tyWPuqQEwqFCIX8L8gikcheVouIiIiIiIiIiIjI0W7FihXMnDkT8L93DrvG23Tu3Jnvfe973HPPPVxyySVtVqO0rpdXbufB1z5mW02aqmQWx/GwLIPXVpfx+FvruWxU70bBjWTWxnE8gpbR4LhpGBgGeB6MNJbybf6HYrOCIqeWQMol45mcGniPMhYwj69QaQytv7ZLYYSKRJZExv/7lrJdDPz7BUyDSNCifV6I6pRNnw55jB/U6cBvjoiIiBzSWhwomTdvXv37y5YtY9myZY3WHKmBEhERERERERERERGRfWFZFnl5eQD1f5aXl9ef79mzJ6tXr26T2qT1vbxyO7MXrGLTjiQFkSDdiyIELYOs41GZyPLh1hpmL1gF0CBUEgkGsOrWhQJ1wSPXAwNcD06x3+YK62m6GmVUkcdmOmJ7AfCyFBlx+rGR71t/43/NGP/hBAAs06CkXS7ry2tJZV0swyA/J0jQMoiELDzXozpl061dhMtG9SZPI25ERESOemZLb+C67l7f1J5PRERERERERERERARKSkpYu3YtAOFwmO7du/P666/Xn1+yZAnt2rVrq/KkFcXTNg++9jGbdiTpVhShYyxMKGBiGAahgEnHWJhuRRE27Ujy4GsfE0/b9dee1LOIgkiQykQWgIpEhpVba6hMZIi4CS4xn6eLUcZGtwPbKcI2QniGQZYgZRSxySums1HOGOdNctxk/X0Lc4OUtMvFMg1yghaWaZBxPGrTDoGAxcBO+Vwzrr9G3YiIiAjQCh1KRERERERERERERETksxk1ahTPP/88d9xxBwDnn38+s2fPJplM4roujz32mMbdHCFeW1XGtpo0BZEgkZDV5JpIyKIgEmBbTZrXV5XxpeM6AzCqfwcef2s9H26twfM8tsfTpLMuAdNgbPB9OhqVVHt5pAiDB47r1o9Q8jzIGGFSpke+V8Pg9HuU5oyof2ZeOEBhbpDjuxXSqziPRMYmNxTgpB5FjOzfgag6k4iIiEidVvuqwPM8li5dyscffwxA7969GTp0aP0XMCIiIiIiIiIiIiIiR7urr76aE044gWQySSQSYfr06axatYr58+cDMH78eO688842rlJaw9vrKqhKZuleFKk/5rj+qJuaVBbXA9PwQyWViQxvr6+oD5REwwEuG9Wbu19cyYef1OB6HjkBC8sy+Ly3ikJq2eB2AGPnfcEwPMA/FA5aJKx8Im6Cfu6KBoGSykSGwtwQ53++W/3zRERERJrSKoGSF154gcmTJ7N+/foGx3v27Mlvf/tbzjrrrNZ4jIiIiIiIiIiIiIjIYW3AgAEMGDCg/uO8vDyee+45qqqqsCyLaDTahtVJa0pmbRzHI2j5qY+KRIZPKlNkHRfb9fA8MAwIJA2yjsdH22saXD9mQEeWrNvBmu212K5L1nXJOBAKpAgYDq4VJGyaeJ4fVDEMME0D1/UwDXAIYOKS46V21ZRxqEraDOyUz6j+HQ7qfoiIiMjhp8WBkjfffJMJEyaQl5fH1VdfzeDBgwFYvnw58+bNY8KECbz88suMGDFiL3cSERERERERERERETk6FRQUtHUJ0soiwQCW5YdFajNZNlUk68fWhAMmBuABWcfD8Tw+2FLNyyu3M2ZAx/p7pDIueWGL/JwwqayL43rg5uK5AWIBD8cwAXA9SNsO0ZBFxnFJZ12CgQwOOSSNHDK2S2UiQ1XSplu7CJeN6k2eRtuIiIjIXrT4q4UZM2bQqVMnSktL6dy5YWu0adOmMWzYMGbMmMELL7zQ0keJiIiIiIiIiIiIiBzWNmzY8JnWlZSUHOBKpLXF0zavrizjnfUVJLM226rTWIbBpzVpKpNZ0lmXcMDEMo36awz8UTUhyySVdXnwtY85uWc7onVhj2TWxnWhY34Yw/Cv+yR5Aomaf5Pv1lBptQP80TmeBwHLpGMsh08qk0ScODVeIW+kerEplSQWCTKwUz6XjerdILQiIiIi0pwWB0pKS0v50Y9+1ChMAtC5c2cuv/xy7r777pY+RkREREREDlF33HEH7777Lu+88w5r166lR48erFu3rtn1paWl3HzzzZSWlmIYBiNGjODOO+9kyJAhjdZu2bKFG264gX/+85/E43EGDx7Mj3/8Y84///wD94JERERERA6gnj171gcD9sRxnINQjbSWl1du58HXPmZbTZqqZBbH8TBNqE07ZGwHwzAImEaDMAmA43nYjkduyKJdXohtNWleX/X/2bvvsKau/w/g74QR9hIUFQVRUcRVxK2IuDduxYHbukXraLWur6Na69Za996zVdyCWmeddU9wT/aSeX5/8COaEhSE5DLer+fh0Zx7cvO+H8Llhpyc8x7NKqS85/L5LCf6uin3vaGogobRB1EsMQgxwgjxMgMk///yOTpyGSyN9GGqk4C4sI+INLKCkX19tDK0gJu9Jeo62SgHqxARERF9TZavGuLj42FqaprudjMzM8THx2f1YYiIiIiIKIf66aefYGVlBVdXV4SFhX2x74ULF+Dh4YGiRYti2rRpAIAlS5agbt26OHfuHCpUqKDsGxISgjp16uDdu3cYNWoU7OzssGXLFnTq1Alr1qxB7969NXlYREREREQaMWnSpDQDShITE/H48WPs378fFSpUQLNmzSRKR9/C//47LDj+AC9CYmFuqIdilobQ+/9BIK/CYvE2IglCCAACukIO2f/PJpKYnIzEJAGFnhxFLAxhpK+DF6GxuPw0VDmgxM3BEqcfvkdYTAIKmikAAB/lRjho0h7tIzfBJuktomXGCBEm0JXpwUwvGYh8A92PYRAFSqGImzd+cqkKPT09CStEREREuVWWB5Q4Oztj27ZtGDJkCHR1VXeXmJiI7du3w9nZOasPQ0REREREOdTjx4/h6OgIAChfvjyioqLS7Tt8+HDo6+vj9OnTKFq0KACgU6dOcHZ2xujRo3H06FFl319++QWBgYH4888/0apVKwBA3759UbNmTfzwww/o2LEjTExMNHhkRERERETZb8qUKelue/LkCWrWrAk3NzftBaIsiYpLxKrTT/AiJBZ2loYw1NdRbtPXlcHB2hhRcUmIiktEUjLwMSERgAwyGaArl8NIP2UwiYWRHoQQSEwWiIlPVO7D3ckGmy88xb03kTA10FXu/7pBVQBAi6jdME8Khm3yWyjkAiZxBoCRBVCwHFB9KGDtqs1yEBERUR4jz+oOBg0ahIsXL6JBgwY4ePAgAgMDERgYiAMHDqBBgwa4ePEiBg8enB1ZiYiIiIgoB0odTPI1jx49wj///IOOHTsqB5MAQNGiRdGxY0ccP34cb968UbZv2bIFJUuWVA4mAQAdHR0MGzYMISEh8PPzy76DICIiIiLKARwdHTFw4EBMnjxZ6iiUQacfvMfbyDiYG+qpDCb5nIlCFzoyQFdHBhMDPVgZ68PaRIFiVoYoY2sKC6OU2UMSkgR05TIY6euq3LefuyPsrAzxIjQW7yI+Ij4xGUIIXNKtgvF647BEdMI5Qw9ElWgKeYUOQP0JQJfNQMn6WqkBERER5V1ZnqGkX79+ePjwIebOnYu///47zfYxY8agb9++WX0YIiIiIiLK5f755x8AQM2aNdNsq1GjBtasWYMrV66gRYsWeP36NV6+fIlu3bqp7Zu6v06dOqX7eM+fP8eLFy9U2m7evAkgZTbFhISEbz6WvCQhIYH1kAhrLx3WXjqsvXRYe2nkh7onJiZ+vRNlStGiRXHnzh2pY1AGXQ4KRXhsAopZGqbbx9RQF2ExciQkCRjoyeFQwFhtv7CYeJgZ6sHN3lKlvX6ZggCAVaef4G1kHF6ExiIxOWXwiZmhMV4Wbgwzd0cU+f9+Snn43ENERETakeUBJQAwe/Zs9O3bF/v370dgYCCAlJHUrVu3hpOTU3Y8BBERERER5XKvXr0CAJXZSVKltr18+TLTfdOzevVqTJ06Ve22qKgohIeHZzB53paYmIjo6GgASLOMKWkWay8d1l46rL10WHtp5Ie6f2m5Q/o2+/btg6Wl5dc7Uo4Qm5CIpCQBPR1Zun0sDPXwRkeGuMRkxCcmq99PfBLCYxNR1tYU7k42abbXL1MQVR2scPrBe1x5GoqY+EQY6evCzd4SdZ1sYKLIm+cYIiIikla2XWE4OTlhzJgx2bU7IiIiIiLKY2JiYgAACoUizTYDAwOVPpnpm56+ffuiSZMmKm03b97EwIEDYWJiAnNz80weQd6U+olpMzMz6OnpSZwmf2HtpcPaS4e1lw5rL438UHcTExOpI+Q606ZNU9seEhKCkydP4tatWxg7dqyWU1F6ouIScep+yiCO2IREGOrpws3BEu7/P4jDUE8XOjoyJCQJ6OuqH1SiI5ehkJkBYoNjEBOXhHcRH2FhpA+9/79fWEw8wmMTYWdliH7ujjBOZ3CIiUIXzSsURvMKhTV5yERERERKHLJKRERERERaYWRkBACIi4tLs+3jx48qfTLTNz3FihVDsWLF1G7T1dXNs2/qfIvUerAm2sfaS4e1lw5rLx3WXhp5ve55deYVTZoyZUq622xtbTF9+nSMGzdOe4EoXf733ymXmQmPTUBSkoCOjgynH77H5gtP0c/dEW4Oljj98D3CYhJQ0CztgPhUyUKgoKkCpga6SBL4z7I1eihra4p+7o7K5W2IiIiIcoJsudo/f/48lixZgocPHyI4OBhCCJXtMpkMjx8/zo6HIiIiIiKiXKpIkSIA1C9Vk9qWupxNZvoSEREREeUmqcvGf04mk8HKyoozvuQg/vffYcHxB3gREgtzQz0UszT8bEaRBNx7E4kFxx9gYL2SKGSqwL03kTA10IWhvk6afX2+nM38zpVx+Wkol60hIiKiXCHLVycbNmxA7969oaenBycnJxQvXjw7chERERERUR5TtWpVACkD0vv166ey7cKFC5DJZKhSpQoAoHDhwihatCguXLiQZj+pbW5ubhpOTERERESU/ezt7aWOQF8RFZeIVaef4EVILOwsDVUGiejrylDQLGWmkRchsdh8/im617TH8lOP/3/wie4Xl7MpaGbAZWuIiIgo15BndQczZsxAmTJl8OTJE/z777/w9/dX+0VERERERPlbqVKl4Obmhp07d+LVq1fK9levXmHnzp3w9PSEra2tsr1r1654/Pgx/vrrL2VbUlISFi9eDAsLCzRv3lyr+TXl0qVLcHFxgampKebMmQMAGDRoEAoUKAALCwsAgImJCa5duyZhSiIiIiKi/OP0g/d4GxkHc0M9tTOOAIChvg7MDXXxNjIOEMDIhk4oa2sKuVyOF6GxePguGi9CYyGXy1HW1hQjGzpxORsiIiLKdbI8Q8nTp0/x66+/Kqekzk/i4+ORkJAAAIiNjZU4DRERERGRNDZu3IinT58CAN6/f4/4+HhMnz4dQMqnL3v06KHsu3DhQtSvXx9169bFsGHDAACLFy9GcnIyfvvtN5X9jh8/Hjt37oS3tzdGjRqFokWLYuvWrfjnn3+watUqmJqaaukINWvChAno0KEDpk6dCgA4e/Ystm7disDAQFhaWgIAoqKiNJ4jLi4Otra2sLGxwb17977YNygoCCVKlMD79+9hbW0NANi0aROGDBmCrVu3onnz5rh+/TpGjhyJa9euwczMDKNGjYKvr6/KftatW4fZs2fj2bNnsLKywqxZs9C9e3eNHSMRERFRTuDp6Znp+8hkMpw4cUIDaUidy0GhCI9NQDFLwy/2szDSx4vQWFx+GoqfW5ZDVQcrnH7wnsvZEBERUZ6R5SsYOzs7xMXFZUeWXGfmzJnKP/oSEREREeVXq1evxqlTp1Tafv75ZwBAvXr1VAaU1KpVCwEBAZg4cSImTpwImUyGWrVqYefOnahUqZLKPgoUKICzZ89i/PjxWLp0KaKiolCuXDls27YNnTt31vyBaUlgYCAGDhyoctvOzk45mERb9uzZg8TERDx69AhnzpxB3bp1M3zfhQsXYtq0afDz80Pt2rURHh6Opk2b4scff8Tx48fx4MEDNG7cGEWKFFF+71avXo05c+Zg8+bNcHV1RXBwMMLCwjR0dEREREQ5x5MnTxAbG4v3798DgHJWutRrIRsbGxgZGUmUjgAgNiERSUkCejqyL/bT05EhMVkgJj4RAGCi0OVyNkRERJSnZHnJm++//x6bN29GUlJSduTJVX766SdERUUhKiqKo8OJiIiIKN8KCAiAEELtV0BAQJr+NWvWxIkTJxAVFYXIyEgcOXIErq6uavddtGhRbNy4ER8+fMDHjx9x9erVXDmYZN68eXBycoKJiQlKlCiBuXPnAgBsbW3x5MkT9OjRAyYmJli4cCH69euHu3fvwsTEBF27dgWQ8onUy5cvAwCmTJmC5s2bY/jw4bCyskLRokWxcuVKlcdLHaBjbm6O7777LkOvV9auXYtu3bqhcePGWL16dYaP7eeff8Yvv/wCf39/1K5dG0DKLCsymQwjRoyArq4uypUrh379+mH58uUAgOTkZEycOBELFy6Em5sb5HI5bGxsULp06Qw/LhEREVFudfLkSRgYGGDEiBF49eoVQkJCEBISglevXmH48OEwNDTEyZMnERgYqPJF2mOopwsdHRkSksQX+yUkCejKZTDS5+wjRERElDdl+SqnSpUq2L17N6pVq4YhQ4agRIkS0NFJu6agu7t7Vh8qx9HX14e+vj4AwNDwy1PfERERERFR/hEVF4XTj0/i6vNLiEmIwZOQp/hp2Vi0r9MZN6/eRNOmTeHi4oI3b97AwcEBc+fORYcOHQAA5ubmmDt3Lm7dupXu/o8dO4auXbti/vz5OHDgADp27IgWLVqgSJEiOHLkCIYPH479+/fDzc0NR44cQdu2bXH79m0UK1ZM7f4CAwNx6tQpzJ49G0FBQejduzcWLVoEMzOzLx7nsGHDcOnSJZw9exaOjo7K9tQBRZ8TQuDGjRsAgPv37+PNmze4ceMGBg0ahLi4ODRo0ADz5s2DjY1NhmpMRERElFv5+vqiVq1amD9/vkq7ra0tFixYgDdv3sDX1xf79u2TJmA+EhWXiFP3U5aoiU1IhKGeLtwcLFG+qBlOP3yPsJgEFDRTpHv/sJh4mBnqwc1eu7MLEhEREWlLlgeUNGjQQPn/fv36QSZTnQJOCAGZTJYvZzAhIiIiIqL8J+Dhcay9tAIJSfHKNrkTcPTVAfjvPore1QagXbt28Pf3R7Nmzb7pMSpWrKhcSqhNmzYwNzfHzZs3UaRIESxatAijR49GtWrVAADNmjVDnTp1sGvXLvj6+qrd3+bNm+Hk5IRq1aqhUqVKUCgU2Lp1q8pSPOocPHgQPj4+KoNJgJRZaBISEvDbb79h2LBhuHfvHlatWoWIiAgAQHBwMADgyJEjuHTpEnR1ddGzZ0/069cP+/fv/6aaEBEREeUWAQEBmD17drrbPTw8MH78eC0myp/877/DqtNP8DYyDuGxCUhKEtDRkeH0w/coYKwPfR053kZ/hKmBLgz1036INjY+CeGxiShrawp3Jw6KJiIiorwpywNK1q5dmx05iIiIiIiIcr2Ah8ex4vySNO1B51/g3qFHiP4Qg61iL5AgQ7du3b75cQoXVl2T3djYGJGRkSmPFRSEyZMnY/r06crtCQkJKFeuHJ49e4Zy5cop2w8dOoSqVati+/btGDp0KABAoVCgS5cuWLVqFQYOHIjNmzerDCyJiopS/v+vv/5Cp06doFAolMv4AICVlRUOHjyIsWPHYubMmbC3t1dZ8sbU1BRAyjKiqTOSTJ48GbVq1UJiYiJ0dTllOBEREeVdMpkMd+/eTXf77du3tZgmf/K//w4Ljj/Ai5BYmBvqoZilIfT+f4mbsJgEPHoXBTNDXZgZ6uFFaCzMDXVhYaT/WZ94hMcmws7KEP3cHWGs4PUrERER5U1Zvsrx8fHJjhxERERERES5WlRcFNZeWpGmPTo4Fhf+uAL3UTVgW84Gcl05Lq26jvjEeDV7ybrixYtj0KBBygEiaXJ+NiAEAPbs2YM3b97gt99+w+LFiwEAsbGxiIiIwL///otu3bqlO/jFxcUFAQEB8PT0REJCAhYuXKjcVqNGDZw+fVp5e8yYMfDw8AAAODk5wcDAIM0Ml/+9TURERJQXNW7cGL///juqVKmCHj16KK+BhBDYsGED/vjjD3h5eUkbMg+LikvEqtNP8CIkFnaWhiqzj+jrylDQTAFTA128CI1FIVMFChU0wYfoeLwIjUVisoCuXAYzQz2UtTVFP3dH1C9TUMKjISIiItIsDpslIiIiIiLKBmcen1RZ5iZVYlwiBAADcwVkOjK8uf0ezy6/RIkCbzSSY9iwYRg8eDCqVasGNzc3xMfH4+LFi7Czs0PJkiXT9F+zZg0aNWqE1atXQ09PT9neqlUrrFq1CosWLfri4zk7O+PUqVOoX78+EhMTsWTJEshkMly9ehUuLi4AgP3792Pt2rUICAgAABgaGsLHxwezZs3Cd999B11dXUyfPh0tW7bk7CRERESU582bNw///PMPevfujfHjx6N06dIAgIcPH+Lt27coVqwY5s2bJ3HKvOv0g/d4GxkHc0M9tUvZAIChvg7MDXURnyzQuWoxKPR0cOVpKGLiE2Gkrws3e0vUdbKBCWcmISIiojyOVztERERERETZ4MrzS2rbzYuYonybMvCfcx4iORmFKxRCsapFEBz9QSM5mjdvjnnz5mHw4MF49OgR9PX1UaVKFSxYsCBN39evX+Pw4cPYvXs3bG1tVQaUjBgxAiNGjMCvv/4KhULxxcd0cnLCqVOnlDOV/PHHH1iyZAn27NmDxMREuLq64sCBAyhfvrzyPvPnz8ewYcPg6OgIfX19NG3aVG1GIiIiorzGzs4O169fx+zZs7F//35cupRyHeno6IhevXph7NixsLCwkDZkHnY5KBThsQkoZmn4xX4WRvp4ERqLW68i8HPLcmheofAX+xMRERHlRRxQkoO0+nF7lvexPBty5ASsxSdZrQXr8AlrkYJ1+IS1+IS1SME6fJJXakGkTTEJMeluq9C2LCq0LavS5mDlCAAICgpSae/Vqxd69eql0iaEUP5/ypQpafb/3320a9cO7dq1+2rmwoULIzY2FuHh4Wm2de/eHd27d1d7PwcHB5VMAFCqVCk8e/ZMeXvNmjVYs2ZNuo9taGiIVatWYdWqVV/NSURERJTXmJubY+bMmZg5c6bUUfKd2IREJCUJ6Ol8eblFPR0ZEpMFYuITtZSMiIiIKOeRSx2AiIiIiIgoLzDSM8pcf31jDSUhIiIiIqL0GOrpQkdHhoQk8cV+CUkCunIZjPT5uVwiIiLKvzighIiIiIiIKBtUKVZNo/2JiIiIiCjr3BwsYW6oh7CYhC/2C4uJh5mhHtzsLbWUjIiIiCjn4YASIiIiIiKibFC3pCf0dPQz1FdfRx/uJetrOBEREREREf2Xu5MNCpkqEB6bgNj4JLV9YuOTEB6biEKmCrg72Wg5IREREVHOwQElRERERERE2cBEYYLe1QZkqG+vagNgrG+i4URERERERPRfJgpd9HN3hJ2VIV6ExuJdxEfEJyZDCIH4xGS8i/iIF6GxsLMyRD93RxgruOQNERER5V+8EiIiIiIiIsomHqUbAgDWXlqBhKT4NNv1dfTRq9oAZT8iIiIiItK++mUKAgBWnX6Ct5FxeBEai8RkAV25DGaGeihra4p+7o7KfkRERET5FQeUEBERERERZSOP0g3hVrwGzjw+iSsv/kFMfDSM9I1RpVg11HWsDxMFZyYhIiIiIpJa/TIFUdXBCqcfvMeVp6GIiU+Ekb4u3OwtUdfJBiacmYSIiIiIA0qIiIiIiIiym4nCBM3KtUazcq2ljkJEREREROkwUeiieYXCaF6hsNRRiIiIiHIkudQBiIiIiIiIiIiIiIiIiIiIiChn4YASIiIiIiIiIiIiIiIiIiIiIlLBASVEREREREREREREREREREREpIIDSoiIiIiIiIiIiIiI8pDk5GTMnz8fZcuWhYGBAYoVK4bRo0cjOjo6W+8fGhqKhQsXonHjxihWrBgMDQ1RpkwZDBgwAM+fP9fEoRERERGRFnFACRERERERERERERFRHuLr64tRo0ahXLlyWLx4MTp27IhFixahVatWSE5Ozrb7X7x4EaNHj4ZMJsPQoUOxZMkSNG/eHJs2bUKFChVw584dTR4mEREREWmYrtQBiIiIiIiIiIiIiIgoe9y+fRuLFy9Gu3btsHv3bmV7iRIlMHz4cGzbtg3e3t7Zcv+yZcvi/v37KFmypMo+WrRogUaNGmHSpEnYtWtXNh8hEREREWkLZyghIiIiIiIiIiIiIsojtm7dCiEERo4cqdLev39/GBkZYdOmTdl2fwcHhzSDSQCgYcOGsLKywq1bt775OIiIiIhIehxQQkRERERERERERESUR/zzzz+Qy+WoVq2aSruBgQEqV66Mf/75R6P3B4Dw8HBERkaiUKFCmT8AIiIiIsoxuOQNEREREREREREREVEe8erVK1hbW0OhUKTZVrRoUZw7dw7x8fHQ19fXyP0BYMaMGUhISICPj0+GMj9//hwvXrxQabt58yYAIDExEQkJCeneNyEh4at98ivWRj3WJX2sTfpYm/SxNuqxLuljbdKnydokJiZ+0/04oISIiIiIiIiIiIiIKI+IiYlROxgESJllJLVPegNCsnr/Xbt2Ye7cuWjatCl69+6docyrV6/G1KlT1W6LiopCeHh4uvdNTExEdHQ0AEBXl295fI61UY91SR9rkz7WJn2sjXqsS/pYm/RpsjZRUVHfdD9+h4iIiIiIiIiIiIiI8ggjIyO8e/dO7baPHz8q+2ji/n5+fujWrRuqVKmC7du3QyaTZShz37590aRJE5W2mzdvYuDAgTAxMYG5uXm69039BK+ZmRn09PQy9Hj5BWujHuuSPtYmfaxN+lgb9ViX9LE26dNkbUxMTL7pfhxQkk1SRwqlTsP3LUJfPshyjisfPmR5H8/On8/yPrIqJ9QiJ9QByHot+Jz4hLVIwTp8wlp8wnNmCj4nPpG6FqnXVKnXWETZJTuu2/OaxMREREVFwcTEhJ8K0TLWXjqsvXRYe+mw9tLID3XntXvOU6RIEdy5cwdxcXFpZhp5+fIlrK2tv7hczbfe//Dhw2jXrh1cXFxw9OhRmJmZZThzsWLFUKxYMZW21OfU3bt3v/jzkx9+zr4Va6Me65I+1iZ9rE36WBv1WJf0sTbp02Rt7t69CyDz1+38DmWTJ0+eAAAGDhwoaY422bGTE7WyYy+Sy3ItWIdPWItP8kAtWIdPWItPeM5MwefEJzmlFqnXWETZJadctxMRERHlNbx2zzmqVq2Ko0eP4tKlS6hbt66y/ePHj7h+/Trc3d2z/f6HDx+Gl5cXypYti+PHj8PS0jLLx8FrdyIiIqLsl9nrdpkQQmgoS77y6tUrHDhwAI6OjjA2NpYkQ+oUgH/88QcqVKggSYacgrVIwTp8wlqkYB0+YS0+YS1SsA6f5IRaREdH48mTJ2jZsiWKFCkiSQbKm3LCdXtOkxN+5vMr1l46rL10WHvpsPbSyA9157V7znPz5k1UqlQJbdu2xe7du5XtixcvxvDhw7Fx40Z0794dAPD48WMkJCSgbNmy33R/ADh69CjatGkDJycnnDx5EgUKFMiW48jotXt++Dn7VqyNeqxL+lib9LE26WNt1GNd0sfapE+TtfnW63bOUJJNihQpggEDBkgdAwBQoUIF1KxZU+oYOQJrkYJ1+IS1SME6fMJafMJapGAdPmEtKC/KSdftOQ1/5qXD2kuHtZcOay8d1l4arDtpU4UKFTBkyBAsWbIE7dq1Q/PmzXH37l0sWrQI9erVg7e3t7JvgwYN8PTpU3z+udPM3P/y5cto06YNhBDo3bs3Dh06lCbP54NPMiOz1+78OUsfa6Me65I+1iZ9rE36WBv1WJf0sTbpy0m14YASIiIiIiIiIiIiIqI8ZMGCBXBwcMCKFStw8OBBWFtbY9iwYZg2bRrkcnm23f/WrVv4+PEjAMDX11ftvr51QAkRERERSY8DSoiIiIiIiIiIiIiI8hAdHR2MHj0ao0eP/mK/oKCgLN2/V69e6NWr1zemJCIiIqKc7utDkYmIiIiIiIiIiIiIiIiIiIgoX+GAkjzEzs4OkydPhp2dndRRJMdapGAdPmEtUrAOn7AWn7AWKViHT1gLovyFP/PSYe2lw9pLh7WXDmsvDdadSPP4c5Y+1kY91iV9rE36WJv0sTbqsS7pY23SlxNrIxNCCKlDEBEREREREREREREREREREVHOwRlKiIiIiIiIiIiIiIiIiIiIiEgFB5QQERERERERERERERERERERkQoOKCEiIiIiIiIiIiIiIiIiIiIiFRxQQkREREREREREREREREREREQqOKCEiIiIiIiIiIiIiIiIiIiIiFRwQAkRERERERERERERERERERERqeCAEiIiIiIionxIJpOp/TIxMZE6Wp4xa9YsdOzYEY6OjpDJZHBwcPhi/4sXL6Jhw4YwNTWFmZkZmjZtiuvXr2sla16Tmdr36tUr3Z+HXbt2aS90HvDgwQNMmjQJNWrUgI2NDUxNTVG5cmXMmDED0dHRafrfv38fXl5esLS0hLGxMerWrYuTJ09KkDz3y0ztp0yZku5zfu7cuRIdQe50//59dOvWDc7OzjA3N4eRkRHKli2LUaNG4fXr12r78zlPlDGZvY5KT0xMDKZNmwYXFxcYGhrCysoKNWvWxN69e7M3sJZkR10SEhKwbNkyVKlSBRYWFrCwsICrqysWLlyI+Pj47A+tBZm9BkmPn58fatWqBWNjY1hZWaFjx44IDAzUYHLNy2ptQkNDsXDhQjRu3BjFihWDoaEhypQpgwEDBuD58+daOALNya7nzec6d+4MmUyG8uXLZ3Na7cqu2iQmJmLRokVwdXWFsbExzM3N4erqij/++EOD6TUnO+oihMCWLVtQq1YtWFtbw9TUFC4uLpg2bRoiIiI0fASak9nr4vTkxfNwVmuTE87DMiGE0MojERERERERUY4hk8lQt25dDBgwQKVdT08PnTt3lihV3iKTyWBlZQVXV1dcuXIFZmZmCAoKUtv3woUL8PDwQNGiRTF06FAAwJIlS/Du3TucO3cOFSpU0GLy3C8zte/VqxfWr1+PjRs3ptnm7u6O4sWLazht3jF+/HgsXboUrVu3Ro0aNaCnpwd/f3/s2LEDFStWxIULF2BoaAgAePz4MapVqwZdXV2MHDkS5ubmWLlyJW7duoVDhw6hYcOGEh9N7pKZ2k+ZMgVTp07F/PnzYW1trbKfKlWqwNnZWYpDyJVOnDiBGTNmoEaNGrCzs4Ouri5u3ryJtWvXwszMDNevX0fBggUB8DlPlFmZ+V2entDQUDRo0AAPHz5E7969UalSJURHR+Pu3bsoU6YMRo4cqZHsmpQddenWrRu2bNmC9u3bo0GDBkhKSsJff/2Fo0ePolOnTti+fbtmwmtQZn4PpmfPnj3o0KEDKlWqhP79+yM8PBwLFiyAjo4OLl++jCJFimjpaLJXVmtz+PBhtGzZEg0aNICnpyesra1x69Yt/PHHH9DX18e5c+dQrlw5LR5R9smO583nDhw4gDZt2kChUMDR0RG3bt3SYHrNyo7axMfHo3Xr1vD390e3bt1Qo0YNJCYm4uHDhzA0NMTMmTO1dDTZJzvqMmHCBMycOROenp7w8vKCnp4eAgICsH37dlSvXh3nz5+HTCbT0hFln8xcF6cnr56Hs1qbHHEeFpQnhIWFicOHD4utW7eKEydOiJiYGKkjadS7d+/Eo0ePVNru3LkjunTpImxtbYVCoRDFihUT33//vXj9+rVEKbXj7du3YsyYMaJZs2aid+/eIiAgQG2/ffv2iRIlSmg5Xc5y7949cf78efH48WOpo5CWRUdHi507d4qxY8eKbt26ibZt24pu3bqJsWPHih07doioqCipI2rF9evXha+vr/j555/FixcvhBBCREREiJ9//lm4u7uLWrVqiZEjR4rnz59LnFSzeN5UFRYWJi5evCiCgoLS7fPkyROxfv16LaYiIm0AIHx8fKSOkad9ft3p4uIi7O3t0+1btWpVYWpqqvwdLYQQL168EKampqJRo0aajJknZab2Pj4+gn8eyR7//POPCAsLS9M+YcIEAUAsXrxY2daxY0chl8vFtWvXlG2RkZGiePHiwsnJSSQnJ2sjcp6RmdpPnjxZABCBgYFaTJi/7NixQwAQs2fPVrbxOU+UOZn5XZ6e7t27C1NTU3H79u1sTCatrNbl5cuXAoDw8vJSaU9OThZ16tQRMplMhISEZEdUrcrM70F14uPjRZEiRUTx4sVFZGSksv3atWtCLpeL/v37Z3tmbclqbQIDA9O8FyOEEMeOHRMARPv27bMtq7ZltTafi4yMFMWKFRPDhg0T9vb2wsXFJTujal121GbixIlCR0dHnDx5UhMRJZHVuiQkJAgjIyPh6uoqkpKSVLZ169ZNAFC5VswL1F0Xq5OXz8PpyWhtcsJ5mH8xyWX27dsnFi5cqNI2ZcoUYWRkJORyufLLyspKrFu3TqKUmte2bVuVi95//vlHmJiYCJlMJuzt7UWNGjWEnZ2dkMlkws7OTuWPsnlJcHCwKF68uJDJZMovuVwuBg0aJBITE1X6btq0ScjlcomSas/ff/8ttm/frtK2bt06UaRIEZWfEWdnZ3H8+HGJUuYsMTEx4unTp1LH0Jh169YJa2trIZfLVX5WPv+ysrISa9eulTqqRt24cUMYGhoqj9nOzk68fv1aVK1aVchkMmFmZiaMjY2FTCYThQoV+uLggtyM501VM2fOFAqFQnlu9PDwUHtxmh9qQZQfpQ4oiYuLU3mxTprxpT/4P3z4UAAQffr0SbOtT58+QiaT5fmB8pqU0QElycnJIjw8PM0f9ijr/v33XwFADBw4UAghRFRUlFAoFMLT0zNN32nTpgkA4uLFi9qOmSf9t/ZCqA4oCQ8PFwkJCRImzJsuXrwoAIjx48cLIficJ8qqbxk4ERgYKORyuRg1apQQQojExMQ8d837LXW5d++eACAGDx6cZlvHjh2Fjo5Onvqwqrrfg+qkvik3bdq0NNs8PT2FmZmZiI+P11RMSWS0Nl9iZWUlypQpk42pcoZvqc3w4cNFkSJFRHh4eJ4YUJKejNYmKipKmJqainbt2gkhUgatRUREaCOiJDJal5iYGCGTyUTz5s3TbBszZowAIB48eKCpmJL473VxevLjeTijtfkSbZ2H5Rqd/oSy3ezZs3H//n3l7QULFmDq1KkoWrQofv75Zyxbtgw//vgjjI2N0adPH/j5+UmYVnP++ecf1KxZU3l71KhRyqmlgoKCcP78eTx//hz79+9HSEgIJk6cKGFazfnll1/w+vVrrFy5EhEREbh9+zbat2+P5cuXo127dkhISJA6otb9/PPPOHLkiPL25s2b0bt3b8TGxqJHjx4YO3YsunbtiufPn6NFixa4cOGChGk169SpU6hWrRqMjIxQokQJTJ06Ve06qHv27EGJEiUkSKh5u3btQu/eveHo6Ij169fjwYMHiI6ORlJSEqKjo/HgwQOsX78eJUuWRN++fbFr1y6pI2vML7/8AiMjIxw7dgy3bt1C8eLF0bZtWzx48ACHDh1CeHg4oqKisHXrVoSGhmLKlClSR9YInjc/OXLkCCZMmAAHBwf4+vqiQ4cOOHv2LKpUqYJTp05JHY+ItGTXrl0wMjKCqakpChYsiGHDhiE8PFzqWPnOP//8AwAqr3FS1ahRA0IIXLlyRdux8h1zc3OYm5vD0NAQjRo1wsWLF6WOlGe8ePECAFCoUCEAwL///ou4uLh0n/PAp58Lypr/1v5zFStWhLm5OQwMDFCrVi0cOnRI2/HyjI8fP+LDhw948eIFjh49ioEDBwIAmjdvDoDPeSIpHD58GMnJyShXrhx69OihvOa1s7PD/PnzpY4nmZIlS6JkyZJYs2YNVq1ahaCgIDx+/Bjz5s3Dnj178OOPP2ZqiY+c7ku/Bz/3tevxiIgIPHjwIPsDSiijtUlPeHg4IiMjv/n+OVlma3Pp0iUsWbIE8+fPh5mZmSajSS6jtTlz5gwiIyNRpUoVjBgxAmZmZjAzM4ONjQ1++uknJCYmaiOu1mS0LoaGhnB3d8fhw4cxe/ZsPHr0CEFBQVi3bh2WLVuG7t27o3Tp0tqIrDFfuy5OT344D39rbdKj1fOwxoesULaysrJSmTLJzs5OuLu7p/k0SVRUlChfvryoU6eOtiNqhUKhUM4mkJCQIHR0dMS8efPU9h07dqwoVKiQFtNpj4uLixgwYECa9rlz5wq5XC6aN28u4uLihBD559PlNjY2YsGCBcrbTk5OokKFCiI0NFSl36tXr4S9vb1o2rSplhNqx927d4VCoRAKhUJUqVJFFClSRMhkMuHq6ipevXql0jcvPzfc3NxE/fr1v/op06SkJOHh4SGqVKmipWTaZ29vLyZMmKC8ffr0aSGTydSO+B00aJAoWrSoNuNpDc+bnzRs2FCUK1dOxMbGKtuuX78uHB0dhZGRkTh27JiyPa/Xgii/qlatmvj111/F3r17xfr160Xnzp0FAFGhQoU89+nNnOBLnyCdO3euACD8/PzSbDt48KAAIP744w8NJ8y7vvbp3XHjxglfX1+xadMmsXfvXjFlyhRhYWEh9PT0VH4f0rdJTEwUNWvWFLq6uuLevXtCCCF27dolAIhly5al6X/79m0BQPz444/ajprnqKu9EELMnz9fDBgwQKxbt07s379fzJkzR/maMa/P3KgpixcvFgCUXw4ODmLTpk3K7XzOE2XNt8zEMXLkSAFA2NjYCGdnZ7FmzRqxadMmUadOHQFATJo0STNhtehblwK6ffu2cHNzUzlv6enpqT1H5Wbp/R5UZ+jQoQKAuHPnTpptS5cuFQDEkSNHNBVV6zJTm/SkzqawevXqbE4nrczWJiEhQVSoUEHlfYa8OkNJZmqzYMEC5Tm4aNGiYtmyZWL79u2idevWAoDo2bOnllJrXmafMy9evBANGzZUOQfLZDIxceLEPLEE4teui9OTH87D31qb9GjzPKyr+SErlJ1iY2NhbGwMAIiOjsbLly8xZ84c6OqqfitTZyj5+eefpYipcVZWVnj79i0AICEhAcnJyXBwcFDb18HBIc9+yjIoKAi+vr5p2kePHg1zc3MMHDgQbdq0wb59+7QfTiKRkZEwNzcHkPLz8vDhQ6xduxYWFhYq/QoXLozvv/8es2bNkiCl5k2bNg3Gxsb4+++/4ezsDCEEfv/9d4wePRru7u7w9/eHnZ2d1DE17vbt21i0aBHk8i9PyCWXy+Ht7Y0RI0ZoKZn2vX37FiVLllTeTv1/5cqV0/T97rvvsHr1am1F0yqeNz+5c+cOfH19YWBgoGyrVKkSLl68iIYNG6J169bYt28fGjduLGFKItKk/86+0LNnT1SsWBETJkzAwoULMWHCBImS5T8xMTEAAIVCkWZb6nk6tQ9lv19++UXltpeXF7y9vVG5cmUMGjQIDx8+lChZ3jBy5EicP38eM2fORJkyZQDwOa8t6mqf2v5fffr0Qfny5ZUz15mYmGgxae7n5eWFsmXLIioqCteuXcOff/6JDx8+KLfzOU+kfZGRkQCA+Ph4nDlzBgUKFAAAdOrUCeXKlcOcOXMwcuRIWFpaShlTEoaGhihdujSqVq0KT09PxMTEYP369Rg6dCiMjY3Rs2dPqSNmi/R+D6qT387TmamNOrt27cLcuXPRtGlT9O7dWwMJpZPZ2vz666949OhRvvhbYmZqk3oODgkJwe3bt5X9O3XqhPr162PDhg0YP348nJ2dNZ5b0zL7nFEoFChRogR69uyJZs2aAQB2796N6dOnw8DAINf/LeZr18XpyQ/n4W+tjTraPg9zyZtcxtHRETdu3ACQcuGnr6+f7vT88fHx0NHR0WY8rfHw8MDmzZuRnJwMQ0NDVKxYEQcOHFDb98CBA+kONsntzMzMlL+Y/6tfv35YuXIljh49itatWyM2NlbL6aRhZ2ennPZKT08POjo6an8BASm/mJKTk7UZT2suXryIwYMHKy/IZDIZBg8ejBMnTiA4OBj16tXDs2fPJE6peSYmJnj58mWG+r58+TJP/9HU1tZWOfUe8GkavlevXqXp++LFCxgZGWktmzbxvPlJeHg4rKys0rRbW1vD398fZcuWhZeXFw4fPixBOiKSypgxY6Cvr4+DBw9KHSVfSf29GxcXl2bbx48fVfqQdpQuXRqdOnXCo0ePcv20ulL6+eefsWTJEgwYMAA//vijsp3Pec1Lr/bpKVCgAL7//nuEhYXh3LlzWkiYt9jZ2aFhw4bw8vLC1KlTsX79eowdO1b5IRY+54m0L3XZlpYtWyoHkwApfy/09vbGx48f8/RS2Ol58+YNqlatiuLFi2PZsmXo0KEDevbsiWPHjqFatWoYOnQoQkNDpY6ZZZn9PZifztOZrc1/+fn5oVu3bqhSpQq2b98OmUymgZTSyGxtHj16hGnTpmHChAlwdHTUQkLpZLY2qefgGjVqpBlkkTpoLSAgINtzaltm6xITE4NatWohIiIC69evR5cuXdClSxfs3LkTnTt3xqRJk3D//n0tJNecr10Xpyc/nIe/tTb/JcV5mANKcplOnTph7dq1uH//PuRyOTp27Ig5c+YgLCxMpd/r16+xbNkyVK1aVZqgGjZx4kQ8evQIrVu3xp07d7BkyRJs374dPj4+OHnyJO7cuYOjR4+iXbt2OHz4MPr06SN1ZI1wcnLC33//ne72Pn36YOXKlThx4gTGjRunxWTSadOmDVavXo13795BV1cXTZs2xdKlS9OsyRcTE4M1a9agQoUKEiXVrNevX6u9iK1VqxaOHTuG0NBQeHh4ICgoSPvhtKhp06b47bff4Ofn98V+Bw8exPz585UjgvOimjVrYsWKFbh16xaCg4MxefJkmJiYYP/+/Xj//r2y39OnT7F8+XJ89913EqbVHJ43PylWrFi6L1AsLS1x4sQJlC1bFm3btsWhQ4e0nI6IpKKnp4ciRYp88yck6NsUKVIEANQOhE1tK1q0qFYzEZQfTODPw7eZMmUKpk+fjt69e2P58uUq2/ic16wv1f5L+JzPPhUrVsR3332HZcuWAeBznkgKqTPz2traptlWuHBhAMgTAycya8WKFQgODkbHjh1V2uVyOTp06IDIyEhcvXpVonTZ41t+D+aX8/S3XiOkOnz4MNq1awcXFxccPXoUZmZmGkgpjW+pzejRo2FlZYW2bdvi0aNHyq/ExETEx8fj0aNHeP36tYaTa9631CY/nIO/pS67du3Cw4cP05yDAaBjx45ITk7+4t+uc6P/XhenJ7+chz+X0dp8TqrzMJe8yWXGjBmDffv2oXr16hgwYAA8PT1x9OhRODo6omXLlspPoP/111+Ii4vDpk2bpI6sEeXKlcPevXvRrVs3VKhQAWZmZpDJZNi0aZPKMQsh0LVrV4wePVrCtJrTpEkTTJs2De/evUPBggXV9unTpw/kcjn69eun5XTSmDBhAnbv3o2qVati3Lhx6N+/P/r37w9nZ2d07dpV+TOyadMmvHz5Ert375Y6skYUKlRI7cwTAFClShUcP34cjRo1Qv369fPsgCsAmD17Nv7++2+0atUKJUuWRI0aNVC0aFEoFArExcXh5cuXuHDhAh4/fozixYunme48L5k4cSKqVq2KSpUqKdt+/vln2NraokyZMqhbty6SkpJw+vRpREdHY9iwYRKm1RyeNz+pWbMm9u/fj9mzZ6vdnjqopGHDhtiyZUue+sQJEaXv48ePePHiBWrUqCF1lHwl9YMA58+fT/P758KFC5DJZKhSpYoU0fK11KVuChUqJHGS3GfKlCmYOnUqfHx8sGrVqjTXERUqVIBCocD58+fT3Df10+Jubm5ayZrXfK32X8LnfPaKjY1FSEgIAD7niaRQrVo1AFCZrTVValt6fxfIy1LflEtKSkqzLfUDef/9YF5u8q2/Bz+/Hm/YsKHKtgsXLsDMzAxOTk7ZnlebsnKNAKS8iZm6XMPx48fz1HJR31qbp0+f4tWrV3BxcVG7vXTp0mjRokW6s+vnBt9am7x+Dv7WuuT1c3B6Pr8uTk9+OA+rk5HapJL0PCwo1wkNDRXdu3cXOjo6Qi6XC5lMlubLwcFB+Pn5SR1V4969eyemTp0qatSoISwtLYWurq4wNTUVzs7OwsfHRxw/flzqiBr18uVLsXz5cnHnzp2v9t2/f7+YMmWKFlJJ7+nTp6JOnTpCJpOp/IzI5XLlbTMzM7FixQqpo2pMmzZthLu7+xf7XLlyRRQoUEB5LsmrQkJCxJgxY4SdnZ3a86WdnZ0YM2aMCA4Oljqqxt24cUMMGDBAdO3aVaxbt07ZPmHCBGFgYCBkMpmwtrYWS5YskTClZvG8+cn+/fuFra2t8Pf3/2K/0NBQUaVKlTx9niDKjz58+KC2/YcffhAAxOzZs7WcKO9zcXER9vb26W53c3MTpqam4uXLl8q2ly9fClNTU9GgQQMtJMy7vlT7qKgoERsbm6b96tWrQl9fXzg7O2s4Xd4zdepUAUD06NFDJCUlpduvQ4cOQi6Xi+vXryvbIiMjRfHixUXp0qVFcnKyNuLmKRmpfUJCgggLC0vT/uzZM2FlZSUKFCggYmJiNB01z3j9+rXa9pMnTwq5XC48PT2VbXzOE327r11HPX36VNy9e1fEx8cr2xITE4W9vb0wMjISL168ULZHRUWJYsWKCQsLCxEVFaXJ2Br3LXVZsGCBACAGDRqk0jc+Pl5UrFhR6Orqirdv32oqskZl9Brk1atX4u7duyI6OlrZFh8fLwoXLiyKFy8uIiMjle3Xr18Xcrlc9O3bV6PZNS0rtRFCiCNHjggDAwNRsWLFdF9L5lZZqc2xY8fEzp0703zZ2NiIYsWKiZ07d4q///5bG4ehEVl93tSuXVvIZDJx5coVZVtiYqKoVq2a0NXVFU+fPtVYdk3KSl327dsnAIjmzZun6d+sWTMBQKVeuUlmrovz23k4q7URQvrzsEwIIbQ3fIWy09OnT3H48GHcv38fkZGRMDQ0hJ2dHapVqwZ3d3fI5VzRiPK306dP49ChQ2p/Rry8vGBhYSF1RI1ZuXIlBg4ciOvXr6NixYrp9rt27RoaNWqE0NBQtaNi85o3b97g1atXiImJgZGREYoUKaJ22r38KDExER8+fEChQoU4EwURUT7g6+uLCxcuoH79+ihevDiioqLg5+cHf39/VK9eHf7+/so1j+nbbdy4EU+fPgUALF68GPHx8crZE+3t7dGjRw9l33PnzqF+/fqws7NTzhS2ePFivH37FmfPnlWZZYy+LqO1v379Opo1awYvLy+ULl0axsbGuHHjBtasWQO5XI6jR4+iTp06kh1HbrN06VIMHToUxYsXx//+9780f5coVKgQGjVqBCBlzflq1apBT08Pvr6+MDMzw8qVK3Hz5k0cPHgQTZo0keIQcq2M1j4sLAwlSpSAl5cXnJ2dYWlpifv372PVqlWIiorC1q1b1U7BTeq1bdsWr1+/hqenJ+zt7fHx40dcuXIF27Ztg5GREQICAlC5cmUAfM4TZVZmrqM8PDxw6tQpBAYGKpfvAgA/Pz+0bt0aRYoUweDBg6Gvr4+1a9fi1q1bWL16da6ctTerdYmIiEDlypURGBiIVq1aoUmTJoiJicGmTZvw77//YsyYMZgzZ47WjyurMnMN0qtXL6xfvx7+/v7w8PBQ9tm5cyc6d+6MSpUqoX///oiIiMD8+fMhk8lw5cqVXLvUQlZrc/nyZdStWxdCCPzyyy+wtrZO8xjdu3fX+HFoQnY8b9RxcHCAiYkJbt26panoGpcdtbl27Rrq1q0LfX19DB8+HAUKFMD27dtx9uxZTJo0CVOnTtXmIWWLrNYlKSkJtWrVwqVLl1C3bl20a9cOALBnzx6cOXMGHTt2xI4dO7R6TNklM9fF+e08nNXa5IjzsNaHsBARkcZ9/PhR3Lt3L0Ozbjx9+lQEBARoIRURERHlFPv27RONGzcWRYoUEQqFQhgZGYlKlSqJGTNmqJ2tgb5NvXr1BAC1X/Xq1UvT/9y5c8LT01MYGxsLExMT0bhx41z76SSpZbT2r1+/Ft27dxdlypQRpqamQldXVxQrVkz07NlT3L17V7oDyKV8fHzSrbu65/2dO3dE69athbm5uTA0NBS1a9cWx44dkyZ8LpfR2n/8+FH07dtXlC9fXlhYWAhdXV1ha2sr2rdvLy5evCjtQeRC27dvFy1atBB2dnZCoVAIAwMDUaZMGTF06FC1n7rlc54o4zJzHZXaNzAwMM1+AgICRP369YWJiYkwNDQUtWrVEn/++ad2DkIDsqMub9++FUOGDBEODg5CT09PGBkZiapVq4oVK1bk2tmSMnMNktpX3Yytf/31l6hevbowNDQUFhYWon379uLRo0faOxANyGpt1q5d+8X75+a3GbPrefNf9vb2wsXFRXPBtSC7anPjxg3RqlUrYW5uLhQKhahcubJYu3at1o4ju2VHXSIiIsSPP/4oypQpI/T19YVCoRDly5cXs2fPFgkJCdo9oGyUmevi/HYezmptcsJ5mDOUUK734sULXLx4Ec+fP0dMTAzMzMxQtmxZ1K5dO998qjIoKAibN2/GhQsX0tShRYsW6NChA/T09KSOSZQjJCUl4caNG2naLS0tUaJECQkSSSc/1+Ldu3cIDw9HqVKllDOyBAYGYt26dQgJCUHNmjXRtWvXfD1bS0JCAiIiIlCgQAGpoxARERERERERERERkQQ4oCSX4gAC4N69exg+fDhOnDih0i6EgEwmg5mZGUaNGoWffvoJOjo6EqXUvF9++QVTpkxBfHy82u0ymQxlypTBzp074eLiouV00omKisL+/fvV/ow0b94ctWrVkjqi1oWFhSnrYGJiInUcrQgJCUHt2rXRoUMH/O9//wMABAcHw8bGJs1AAQsLCzx69AiWlpZSRNU41iJFcnIyBg4ciLVr10IIgTJlysDPzw8hISFwd3dHTEwMgJRzZ8OGDeHn55enf4fcu3cPS5YsQWhoKFq0aAFvb28kJSVh6NChWL16NZKSklCqVCksXboUDRs2lDouERERERERERERERFpEQeU5EIcQADcv38fNWvWRFxcHOrWrQsjIyNcvHgRwcHBGDduHD5+/Ag/Pz/cvn0brVq1wr59+/Lkp8w3btwIHx8fNG3aFD179oSRkRFOnjyJVatWYfXq1ahYsSIOHjyIOXPmQC6X4/r167C1tZU6tsZt3boVw4YNQ2hoKFJPcTKZTOX/Hh4e2LBhQ65dcy2jtmzZgrVr1+LSpUuIiopSthcuXBgtWrTA6NGj4eTkJGFCzfrtt98wceJEPHv2DDY2NgA+DaLo0aMHHB0dAaQMMvjll1/wyy+/wNfXV8rIGsNapFi/fj169+6N5s2bo3jx4li7di3q1auHhIQEVK9eHT4+PoiNjcWiRYuwfv16LF26FN9//73UsTXi8ePHcHV1RWRkJICUc+Nvv/2G0NBQLFu2DA0bNkRsbCxOnDiBxMREXL16Fc7OzhKnJiIiIiIiIiIiIiIibeGAklyGAwhSdOjQAWfPnsW5c+eUyzLExsaiffv2+PjxI06ePAkAWLx4MUaMGIHff/8dAwcOlDKyRlStWhVmZmZpZmmZM2cO5s2bhxcvXkBXVxeBgYGoUqUKunXrhsWLF0uUVjsOHTqEli1bwsXFBd7e3sqfkcOHD2PZsmUoXLgwDh48iNWrV6NYsWK4fPkyzMzMpI6d7eLi4tC2bVscOXIEn5/m9fX1UalSJQQFBeH9+/cwMDDAqlWr4O3tLWFazWnYsCEsLCywa9cuZVvqIIrjx4/D09NT2d6pUyeEhITg+PHjUkTVONYiRa1atWBmZobDhw8DAJYsWYIRI0agc+fO2LJli0rfatWqwcDAAKdPn5Yiqsb169cPf/75J/z8/FC8eHF07doVd+/eReHChXHgwAEULlwYAHDnzh1Ur14dnTt3xqpVqyROTURERERERERERERE2iKXOgBlzqJFi1C/fn34+fmhS5cuaN26NRYsWIBJkyZhxIgRKF26NH744QdcvHgR8fHxmDFjhtSRNSIgIACDBw9WDiYBAENDQ0ydOhWnTp3CkydPAADDhg1DmzZtsGbNGqmiatSdO3fQrl27NO3t27fHu3fvcOPGDQBAiRIl0K9fPxw4cEDbEbVu1qxZ+O6773Dt2jWMHz8ew4cPx759+zBixAjMmDEDzZo1w5IlS3DmzBm8fPkSv/zyi9SRNWLWrFk4cuQIZsyYgVevXiEsLAx79uyBlZUVWrZsibdv3+LSpUtwdXVFr169cOXKFakja8TNmzdRs2bNDPWtWrUqbt26peFE0mEtUjx69AgtWrRQ3m7atCmEECptqby8vHD79m1txtOqU6dOoW/fvnBzc0PBggUxZcoUvHnzBl26dFEOJgGAcuXKoUePHggICJAuLBERERERERERERERaR0HlOQyHECQIioqSrlkw+dsbGwghMCrV6+UbU2aNMHdu3e1GU9r9PT0EBMTk6Y9OjoaMplMZVkkZ2dnvHnzRpvxJHH9+nV0794dOjo6Ku29e/dGYGCg8mfEzc0NvXr1wp49e6SIqXGbN2+Gj48Pxo8fD1tbW5iZmcHLywsLFizAzJkzERERATc3N5w4cQKOjo55dmBNaGgorK2tVdpMTEywePFilC1bVqXdxsYGoaGh2oynVaxFisjISJiYmChvW1paAoDKAIpUtra2yuVg8qIXL16gdOnSytup/y9XrlyavuXLl8fLly+1lo2IiIiIiIiIiIiIiKTHASW5DAcQpHB0dFS7FMPx48chk8lQpEgRZVteXtXJzc0NK1asSPOG57x586Crq6vypuCHDx/y5NIu/5WcnPzF7VFRUcr/V6lSBc+ePdN0JEk8f/5c7WwUNWrUQFxcnHLWBYVCAR8fnzy7pIexsTHCw8NV2hQKBYYMGaJyngCAiIgIGBkZaTOeVrEWKaytrfHu3TvlbT09PVSpUkXt+fHt27ewsLDQYjrtMjIyQmxsrPK2np4egJQZv/5LJpOlGahHRERERERERERERER5GweU5DIcQJCie/fu2Lt3LwYPHoxLly7h1q1bWLRoEX744QdUrlwZjo6Oyr537tyBg4ODdGE16KeffsKTJ09QpkwZDB48GD/88APc3NywceNG9OnTB+bm5sq+Z86cQfny5SVMqx2VKlXCpk2bkJSUpNK+du1ayOVyODk5KdvCw8Pz7JvmFhYWCAoKStMeFBQEmUwGhUKhbCtUqFCagQZ5RcmSJXHhwoUM9T1//jxKliyp4UTSYS1SuLi4KGcqAgAzMzP8888/cHNzS9P31q1bKr9P8ho7Ozs8ffpUedvc3Bx//fUXKleunKZvUFAQChYsqMV0REREREREREREREQkNV2pA1Dm/PTTT2jUqBHKlCkDLy8vGBkZISAgANeuXcOAAQPyzQCCsWPH4u+//8by5cvxxx9/AEiZiaRgwYLYsGGDSt8nT56gc+fOUsTUOE9PT2zatAnDhw/H8uXLAQA6Ojro168fFi5cqNK3ZcuWapcxyGtGjhyJzp07o0qVKvD29oaRkRFOnjyJ/fv3o127dipLJV26dCnNUh95RYMGDbBkyRK0bt0a1atXBwC8e/cOY8eOhbm5OSpUqKDs++LFizz7RnGTJk3w22+/4cGDByqDif7r3r172LNnD8aOHavFdNrFWqTw9vbG1atXv9ovJCQEe/fuxdChQ7WQShqurq44f/688rauri5atGihtu/BgwfVDrohIiIiIiIiIiIiIqK8Syby8nogedTWrVsxfPhwBAcHA0gZQNCnTx8sXrwY+vr6yn4rVqxAuXLlUKdOHamiaty+ffsQEBCAuLg4uLi4oHv37nl6eYL0JCYm4tatW4iLi0OZMmXyZQ0+N3v2bEyePFllCahGjRphx44dKoOuJk2aBFdXV3h5eUmQUrOePHmCKlWqIDIyEk5OTjAyMsLdu3cRGxuLxYsXY8iQIcq+1apVQ7FixbB7924JE2vG27dvUaZMGVhaWmL16tXw9PRM0+fEiRPo27cvIiIicPfuXRQqVEiCpJrHWmROUlISoqKiYGRkpFwKJq95//49QkNDvzjACEgZjDZr1iy0atVK7fOGiIiIiIiIiIiIiIjyJg4oyaU4gCBz7ty5ky9m56BP3r17h3PnzikHG+XV2Xq+5P79+xg3bpzKoKvRo0eja9euKv3u3LkDa2vrPDtLyeHDh9GhQwfExsbC3t4elSpVgpmZGSIiInDjxg08ffoUhoaG2LNnDxo3bix1XI1iLYiIiIiIiIiIiIiIiCij5FIHoMx5+fIlgJRp6StXrozq1at/cTDJr7/+qqVk2jV8+PAM971z5w4/UZ2PREREAAAKFiwILy8vdO7c+YuDSbZt26ataFpXpkwZ7Nu3D2FhYYiNjcXly5fTDCYBgHLlyuXZwSQA0LRpU1y6dAleXl54/fo19u/fj40bN2L//v14/fo12rZti0uXLuWLARSshXpJSUm4evVqmq/AwECpoxEREZEWBQUFQSaTYcqUKVJH+WYymQy9evWSOoZaHh4ekMlkkMlk0NXN/ArMb968Ud4/Jx8nERERUX4zZcoUyGQyBAUFSZqjV69ekMlkkmZYt24dZDIZAgICJM0hFW09FxwcHODh4ZGhvpn9nuSU5zNRTpL5V/AkqUaNGuHvv/+GlZXVV/vOnTsX48ePx5gxY7SQTLuWLFkCKyurr/6h7969e/D09FQOMsjPZsyYgcmTJyMxMVHqKBrVokULHDt2DAYGBl/tu337dvTs2RNdunTRQjKSUrly5bB7927ExcXh4cOHiIiIgJmZGUqXLg2FQiF1PK3K77UICQlB7dq10aFDB/zvf/8DAISFhcHNzS3NC04LCws8evQIlpaWUkTViu3bt8PS0lI5iCgyMhLdunVL08/R0RELFizQcjoiIsrrrl+/jn379qFXr15wcHCQfD+5QVhYGBYsWAAPD48M/wE1J7G2tsb8+fMhl2f+800WFhbYuHEjAKBHjx7ZHY2IiIiIvmDfvn24fv265IOv161bh7CwMIwcOVLSHJo0ZcoUVK5cGV5eXlJHybMCAgIQEBCAkSNHcgUIogzggJJc5vHjx2jatClOnjwJExOTdPvNnz8fY8eOhaurqxbTaU/qG4EFChTAsGHD1Pa5f/8+6tevj/DwcOzbt0+7AXOo/LDC1blz59C+fXvs37//i59627VrF7p3745ixYppMV3O9Pz5cwQGBsLd3V3qKBqnUCjy5fJH6uTXWqxduxZBQUFqZ7rq3r07HB0dAQDJycn45ZdfsG7dOvj6+mo7plb4+/vD29sbu3fvVrbFx8fjwIEDafrKZDK0bdsW9erV02ZEIiLK465fv46pU6fCw8MjywNKsmM/uUFYWBimTp0KAGoHlMTGxkJHR0fLqTLO2NgY3bt3/6b7GhgYKO/LASVERERE2rVv3z6sX79e7YCSiRMnYvz48Vr5sNq6desQFBSkdkDJypUrsXz5co1n+JIePXqgS5cu0NfX/+Z9TJ06FT4+PhxQ8gX379/P0mw0AQEBmDp1Knr16sUBJUQZwAEluczGjRvh7e2N1q1b49ChQ2p/QS9cuBCjR4/Gd999h+PHj0uQUvM2b96MiIgI+Pr6wsLCIs0fkx48eID69esjLCwMe/fuRZMmTSRKqlnPnj3LcN+wsDDNBclBfv31V/zwww/o2bMntmzZorbP7t274e3tjaJFi+bbqec+t2HDBkyaNAlJSUlSR9GqhIQEnD9/HpUqVYK5ubnUcSSVn2px6NAhtGjRAjY2Nmm2+fj4qCyRdvfuXRw8eDDPDijZvHkznJyc1L44PX78uLIWQgiUKVMGGzdu5IASIiKiL0hKSkJcXByMjIwky5CRmRqJiIiIKPeIjIyEqamp1DG+SFdX95uWNMxuenp60NPTkzSDjo5Ojh7gnRueTxmRH2baJspJMj/HKEmqU6dOWLZsGQICAtC5c2ckJyerbF+0aBF8fX1RuXJlHD9+PM+OrNPT08PevXtRvXp19O3bF3/++ady28OHD+Hh4YGQkBDs2bMHTZs2lTCpZjk4OKBEiRIZ+po3b57UcbVi1KhRmDBhArZt24ZBgwal2b5nzx507doVRYoUwalTp1C8eHEJUlJOEBISgvr16+PKlStSR5FcfqrFzZs3UbNmzQz1rVq1Km7duqXhRNI5d+4cWrZs+dV+MpkMHTp0wLlz57SQioiI8ospU6agd+/eAID69etDJpNBJpOhV69eyj4fPnzAkCFDUKxYMejr66NYsWIYMmQIgoODM7yfyMhITJw4EdWrV4e1tTUUCgVKlSqF8ePHIyYm5pvzp67Dffz4cfzvf/9DyZIlYWBggB07dgBAmmP57/0+H9ieukb3/fv38dNPP8HOzg4KhQKVKlWCn5+fsl9AQABKlCgBIOVTi6nH+vmsLOoeN7Xt5MmTqFmzJoyMjGBnZ4fZs2cDAEJDQ9G3b18ULFgQRkZGaNmyJV69epUme3h4OMaNG4dSpUpBoVDAxsYGXbt2xZMnT76xip88f/4cffr0gb29PRQKBQoWLIhatWph/fr1Wd43ERERkVQ+v2acMmWK8lqnYsWK2LZtW5r+Dg4O8PDwwLVr19CkSROYm5ujYsWKyu0PHz5Ejx49ULhwYejr68PBwQFjxoxBdHS0yn7u3buHwYMHw8XFBaampjAyMkKVKlWwatUqtTkjIiIwYcIEODs7w8DAAAUKFECdOnWUGT08PJTXZanXoDKZDOvWrQPw6Xo2KCgIAPD7779DJpOpvG+TKjk5GXZ2dqhcubKy7ejRo+jcuTMcHR1haGgICwsLNG7cGKdOnUpTn1OnTuHp06cqOVKvrXv16qV21op///0Xbdu2RYECBWBgYIBy5cphzpw5aT5gmXr/8PBwDBo0CAULFoSBgQFq166Nixcvqq3df6m73k9tO3nyJObOnYuSJUtCoVDAyclJ5Xo3KChImX/9+vUqx/i548ePo3HjxrCwsICBgQEqVqyodmaW9J5Phw4dgkwmw6JFi9QeQ82aNWFjY4OEhAQAwKVLl9CrVy84OTnByMgIpqamqF27Nvbu3ZuhmqiT+nomMDBQ2fb69WvIZDLI5XKEhIQo2+/evQuZTKZ8/fL5sf3XypUrUbZsWeXrvgULFqSZub9Xr17KWR9LlCihrPF/Z9+Ji4v74uszovxE+iGDlGkDBgxAWFgYxo8fj169emHDhg0AgCVLlmDkyJGoVKkSTpw4AUtLS4mTapahoSH8/PxQr149dOnSBX5+frCzs1MZTNKsWTOpY2qUrq4uSpUqhQYNGny17+XLl3Hp0iUtpJLe//73P4SFhWHZsmWwtLTEzJkzAaRMy9elSxcULlwYAQEBsLe3lzip5qSeFzLi2rVrGkySs+WHZaAyKr/UIjQ0FNbW1iptJiYmWLx4McqWLavSbmNjg9DQUG3G06rnz5/DyclJpU0ul6NAgQJppuUsXrw4nj9/rs14RESUx7Vr1w6vX7/GihUr8NNPP8HZ2RkAULJkSQApgxdq1aqFR48eoU+fPnB1dcW1a9fw+++/4+TJk7h06RJMTU2/up+XL19i1apVaN++Pby9vaGrq4tTp05hzpw5uHbtGo4cOZKl4/jhhx+QkJCA/v37w8zMDGXKlPnmffn4+EBPTw8//PAD4uPjsWDBAnh5eeHBgwdwcHCAs7Mz5s+fD19fX7Rt2xbt2rUDgC8uh5vq2rVr+OuvvzBgwAD07NkTO3bswPjx42FgYID169fDwcEBU6ZMwaNHj7Bo0SL07NlTZcbT1O/Hs2fP0KdPH7i4uOD169dYtmwZqlevjsuXL3/z66vExEQ0atQIL1++xODBg+Hk5ITw8HD8+++/OHPmDHx8fL5pv0REREQ5xbhx4xAdHY3BgwcDSFmSuWvXrvj48WOawcDPnj2Dp6cnOnbsiPbt2yMqKgoAcOXKFXh6esLCwgIDBw5E0aJFcePGDSxatAhnz57FqVOnlLNzBAQE4PTp02jZsiVKlCiB6Oho7Ny5E/3798f79+/x448/Kh8vLCwMderUwe3bt9GhQwcMGjQISUlJuHbtGg4cOIAuXbpgwoQJSE5OxpkzZ7Bx40blfWvVqqX2eLt06QJfX19s2LABrVu3Vtl24sQJvHz5EqNHj1a2rVu3DiEhIejZsyfs7OyU1/ANGjSAv78/6tatCwBYsGABfvzxR3z48AHz589X3j/1NYA6ly9fRr169aCnp4chQ4bA1tYWf/31F8aNG4cbN25g8+bNae7TpEkT2NjYYNKkSQgODsa8efPQokULBAYGZml2j59++gmxsbEYOHAgFAoFfv/9d/Tq1QulSpVC7dq1YWNjg40bN6JHjx6oW7cuBgwYkGYfK1aswPfff48aNWpgwoQJMDY2xrFjxzBo0CA8fvwYv/76q0p/dc+nxo0bw9bWFhs2bEizLPjDhw9x4cIFDB8+XPl82rt3L+7du4dOnTrB3t4ewcHBWL9+Pdq1a4fNmzfD29s707Xw9PTElClTcPLkSfTt2xdAynNDLpcjOTkZ/v7+aN++PQDg5MmTyvt8yYIFC+Dr64tKlSph5syZiImJwdy5c1GwYEGVfgMHDkRERAT27t2L+fPnK/9W/PngLeDrr8+I8hVBudb48eOFTCYTw4cPF0uWLBEymUxUqlRJfPjwQepoWvXmzRtRunRpYWZmJooWLSoUCoX466+/pI6lFZUqVRI1a9bMUN/p06cLuVyu4UQ5S/fu3YVcLhe//vqr2Ldvn9DX1xfFihUTT548kTqaxslkMiGXy4VMJsvQV357bgiRcu6QyWTixIkTUkeRXH6qhYWFhVi4cGGG+i5cuFBYWFhoOJF0FAqFWL16dYb6rlq1SigUCg0nIiKi/Gbt2rUCgPD390+z7aeffhIAxNKlS1XalyxZIgCIiRMnZmg/cXFxIj4+Pk37xIkTBQBx8eJFZVtgYKAAICZPnpzh7E5OTiI6OjrNdgDCx8cn3ft9nnXy5MkCgGjRooVITk5Wtl+6dEkAEOPHj89wRnWPC0DIZDJx4cIFZVtcXJywtbUVMplMDBs2TKW/r6+vACDu3bunbBs+fLgwMDAQ169fV+kbFBQkTE1N1R7rf9WrV0/Y29unab9x44YAIGbPnv3VfXx+TBl5TCIiIiIppV77FS9eXISFhSnbw8LCRPHixYWlpaWIiYlRttvb2wsAYuXKlWn2VbFiRVGmTBkRERGh0r5nzx4BQKxdu1bZFhUVleb+SUlJol69esLMzEzl+njQoEECgPjjjz/U3ieVj4+PSO8txdTr2cDAQGVbhw4dhEKhECEhISp9u3fvLnR1dcXbt2+/mPfNmzeiQIEColmzZirt6V1TppexVq1aQkdHR9y4cUPZlpycLDp27CgAiOPHj6e5/6BBg1T2sWPHDgFALF++XO3jfk7d9X5qW+XKlUVcXJyy/cWLF0JfX1906dJFZR/pXeu+evVKKBQK0bVr1zTbhg8fLuRyuXj8+LGy7UvPpx9++EEAELdv31ZpT32ddOXKFWWbuu9PdHS0cHJyEs7Ozirt6p4L6sTHxwsjIyPh7e2tbOvdu7dwdXUVzs7OKt+Ddu3aCXNzc5GYmKhybPXq1VPeDg0NFUZGRsLZ2Vnl9dnz58+FsbFxuq/B1OXMzOszovyCS97kYrNmzcLAgQOxePFiDB8+HBUrVsTJkydRoEABqaNpVaFChXD8+HGYm5sjODgYu3btytAU/nmBq6srbty4kWbpI0qxbt06tGzZEuPGjUOnTp1QqFAhnDp1SjlNdF5mbGyMWrVqYefOnV/96tKli9RxibSmZMmSuHDhQob6nj9/Xvnp5rzIxsYmw1PUP3nyBDY2NhpORERE9MnevXthY2OT5lN5AwcOhI2NTYanV9bX11d+si4xMRGhoaH48OEDGjZsCAAZnro6PYMGDYKRkVGW9pFqxIgRKtNZV61aFSYmJnj48GGW912zZk1Ur15deVtfXx/VqlWDECLNpxJTPwGa+rhCCGzevBnu7u4oWrQoPnz4oPwyNjZGjRo1cPTo0W/OZm5uDgDw9/fHu3fvvnk/RERERDnVoEGDlNc8QMr1z/fff4/Q0FCVpVEAwMrKSrmkY6qbN2/i33//hbe3N+Li4lSux+rUqQNjY2OV6zFjY2Pl/z9+/Ijg4GCEhISgcePGiIiIwL179wCkLD+zbds2ODs7q50NQy7/9rcQfXx8EBcXh+3btyvboqKisHfvXjRt2lRl1ojP80ZFRSE4OBg6OjqoXr16lq7X3717h3PnzqF169Yqs0/IZDJMmDABANS+rvD19VW5nTozRlavywcPHqwyK3DRokXh5OSU4f3u2rULcXFx6Nu3r8pz4MOHD2jVqhWSk5NVZhkE1D+fAChnAfx8pnMhBDZt2oTy5cvD1dVV2f759ycmJgbBwcGIiYmBp6cn7t69i4iIiIwV4DN6enqoU6cO/P39lW3+/v5o0KABGjRogBMnTigznTp1CvXq1YOOjk66+zt69ChiYmIwZMgQlddndnZ26NatW6bzAZp9fUaU23DJm1xmz549KrcbNGiAY8eO4c2bNxg0aFCaiw8Aymlw8xJTU9M068bFx8cjMTFR7fRaqeve5TXNmjVDYGAgXr16BTs7uy/2dXd3x6RJk7SUTDpXr15VuT127Fjcvn0bb968wdy5cxEaGppmCYvPL47yikqVKiEsLEw5LdyXpL6AyG9MTEwwefJkODo6Sh1FcvmpFk2aNMFvv/2GBw8epFnu5XP37t3Dnj17MHbsWC2m065q1aphz549mD59+hf7CSGwZ88eVKtWTUvJiIiIgMDAQLi5uUFXV/XPFrq6unByckpz3f8ly5Ytw/Lly3H79u00g/Gzurzdl64nMkvdtViBAgUQHByskX2nLpP73wH3qe2pj/v+/XsEBwfj6NGj6Q4wzcqbDfb29pgwYQJmzZqFwoULo3LlymjQoAE6duyIqlWrfvN+iYiIiHIKdUuylCtXDgDSfNinZMmSad44v3v3LgBg8uTJmDx5strHePv2rfL/UVFRmDJlCnbs2KF2CePUa+APHz4gNDQUTZs2zcTRZEzqoJENGzbg+++/BwDs3r0b0dHR6Nmzp0rfx48fY8KECThy5AjCwsJUtv33faDMCAwMBAC4uLik2ebs7Ay5XK72w1b/vXZO/RB3Vq/L07vef/r0aYbun/o8SB0cr87nzwNA/fMJgHLQyObNmzFz5kzI5XKcPn0aQUFBmDNnjkrfd+/eYeLEidi/f7/aAeBhYWEwMzPL0DF8ztPTE0ePHsXdu3ehUCgQFBQET09PxMbGYsmSJXj58qXytcjXlrtJ/T7+d0lz4NPPWmZp8vUZUW7DASW5TIcOHdL8AhVCAIBy/b3P22UyGZKSkrSWT1uqVKmSpQuJvKJjx47o2LFjhvrWrVtX+UmzvMzNzS3dn5GuXbuqvU9e/BlxdXXF77//jtjYWBgaGn61f2qN8hNjY+N0X4DlN/mpFsOHD8fSpUvRpEkTrF69Wu2LkRMnTqBv374wNjbG0KFDJUipHb169UKbNm3w448/YtasWen2+/HHH/HgwYM0LyaJiIhyg3nz5mH06NFo3Lgxhg8fjiJFikBfXx8vX75Er169sjzbY2ZnJ0lMTEx3W3qfuMuOa/UvfZrva4+b+m/Dhg0xbty4LGdRZ/r06ejTpw8OHjyIM2fOYNWqVfj1118xduxYzJ49WyOPSURERJQTqbu+TL0eGz16dLqDP1IHBQOAt7c3Dhw4gAEDBsDd3R0FChSAjo4O/Pz8MH/+fK3MeK6rqwtvb28sWLAAjx49QqlSpbBhwwZYWlqidevWyn5RUVFwd3dHdHQ0Ro4ciQoVKsDU1BRyuRyzZs3CyZMnNZ71vzR1XZ7V/ab227BhAwoXLqy2z38HQXzp9UrPnj0xcuRInDx5Eg0bNsSGDRugo6OD7t27qzxm48aNcffuXYwYMQJubm4wNzeHjo4O1q5diy1btnzz8yn177InT56EQqGAnp4e6tati/j4eMjlcpw4cQIfPnxQ6atNmnx9RpTbcEBJLrN27VqpI+QI6mZioa+7c+fON4/GzC0mTZrEwUYAevTogQIFCiAqKuqrA0p69OiBOnXqaClZznD//n0EBAQgNDQUBQsWROPGjb86y09eld9qUahQIWzbtg0dOnRAo0aNYG9vj0qVKsHMzAwRERG4ceMGnj59CkNDQ+zZsweFChWSOrLGtGrVCm3atMGcOXNw7tw59OnTB5UrV1bW4vr161i9ejXOnj2LNm3aoFWrVlJHJiKiPOZL1+2Ojo64f/8+EhMTVWYpSUxMxIMHD1T+UPql/WzcuBEODg44dOiQyiwahw8fzmL6L7OyskJISEia9owuN5ceKV7r2NjYwMLCAhEREV/8NGRWOTo6YtiwYRg2bBg+fvyIJk2aYM6cORg9erTKlOhEREREuc3du3fRpk0blbY7d+4AUD8Lwn+VLl0aQMob3F+7HgsLC8OBAwfQo0cPLF++XGXbf5dDsba2hqWlJW7cuPHVDN9yHerj44MFCxZgw4YN6N+/PwICAjBgwAAoFAplnxMnTuDVq1dYs2ZNmqVZJk6cmKUcqTPx3b59O822e/fuITk5OVfN2Jz6PLC2ts6W63Jvb2+MGTMGGzZsQO3atbFr1y40atRIZbDKv//+ixs3bmDSpEmYOnWqyv1XrVqVpcd3dXWFubk5Tpw4AYVCgerVq8PY2BjGxsb47rvvcOLECYSEhKBgwYIoX778F/eV+n28d+8eGjRooLIt9Wftc3wPiShzOKAkl0ld14wISPmk/aJFizLU986dO/D09MSbN280nEpaU6ZMkTpCjlC1atUMTw9dvHhxFC9eXMOJpLFy5Uo8ffpUuaRHUlIS+vXrp1wbMnU0sZ6eHiZMmJCnl4ViLT5p2rQpLl26hJ9//hl+fn4ICgpSblMoFGjbti2mTZuW5wfgAcDWrVvx/fffY8OGDfj777/TbBdCoGfPnmn+AEFERJQdTExMAEDtwAsvLy/MnDkTq1atUk6RDaRc07x//x4DBw7M0H50dHQgk8lUPkWWmJiIX375JduOQx0nJyecP38eMTExyk8FhoaGZvlDIl86Vk2Ry+Xo1q0bli5dil27dqFDhw5p+rx79+6bB32Eh4fDyMgIenp6yjYDAwM4Ozvj9OnTyoHPRERERLnV77//jkGDBsHc3BxAyvXP8uXLYWFhgXr16n31/t999x3Kly+P5cuXY+DAgWkGQSQmJiIiIgJWVlbKWRX+O4vC69ev0wwAkMvl6Nq1K5YtW4bVq1ejb9++KttTZ8EHVK9DraysMnTclStXRsWKFbFp0yYYGBggOTk5zXtc6eU9evQoLl68mGafJiYmCA0NVcmWnoIFC6JWrVr466+/cOvWLeWgBCGEcrbetm3bZuhYtMnExETt9X6nTp3w008/YfLkyfDw8EjzQdLw8HAYGBioDNj5EhsbGzRr1gx79uyBu7s7IiIiMvz9uXXrFvbu3ZuZw0pDR0cH9erVw6lTp6Cnp6fyGs/T0xNbtmxBZGQkmjZt+tXvdaNGjWBoaIilS5eid+/eytdgL168wJYtW9L0//z57ODgkKXjIMoPOKCE8pyIiAiMHDkSY8eOVbteWl6yZMkSWFlZfXUQxb179+Dp6YmIiAjtBCPJRUREZGrdwm3btqFLly4aTCSN5cuXw8PDQ3l70qRJWL9+PTw8PNCzZ08ULlwYL168wIoVKzB16lSUKFECPXr0kC6wBrEWqsqVK4fdu3cjLi4ODx8+VP7MlC5dOsMvuvICAwMDrFu3DqNHj8bu3btx69YtZS3Kly+P9u3bo0KFClLHJCKiPKpq1aqQy+WYMWMGQkNDYWxsjBIlSqB69eoYO3Ysdu7ciSFDhuDq1av47rvvcO3aNaxevRplypTB2LFjM7SfDh064Mcff0SzZs3Qrl07REREYMuWLSqDFzRh6NCh6N69Ozw9PdGjRw+EhYVh5cqVsLe3z9Ig/wIFCqBUqVLYtm0bSpYsiUKFCsHY2FjjM4nNmDEDZ8+eRadOndCpUyfUqFED+vr6ePr0Kfz8/FClShWsW7fum/bt7++PAQMGoH379ihTpgxMTExw5coVrFq1CtWrV0eZMmWy92CIiIiItMza2hrVq1dXzsCxdu1aPHv2DKtWrcrQEooymQwbN26Ep6cnKlasiD59+sDFxQUxMTF49OgR9uzZg1mzZqFXr14wNTVF48aNsWnTJhgaGqJq1ap4+vQp/vjjD5QoUQLBwcEq+54+fTpOnjyJfv364ejRo6hTpw6EELh27RoSExOxceNGAECNGjWwZMkSDB48GC1atICenh6qV6+unAUkPT4+Phg9ejRmz54NJycn1KhRQ2V7nTp1YGtri9GjRyMoKAh2dna4fv06Nm7ciAoVKuDmzZsq/WvUqIEDBw5g6NChqFWrFnR0dODp6ZnuAOSFCxeiXr16qFu3LoYMGQJbW1scOHAAR44cgbe3d5rZLHKCGjVq4Pjx45g9ezaKFy8OmUyGLl26wM7ODr///jv69esHZ2dn9OjRA/b29nj//j1u3ryJffv24c6dO5kaIOHj44M///wTo0ePhrm5Oby8vFS2Ozs7w8XFBXPmzEFMTAzKlCmDBw8e4I8//kCFChVw5cqVLB2rp6cn/vzzT+X/P2//9ddf07Snx9LSEv/73//www8/oFatWujZsydiYmKwfPlylC5dGteuXVPpn/o8HDduHLp16wYDAwOUL1/+qzOhEOVbgvKMN2/eCLlcLk6cOCF1FEnlpzp07NhRyOVysWjRonT73Lt3T9ja2goDAwNx+PBhLabLed6/fy9KlCghzp07J3UUjatTp46IjY3NUN9t27YJPT09DSeShrm5ufj999+Vt21sbISXl1eafklJSaJ27drC1dVVm/G0irUgIiKinGjdunXC2dlZ6OnpCQDCx8dHue3du3di0KBBomjRokJXV1cULVpUDB48WLx//z7D+0lMTBQzZ84UJUuWFPr6+qJ48eJizJgx4s6dOwKAmDx5snIfgYGBadrSs3btWgFA+Pv7p9tnzpw5onjx4kJfX1+ULVtWrF69Wu39Jk+eLACIwMDANPuwt7cX9erVU2m7ePGiqFWrljAyMhIAhL29vXLbf2uYXpsQQvj4+Ah1fxby9/cXAMTatWtV2qOjo8W0adNE+fLlhYGBgTAxMRFly5YV/fr1ExcuXEi3Dqnq1aunkjXVkydPxMCBA0XZsmWFqampMDIyEmXLlhU///yzCAsLU7uv9I6JiIiIKCdJvfY7duyYmDRpkihWrJjQ19cX5cuXF5s3b07TX9213+eCgoLEwIEDhb29vdDT0xNWVlbC1dVVjB8/Xjx79kzZ7/3796Jv376icOHCQqFQiPLly4sVK1akew0bGhoqxowZI0qWLKncb506dcT27duVfZKSksTo0aNF0aJFhVwuV7le/NL17Js3b4Surq4AIKZPn672uG7cuCGaNGkiLCwshImJiahXr544ffq02uvV6Oho0adPH1GwYEFljtTjSe/69vr166JNmzbC0tJSeW0+e/ZskZiYqNIvvfsLkfHrT3U1/tJrB3XXyA8ePBCNGjUSpqamAkCaTH///bfw8vISNjY2Qk9PTxQuXFh4eHiIuXPnqrwn8LXnkxBCxMXFCSsrKwFA9OvXT22foKAg0aFDB2FtbS0MDQ1F1apVxZ49e9R+37/0XFDn33//FQCEoaGhiIuLU7ZHRUUpX9s9ePAgzf3SO7bly5cLJycnoa+vL0qWLCnmz58v1qxZo7b+s2fPFiVKlFA+P1NfB2b29RlRfiAT4j/zFFGu9fbtWxQuXBjHjx/P0Ii9vCo/1SEhIQGtWrXC8ePHsXbt2jSzCTx48AAeHh4IDQ3F3r170bRpU4mS5gz56bmho6ODpk2bYv/+/Sprzv/Xrl270LVrVxQrVizL67nnRCYmJliwYAH69euHqKgomJmZYceOHWqn6V6yZAnGjh2LmJgYCZJqHmvxdQkJCTh//jwqVaqknIKUiIiIiLKHh4cHnjx5gqtXr0Imk6FAgQKZur8QQvmJWhsbG/j4+HzzrChERERE2rBu3Tr07t0b/v7+KjMHExER5SZyqQMQacLX1lPLK/T09LB3715Ur14dffv2VU4NBgAPHz6Eh4cHQkJCsGfPnnw/mCS/+fXXX3Ho0CH07Nkz3T67d++Gt7c3ihYtioCAAO2F06IyZcrgwoULAFIGVKS3/iQABAcHw8DAQJvxtIq1+LqQkBDUr18/y1M15gWhoaHw9PRMMx0kERERUVY8f/4cNjY2KFSoUKbv+/btW9jY2MDGxkYDyYiIiIiIiIhInfQ/tk6Ui/134p24uDgoFAqJ0miWoaEh/Pz8UK9ePXTp0gV+fn6ws7NTGUzSrFkzqWOSlo0aNQqhoaGYMWMGzM3N8fvvv6ts37NnD7p27YoiRYrg1KlTKF68uERJNcvHxwfjxo1Dz5494e7ujt69e2P69Onw9PREqVKllP2uX7+ORYsWwd3dXcK0msVaZAwnbksRHx+PgIAAhIaGSh2FiIiI8ojffvtNeW0hl2f+801WVlY4duyY8naRIkWyLRsRERERERERqccBJXmIvr4+6tWrB0tLS6mjSMrGxgaBgYEoXLgwrly5gtWrV2P79u3KqXHzInNzcxw5cgR169ZFmzZtYGpqiuDgYOzatQvNmzeXOl6OoaOjA3t7exgaGkodRSv+97//ISwsDMuWLYOlpSVmzpwJANi3bx+6dOmCwoULIyAgAPb29hIn1ZwhQ4bAz88PDRs2hJeXF1xdXREXF4dy5cqhRo0asLW1xYsXL3Dp0iUYGRlh+vTpUkfWGNaCiIiIiKRUpUqVLN1fX18fDRs2zKY0RERERERERJQRMsGP4uZZ8fHx2LNnD7p06SJ1FK0KCQnBpk2bsGbNGty8eRNCCDg5OeHevXtSR9O4Z8+eoU6dOnj//j127tyJli1bSh2JcoAePXpgy5YtmD17NkqXLo1OnTqhUKFCOHXqFEqUKCF1PI1LSEjAtGnTsGjRIkRGRqrtU7t2bSxevBiVK1fWbjgtYy2+7O3btyhcuDCOHz8OT09PqeNI6u3btyhSpAiOHTuW72tBRERERERERERERJRfcUBJHnTt2jWsWbMGW7ZsQVhYGJKSkqSOpBVHjhzBmjVr8OeffyI+Ph5OTk7o2rUr2rdvDxcXF6njaYSpqSlkMplKW3x8PBITE2FkZJSmv0wmQ3h4uLbiSeb58+fYs2cP9PX10b59exQsWBDPnz/HuHHjcPLkSURGRsLNzQ3Tp09H3bp1pY6rcUlJSWjXrh0OHDgAXV3dfDWY5HMxMTE4c+YM7t+/j8jISBgaGsLOzg7VqlWDg4OD1PG0irVQLzo6GnPnzoWPj0++rgPAwTVERERERERERERERMQBJXlGWFiYclaOGzduQAiBKlWqoH379hg/frzU8TQmKCgIa9aswfr16/HixQtYW1ujcePG2LJlC3bu3Il27dpJHVGjPDw80gwo+Rp/f38NpckZ7t27hxo1aiAyMhJCCBQsWBCnT59Gs2bNEBgYCHNzcyQmJiI6Ohr6+vo4e/ZslqdezomuXr2qcjs2NhY+Pj548+YN1qxZg1KlSqW5j6urq7biEREREREREREREREREVEOxwEludzx48exevVq7N+/Hx8/foRMJsPAgQMxbtw42NvbSx1PYzZv3ow1a9bg1KlT0NHRQcuWLeHj44PmzZvj6dOnKF26NHbt2pXnB5RQWn369MG2bdswe/ZsFC5cGBMnToSenh4+fPiA/fv3o1q1agCAY8eOoV27dmjatCl27twpcersJ5fL0ww2Sj3dpzcIKb/MZkSkzv379xEQEIDQ0FAULFgQjRs3hp2dndSxiIiIiIiIiIiIiIiIJKMrdQDKvGfPnmHt2rVYt24dnj17Bmtra3z//feoW7cu2rdvj4YNG+bpwSQA0KNHDzg6OmLBggXo2rUrChQoIHUkyiFOnTqF/v37Y9iwYQAAY2NjtGjRAnPmzFEOJgGARo0aoX///tiyZYtUUTVq0qRJmZ69Ji87d+4c1q5diwsXLuD58+eIiYmBmZkZypYtixYtWmDgwIGwsrKSOqZW5PdarFy5Ek+fPsX06dMBpAyk6tevHzZs2ADg08ArPT09TJgwAZMmTZIsq7aEh4fj/v37KFSoULrXD4GBgThz5gx69uyp5XRERERERERERERERCQVDijJZRo1agR/f3/lrByLFi1Cs2bNoKuri8ePH0sdT2sUCgWCgoKwf/9+WFpaol27djA0NJQ6Vo4SERGBkSNHYuzYsShbtqzUcbTm1atXqFixovJ2hQoVAADlypVL07d8+fIIDg7WWjZtmjJlitQRcoTk5GQMHjwYK1euxOcTcunq6kJHRwfnz5/HuXPnsHDhQmzduhX169eXMK1msRYpli9fDg8PD+XtSZMmYf369fDw8EDPnj1RuHBhvHjxAitWrMDUqVNRokQJ9OjRQ7rAGjZr1ixMnToVCQkJAAB3d3esWrUKJUuWVOl37tw59O7dmwNKiIiIiIiIiIiIiIjyEbnUAShzTpw4AUdHR1y4cAG7d+9Gq1atoKub/8YFvX79GgsWLEBwcDB69OgBW1tb9O3bF6dPnwZXcUoRGxuL9evX49WrV1JH0aq4uDiVwUWp/zcwMEjTV6FQIDk5WWvZSPsWL16MFStWoH///jh37hyuX7+OefPmwcjICDNnzkRERAR27NgBMzMztGrVCg8fPpQ6ssawFikeP36M0qVLK2+vXLkSbdq0wcmTJ9GrVy80adIEffv2xfnz51GzZk0sWLBAurAaduTIEUyYMAEODg7w9fVFhw4dcPbsWVSpUgWnTp2SOh4REREREREREREREUmMA0pymY4dO+LZs2eoWrUqGjVqhE2bNiEmJkbqWFpnYWGBoUOH4urVq7h8+TK6d++OvXv3on79+qhTpw5kMhnCw8Oljik5Dq4hdT58+ABHR0ecP39e6igat3LlSrRt2xbLly9HjRo1ULFiRYwcORK//vorRo0aBR0dHXTo0AHnz5+HhYUFZsyYIXVkjWEtUiQmJioHYkZFReHDhw/o1q1bmn5yuRxdunTB3bt3tR1Ra+bOnQtnZ2dcv34dc+fOxfbt2/HPP/+gQIECaN68OY4fPy51RCIiIiIiIiIiIiIikhAHlOQy27dvx6tXr/Dbb7/h3bt36NmzJ2xtbdGnTx+cOXNG6niScHV1xdKlS/H69Wts3LgRLi4uAIB+/fqhcuXKmD59Om7fvi1xSmnIZDKpI0jCz88P8+bNw7x58/D7779DJpNh586dyrbULz8/P6mjSiIpKQlBQUGIjY2VOorGPX78GI0aNUrT3qhRI0RGRuLGjRsAgAIFCqBv3755+g101iJFmTJlcOHCBQCAiYkJTExMEBISorZvcHCw2tmN8oo7d+6gd+/eKsdYqVIlXLx4EaVLl0br1q1x9OhRCRMSEREREREREREREZGU8t9aKXmAlZUVRowYgREjRuCff/7BmjVrsG3bNqxbtw4ymQz79u1DyZIlUalSJamjapVCoYC3tze8vb0RFBSENWvWYP369Zg0aRKmTJmCxMREqSNq3X9nKImLi4NCoZAojfZs2bIFW7ZsUWn7448/1PbNr4Nu8gtjY2N8+PAhTfuHDx8gk8lUvv8ODg5q++YVrEUKHx8fjBs3Dj179oS7uzt69+6N6dOnw9PTE6VKlVL2u379OhYtWgR3d3cJ02pWeHg4rKys0rRbW1vD398fDRo0gJeXF/bs2SNBOiIiIiIiIiIiIiIikhoHlORyVatWRdWqVTF//nzs3LkTa9aswebNm7F582Y4ODigffv2mDNnjtQxtc7BwQHTpk3D1KlTceTIEaxZs0bqSFpnY2ODwMBAFC5cGFeuXMHq1auxfft2BAcHSx1No/z9/aWOQDlI7dq1sXTpUvTo0QP29vYAUgZWTZo0CQYGBqhQoYKy75s3b1CgQAGpomoca5FiyJAh8PPzQ8OGDeHl5QVXV1fExcWhXLlyqFGjBmxtbfHixQtcunQJRkZGmD59utSRNaZYsWK4f/++2m2WlpY4ceIEGjRogLZt26J9+/ZaTkdERERERERERERERFKTif9OYUC53pMnT7B69WqsW7cOb968QVJSktSRSAIhISHYtGkT1qxZg5s3b0IIAScnJ9y7d0/qaCSxDx8+oGrVqtiyZQtq1qwpdRyNunr1KmrXrg0dHR24u7vDyMgIFy5cwOvXrzFhwgRMmzZN2dfT0xOGhoY4ePCghIk1h7X4JCEhAdOmTcOiRYsQGRmptk/t2rWxePFiVK5cWbvhtKhPnz44d+7cF38vhIaGomHDhrh27RpkMhmvKYiIiIiIiIiIiIiI8hEOKMnDkpOTcfjwYTRv3lzqKKRFqTOy/Pnnn4iPj4eTkxO6du2K9u3bw8XFRep4RFp3+vRpDB8+HP/++y+AlOU8fH19MX78eJVlXg4dOgQHBwc4OztLFVXjWAtVMTExOHPmDO7fv4/IyEgYGhrCzs4O1apVg4ODg9TxNO7PP//EwIEDsXXrVnh4eKTbLywsTDmohANKiIiIiIiIiIiIiIjyDw4oyUM+fPiAatWqYfPmzXl+1gFSFRQUhDVr1mD9+vV48eIFrK2t0bhxY2zZsgU7d+5Eu3btpI5IEnj+/Dn27NkDfX19tG/fHgULFsTz588xbtw4nDx5EpGRkXBzc8P06dNRt25dqeNqXFhYGOLi4lCoUCGpo0iOtSAiIiIiIiIiIiIiIqKvkUsdgLJPUlISgoKCEBsbK3UU0pLNmzejQYMGKFWqFGbPng03Nzfs3bsXL1++xJQpU8DxYvnXvXv3UKFCBYwaNQpDhgxBxYoV8eDBA3h4eGDbtm2Ii4uDXC7HmTNn0KhRI1y5ckXqyBpx69Yt5f8tLCy+OoBi2LBhmo4kGdaCiIiIiIiIiIiIiIiIMoMDSohysR49euDp06dYsGABXr16hd27d6N169bQ1dWVOhpJbM6cOYiPj8eCBQuwY8cOWFhYoH379oiJicGFCxcQGhqKyMhIHDlyBHp6evjll1+kjqwRTZo0QVBQUIb6Dh8+HMuWLdNsIAmxFqrOnTuH/v37o0KFCrCwsIC+vj6sra1Rp04dzJo1CyEhIVJH1Kr4+HhcuXIF+/btw5YtW3DgwAE8evRI6lhERERERERERERERCQhvutMlIspFAoEBQVh//79sLS0RLt27WBoaCh1LMoBTp06hf79+ytnmTA2NkaLFi0wZ84cVKtWTdmvUaNG6N+/P7Zs2SJVVI2KiopCw4YNcfbs2S/OyOHr64slS5agadOmWkynXaxFiuTkZAwePBgrV65UmcVJV1cXOjo6OH/+PM6dO4eFCxdi69atqF+/voRpNe/du3eYOHEitm3bhujo6DTbS5cujQkTJqBHjx4SpCMiIiIiIiIiIiIiIilxhpI8REdHB/b29hxQkI+8fv0aCxYsQHBwMHr06AFbW1v07dsXp0+f5nI3+dyrV69QsWJF5e0KFSoAAMqVK5emb/ny5REcHKy1bNr0559/4tWrV2jcuDHCwsLU9hk1ahQWLlyIJk2aYN++fVrNp02sRYrFixdjxYoV6N+/P86dO4fr169j3rx5MDIywsyZMxEREYEdO3bAzMwMrVq1wsOHD6WOrDGvXr1CtWrVsHr1ahQpUgSVKlWCQqGAnp4evv/+e7Rv3x7BwcHo1asXBg8eLHVcIiIiIiIiIiIiIiLSMg4oyUOsra0RGBiImjVrSh2FtMTCwgJDhw7F1atXcfnyZXTv3h179+5F/fr1UadOHchkMoSHh0sdkyQQFxenMrgs9f8GBgZp+ioUCiQnJ2stmzbVq1cPO3bswN27d9G8eXPExMSobB89ejQWLFiAxo0bY//+/dDX15coqeaxFilWrlyJtm3bYvny5ahRowYqVqyIkSNH4tdff8WoUaOgo6ODDh064Pz587CwsMCMGTOkjqwxEydOREhICP7++2/cu3cPV69exePHj1G5cmW8e/cOO3bswMuXLzFy5Ej88ccf2LFjh9SRiYiIiIiIiIiIiIhIizigJI949OgRzp49y8ED+ZirqyuWLl2K169fY+PGjXBxcQEA9OvXD5UrV8b06dNx+/ZtiVMSaV/Lli2xbt06XLx4EV5eXkhISAAAjBkzBvPnz0ejRo3y9ACKz7EWwOPHj9GoUaM07Y0aNUJkZCRu3LgBAChQoAD69u2L48ePazui1hw6dAhDhw5VGYhauHBhzJkzB3v27MHLly+hr6+P3377DR4eHli2bJmEaYmIiIiIiIiIiIiISNt0pQ5AWXPgwAGMGDECQUFBAIBjx47B09MT7969Q61atfDLL7+gQ4cO0oYkrVIoFPD29oa3tzeCgoKwZs0arF+/HpMmTcKUKVOQmJgodUTSEj8/P7x58wYAEBMTA5lMhp07d+L69esq/a5cuSJBOu3y9vZGeHg4hgwZgq5du8LR0RG//fYbGjRogP3790OhUEgdUWvyey2MjY3x4cOHNO0fPnyATCaDTCZTtjk4OKjtm1eEhoaiRIkSadodHBwghEBgYCCKFi0KAPDy8sLPP/+s7YhERERERERERERERCQhDijJxQICAtC2bVtUrlwZPj4+mDJlinJbwYIFUbJkSWzbto0DSvIxBwcHTJs2DVOnTsWRI0ewZs0aqSORFm3ZsgVbtmxRafvjjz/U9v38TfS8atCgQQgLC8OECRMgk8nQoEEDHDhwIM8PoFAnP9eidu3aWLp0KXr06AF7e3sAKUtETZo0CQYGBqhQoYKy75s3b1CgQAGpomqcnZ0dLly4gP79+6u0X7hwATKZDNbW1so2hULBAYlERERERERERERERPkMB5TkYtOmTUOlSpVw8eJFhIaGqgwoAYCaNWtiw4YN0oSjHEUmk6Fp06Zo2rSp1FFIS/z9/aWOkCPMmzdP5bZCoYCjoyPevHmDevXqYenSpSrbZTIZfH19tRlRa1iLFJMnT0bt2rXh4uICd3d3GBkZ4cKFC3j9+jUmTJgAQ0NDZd9jx46hcuXK0oXVsHbt2mHevHkoU6YMevXqBSMjI5w8eRK+vr4oWbIkypYtq+z78OFD2NnZSZiWiIiIiIiIiIiIiIi0TSaEEFKHoG9jamqKadOmwdfXF8HBwbCxscHx48fh6ekJAFi1ahWGDx+OmJgYiZMSEUlDLpdnqr9MJkNSUpKG0kiLtfjk9OnTGD58OP79918AgLW1NXx9fTF+/HiV2XoOHToEBwcHODs7SxVVo6Kjo+Hu7o5r166pHLehoSH8/Pzg7u6ubHNzc0OtWrWwaNEiKaISEREREREREREREZEEOENJLpacnPzF5Qk+fPgAfX19LSYiIspZOFPLJ6zFJ+7u7rh+/TrCwsIQFxeHQoUKqe3XrFkzLSfTLmNjY5w9exbLly9HQEAA4uLi4OLigkGDBqFkyZIqfS9fvixRSiIiIiIiIiIiIiIikgpnKMnF3NzcULp0aWzdulXtDCV16tSBjo4OTp06JXFSIiKinOHWrVsoX758hvsPGzYMixcv1mAiIiIiIiIiIiIiIiKinClz899TjtK3b1/s2rULq1evRnJyMoCUJQpiYmIwfPhwnD9/HgMGDJA4JRFRzpScnIxnz54hPj5e6iiSy0+1aNKkCYKCgjLUd/jw4Vi2bJlmAxEREREREREREREREeVQHFCSiw0aNAidO3dG//79Ubp0achkMnTt2hXm5uZYsmQJevXqhW7dukkdk4goR3r//j1KlCiBv//+W+ookstPtYiKikLDhg3x9u3bL/bz9fXFkiVL0KRJEy0ly9nOnDmDadOmSR2DiIiIiIiIiIiIiIi0iANKcrlNmzZh9+7daNCgAcqWLQsrKys0b94cO3fuxOrVq6WOR0SUo3HVt0/ySy3+/PNPvHr1Co0bN0ZYWJjaPqNGjcLChQvRpEkT7Nu3T6v5cqrTp09j6tSpUscgIiIiIiIiIiIiIiIt0pU6AGVd27Zt0bZtW6ljEBER5Xj16tXDjh070K5dOzRv3hzHjx+HkZGRcvvo0aOxYMECNG7cGPv374e+vr6EaYmIiIiIiIiIiIiIiKTDASVERESUr7Rs2RLr1q1Djx494OXlhYMHD0JPTw9jxozB/Pnz0ahRo3wxmKRPnz4Z7nvjxg0NJiEiIiIiIiIiIiIiopyIA0pyuejoaGzZsgUPHz5EcHBwmiULZDIZl74hIlLD0NAQPj4+KFKkiNRRJJcfa+Ht7Y3w8HAMGTIEXbt2haOjI3777Tc0aNAA+/fvh0KhkDqixq1btw4ymSzDyx3JZDINJyIiIiIiIiIiIiIiopxEJjL6LgLlOJcuXULLli3x4cOHdPvIZDIkJSVpMRUREVHuMWvWLEyYMAEymQyenp44cOBAvhhMAgAFCxZEtWrVsH79+q/2/e233zB79mxeUxARERERERERERER5SOcoSQXGzVqFOLj47Fjxw54enrCyspK6khERDlKUlISJkyYAAcHB3z//ffp9vv999/x/PlzzJgxI8/OwsBapJg3b57KbYVCAUdHR7x58wb16tXD0qVLVbbLZDL4+vpqM6LWVKlSBbdu3UKBAgW+2tfY2FgLiYiIiIiIiIiIiIiIKCfhgJJc7MqVK/jpp5/QoUMHqaMQEeVImzZtwq+//opLly59sV+1atUwdOhQlC9fHt7e3lpKp12sRYoffvgh3W2TJk1K05aXB5S4urriyJEjCA4O/uqgEiFEhpfGISIiIiIiIiIiIiKivIFL3uRihQoVwuTJkzF48GCpoxAR5UgtWrRAYmIijhw58tW+zZs3h46ODv766y8tJNM+1iLFqVOnMn2fevXqaSCJ9N68eYP79+/Dzc2NM5AQEREREREREREREVEanKEkF2vXrh2OHDnCASVEROm4cuUKRo8enaG+9evXT7McSl7CWqTIq4NDvoWtrS1sbW2ljkFERERERERERERERDmUXOoA9O1mz56Nd+/eYdiwYXj8+DGnoici+o+QkBAULFgwQ31tbGwQEhKi4UTSYS2+Ljk5Gc+ePUN8fLzUUbTi1q1bmeo/bNgwDSUhIiIiIiIiIiIiIqKciANKcjELCwtcunQJy5Ytg5OTE3R1daGjo6PypavLSWiIKP8yNTXFhw8fMtQ3ODgYJiYmGk4kHdbi696/f48SJUrg77//ljqKVjRp0gRBQUEZ6jt8+HAsW7ZMs4GIiIiIiIiIiIiIiChH4WiDXKxnz56QyWRSxyAiyrFcXFxw9OjRDC31cuzYMbi4uGghlTRYi4zJT7N9RUVFoWHDhjh79iwKFSqUbj9fX18sWbIETZs21WI6IiIiIiIiIiIiIiKSGgeU5GLr1q2TOgIRUY7Wrl07jB49Gvv370ebNm3S7ffnn3/i2LFjmDdvnhbTaRdrQf/1559/olmzZmjcuDFOnToFCwuLNH1GjRqFhQsXokmTJti3b5/WMxIRERERERERERERkXS45A0REeVZAwcORKlSpdCpUydMmDAhzfIeQUFBmDhxIjp16gQnJycMHDhQmqBawFrQf9WrVw87duzA3bt30bx5c8TExKhsHz16NBYsWIDG/9fe/cdaXdd/AH+ee+kK6gVvCQRORe7F28SlJj9ciSJqMLWhl2SDgcWlvCSuFCbWrs1JM82SZM1sU/IqzLWklF8VcA3INcwE/SP7ASg0yi7zgoRNjN3r+f7RvPuezMSUe+6Vx2O72/28z+ucz3OfP4A/nrzOpz+dFStWpKqqqkxJAQAAAACAcigUj6bd7gAcdXbs2JErrrgi27ZtS6FQSP/+/VNdXZ1XX301Bw4cSLFYTH19fVavXp3a2tpyxz2iPIv/7sCBA/nKV76Sm2++OR/72MfKHafbPPLII5k5c2YuvvjirFmzJh/60Idy00035e67786ll16alStX5phjjil3TAAAAAAAoJsplPQiFRUVqaioyGuvvZaqqqpUVFSkUCj81/cUCoV0dHR0U0KAnun111/P/fffn+XLl+f555/PgQMH0r9//5x55pmZMmVKvvCFL6Rfv37ljtktPAv+k/vuuy9z585NQ0NDhg8fnu985zu5+OKLs2rVqvTt27fc8QAAAAAAgDJQKOlFPv/5z6dQKOSBBx5IZWVl1/U7efDBB7shHQD0fJ2dnWlubs6wYcMyZ86ct5277777snv37tx+++2H9XftB8Edd9yR5ubmFAqFTJgwIatXr7aZBAAAAAAAjmIKJQDAUeOhhx5KY2Njnn766Zx77rlvO7dly5aMGTMmS5cuzfTp07sxYfdZtGjRW86+//3vp62tLV/96ldz7LHHlrxWKBRy4403dlc8AAAAAACgzBRKepHKysosW7Ys06ZNS5I0NjamqakpY8eOLXMyAOgdLr/88nR0dGTt2rXvOHvZZZelsrIyq1at6oZk3a+iouJdzRcKhXR2dh6hNAAAAAAAQE/Tp9wBOHwVFRXp6Ojoum5packll1yiUAIAh2nLli2ZP3/+Yc1edNFF/3GLxwfFhg0byh0BAAAAAADowRRKepFTTjklTz75ZGbOnNl1VigUypgIAHqXffv2ZdCgQYc1O3DgwOzbt+8IJyqfCy+8sNwRAAAAAACAHkyhpBeZOXNmFi5cmOXLl+eEE05Iktxwww1pbm5+2/cUCoW88MIL3ZQQAHq26urqtLe3H9bs3r17c/zxxx/hRD3TG2+8kb/85S/56Ec/mqqqqnLHAQAAAAAAyqBQLBaL5Q7B4SkWi2lpaUlra2va2tqycePG1NfXZ/Dgwf/1fVbaA8C/XHDBBenXr1/Wrl37jrOTJk3Ka6+9ll/96lfdkKxn2bNnT4YOHZr169dnwoQJ5Y4DAAAAAACUgQ0lvUihUMisWbMya9asJElFRUVuueWWTJ8+vczJAKB3aGhoyPz587NixYpMnjz5bedWrlyZ9evXZ9GiRd2YrmfROQYAAAAAgKNbRbkDcPgmTJiQJ554ouv6wQcfzIgRI8qYCAB6l6amptTV1WXq1Klpbm7Orl27Sl7ftWtXbrnllkydOjWnn356mpqayhMUAAAAAACgzBRKepGNGzdmz549XdeNjY3Zvn17GRMBQO/Sr1+/rFmzJqeddlruuOOO1NbWpqamJqecckpqampSW1ubb37zmznttNOyevXq9O3bt9yRAQAAAAAAykKhpBcZMmRIdu7c2XVtFT0AvHt1dXV57rnnsnjx4px//vmprKxMW1tbKisrM27cuCxevDhbt25NbW1tuaOWTb9+/fK5z30uQ4cOLXcUAAAAAACgTApFrYRe45prrsmjjz6aSZMmpaamJi0tLbngggsyfPjwt31PoVDIkiVLujElAAAAAAAAANDbKZT0Inv37s28efPS2tqatra2JO+8paRQKKSzs7M74gEAvUhnZ2eam5szbNiwzJkz523n7rvvvuzevTu33357CoVCNyYEAAAAAADKSaGkF6uoqMiyZcsyffr0ckcBAHqZhx56KI2NjXn66adz7rnnvu3cli1bMmbMmCxdutS/OQAAAAAA4CiiUNKL3XbbbZkyZUrOPPPMckcBAHqZyy+/PB0dHVm7du07zl522WWprKzMqlWruiEZAAAAAADQE/QpdwD+d7feemuS5JlnnslvfvObvPLKK3njjTdKZgqFQr7+9a+XIx4A0INt2bIl8+fPP6zZiy66KIsWLTrCiQAAAAAAgJ5EoaQXO3jwYBoaGrJu3boUi8UUCoW8uXDmzd8VSgCA/2Tfvn0ZNGjQYc0OHDgw+/btO8KJAAAAAACAnqSi3AH43y1cuDDr1q1Lc3NzNmzYkGKxmIceeig///nPM27cuIwePTq///3vyx0TAOiBqqur097eflize/fuzfHHH3+EEwEAAAAAAD2JQkkvtnz58lx99dVZuHBhzjzzzCTJSSedlIkTJ6a1tTWHDh1KS0tLeUMCAD3SyJEjs27dusOaXb9+fUaOHHmEEwEAAAAAAD2JQkkvtnv37lx44YVJksrKyiTJoUOHkiR9+vTJtGnT8qMf/ahs+QCAnquhoSGtra1ZsWLFf51buXJl1q9fnylTpnRTMgAAAAAAoCdQKOnFqqur09HR0fV7RUVFXnrppa7XBwwYkLa2tnLFAwB6sKamptTV1WXq1Klpbm7Orl27Sl7ftWtXbrnllkydOjWnn356mpqayhMUAAAAAAAoi0KxWCyWOwT/m/POOy+jR4/O9773vSTJWWedlZNOOik/+9nPUiwWM2nSpOzcuTPbtm0rc1IAoCfasWNHrrjiimzbti2FQiH9+/dPdXV1Xn311Rw4cCDFYjH19fVZvXp1amtryx0XAAAAAADoRjaU9GKXXHJJfvKTn6SzszPJv/6n8S9+8YvU1tZmxIgRaW1tzezZs8ucEgDoqerq6vLcc89l8eLFOf/881NZWZm2trZUVlZm3LhxWbya9xElAAAHaklEQVR4cbZu3apMAgAAAAAARyEbSnqxf/zjH/nrX/+a2tra9OnTJ0myaNGiLFu2LJWVlfnsZz+bBQsWpFAolDkpAAAAAAAAANCbKJQAAAAAAAAAAFDCV94AAAAAAAAAAFBCoQQAAAAAAAAAgBIKJQAAAAAAAAAAlFAoAQAAAAAAAACghEIJAAAAAAAAAAAlFEoAjlLjx4/PsGHDuvWew4YNy/jx47v1ngAAAAAAAMC7p1AC8AHW0tKSe+65p9wxAAAAAAAAgF5GoQTgA0yhBAAAAAAAAPhfKJQAAAAAAAAAAFBCoQSgG7W0tKRQKOSJJ57IwoULc+qpp6Zfv34ZO3ZsnnrqqSTJpk2bcv755+e4447LkCFD8o1vfOMtn/PMM8/kqquuyoknnphjjjkm9fX1uf3229PR0dE1M2zYsGzatCl//vOfUygUun42btxY8lkvvfRSpk2blpqamhx77LGZOHFitm3b9pZ7tre3Z+7cuTn55JNTVVWVk08+OXPnzs3evXvfMrt79+5MnTo1AwYMSP/+/fOZz3wmL7zwwnt8egAAAAAAAEB3KRSLxWK5QwAcLVpaWjJr1qyMGjUqnZ2dmTFjRg4dOpS77747//znP/Pwww9n9uzZufbaa3PKKafkxz/+cTZu3JilS5dmxowZSZI1a9akoaEhdXV1mTFjRj784Q9n8+bNWbp0aRoaGvLoo48mSR5//PF87WtfS3t7e7773e92Zbj00kszePDgjB8/Ps8//3wGDBiQ8847L5/61Keyc+fOLF68OMOHD8/vfve7VFZWJkn+/ve/Z/To0dmxY0caGxvziU98Is8++2yWLFmS+vr6PP3006murk6S7N+/P+ecc052796dOXPm5IwzzsimTZuyefPmHDx4MCNHjnxLqQUAAAAAAADoWfqUOwDA0aizszNPPfVUqqqqkiRnnHFGJk+enKuvvjqbN2/OqFGjkiSzZ8/OqaeemnvvvTczZszI66+/ntmzZ2fs2LH55S9/mT59/vXHeFNTU84666zMmzcvGzduzPjx43PllVfmnnvuycGDB7vKKP+uvb09N910UxYsWNB1NnDgwCxYsCCtra2ZOHFikuSuu+7K9u3bc++99+a6667rmj377LNz/fXX56677urapHLXXXdl165d+eEPf5hZs2YlSa677rrccMMNWbx48fv8JAEAAAAAAIAjwVfeAJTBl770pa4ySZKMGzcuSTJ27NiuMkmSVFVVZcyYMdm+fXuSZP369dmzZ09mzZqV/fv3p729vevnsssuS5KsW7fusHNUVFTky1/+csnZhAkTkqTrnkny2GOPZeDAgbn22mtLZpuamjJw4MA89thjXWePP/54Bg8enGuuuaZk9uabbz7sXAAAAAAAAEB52VACUAbDhw8vua6pqUmSnHbaaW+Zrampyd69e5Mkf/jDH5IkjY2Nb/vZe/bsOewcQ4cOTd++fUvOPvKRjyRJ1z2TZOfOnRk1alTXRpQ39enTJ6effnq2bt3adfbiiy9m9OjRXV+X86YhQ4bkhBNOOOxsAAAAAAAAQPkolACUwb+XLd7p/E3FYjFJ8u1vfztnn332f5wZOnToe87x/+8FAAAAAAAAHH0USgB6kREjRiRJjjvuuFxyySXvOF8oFN6X+w4fPjx/+tOf0tHRUbKlpKOjI9u2bSvZuDJ8+PBs3749nZ2dJYWVv/3tb9m/f//7kgcAAAAAAAA4sirKHQCAwzdx4sQMGjQod955Z/bt2/eW1w8ePJhXX3216/r444/PK6+88p63jVx55ZV5+eWX88ADD5Sc33///Xn55Zdz1VVXdZ1Nnjw5e/bsycMPP1wy+61vfes9ZQAAAAAAAAC6jw0lAL3Icccdl4cffjhXXnll6uvr09jYmLq6uuzfvz9//OMf89Of/jSPPfZYxo8fnyQ577zzsnr16lx//fX55Cc/mcrKykyYMCGDBg16V/ddsGBBHn300cydOzdbt27NOeeck2effTZLlixJfX19FixYUDL7yCOP5Itf/GK2bNmSkSNHZuPGjdm8eXNOPPHE9/NxAAAAAAAAAEeIQglALzNx4sT89re/zZ133plly5bl5ZdfTk1NTWprazNv3rx8/OMf75q98cYb8+KLL2b58uX5wQ9+kDfeeCMbNmx414WSAQMG5Ne//nVuvfXWrFy5Mg8++GAGDx6cOXPm5Lbbbkt1dXXXbE1NTZ588snMmzeva0vJhRdemA0bNuTiiy9+fx4CAAAAAAAAcEQViu/1exAAAAAAAAAAAPhAqSh3AAAAAAAAAAAAehaFEgAAAAAAAAAASiiUAAAAAAAAAABQQqEEAAAAAAAAAIASCiUAAAAAAAAAAJRQKAEAAAAAAAAAoIRCCQAAAAAAAAAAJRRKAAAAAAAAAAAooVACAAAAAAAAAEAJhRIAAAAAAAAAAEoolAAAAAAAAAAAUEKhBAAAAAAAAACAEgolAAAAAAAAAACUUCgBAAAAAAAAAKCEQgkAAAAAAAAAACUUSgAAAAAAAAAAKPF/7rg1lWQ30zwAAAAASUVORK5CYII=", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_methods = [name for name in benchmark_table.index if name != \"interval\"]\n", + "fig, axes = plt.subplots(1, 3, figsize=(17, 4.8))\n", + "\n", + "benchmark_table.loc[plot_methods, [\"L2_normalized_width\", \"W12_normalized_width\"]].plot.bar(\n", + " ax=axes[0], logy=True, color=[\"#4C78A8\", \"#E45756\"]\n", + ")\n", + "axes[0].axhline(benchmark_table.loc[\"interval\", \"W12_normalized_width\"], color=\"black\", linestyle=\"--\", label=\"interval W12\")\n", + "axes[0].set_ylabel(\"normalized norm-interval width (log scale)\")\n", + "axes[0].legend(fontsize=8)\n", + "\n", + "axes[1].scatter(\n", + " benchmark_table.loc[plot_methods, \"total_s\"],\n", + " benchmark_table.loc[plot_methods, \"W12_normalized_width\"],\n", + " color=\"#59A14F\",\n", + ")\n", + "for method in plot_methods:\n", + " axes[1].annotate(method, (benchmark_table.loc[method, \"total_s\"], benchmark_table.loc[method, \"W12_normalized_width\"]), fontsize=7)\n", + "axes[1].set_xlabel(\"total runtime [s]\")\n", + "axes[1].set_ylabel(\"normalized W12 width\")\n", + "axes[1].grid(alpha=0.25)\n", + "\n", + "representative = per_neuron_table.loc[\"B-K96\"].reset_index()\n", + "for layer, group in representative.groupby(\"layer\"):\n", + " axes[2].scatter(group[\"preactivation_width\"], group[\"tanh_prime_selected_rho\"], label=f\"layer {layer}\", alpha=0.75)\n", + "axes[2].set_xlabel(\"preactivation interval width\")\n", + "axes[2].set_ylabel(\"quadratic tanh-prime approximation radius\")\n", + "axes[2].legend()\n", + "axes[2].grid(alpha=0.25)\n", + "\n", + "fig.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "584e1f7c", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABowAAAI7CAYAAADWJbmZAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAT/gAAE/4BB5Q5hAABAABJREFUeJzs3Xd8VFX6x/HP1EzqpE8SQkmhgyCo6CpKs6wVcG3Y1gL6s7uuXdG1rWtbRdYuIvZVXLF3XHVVRBFBeq+hpU+mz9zfH5NMMiRUgQT4vl+vvMK995wz5454mbnPfZ5jMgzDQERERERERERERERERPZb5taegIiIiIiIiIiIiIiIiLQuBYxERERERERERERERET2cwoYiYiIiIiIiIiIiIiI7OcUMBIREREREREREREREdnPKWAkIiIiIiIiIiIiIiKyn1PASEREREREREREREREZD+ngJGIiIiIiIiIiIiIiMh+TgEjERERERERERERERGR/ZwCRiIiIiIiIiIiIiIiIvs5BYxERERERERERERERET2c9bWnsDeau3atbz//vsUFxeTnJzc2tMREREREZGdUFdXx9KlSznxxBMpKCho7em0CfquIyIiIiKy99uZ7zoKGO2k999/n0suuaS1pyEiIiIiIrvA008/zZgxY1p7Gm2CvuuIiIiIiOw7duS7jgJGO6m4uBiIvtm9e/du1bmEQiHcbjcpKSlYrfpPKiKyq+j6KiKy+7SVa+zs2bO55JJLYp/vRd91RET2B7q+iojsHm3p+roz33X0L8JOaijN0Lt3bw477LBWnUswGKS6uhqn04nNZmvVuYiI7Et0fRUR2X3a2jVWpdca6buOiMi+T9dXEZHdoy1eX3fku455N85DRERERERERERERERE9gIKGImIiIiIiIiIiIiIiOznFDASERERERERERERERHZzylgJCIiIiIiIiIiIiIisp9TwEhERERERERERERERGQ/Z23tCYiIiIiIiMjexzAMqqurqa2txe/3YxjGLn+NSCRCKBSivLwcs1nPO0qUyWQiISGB1NRUnE4nJpOptackIiIisk9QwEhERERERER2iGEYrF27lpqaGgDMZvNuCeiYTCasVqsCAhInHA7jdrtxu93U1dVRUFCgvyMiIiIiu4ACRiIiIiIiIrJDqqurqampISEhgfz8fBwOx265YR+JRAiHw1gsFmUYSYxhGPh8PsrKyqipqSElJQWn09na0xIRERHZ6+kTt4iIiIiIiOyQ2tpaAPLz80lMTFR2h+xRJpOJxMRE8vPzAWKZbiIiIiLy+yhgJCIiIiIiIjvE7/djNptxOBytPRXZjzkcDsxmM36/v7WnIiIiIrJPUMBIREREREREdohhGJjNZmUWSasymUyYTCYMw2jtqYiIiIjsExQwEhEREREREZG9koKWIiIiIruOAkYiIiIiIiIiIiIiIiL7OQWMRERERERERERERERE9nPW1p6AiIiIiIjsB2a+Cj2Ggz2p5eMBD8x9B/qO2pOzEhERERER+d2qPUHe/HkVn81dR1Wdn/TkBI7pmc+f+hXiTLK19vS2mzKMRERERERk95r5Krzzf/DaGdHA0OYCnuixd/4v2lb2a9WeIM99s5Qzn/mek8b/j3MmTGfC/5ZR7Qm29tQAqKysJDExEZPJxEsvvbRDfQcNGkRKSkqz/YFAgNNPPx2TycQVV1yBYRgABINB7rvvPrp3705CQgJZWVmceuqpzJ8/f4uvMXfuXEaNGkV+fj4JCQkUFhYyYsQI1q9fv2MnKiIiIiLb5d/TVzHgvs+554N5TFtWyYINHqYtq+Tu9+cy4L7P+ff0Va09xe2mDCMREREREdm9egyn6qWnSVv0DebXzoCz3gBT/VN2QQ+8eQ6RRd9Q4+1Leo/hrTpVaV3/nr6KsVN+wxeKxO2ftqyShz5ZyF2n9OL0g9u30uyiXnnlFfx+P0VFRUyYMIFzzz33d43n8XgYOXIkn3zyCbfddht33303AIZhcMopp/DRRx8xfPhwrrzySjZu3MgTTzzBYYcdxv/+9z969OgRN9Ynn3zC8OHDKSkp4aqrrsLlcrFhwwa+//57ampqcLlcv2uuIiIiIhLv39NXccPkWVs87gtFYsdb+3Ps9lDASEREREREdquqDz6l7IMNVHcspj31QaPTXoagF8u//0xk8besml6MZ8UGOOJT0kcMb+0pSyvYW75sP//88wwePJhTTjmFa665hqVLl1JcXLxTY1VVVXHCCSfw/fff889//pNrrrkmdmzKlCl89NFHjBkzhqeffjq2/9xzz6VXr15cddVVfP7557H9GzZsYNSoUQwaNIh3330Xm23vKX0iIiIisjeq9gQZO+W37Wo79t3fOLZnXpsvT6eSdCIiIiIislulHXcsSYcOwLPCy6of2hNZ9A3WF47FOeVciAWLvCQdeihpxx3b2tOVVrCjX7ZbqzzdjBkzmDlzJueffz6jRo3CarUyYcKEnRpr/fr1DBo0iGnTpjFhwoS4YBHA1KlTAbjgggvi9hcXFzNw4EC++OILVq5cGdv/1FNPUVFRwQMPPIDNZsPj8RAMto0yfiIiIiL7ordmrG6WGb8lvmCEyTNW7+YZ/X4KGImIiIiIyK5lGFC5AuZOgc//hvmtUdRk/4o9N4BndYhVX2dilM3Hsnomq37shGeFF/uAAdScfTrmxMTWnr20gr3ly/bzzz9PSkoKp556KtnZ2Zx44om8+OKLRCLbN/cGK1asYODAgcyfP58333yTP//5z83a+P1+AJKSkpoda9g3bdq02L4PP/yQtLQ0qqqq6Nu3L8nJyTgcDgYOHMj06dN3aH4iIiIism2fzV23g+3b/pqSKkknIiIiIiI7zzDAZIrfN/FEWPFtbHNOVS6flnWlsFMyBwbX49mQwPLPsjFbDbzlfuwHH8TPRXmseu5fmO12eh41dA+fhOxKZz/3A2sqvTvUZ121b4faP/DxfCZ9v3yH+rTLSOSViw/doT5N+Xw+Xn31VU499VSSk5MBOP/88/nPf/7DJ598wh//+MftGicQCHDEEUdQVVXFBx98wNChLf9979mzJwBffvklBxxwQGy/x+OJBYpWrWpcQHnBggWEQiGOO+44TjvtNG6//XaWL1/OPffcw6BBg/jxxx9jY4qIiIjI71frC8X+nIab0yxfM8w8g1STh1ojic8i/XkrPJAaUgCo8bX97G8FjEREREREZPsYBlQshbKZUPYrrJ0JmxbBNbPB0uSrRWYnQvO/w1tpw1dhI6kyTHaSl9Wk482wc1DVOvzV0drdtuwgP6dUsGpeJR16HUCXQw9vlVPbm1VUVHDffffxzjvvsHr1alJTU+nVqxd33XUXAwcO3OPzWVPpZXm5Z7e+hi8U2e2vsbm3336bqqoqzj///Ni+448/npycHCZMmLDdAaNwOMyGDRvIysqiffstr8V0zjnncM899zB27FiSk5MZNmwYmzZt4o477mDTpk1ANHjUoLa2lnA4zNlnn83EiRNj+/v378/gwYO56667eOONN3bwrEVERERkS1Id0e9Ap1m+4i7rCySa4gNCh1nmcr31dcaGLuDN8CDSHG17/SJQwEhERERERLakfAms/SUaIFo7E8pmgb+6hXaLILd7bHP584vwLsyLa9LfVMZPRXmUpybxU1EeBy2Llm+YntOJjRUmOmTB8Guux5bg2I0ntO9ZsWIFgwYNwu12c9FFF9GlSxeqq6uZNWsWa9asaZU5tcvY8bKC66p9212SDsBhNZPn3LG/Kzszr6aef/55cnJyKCwsZPHixbH9xxxzDG+++SabNm0iIyODjRs3xvVLTEzE6XTGbb/66qucdtppDBo0iKlTp9K1a9dmr5eRkcHnn3/Oeeedx5gxY2L7jzrqKG688Ubuuece0tLS4sZ1u93NytsNGjSIDh068NVXX/2u8xcRERGReEf3yKP9ird50PYMhtFyGwdBHrQ9A0D3HpftwdntHAWMRERERET2d5EI1JaBs138/ldOg4olsU3DgJDXjK/Shq/Cjq/SRtCfRNGfK2lalM7asRssXBU3lCPNwpG+ZXxlK6E8NYkfepdgxkSlKUJO0M/J6T9je/s8OOsNsDdfs0Vads455xAKhZg1axb5+fmtPR2AnSr79vy3y7j7/bnb3f6G47px4RFFO/w6O2vZsmVMnToVwzDo0qVLi21efvllhg8fTlFR/LzOP//8uIwfgJNPPpm33nqLP/3pT7GgUbdu3ZqN2bt3b3755RcWL17M2rVrKSgooLS0lBtuuAEgrk9hYSHz588nLy+v2Tj5+fnMmDFjR09bRERERLbilK5JjPrsBQwDaiwmpqSk8FVSIm6zmZRIhMEeLye73aSFDe6yTiTQ84bWnvI2KWAkIiIiIrI/iYShfHF9xtDMaGm5sllgtsCNy2PrERmGQSipO75Za/BW2PBV2fBVJRL2bP7oXISQo5imxRWSDz+cCODv1IHatBQqN8xl48JfWO9tT8hkBqDaZAAGmSYr/eYtZV1dJ9qbv8E89x3oO2p3vwv7hK+//ppvv/2WcePGkZ+fTzAYJBgMkpS09wXc/tSvkAc/nr9dWUYOm5lT+xfugVk1euGFFzAMg2effZb09PRmx2+77TYmTJjApZdeymeffRZ3rKCgoMUxTzrpJCZPnsypp57K4MGD+fLLL+nevXuLbUtLSyktLY1tf/TRR6SlpXH44Y0lHA855BDmz5/P6tWr6dWrV1z/1atXk5ubu72nKyIiIiLbYBgG300ex8mmIP9JSeberAz8ZnNcm+mJDh7LcHJreSUj3HUkLngLDv2/Vprx9mnzAaNIJMJjjz3G008/zfLly8nJyeH000/nrrvuii00uiV33nknf/vb37Z43Gq1Egy2/YWmRERERER2WiQCs//dJEA0C4J1cU0MA8IBE9bK5ZDZmB2x7LlFhKsym7ZsNrytfXtCGzdia5LVMK1iLQuq1hD6cWmTlk7i0pDq2YuLSExIxzNtGqtSDqN91xGYmzeTFnz44YcAdOjQgZNOOomPPvqIcDhM586dGTt2LOecc04rz3D7OZNs3HVKL26YPGubbe86uRfOxD1X/z0SiTBx4kR69+7NxRdf3GKbOXPmcOeddzJ79myGDRu23WOfeOKJvP3223FBox49emy1z+OPP85vv/3GHXfcEfed+Nxzz2XSpEk89dRTHHfccbH97733HmvWrGH06NHbPS8RERER2bqnv15KnzVf8h9nMmNzsthSTTq/yRQ9DoyY/4ECRr/Xtddey7hx4xgxYgTXXXcd8+bNY9y4cfzyyy98/vnnmM1b/jo5cuTIuKewGsyaNYsHH3yQk046aXdOXURERERkzwmHomsJuTdA8VGN+81m+PxvULsWiH6PCdZZ6svK2fBV2vFVOzAnJFB6feOXHJPJhOOAvtR9/U1sn61jB2w9uuMtbEdNSiKVQT/l68s4c7Mb3CazmVDAH7/PMEgzWQk5U6mrriKrfUfMVhvrlizi5+696M8APNOmUfPxJ6SPGL7r35990IIFCwAYPXo0nTt35sUXXyQQCPDwww9z7rnnEgwGueCCC7Y6xqpVq1i9enXcvtmzZwMQCoW2+IBdJBLBZDIRiWz/ukPb8qf+7YgYEe54dy7+FjKNHDYzd57UI9puF77utnz88cesWrWKCy+8cIuvO2LECO68806ee+45+vfvv80xm47zxz/+MVaebvDgwXz++ef07NkTgBNOOIHi4uJYEOmzzz5jypQpHH/88dx8881x4wwZMoQzzzyT119/nT/+8Y+ceOKJrFixgvHjx5Ofn8/YsWP36Pu2pxiGgWEYehhUZAcFg8GtXudFRGTLPp27nn98PJ/XEuq4Nysj+iXL1MLTcRDdbxjcl5XBEG8lSXvwuhsKhXa4T5sOGM2ZM4fHH3+ckSNHMnny5Nj+oqIirrrqKl5//XVGjdpyuYoDDjiAAw44oNn+Sy65BICLLrpo109aRERERGR3C4dg04LGrKG1M2HdbAh5IbUArpsXa2qEQtRWdsQ3pxZfZQK+qgTCvs2ffjMIe32ETBlxXxCSTzoJd5fO1CQ7qAx42bh6JZtWL8VYuziud8Xa1WS37xjbLujSnU2rVuAqKiG3qARXUSn8+DP/nf8Lq+b9RodeB3DCtbdQU1PNf58dz6q5s6F7L4aceLeCRTugtrYWgNTUVKZOnYrdbgdg+PDhFBcXc8stt3D++edv9SG7559/fotVGdxuN9XV1S0eC4VCWK1WwuHw7zyLeKceWMCwbjm8/ctavpy/gRpfiNQEK8N65DKibwHORNsuf81tef7554HoukNbeu3u3bvTuXNn3njjDR588EESExNbbGfUP3m6+TjHHnssb775JqeddhpDhw7lk08+oVevXgwYMIA333yTF198EYiuWTRu3DhGjx6N2WxuNs6ECRPo3bs3EydO5C9/+QtOp5ORI0dy11134XK59vh7tycYhkEoFNri31URaVkoFKKuLppxbLW26duDIiJtyrx1dfzlzbkYBnyWYsNvbjmzKI7JhM9k4j92OHEPfmZxu9073KdN/4vw2muvYRgG11xzTdz+0aNHc9NNN/Hyyy9vNWDUkrq6Ol5//XUKCwvj0vRFRERERNq0oBc+vT0aIFr3WzQ41IRhQNBtIbxpI4m16yHVFT1gNlP24QYintSGlvHjmkzYO3XC2r07htcLGRmxQ+9++QEVa+OzTzZnTUigZtOGuIBR7yHH0HvIMY1T9/t4Z9GEWLBo+A1jwWzBak/gpOtu5YN/3sfK32Yx1WJmuP9EbAmOHX579kcNQYmzzjorFiwCyMjI4OSTT2bSpEksWLBgi+viQPQhumOPPTZu3+zZs7nkkktISUnB6XS22K+8vByTyYTFYtkFZxIvM8XCxQOLuXhgMYZhEIlEMJvNmLb01OZu9uabb25Xu/nz52+zzdSpU7d47IQTTsDj8cTtGzt2LGPHjt2u1wewWCzcdNNN3HTTTdvdZ29nMpmwWq1b/LsqIi1ryCxKS0vDZttzZT5FRPZmZdU+rpsyM5YN/3NWGrBpu/t/nZrC2XvwM0tKSsoO92nTAaPp06djNps55JBD4vY7HA769u3L9OnTd3jMN998k5qaGq666qrd8uVGRERERGSnhYOwYV40KIQJ+p3beMzqgNlvgq8Kw4BArSVaTq7Chq/Kjq8ygUjAwF6QRYk1IdbNZDbj6NkLz/TpYDZjLy7C1LULdfkuqhPtVHjq2LhqBb7qtVyWnx83nZxOxXEBI3tiUpOsoRJyi0rJKCjAbN765+qFP/yPlb/NigWLbAmO2I0qW0ICw28YyzsP3MXK32ax8If/0fOoob/7rdwfFBYWApDXZP2oBvn1/y0rKyu3Okb79u1p3759i8esVusWbyI2ZC1tLXtpV2gooWYymXb7a8neyWQyYTKZdMNbZCc0XOf1/4+IyLbV+UNc+spMNtRGS28P6+6i2pkO1dsfMHInJO/Ra+7OZJC26YDR2rVryc7OJiEhodmxdu3a8d133xEIBOKeptuW559/HpPJxIUXXrjdfXa2rveeorqzIiK7h66vIrJbhQOwYR6mdb9iKvs1+nvDXEzhAABGZjGh3mc2Nq+tpWpOPr4VNvxVdiKB5mXlAAJlFfgDJszW6LXLMAxq/3g0mw46gEqPm42rVlC7eiGsXthsSuVr1+DMdcW2iw8aQEpmFjmdSsgtKsaZ48K02U37cDhCOLz1dVG6/OFIwqEQpQMOB7OFYDAYd4212WyccO0tLJ72P7r84cg9et3dmbrebcUhhxzCU0891ey7ChDbl5ubu6enJSIiIiKyTwlHDK5+fSZzy2oA6JGfxmNn9OHKtzbs0DipjoxtN2plbTpg5PF4WgwWQTTLqKHN9gaMFixYwLfffsvQoUMpKira7nnsbF3vPUV1Z0VEdg9dX0VkdzB5NpH23kVYyxdiigTijhkR8Nda8VXYMK1cS3DDaoyEaCk5IxSicqYHgjaalZWzWDB36kiwuIhgYTuqq6sxNwm6fPPfz/BUVWxxTo7UNLI6dKKqohyalIPL7dqT3K49Y9s19Wvm7IyCPv3x+Hzg8wEtX2ML+vTf45+td6aud1sxfPhwrr76al5++WVuu+22WMmJsrIy3nnnHbp06UJpaWkrz1JEREREZO92/0fz+HzeegByUxN4/s8Hkfz9QwzesILpWdsfBBrcfvDumuIu06bvfiUlJbFhQ8tROl/9F82kpKTtHq9hsdKLL754h+axs3W99xTVnRUR2T10fRWRnRLyYVo/N5oxtO5XjLRCIgP/2ng8NQVr1VIIBfDVWPFV2up/HPiqbBjBaDAooUsR7bPzoUm5t9quXfHPn4+tpJhQ51Jqc7KosZkpr61m48rl+DYsJzlYw0WblZZzFZewbEY0YJSSlU1up2JyO5WQ06mY3E7FJGdk7v73ZTNt5Rq7M3W924qMjAweeughLrnkEg499FAuvPBCAoEATz75JIFAgMcff7y1pygiIiIisld77ceVPPvNMgAcNjPPn38w+cvegf/ezxFWCw9kpsN2rLXpsDg4ufTk3TvZXaBNB4wKCgqYO3cufr+/WabRmjVryM7O3u7solAoxKRJk8jKymLEiBE7NI+dreu9J6nurIjI7qHrq4hsVdAL6+fA2l+i6w6t/RU2zoNIkzJnrl5Yhtwc2wys3sCKL/PwrfdjhFouKwcQWLYaq8mMqf76EwoEWD70CDb1KmHDimX4Vy2AVc2nVFdZQaDOTXJ645Nuh5zyJ/oddxK5RSUkpbWdheHbwjV2b88gHTNmDNnZ2TzwwAPcfvvtmM1mDjvsMF599VUOP/zw1p6eiIiIiMhe63+LN3H7O7/Fth89oy+9Q7Ph3SvxmEzcnJu9XcEigFsG3EKaPW13TXWXadPfjg4++GA+/fRTfvzxRwYOHBjb7/P5mDlzJkceeeR2j/Xee++xfv16rr766i2WuRMRERER2SLDiP8ysH4uPHUEGOH4ZhHwV9dnDlXYsWeWkTkmCJZoUMSalYm3LACRzYJFNhv2Lp0JlBRTm5WBrV0BWBqziyw2G/N++gGfu4WycCYTmfntyC0qwVVUgrlJP4DCbj2b95F9xsiRIxk5cmRrT0NEREREZJ+xeIObS1/+mVD997Ybj+vGcXlueO5swpEgN+ZmM6c+zpCbmEt1oBp/2N9sHIfFwS0DbmFE5x1LYmktbTpgdMYZZ3Dffffx6KOPxgWMnn32WTweD2effXZs35IlSwgGg3Tr1q3FsRrK0V100UW7d9IiIiIisvcL1MG62bB2ZjRzqOxXKDoK/nh/Y5vMYoyICX+VFW+FPRogqnLgr7RihBuDQYn9+5NpacygMScm4ujenbAJ/CVF1GY6qTJBeWU5m1atILRiHqyA9LX5HHjOn2P9TCYTuUUlrJozi6zCDriKSsht+OlUjN2RuPvfFxERERERkX1cRV2Ai16cTq0vWjni9IMKufSgNHh+GPiqeCgzna+So0vlZCdm8/LxL5NkS+LdJe/y5covqfZW40x0MrTDUE4qOQlnQtup8rAtbTpg1Lt3by6//HLGjx/PyJEjOf7445k3bx7jxo3jqKOOYtSoUbG2Q4cOZcWKFRjG5mU9YO3atXz88ccccsgh9O7de0+egoiIiIi0dUFvY2Co4femhdFUoabsyXGb3rkLWPGWKy44FBW/HVi2DMMwMNVnJ9VVVfJtaQGbVq0ksnweLG95WlXryvB76khIanzd4/7vGhypqdjsypgXERERERHZ1fyhMJe+9DMryj0AHFqcyT0ndsb0ynCoXM4raSm87IyWlku0JjJ+6HjyU6JryJ7b41zO7Hwm1dXVOJ3OvXJ5gzYdMAJ49NFH6dSpE8888wwffPAB2dnZXHnlldx1112YzebtGmPixImEw2Euvvji3TxbEREREWnT/LUQCkByVuO+NTNg4vFxzSJh8Ffb8FXU/1Q5SOkVJOeixrJ09o4dmwWLTAkJWLp1xdupIzXpKVQZYTKKS+nSpJRdYloaFWVriIRDcX2tNjs5HYvILS6NZQ/ZEhxxbVKzsnfFuyAiIiIiIiKbMQyDm9+ezY/LKwAoyk7mqbMPxP7e/8GqaUxNSuQfmZkAmE1mHjjyAXpm7Vvlv9t8wMhisXDddddx3XXXbbXd8uXLt3jslltu4ZZbbtnFMxMRERGRNs1XDWWzmmQO/Qrli+EPV8Ixd8eaRTK74i+PBoa8lTZ8VYn4q8ywWYKRJdIlbg0jS3o6CcOGUpOcSK0zlcpIkE2bNlC5bi0snxtrl1NVweFnnhvbNpsttOvag1DAj6uoNLbuUGa79s3WHhIREREREZE944mvlvD2jDUAOBNtTPjzwaT/8CDMeZs5dhs35mRj1H8lvPHgGxnUflDrTXY3afMBIxERERGR7VK5HOa+2xggqljSrEkkBMayn2kalqn7aTarP8vZ6tCmxEQMuz1u38YVy/jPxuWwcevTcldWEAmH44JBf7r17q30EBERERERkT3pg1llPPjJAgBsFhNPndOfolX/gW8eYq3VwhV5uXjN0WjROd3PYVT3UVsbbq+lgJGIiIiI7F28ldHMIWchZJU07t+0CD67PbYZCZnwVVmjJeUq7fiqHPirzOSckknTwm6OnvElBExJSRjdu+Jp346alEQqgwE2biij9KBedGzSLqOgELPFQiQcju1LTE3DVdyYNeQqLiUtxxVbv0hERERERETalpmrqvjLv2fGtu8d0ZvDTL/Be1dTazJxuSuHTfUPAA5uP5i/HvTXVprp7qeAkYiIiIi0XZ6KaCm5WFm5mdFMIoCjboLBN8eaBsK5uBcmx9Yc8lebwWg+pK8uI247nJxE3WkjqLZbqAz42bh+LZ7qKlhWE9du/bL4jCWrzUafY47HkZxCblF03aGUzCwFh0RERERERPYSa6q8XPziT/hD0ZrklxxVzOkdPfDcuQQjIa7Ny2VxfbWJXlm9uH/g/VjM+24pcXNrT0BEREREJM7yb+Hf58GjB8ADRfDScPj8Tpj7DlQuJxw04dlgJzj/x7hu3iWrWT/DSfXypOgaRJsFi0wpKUQO7k+oc2nc/k2rlvPfhbOY+dsvrFg4Nxos2kxajot0V36z/UP+fAl/OO1sSg8aQGpWtoJFIruCtxK+/xdMPBHTM0dhefkU+OHJ6P5W9NVXX2EymeJ+HA4HxcXFXHDBBcybN2+7xrnzzjsxmUz89NNPzY498sgjmM1mDj/8cKqqqmL733zzTf7whz+QnJxMamoqAwcO5MMPP9zia1RUVPDXv/6V0tJSHA4HOTk5DB48mG+++WaHz1tERERkX+X2h7ho4nQ2uf0AHNvTxY2HZ8Irf8LwV3NXdibTEh0AFCQX8PjQx0myJbXmlHc7ZRiJiIiIyJ5Xt6kxY6jLsZDXu/GYpxzmTgEgHDThq7TVl5Wz4atKJFAdDcrkFvckq8mQiU1Ky5nS0gh264K7wEVNYgKVPg8by9YQ8FZxUGYqnZr0y+1UHDe19Lx8XEUNZeVKyS0uITEldZeevohswYyX4MO/QsgHgKn+hxX/gy/vguMfgn7ntuYMOeusszj++OMB8Hq9zJo1i+eee47Jkycze/ZsOnbsuI0RWnb77bdzzz33cOyxx/L222+TlBS9GfGPf/yDm266iQMPPJC7746uf/byyy9z4okn8tJLL3H22WfHjbNixQoGDRqE2+3moosuokuXLlRXVzNr1izWrFnzO85cREREZN8Rjhhc9dovzF9XC0Cvdmn8c2RXzK8Oh6qVPOtM453UFABSbak8MewJshOztzLivkEBIxERERHZvdwb40vKrZ0JNasbj5utcQGjutUGVd+lR4NDNVvO2PGtD8RtVxlhlp85nAqPm03r1hLyV8Ky5hkJGzYrLWdPTOLoMVeSkV9AbqdiEpKSd+IkReR3m/ESvHvFlo+HfI3HWzFo1K9fP84555y4fZ07d+bqq6/m7bff5tprr92h8QzD4KqrrmL8+PGcfvrpvPTSS9jry56sX7+esWPH0qtXL6ZNm4bNZgPgyiuvpF+/flx55ZWcdNJJpKWlxcY755xzCIVCzJo1i/z85pmRIiIiIgL3fDCXL+dvACAvzcHz5/Un6YNLYc1PfJCcxOOZ6QBYTVb+OfiflKSXbGW0fYcCRiIiIiKye8yYBFP/DrVrmx0KBxozh1LzfsB+ROOxYG2YmpVbSPNPT8ffrQtuVzaJhx0ad6iuuoq582a32M1ssZBV2IHcohIKu/VsdvyAocdu/3mJyK7nrYxmFm2Pj66H7idCYsa22+4hBQUFALFAz/YKhUJccMEFvPzyy4wePZqnnnoKs7mxcvx3331HIBDg7LPPjgWLAGw2G6NGjeKWW25hypQpnHtuNID29ddf8+233zJu3Djy8/MJBoMEg8FYtpKIiIiIwEs/rOCF/y0HINFm4bnzD8I1/QGYO4WfExK4PaexlsWdf7iTAfkDWmmme54CRiIiIiKyc2rKohlDZb9Gs4YG3QgFBzYet9ihdi1hvwlvpb2xtFxVIsHaxmbm44fR9Baro1cvAIysTHxdOuPOzaLaZqGiroaK9WVEvOWwvBzzQf3iSsu5iqJPfFmsVrI7FOEqKsFVHC0tl92+I9YdvJErInvQzNdiZei2KeiFX1+HQ/9v985pCzweD5s2bQKiJel+++03br31VrKzszn11FO3exyfz8epp57Ku+++yw033MA//vGPZm38/mg9/ZYCPg37fvjhh1jAqGFdow4dOnDSSSfx0UcfEQ6H6dy5M2PHjm2WGSUiIiKyv/nvwo3c+e4cAEwmGHfWgfRa9w58+0+WW61c7cohWL827aV9LuWU0lNacbZ7ngJGIiIiIrJ1hgG1ZfEl5cpmgnt9fLuSIXEBo4ppZVR8UBAXHGqJb/HKuO0Vm9bx/TFHULVhPYZnEyzf1GK/9ZuVlktypnPeg+PJLCjEYtXHXJFW8+LJUL1qx/rUNM9E3KrP74Qfn9mxPs72cP67O9anBXfccQd33HFH3L4ePXrwzTffkJeXt93jnH/++SxdupT777+fG2+8scU2PevXZvvyyy+56qqr4o5NnToVgFWrGt/rBQsWADB69Gg6d+7Miy++SCAQ4OGHH+bcc88lGAxywQUXbPccRURERPYli9bXcsUrMwhHDABu+WN3jk6YA29eS4XZzGV5OVRbotneJxafyGV9LmvN6bYKfZMWERERkUZG9IMzpiZrB31yK/zwr2ZNQz5zY9ZQpY2ckt9IaJqpn5zbYrAonJuDr7SY2uxMqqwm+p9wXNxxs9VG5fqyZv3siYnkdioht6gEV1EJeaVdmrXJ6dBpe85SRHan6lVQsXT3vkbIt/tfYwvGjBnDaaedBkSzhObOncvDDz/M8ccfz9SpU+nYseN2jVNWVobVaqVLl+bXsga9e/fm6KOPZsqUKdxwww2xYM/EiRP56KOPgGjGU4Pa2uhFNzU1lalTp8ZK5A0fPpzi4mJuueUWzj///LiydyIiIiL7g3K3nwtfnE6tPwTAWYe05+KuPphwPj4iXOXKZVV9CeD+rv787Q9/w2Ta8pq6+yoFjERERET2V4YRvbEblzn0K5z7H8g/oLFddudocKjChrc+OOSrSiRUFz9cSuoQEppsJ/bsSSjPhbekiNqsdKrMUFFdSU3FJqir/wHarV9Lhyb9XMWlOJJTyC0uxVXUGCBKd+Vj0k1OkbbP2X7H+9Ss3f6SdABWB6QV7Nhr7My8WtC5c2eGDRsW2z7xxBM56qijOPTQQ7nxxht5/fXX2bhxI+FwONbGYrGQk5MTN86ECRO4/vrrOeOMM3j99dcZOXJki6/3xhtvcPHFF/PQQw/x4IMPAtCpUyf+9a9/MXr0aNLS0mJtExMTATjrrLPi1lPKyMjg5JNPZtKkSSxYsIDu3bv//jdCREREZC/hC4YZ89LPrKrwAnB4aRZ3Dc3BNOEYIv4abs3J4ldH9Ntsp7ROPDb4MeyW/bOkuQJGIiIiIvuL6tWw5uf4AJG3onm7splxAaOyf8+k6t2tl1kyAP+yZXH7fp4/m2muFHBvjP5sQWVZfCmq1OwcLnv+tf3yaS6RfcLOlH37/gn45Obtbz/szlZbw6glAwYMwOl08uWXXwJw8MEHs2LFitjxjh07snz58rg+paWlfPXVVwwaNIgzzjiD1157jT/96U/Nxs7IyGDy5MmsX7+ehQsXkpKSQp8+ffj4448B6NatW6xtYWEhQIul8fLz8wGorKz8fScrIiIishcxDIMbJ8/i5xXRz0DFOck8cVp3bG8Mh+qVPJbh5NOUZAAyHZk8MewJnAnOVpxx61LASERERGRfYxhQuRzSO0LTjJwv7oJZb8Q1C3kbysrZY5lDHQ+rpumzVPauBwCfNvYDAoUFeIs6UpueRiVhyis2ccrx8aXlMtsVNptaSlY2rqISXEWlscyh5IzMuDYKFInsh/qeBV/8bfuyjGyJ0Oes3T+nHRQKhfD7/QC88soreL3e2LGGzJ/NlZSU8N///pdBgwZx1llnYRhGrNzd5lwuFy6XK7b94YcfAnD88cfH9h1yyCE89dRTrF69uln/hn25ubk7eGYiIiIie69xXyxmyszoQ4oZSTZeOL8/zo8uhbUzeDM1mQnp0eBQgiWBcUPG0T5112Sl760UMBIRERHZmxlGdB2PWEm5mdGycr5quOJnyC6NNQ06SvGtduCrrC8tV5VI2GtsPiA+usYFjEIlxVQOPIwaZyqVRojy8o34vR6o2RD9qbdh2WLadW0sc5TfuStdBhweCwzlFpWQ5EzfDW+CiOz1EjPg+Ifg3Su23faPD0Ji+m6f0o747LPPqKur4/DDDweI/d4excXFfPXVVwwePJhRo0ZhGAann376Vvv89NNPPPfccxx11FEcccQRsf3Dhw/n6quv5uWXX+a2224jJSUFiK6X9M4779ClSxdKS0u3NKyIiIjIPuXdX9fyz88XAmCzmHj63IPoOOMfMP99vk10cG9W9OFFEyb+PvDv9Mnp05rTbRMUMBIRERHZ2yz8BJZ/Ux8gmgX+6rjDDZlDtrKZcQGjdZN/w/2/ptk88cGiCODv2J6wzxu3f9aiOczaLDi0uYz8dpgtlvh9eQWc9JcdKDElIvu3fudGf3/415YzjWyJ0WBRQ7tWMmPGDF5++WUA/H4/c+bM4ZlnnsFms3HPPffs1JgNQaNBgwYxatQoIpEIZ555JgC33347ixYt4pBDDsHpdDJjxgxeeOEF2rVrx0svvRQ3TkZGBg899BCXXHIJhx56KBdeeCGBQIAnn3ySQCDA448//vtOXkRERGQv8fOKSv765q+x7ftHHsAhm/4D3z3OAruN63KzCddXt/hL/79wdMejW2uqbYoCRiIiIiJtUSQC5Yth/W/QcwQ0LdP2y0sw7z0gGhwK1lnqy8rZ8FXa8VU5CPsMOo9Kjvuw5zhwAO7/TY8ObwJvpw542rejNjWZylCA8vKNhIIBCvv0jpuKq6gx6GQymclsV4iruDSWNZTTsZiEpKTd9laIyH6k37nQ/USY+Ros+BDDV42RkAbdTsDc96xoJlIre+2113jttdcAMJvNZGVlccwxx3DzzTdz8MEH7/S4RUVFsfJ055xzDpFIhFGjRtGvXz+++OILPv30UzweDx06dODKK6/k5ptvJj09vdk4Y8aMITs7mwceeIDbb78ds9nMYYcdxquvvrpDmU8iIiIie6tVFR7GTPqJQCgCwOWDSzg1bT68ej3rLRYuc+XgqS/ffkbXMzi/5/mtOd02RQEjERERkdYWCUeDQw0l5dbOhHWzIOCOHm8/AJztYs1rN2bi/TU1LjgUL7rtq0kmpcne9bmZLD1+CJVBPxXlG4mEw1C9AeITlFi/bDG5nYpj2x1692Xohf9XHxzqhC3BsavOXESkucQMOOwyOOwyjEiEcDiMxWKJX5OtFQwaNAjD2Px6u+PuvPNO7rzzzhaPderUieXLl8ftGzFiBCNGjNih1xg5ciQjR47cyRmKiIiI7L1qfEEuenE65XUBAI7vncd1BwThhT9TR4QrXC42WKNhkYHtBnLTITdpHd0mFDASERERaS1f3Q9LpsK62RCsiztkGBB0W/DXWEktmxkXMKr4YSOeeakNLeP6hSxmvEUdqWuXT8fUlLhjqzasZdGaFS1OxWKzkdOxCFdRCZn5hXHH0l159D32hJ07RxEREREREZE9IBSOcMWrv7BwffThyz6FTh4+zoX5xWMIBWr5qyuH+QnRFXu7ZXbjwaMexGpWiKQpvRsiIiIiu0s4BJsWRrOGKpbBkFvjj6+eDqt+iAWHvA0l5Srt+CoTiASiwaAudxTTdHWgxL798fw0k6DFgqe4I3UFedQk2qnwe6mqLI9Gm6rW08flommhuNyiUuZ+MxVbgoPcomJyi0pwFZWSW1RCZkEhFqs+GoqIiIiIiMje6a735/L1wo0AFDgdPDuqO4lvjsCoWcP9WRl8m5QIQG5SLuOHjCfZltya022TdFdAREREZFcIh2DTgs3Kys2GkLexzaH/B0mZsc2NP5vx/JyNr6oxONSocdu3ppbkJkk/s+yw7KiDqKmqjO6oWg9Vzae0YdliMgsaM5O6HX4knfr2IyO/ALPZ0ryDiIiIiIiIyF5o4v+WMen7aEWNZLuF58/rR+4nl0HZr0xKS+WNtGiVjiRrEk8MfQJXsqs1p9tmKWAkIiIi8nu9NAJWfAchX9xuIwIBtxVfhY2Q10xW2a9QMjh2vG6tCe8GO5uXlfPZbXiLOuLOy8GTnEhJh/Zxxz2hQGOwqAlHahquohJcRSXkFpVS2L1n3PHk9AyS01t/wXgRERERERGRXWXq/A3c9f5cAMwmeHzUgXSf/QAs+JDPkhJ5KCv6PdhisvDwoIfpmtm1NafbpilgJCIiIrI1oQBsnNeYOeSrhj9N2KyNHyPgI1AbDQ55K234KhPwV9mJBOuDQRYLGXkH0XTJ9oSevalYsABPp464XdlU261U1NXgqXMDBlRtgCrw223Ym/RzFZWyduH8+sBQSX1puRJSs3K0WKeIiIiIiIjsN+avq+HK134hUv/V+7YTejCkegr88AS/Jti5OTc71vbWQ2/liHZHtNJM9w4KGImIiIg0CAVgwxwo+7UxQLR+DoQDjW1MZjj5cbBHax0bkQgr367Du7wdRnDLZeUIh/EvWUbiAQfEdn1SuZryru2BSDQ4tAWbVi4nNavxQ+7BJ5/KgBGn7/x5ioiIiIiIiOzlNtb6uWjiT7j9IQDOObQDF+QsgNdvZJXVwlWuHPz1D1Ve0OsCTutyWmtOd6+ggJGIiIjsnyIRMDfJ94mE4YFiCNTGNTMi4K+JZg75Km0YWMmvWAp5vQEwmc2ELZkYwcaAjwHUJSfi6dQed04mVRYzQbOZrt26xY2dlptH+do1cfvSXfnRjKHi0mj2UKdiktKccW3MFq0/JCIiIiIiIvsvXzDM6Ek/saYqum7wwM7Z3HlwBNPEC6k2wWWuXCrqvzsf0/EYrul3TSvOdu+hgJGIiIjs+4K+aKZQ2S+NmUO2JLjo08Y2ZgtGRgn++b/hq7BHy8pVJeCvsmKEGpuZHA7ysrvTUPjNMAx83buwwWpQm5VBlRkqaqsJBQNAGCo3xvr6/T4S7Y3F5YoOPIiE5JToukPFpeR0KsaRnLI73wkRERERERGRvVokYvDXN39l5qoqADrnpvDESXlYXzqWQLCOa/JyWW63AdAnpw/3HnEvZpN5KyNKAwWMREREZN8SDsHaX6JBobKZsPbX6BpEkVBcM8NsxxQKgDUawIl4vSx8ohwjmLvloU0QMENw7VrsHTrUD2TwRdkyApYQVG1ssZ/JbCarsAN1VZUkpqbF9h943EkceNxJv+t0RURERERERPYnj36+kPdnlQGQlWznhVHdSX17JEZtGXfkZPFTogOAwpRCxg0Zh8PqaM3p7lUUMBIREZG9V8ADnk2Q3qFxXzgAE46J1pKrFwmDv9oWLStXYcNX7cDiTKGDpxzS8gEwJyZiKygksGJFdBiTidq0FOo6FVKbnkYVEapqqsjIb0e/Do2vZzKbyS0qZvXc3wCwWK1kd+gULStXVEJuUQnZHTphsyfsgTdEREREREREZN/1n19WM+7LxQDYLWaeOacPhV9cDutm80S6k/dTousNOxOcPDnsSTIdma053b2OAkYiIiKydwjUwbrZjSXl1s6ETQug/aFw4UexZobJhi9cgm/pGnwVNrxVDvxVFojED2cO2TFS82Kl5YJ+H6sP7EV5xzwqjRDVtdUYhgEEoGpTrF/F2jUEfT5sjsYnlA487iR6DBwSDQ6174DFattd74KIiIiIiIjIfmn68gpufGt2bPuBU3vTf+4DsOhTpqQk81RGdP1fm9nGY4Mfo5OzUyvNdO+lgJGIiIi0Td4qmPlqfWm5X2HTwrisIYhWmQsvnY0tEgFztB5xuKaG5f+uA9JbHDZoMVPjTMXWoQORmhoszugHSrPFyi+rlxAOBlvsZ09MJLdTNGMoGPDHBYy6DDj8956tiIiIiIiIiGzBynIPl7z0M4Fw9L7AVUM7MzzwHkx/lmmOBO7Mbswkuvvwu+nv6t9aU92rKWAkIiIirctXA+tmgdkKHQ5t3G9E4JObY5uRkAlfVX1JuUo7vqoE/FUWUnrk0z7kBXs07dyalYU1L4/QunX4rRZq01Opa9+OmtQkqkJBautqAXAVtWNAfbAIoqXkcjp0Yt2SRSQkJ9eXkyuN/c7Iy8dk1iKZIiIiIiIiIntStTfIBRN/pKIuAMBJfQq4tnAhvHEzS2xWrnXlEDJF64dceeCVnFB8QmtOd6+mgJGIiIjsOb5qKJvVWFKubCaUR2sPUzoMzpkcaxrym6hZVYhvjRtflQN/tRmMFobcGI4FiwDcFeXMPLAb5VW51HnqojsjPqj2xfXbtHIZ4VAIi7Xx49Cwiy/HkZJCWo4Lk8mEiIjsedX+aqYsnsJXq7+iNlBLijWFwR0Gc0rpKTgTnNseYDfyeDw888wzTJ48mTlz5lBbW0tmZib9+/fn9NNP55xzzsFq3frX7K+++orBgwfz4IMP8te//jXu2H//+19OPvlkkpKS+OSTTzjggAMAmDFjBn/729/49ttvqauro6SkhNGjR3PllVdisViavUYoFOKJJ55g4sSJLFiwAKvVSklJCZdccgmXXHLJDp3zunXrePzxx/n4449ZsmQJdXV1ZGRk0Lt3b/74xz9ywQUXkJWVFWv/4osv8uqrrzJnzhw2bdpEamoqpaWlXHLJJZx77rlx812zZg2TJk3i448/ZuHChdTU1NCpUyeOP/54brrpprhxRUREZP8UDEe4/JUZLNkY/X5/YId0Hjo8guml0WyymLjMlUtt/cOdI0pHMLr36Nac7l5PASMRERHZvSqWwhd3R4NDFUubHY4Eo5lDJs9MEg0D6gM1kToP6/8XAZLi2huA12alNtNJXbt8nAXtKA0GMdmi6wYlJCezqmwNxmbl6wBSMrPILSqJZQ1tHoFyFZfuijMWEZGd9J9F/+HeaffiD/vj9v+04SfG/TKOWwfcyojOI1plbosXL+aEE05g4cKFDBs2jJtvvpns7Gw2bNjA559/zgUXXMDcuXN54IEHdmr8999/n9NOO428vDw+//xzSkpKAPj666855phjcDqdXHXVVeTk5PDZZ59x7bXXMnfuXJ555pm4cQKBACeffDJTp07l7LPP5tJLLyUUCrFo0SJWrFixQ3P6+OOPOfPMM/F4PIwcOZJzzz0Xp9PJpk2b+P7777n11lt59tlnWbBgQazPjBkzyMjI4PLLLyc3Nxe3280HH3zABRdcwDfffMPzzz8fa/vee+9x5513csIJJ3D99deTmprKjz/+yKOPPsrrr7/O9OnTycvL26n3U0RERPZ+hmFwx7tz+HZxdF3hdumJPHdKPgmv/RFvyMuV+bmstUVDHAPyB3D7Ybfr4c/fSQEjERER+f28ldF1htbOhKxS6H5i4zGrA+a8DUA4aMJfacNbWV9arspBoDr6JFDqHw6g0IiAKfrksa1dAWankxqvJxocKsijJimBSr8Xf6D+RmLYS2GKIxYsArAlOMhsV0jQ768PDJXgKi4lt1MxyekZe+TtEBGRHfefRf9h7Hdjt3jcH/bHju/poJHX6+XEE09k6dKlTJ48mZEjR8Ydv/HGG5k+fTrTp0/fqfFfffVVzj//fLp27cqnn35KQUFB7NhVV12F2Wzm+++/p7i4GIDLLruMSy65hGeeeYbzzjuPI444Itb+7rvv5vPPP+ezzz5j8ODBOzUfgDlz5nDqqaeSlZXF999/T/fu3Zu1Wb9+PePGjYvb99hjjzVrd/XVV3PCCSfwwgsvcO+998aCQAMHDmTFihVxQaHRo0czYMAARo8ezUMPPcRDDz200+cgIiIie7fnv13Gq9NWApCSYGXiqG5kTfkTYfc6bsrN5reEBABK00t5ZNAj2My2rQ0n20EBIxEREdkxnopotlBDgKhsJlQubzze/aS4gJF/g4dN0/PwbTIRqN7ykz6+1VVgbixTs37pYj7pUkAwEK1RTNgLtd5m/TYsX4oRicStL3T23/+JzZ6wc+cnIrKL+P1+pk2bRn5+Pp07d27t6bRp1f5q7p1273a1vW/afQzpMGSPlqd77rnnWLBgATfeeGOzYFGDgw8+mIMPPniHx37yySe5/PLLOfjgg/noo4/IzGxcsLmyspJff/2VY489NhYsavDnP/+ZZ555hhdeeCEWMKqrq+Oxxx7jlFNOYfDgwRiGgdvtJjU1dYfnNXbsWDweD++8806LwSIAl8vFvfdu33+3jh07YhgG1dXVsQBRz549W2x7xhlnMHr0aH777bcdnreIiIjsGz6fu557P5wHgNkE/zqzN52/vhI2zOHhzHS+TI5WI8lOzOZfQ/9Fmj2tNae7z1DASERERLbPyh/g7dFQtbLZoXDAhK/Shq/SRkLNL6ScEX+8Zok5bjsCuB12arPSceflUuOw0aFzD0oMI5Y+7nTlNQaL6plMZjLbFcZKyrmKSsjpVBwXLAIULBKRNsFisTB06FAefvhhBYy24d0l7zYrQ7clvrCP95a8xzk9ztnNs2r01ltvATBmzJhdOu7f//53brnlFoYMGcKUKVNISUmJO+73R9+TpKSkZn0b9v3www+xfd988w21tbX079+fq6++mgkTJuB2u8nOzmb06NHcdddd21xjCcDn8/HBBx/QsWNHjj766J06t+rqaoLBIJWVlXzyySdMmDCBLl26UFq67fKvq1evBqIBKREREdn/zF1bw1Wv/4JRX0X+zpN6cNSSB2Hx57yamsJLzmhwKNGayPgh4ylIKdjKaLIjFDASERGRqLpN9RlDv0R/dz0eDjy78XhyDlStjAaHKmz4Ku14K2z4qh0Eaxozh5ztj6Hp7S5zu3ZUp6dRk5pEnSuHaruFSm8dkUj9GkMhL7i9JBvBuFrDiSmpFB14EEnOdFzF9cGhDkXYHI7d+jaIiOwqVquVvLw8DMPYduN9yMWfXkyZu2yH+qz3rN+h9o/OeJTX5r+2Q33yU/J57pjndqhPg99++420tLRmWT6/x5NPPsnSpUsZPnw4r7/+OgkJzR92cLlcZGdn88MPP+D1eklMTIwdmzp1KgCrVq2K7WtYS+jRRx/FbrfzwAMPkJWVxSuvvMLf//531qxZw4svvrjNuS1atAi/30+fPn2aHfP5fLjd7rh96enpzQJRQ4cO5eeffwbAZDIxbNgwnnrqKSwWC9tyxx13AHD++edvs62IiIjsWzbU+Ljoxel4AmEA/vyHTpzHB/DTBL5KTOQfWdFS8yZM/GPgP+iZ3XLGsuwcBYxERET2R+4N9cGhX6Ml5dbOhJrV8W3syXEBI/e8MtZ92I5gzZZvfIbMJmoWLqXpsz0r587ifx1zohtBDwSb97PYbLRUrG7kTXdu3/mIiLRRp512Gv/+97+58sorMW+WDbmvKnOXsbK2eTbqruQP+3f7azRVU1Ozy7NdysqiQbWSkpIWg0UQDbRce+213HrrrYwcOZK77rqL7OxsPv/8c+644w6sVisejyfWvra2FoCKigrmzJlD165dATj99NMZPHgwkyZN4qabbtpiibkGNTU1AKSlNS/t8txzz3HllVfG7Zs+fToHHXRQ3L4nnniCmpoaysrK+OCDD1i/fj2VlZVbfV2Ahx9+mDfffJMxY8YwZMiQbbYXERGRfYc3EObiST9RVu0DYFDXHG4vWQJv3sYcu40bcrOI1D9oeuMhNzK4w86v1ygtU8BIRERkXxeJQNOblLP+HS0tt5mQ3xzNHKqIlpZL8y0krckyDRZnelywKGg2U5NopzYnE3d2JtVWM9XeOg474aS4cXOLSuK2rQkJ5HYsJreoBFdRCa7iUjLbtceyHSVyRET2NhdffDFTp07l6KOP5pprrqFz584tlhfr0KFDK8xu98hPyd/hPus967e7JB1AgiUBV9KOBXB2Zl4N0tLSYsGY7VFdXY3XG7/uXk5OTlx2zU033cR///tfHn74YQzD4OGHH25xrJtuugmPx8PDDz/MIYccAkBKSgqPPPIIt956K6FQKNa2IQPp0EMPjQWLGpx33nl89dVXfPXVV3Tv3h23290sUygzMxO73R4LFDUEjpoaPnw43bp1A2DSpEm89NJLLc67Ya4A5557LjfffDNHHnkks2bNoqSkpMU+zz33HNdffz0nnHAC48ePb7GNiIiI7JsiEYO//Hsms1ZXA9DVlcq/BhlYXhlDmcXMFa5cvPX3Ns7pfg5ndz97a8PJTtKdGRERkX1JTVljxlDD70E3wkEXNrbJ7U7IZ46uOVQfHPJWJRKKv2eE9cghNH2uuC4tmWUl7XFnOKmymKj1NT7RTNAbyxzatHFD3DgpGVkMGHEGWe0KyS0qJaOgALN52+VoRET2Bb169cJkMmEYBl999dUW24XD4T03qd1sZ8q+vTT3JR6Y/sB2t7+m3zV7dA2jXr168fXXX7N06dLtKkt39dVXNyv9tmzZMjp16hTbTkpK4v333+ekk07ikUceIRKJ8M9//rPZWGazmXvuuYebb76Z2bNnYxgGffr0IRKJcMkll3DooYfG2hYWFgKQl5fXbJz8/GjArCHL56GHHuJvf/tbXJupU6cyaNAgOnfuTEJCAr/++muzcQoLC2Ov8+23327zvWhw/vnnc//99zNx4kTuvvvuZscnTJjAmDFjOOaYY5g8eTI2m227xxYREZG930OfLuCj39YBkJ1iZ+JIF8n/PpHasI/LClxsskbvIwxuP5i/HvTX1pzqPk0BIxERkb1V3SZYNS2+tJy7+RoQxppfMDWpElM5dTbr3ml+I6mB32qhOjEB06pVNG21ac0q5qXY44JDTTlSUsktKiG/c/wTzSaTiSPOPHfHzk1EZB8xduzYuPXZpGUnl5zMYzMe264sI4fFwcmlJ++BWTU69dRT+frrr3nuuee47777ttn+hhtu4Jxz4gNaLQVxEhMTee+99zj55JN59NFHMQyDRx99tMUxk5OT44JDb731FoZhcPzxx8f2NWT1rF69uln/hn25ublANOPoiCOOiGvTsGaRw+HghBNO4O233+azzz7j6KOP3tYpb1NDxlVFRUWzYxMmTODiiy9m2LBhvPPOO1ss0SciIiL7pjd/WsUTXy0BwG4189yZXch//wyCdRv4S14Oi+12AHpm9eT+gfdj0UOou40CRiIiIm2dYUD1anCkgcPZuP+3t+Gj6+OaBr3mWNaQr8KOr8pBrqMO5ymNbRJKO0eHBXw2CzWJCdTmZFGb4aTKFMEbiN6sGzT8hLixXU1KyyU503EVlZBbVIqruARXUSmp2Tm6KSoispk777yztaewV3AmOLl1wK2M/W7sNtveMuAW0uzN19bZnS6++GKeeOIJHnroIQYMGMApp5zSrM3PP//MtGnTuOyyy+jRowc9evTYrrETExN59913OeWUU3jssccwDIPHHntsq33Ky8u55ZZbyM7O5tJLL43tLyoq4vDDD+e7775jxowZ9OvXD4hmsD377LNYrVaOOeYYAIqLi7eaLXXXXXfx8ccfc9FFF/HJJ5+0uO6RYcSvaxgKhaiuriYrK6tZ28cffxwgLugFMHHiREaPHs2QIUOYMmUKDodjq+cuIiIi+5YflpZzy39mx7YfObUHfb+7GmPjPO7JzuSH+pK7BckFjB86niRb8/LOsusoYCQiItKWGAZUr4ovKVc2EzzlMOJp6HNmrGkosRjvmoRoYKjShrfKQdjTfEifuTtNwkys8XuZcXAvqowwvmCg8UAwfq2F9cuWxG2nu/IZfsNYXEUlJGdkKjgkIiK71IjOIwC4d9q9LWYaOSwObhlwS6zdntRQPu6EE05g+PDhHHPMMRx99NFkZWWxceNGpk6dyieffMINN9ywU+M3DRqNGzeOSCQSC7B8+OGHPPjggxx99NHk5eWxYsUKnnvuOSorK3n33XfJzs6OG+vxxx9n4MCBDBs2jKuuuoqsrCzeeOMNfvzxR8aOHbvd62X17NmTyZMnc+aZZ9KnTx9GjhzJYYcdRlpaGhs3bmT69OlMmTIFp9NJRkYGAG63m8LCQkaMGEGvXr1wuVysW7eOd955h59++omhQ4cyatSo2Gu8++67XHTRRaSlpXHGGWcwefLkuDmkpKQwfPjwnXpPRUREpO1btqmOS1/+mWA4+hDKX4Z15sRVD8HSqTzvTOPt1BQAUm2p/Gvov8hOzN7acLILKGAkIiLS2lZNhwUfNJaW88aXajEMCHnMWJb9hLlJwKjqm/ls/Kb5E7wGUJdgi2UO9SRC02XB/UE/6wLeZv0AnK48XJ1KyC0qobB7r7hjJrOZkv6HtNhPRES2rLa2ln/+8598+umnrF+/nkmTJnHYYYexadMmnnjiCU4//XS6devW2tNsE0Z0HsGQDkN4d8m7fLXqK2oCNaTYUhjSfggnl56MM8G5zTF2l9LSUn755ReefvppJk+ezL333ovb7SYzM5ODDjqIF198MS4YsqMcDgdTpkxh+PDhjB8/nkgkwvjx4+nUqRMOh4PHH3+c8vJysrOzGTp0KLfddhtdu3ZtNs6BBx7Id999x2233cajjz6Kz+eje/fuvPDCC/z5z3/eoTkdd9xxzJs3j/Hjx/PRRx/x0Ucf4fF4yMjIoFevXtx3331ccMEFsYyipKQkLr/8cr7++ms+/fRTqqqqSE1NpWfPnowfP54xY8ZgsTSWkJkxYwaRSISqqirGjBnT7PU7duyogJGIiMg+qsoT4KKJ06nyRGvejziwHVc6PoRvJ/FhchKPZaYDYDVZeWTwI5RmlLbibPcfChiJiIg0mPkq9BgO9i2kNwc8MPcd6LsTN4MMAyqXRYNCRUdCcpOnYlb8D779Z6xZyGPB21BWrtKGr8pB2AuFPTJIbTKk44C+RIA6h53qRDvu3GxqUpOpigQJNVk8vejAA+KmkltUAiYTGfntcBWVxErL5XYqxpGSsuPnJiIiW7Rx40aOOOIIli5dSmlpKUuXLo2t5ZKdnc2LL75IVVUVjzzySCvPtO1wJjg5t8e5nNvjXCKRCOFwGIvFgtlsbu2pkZSUxLXXXsu1116702MMGjSoWSm3Bg6Hg48//jhuX48ePfjoo4926DUOOOAA3n333Z2eY1P5+fnce++93Hvvvdtsa7fbeeihh7Z77DvvvFNlG0VERPZDgVCE/3t5Bks31QFwUMcMHuixFNPkO5iRkMBtOY0Px449bCyH5h+6paFkF1PASEREBKLBonf+D359Dc56o3nQKOCB186AZV9Ht7cWNIpEosGhpiXlyn4FX3X0+OmToEfj2geeyhTcv6biq7Tjq0og7Gt5WF9tWlzAaNbKxfx4YGfCkUjjzmDzzuuXLY7bzu7QkStfeAN7our+iojsbrfddhvr1q1j2rRpdOjQgdzc3Ljjp5xyCl988UUrzU5EREREZM8yDIPb3/mN75eWA9AhM4nnhxrY/v1/rLBaucqVTbC+BP6YA8a0Sjni/ZkCRiIiIgA9hlP10tOkLfoG82tnRINGJlv0WNADb55DZNE31Hj7kt5jeMtj/PJKNOBUNgv81XGHDAOCbgu+ShvJS6ZjaRIwci+ooHxealz7sMlEbaKd6sQE3DlZ1KQkMjQv/iZjYnZ2fLAIMFusZLfvSG595pCruJTsDh3j25gtChaJiOwh77//Ppdddhn9+vWjvLy82fHi4mImTpy45ycmIiIiItIKnvl6KW/8tAqAVIeVSSNzcb59MpWRAJcVuKiuL197fNHxXNH3itac6n5JASMRERGg6oNPKftgA9Udi2lPfdDotJch6MXy7z8TWfwtq6YX41mxATreQ3pXExzxF0husoZQ1UpY/k1ccMhbYa8vK5dApH797g6n9iS5yWubu/WgItkRW3OoJtlBTSgQX64m6KcuLyduznklnckv7UpuUUksQJTVviNWm233vVEiIrJDNm3aRGnpluutm81mfL4tpJaKiIiIiOxDPpmzjvs/ng+AxWzimT+V0unjM/F7N3FVnouV9fcz+uX24+7D78ZUn2kke44CRiIiIkDaccfy6+Q3yPr5V1YRDRpZJh5Hhr8Oyley6ttcatf7KO/ooOuGJ6DCgOJB0Pno2BiVszzUfJkdDQ4FWl6bIGg241lTGxcw+n7+rywsbdekkb9ZP1uCA5+7Nm5ffmlXRt378O85bRER2c3y8vJYsmTJFo//8ssvdOjQYQ/OSERERERkz/ttTTXXvD6Thmdj7zmpC4f9fA2RTQu5NSeLmY4EADqldWLckHHYLfZWnO3+SwEjERHZPwW90YygyhVQOox5P37HT8Facvp2pd/MBazyZtL+yHmYgFVfZ1K70cqMri42JiSTtDxAh9oacv4wHXOTgFGgzoFngx2IfvrxW8zUJCVQneTAnZ1JjcOGOxTkvAP7xk0lr0s3Fk7/PradkJQczRgqLo1lDqXn5WM2W/bAGyMiIrvS8ccfz/PPP8+VV16J3R7/pXfatGlMmjSJa665pnUmJyIiIiKyB6yr9nHRi9PxBsMAXHR4J85a/09Y9jXjMpx8khJ9rDYjIYMnhj6BM8HZmtPdrylgJCIi+66qlVC5PBoUqlrR5PdycK9vbHfdArocejhzv/6Clb/NYkbPDvSbs5Lln2cTCZnw1dn4qSSP8oQksmo9JC4zqDBSSM0YQtOVgCqys1iUl4k7O4Nquw1POBg/n1B0uzISpGlxuQ69+3LI8NNwFZWQW1SKM9eltGsRkX3EHXfcwbvvvsuBBx7IySefjMlk4sUXX+TZZ5/l7bffpqCggBtvvLG1pymy1zIMQ5+bRERE2jBPIMRFL05nfU20msqw7rncmvYRTH2Zt1KTeT49Ghyym+2MGzKO9mntW3O6+z0FjEREZO9kGODe0BgIioSg71nxbV44HqpXtdg9EoZgnYWQx0Jy5QpsHfIYfsNY3rn/TlbO/Y2fivI4aNk6AH4qyqM8ORos6r9sHQGrhRqHHdf8xSQdfGhszPkV61jqyohubB4sApIzMnEVlZCQmBy331WfRSQiIvuevLw8fvjhB6644gomTJiAYRi89NJLmEwmjj/+eJ588kkyMzNbe5o7zGQyEQ6HdbNeWpVhGBiGgdlsbu2piIiISAsiEYNrXp/JnLU1AHTPT2P8AcswT7mb/yU6uCer8XPwfQPvo29u31aaqTRQwEhERNq2QB0s/mKzDKEV0eyhkLexXVphXMDIMAxClnYEN64n6LYQqLMQdFsJeGwEPXZC7sY1hrqOzcAM2EwRhrf/jTdm1bI+NZVpJQVETFCT5MARCGK2m/mqb2cCkWgKdf9jG8vRAbhKurD0l5+i08nJJbdTNBCUW1xCbqcSUjL2vhuCIiLy+7Vv354pU6ZQU1PDggULMAyD0tLSvTJQ1CAhIQG3243P5yMxMbG1pyP7KZ/PRyQSISkpaduNRUREZI/7x8fz+XRutMJLTmoCLx0dwTH5ChbYbFyXm024/sGjv/T/C8d2OrY1pyr12nzAKBKJ8Nhjj/H000+zfPlycnJyOP3007nrrrtITk7e9gBARUUF9913H++88w6rV68mNTWVXr16cddddzFw4MDdfAYiIrJFQV808NNQJq5qBbh6QZ8zG9v43fDvc1vsHg6YCNZZCLitYNpEWsgP1ugiiSaTiaUT1xPxZLfQszFYFDaZCLjBEfDAa2dgWfotCb4+mJMiVCU7Yu18dhs+bNHUpHqb1qwiNdcV2+5+xFEUdO1ObqdiktJUb1dEROKlpaVx8MEHt/Y0donU1FTcbjdlZWXk5+fjcDiUaSR7jGEY+Hw+ysrKgOj/WyIiItK2vP7jSp7+eikADpuZScOzyX5vBBsIcXmei7r6DOHTupzGn3v+uRVnKk21+YDRtddey7hx4xgxYgTXXXcd8+bNY9y4cfzyyy98/vnn20w9X7FiBYMGDcLtdnPRRRfRpUsXqqurmTVrFmvWrNlDZyEiImyYB7+93RgYqlwB7nXN2/U4JS5gFDEl49mQSrA6TKDOStBtIeh1EHCbifgbAz8JnQpIM8X/m2DrVIxv7jx8NiseuxVPgg2vIwFfehoeh506DDCb6ZqeDnPfIbLoG1ZNL8ZjNhNp6d8XwyAzwYurcw9y+w0ls11h3OGM/HZk5Lf7Pe+SiIjs5VauXLlT/Tp06LCLZ7J7OZ1O6urqqKmpYfny5ZjNZkwm0y4PGjWUHNsdY8veqeHvRCQSAaLBIgWMRERE2pbvFm/itnd+i20/PrwT3b88B4+vkivyXay3RsMSh7c7nFsG3KLPeW1Imw4YzZkzh8cff5yRI0cyefLk2P6ioiKuuuoqXn/9dUaNGrXVMc455xxCoRCzZs0iPz9/d09ZRGT/YhhQt7FJqbjljcGgw6+G0qGNbcuXwNcPNOse8poJ1geCAnUWHJ6FpJ7e2Ca4YSOrvkxt6cUb25jN+MurMcxWGj5iBAN+PnfaqetbSsQwmndvWGMoDEZqCpGuI1g17yVqV66g7oBCIILNkYjZasXvriXdZKX/b4vI6NaB9tc/iFnld0REpAWdOnXaqS+84XB4243aEJPJREFBASkpKdTU1OD3+zFa+vf2dzIMg1AohNVq1Y0EAaJ/98xmM0lJSbFgkf5uiIiItB1LNrq59OWfCUWinw1vPLqIo2ddR6h8Mde7cpiXYAega0ZXHj7qYazmNh2i2O+06f8ar732GoZhcM0118TtHz16NDfddBMvv/zyVgNGX3/9Nd9++y3jxo0jPz+fYDBIMBhUfWMRkd9j5qswd0pjkCjoabld6bC4gJG/2oJ7fjIBt5Wgx0bQk0CwFoxw/M0lZ+mRNA0P2doVEAF8diseuw2P3Yo3JRmfMxWP3UadESYQDtGhUwl9DAPqbxjY7AkEzaaWg0WALcGB05WHMzePgM+H56uvqZ27ghl9u+KNBOnQ6wBOuPYWamqq+e+z41k9dzazDuhCv5kLqPn4E9JHDP8db6KIiOyrxo4d2+zm9bvvvsvMmTM5+uij6dGjBxB9OO6LL76gb9++nHTSSa0x1d/NZDLhdDpxOndfGdZgMEh1dTVOpxObzbbbXkdEREREfr/KugAXTpxOjS8EwKkHtuPS6kcxVvyP+7My+Dop+vBtbmIu44eOJ9m2fUvOyJ7TpgNG06dPx2w2c8ghh8Ttdzgc9O3bl+nTp2+1/4cffghEyzucdNJJfPTRR4TDYTp37szYsWM555xztmseq1atYvXq1XH7Zs+eDUAoFCIYDG7vKe0WwWCwTcxDRPZSIR9Ur8JUtRJT1QqoWompfl0hU/VKQhf/F9IKYs3NGxZgWfhxbDsShmCdpUmWkJVgnZX0dotJOKTxuuReWc2GmU1vKBlxfwpazHjsNow1a8hpcj1bvWAun/QtbeGp5QiE/I3jBwOEwmFo8oR2fuduBLwenLku0nLz6n+7cOa4SExzxt3QMx89jFlffsDGDWUU9ujNCdfeAmYzmC0cd9UNfDzugWjQaNjhFB89TNdcEZHfqa18hg2FQrt0vDvvvDNu+9VXX2XZsmX8/PPP9O3bN+7YjBkzGDp0KF26dNnp19tSZkVycjJut3unxxURERER2RH+UJhLXv6ZFeXRB4sPKcrkH7mfYPrvG7yYlsobadHHg5OsSfxr2L/IS85rzenKFrTpgNHatWvJzs4mISGh2bF27drx3XffEQgEsNvtLfZfsGABEM1I6ty5My+++CKBQICHH36Yc889l2AwyAUXXLDNeTz//PP87W9/a/GY2+2murp6B85q1wuFQtTV1QFgtbbp/6Qi0hoiYUy+Soyk7LjdqZ9chXXtT1jq1m+1e92auYSMxic+wr9WEv4hnYDPQdBtJeSOtNjPsPQlscn10ZueSV1DllCCDW96Gt6UJDxWKx4jRLA+0NPniMPIbdLPsNq2WOLG5nCQmp1LSnYO6XkFza7HR158eYv9gkCwpiZu3+IfvqVsQxl5Xbpz1Ogr8Ph8cdfXo0ZfwRdPPkrZwnnM+upzSg89YgvvmIiIbI+28hl2dwdV7r//fq644opmwSKAfv36cfnll/P3v/+ds846a6dfY+DAgYwZMyZun7JxRERERGRPMQyDW97+jR+XVQDQKSuJF/otw/rBfXyelMjDmekAWEwWHjrqIbpldmvF2crWtOnogsfjaTFYBNEso4Y2WwoY1dbWApCamsrUqVNj7YYPH05xcTG33HIL559/PuaWFjZv4qKLLuLYY4+N2zd79mwuueQSUlJSdmsJhu3R8FRmWlqavhiK7I8MAzybGrOCGjKFqldiqlwBNashKZvQ1b/FdbMEazDXB4vCARPBOku0XFydhaDbSsCbQNBjp3BEBHOT69zGmjSqlzeU9mwMFhlAwGLGm2DDY7dRWr4p7vo4b8NqfuzecbPJhyEUv2ZDMBSI65eSnERhj95xWULOnGimkCMldZfVrO9/7AkkJSZSOuBwbPX/9mx+fR1x4x0snvY/uh85ZJe8pojI/qytfIZNSUnZreMvWrSI3NzcLR53uVwsWrTod71GcXHxdldPEBERERHZ1Z787xImz4hW6EpzWHnlmAjJ717NrAQ7N+VkYdTfu7llwC0MLBzYmlOVbWjTAaOkpCQ2bNjQ4jGfzxdrsyWJ9QuSn3XWWXFBpYyMDE4++WQmTZrEggUL6N69+1bn0b59e9q3b9/iMavV2iaCNA3zaAtzEZHdwF8bXTPIbIHcJtcsbyU80hOCdVvtbtSsw0YYbI7YvrVfBvAvKyRQayLi29JC2xGMhJK4a0sg18WG1CS8iQn4MjPwJifisZhwh4KEwo1lfUqOPTquX1bHohZfwZGSijM3D6crj/RcFwVde8T1s9lsnHHH37d6frvKAUOPbbav6fXVZrO12EZERHZOW/gMu7uzm/Lz83n77be5/PLLmz3kEIlEmDx5Mnl5v78cRyAQIBAI7PYAmIiIiIhIUx/NLuOBj6OVvqxmExNPyaTdx6ex2hTmSlce/vpkjQt6XsDpXU9vzanKdmjTAaOCggLmzp2L3+9vlmm0Zs0asrOzt5hdBFBYWAjQ4hew/Px8ACorK3fhjEVEdlIoAFUroWp5NDBUtSL+tzea0ku3E+HMVxr7OdIBA8OAkNcczQyqs9RnCdkI+BIJ1lmwJDkovtkbFzDy16Xh27gqbhoG4Lda8NqteBLsBDKcdKzP1mzwc9U6lhfnN5m7D1pY/qF6/bq47dxOxfQ5+o+x4JCzPlvIkawbWyIisu8aPXo0t956K8ceeyx/+ctf6Nq1KwDz58/nkUce4ZtvvuGee+75Xa/x1ltv8fLLLxMOh8nJyeGMM87gnnvu2a5KCFqvVURk/6Trq4jsCrPXVHPtv2fGtu8/Lp8Dv76Aan8Vl+XnUWGxADCs/TAuP+Dy/eKa05aurzuzXmubDhgdfPDBfPrpp/z4448MHNiYqubz+Zg5cyZHHnnkVvsfcsghPPXUU82+AAGxfVsrDyEisstEwlBb1hgEyu8Lrh6Nx5d8Ca+dsdUhwgETxpplcRfu4MaNrPwon2CVDyPU8jo/ECYcDGM40ml4rtkwDDZkZ1BR6MKf7sSbmECdGdxBP+FwfLbRkB7xWZjp7drDb7/G7UtMc5K+WSCosEevuDbZ7TsybAtrComIiOyrbrrpJtavX8/jjz/OF1980ez45Zdfzs0337zT4x9yyCGcdtpplJaWUlNTw4cffsj48eP573//y3fffbfNjCOt1yoisn/S9VVEfq91NX5GvzIHXzC6XMGf+2dxyry/EqpcxrV5uSyzR6sI9EjvwV97/pXamtqtDbfPaEvX151Zr7VN/4twxhlncN999/Hoo4/GBYyeffZZPB4PZ599dmzfkiVLCAaDdOvWuGDW8OHDufrqq3n55Ze57bbbYl+WysrKeOedd+jSpQulpaV77oREZN9XvgTKZjbPEqpaBZEmTxYMuzM+YJTRkUiYaGZQnbVxPSGvg6A3gUANRHxhnEcWUtDk5azp6QTKfdF1jOo1ZAl5Emx4Ex34sjJIcKbTxePBlJwMgMlk4qeQG19WChCGgGeLp1S9YT2OosabTZ0POSy6jpArLxokynVhT9xyeVAREZH9mclk4tFHH+Wyyy5jypQpLF26FIiuO3TyySfHMo521rRp0+K2zzvvPA444ABuvfVWHnvsMW699dat9td6rSIi+yddX0Xk96jzh/jry3Mpr4teS47ulsPtxlOYy37i1uwspidGK9wUphQybsg4Mh2ZrTndPaotXV93plx1mw4Y9e7dm8svv5zx48czcuRIjj/+eObNm8e4ceM46qijGDVqVKzt0KFDWbFiBUaTm6YZGRk89NBDXHLJJRx66KFceOGFBAIBnnzySQKBAI8//nhrnJaI7K0a1hFqGgg64i+Q6mpsM/NV+OahFrvHysbVWbEtX4DtiMZjNdMXs+atgmi0p5nGjJ+g1xF3xOvzsuqAbtSZwOuwU4dBnd9HOBKfJZSRlYa5PljUIN2Vxzp349MdyekZpOW64jKF0nPzyMxvF9evQ68+dOjVp8VzFBERkZZ16dKF66+/fo+81vXXX8/f/vY3Pvjgg20GjLReq4jI/kvXVxHZGeGIwXVvzWT+uug9pZ4FaTxR+CmWbyfzZHoa76VG7z+l2dN4YtgTuJreN9tPtJXr685kOLXpgBHAo48+SqdOnXjmmWf44IMPyM7O5sorr+Suu+7CXL9g1taMGTOG7OxsHnjgAW6//XbMZjOHHXYYr776KocffvgeOAMR2Sv99jaU/doYHKpc3riOUFPdTowLGIUT8ghWWqPZQfXZQgFfMsE6G8GaMEYomqab16crGU2GseYXxgWLDMBns+KxW/E60/BnpuNNTqSwqAMdm/QLeL3MJhDt4A1s8XRqNqzDiEQwNbluHnbaKIxIJFpCLseFzeHYYn8RERHZe9hsNgoKCti0aVNrT0VERERE9jH3fTiPL+ZvAMCVlsArBy/D9smDvJuSzBMZ6QBYzVYeHfwoRc6iVpyp7Iw2HzCyWCxcd911XHfddVttt3z58i0eGzlyJCNHjtzFMxORvU4kEl1HqKo+ANSQJeSrhrNei28740VY+lXzIZqWjXNbcK5bhLmosWTmxo8WUvlJS2ujxS90F6yJzwAqjwSYf1BvPHYrHiOC2+chEok0aREGn5uUnPgU3tSsbExmM0Z925SMzCbrCOWR3vBnVx6YTHF9iw88uMW3SURERHat77//nvHjx7No0SLKy8vjqiJAtGzdkiVLdtnr+Xw+Vq9ezaGHHrrLxhQREREReWXaCp7/dhkAiTYLrx8dIv2jvzDdkcAd2Y33rO4+/G4OztN9p71Rmw8YiYjstFAAPrqhMUuoehWEt5CFE/CAvXEdHm9tBv5lidHgkCeRgC+RoNtEqDYYlwmUmHwgTfNybJ17Au8DEAF8diueBBv+rEx86U68iQnUmeEPHdvTNKzk9vtYGvRsHldqxl1RHrdtsVo59Za7SMnMIi0nF5s9YZtvi4iIiOw5kyZN4oILLsBms9GlSxc6dOiwy8YuLy8nKyur2f7bb7+dUCjESSedtMteS0RERET2b98s2sjYKXOA6DPJzx6fStEXZ7HUYnB1bg6h+geVL+97OScWn9iaU5XfQQEjEdm7+N3xawg1/e3qCac+29jWYoPZb0GgNm6IcMBEsM4SKxsXCdvIqS2DrJJYm3WfV+Cb17RoXKjZVAwgsG4jjgMa9y2NBFh4RH/qwiHc3rr4J4gNP3j8AHjS4hedS8/Ni/7BZCI1KztuHSGnKy+2nZia1mweHXv33epbJiIiIq3n3nvvpWvXrnz++ecUFBTs0rHvuecefvjhBwYPHkyHDh1wu918+OGHTJ06lQEDBnDllVfu0tcTERERkf3T4g21XPbKDMKR6H2uO4e4OOLHi9gUqOWygjxqLdElEE4pOYVLDrikNacqv5MCRiLStoQCEAmCPblxX9kseO/qaGDIU77lvpuVXIsEAlSvchEoMxH0pxD02AnURIh444M/JpuN7Iwimva2F5Xim7eIiAm8NhveJAf+3Gy8aal4HXbqiFDr83Jh1y5xY7nNsK62aqunaE9MIhSMz3TK6VjEhY8+TVpOLharFhwVERHZV6xYsYIHH3xwlweLAAYNGsTcuXN58cUXKS8vx2Kx0LlzZ+69917+8pe/4ND6hCIiIiLyO5W7/VwwcTq1vuj9tLP75XLeipvxVq3gqrxc1tiiIYYBeQO447A7MG12f072LgoYicieFbeOUAtZQjVrYMhtcORfG/tYE2DtjNimYUDIa46tIxSosxD0JWNvZyK7yUuZzGbW/dcHkYZsnubl6AwgGA4R2rgRm8sV2/9joon1h/WhzuOO7xCsi/7UqwuHaJor5HTlYTKbScvOiWUHbb6WkCM5pdk/nla7nYz8dtv5JoqIiMjeorCwEL/fv1vGPuWUUzjllFN2y9giIiIiIv5QmEte+plVFV4A/lCUwV38i8jqH7k5N5vZjujSCCXOEh4Z/Ag2ix6C3tspYCQiu5ZhgLcSKpdDdmdISG089sOT8NnYLa8j1KBqRdymvyJM5S/ZBHxJBN0WgtVBjFCkWbdEZ6f4gJHNhi0/H9/atXjtVrzpTgI5WXhTk/HYrNQZYWq9daQ4M+i9Wf1/r4nmwaImHCmpOHNdhEPxiw71HnosfY4+HotVl1cRERGBSy+9lFdeeYVrr70Wi8XS2tMREREREdkuhmFw0+TZ/LSiEoDi7GQmdPwUyw//4cHMdL5Ijq4FnuXI4l/D/kWavfkyCrL30R1NEdlxgTqoWhnNCKpc3jxLqGHNoPOmQPGgxn4JabFgUSQMwTpLY5aQx15fNs5GRjKkN3m5iC9I5QI70XWEtrCWkNVCaOP6uP11VZV83imHuuym5VhC4KmOa1dbVYFhNsWVpMssKMRbW9MsO8iZm4cz14UjOX4NogY2e8KW3zcRERHZ7/Tv35/JkydzyCGHcPnll1NUVNRi4OjII49shdmJiIiIiLRs/JeL+c8vawBIT7Lx70MW4/jyn7yWmsIkZzQ45LA4GD90PO1SVDVnX6GAkYg0Fw5C9apo8Ke2DPqOij/+xrmw5IutDmEYQMUKTMWN+6pnVVD5bRHBWgjV+KORnjgRwI8/0j5ur62wkLDJhCfBhj83G19WJr7kJDw2C+5wEHedm1AoSPcjBtG7Sb/E1DQ8njq2JDE1LRYECvn92BOTYseOu+zarZ6fiIiIyPYYOnRo7M8XX3xxs7K0hmFgMpkIh8N7emoiIiIiIi16f9ZaHv5sIQA2i4nXhvrI/uJG/pvo4P6sDABMmPjHkf+gV3av1pyq7GIKGInsrwwDVv7Q8lpCNWvAaFLyrfvJkNAkoyajIwDhgIlgnYWA2xr97Ush6Esk6DYTrArQ8ahkEpu8ZNiUjnd1yzX8DcBvteDNcJIQiG+zdPECPjmgSeQp4oVab7MxqjfEZxiZLRY69OqD2WxuzA5y5ZFenyXUNEAkIiIisju88MILrT0FEREREZHt9svKSq7796+x7fFHJ9H96/OYazVxfW42kfoHoG44+AaGdBjSWtOU3UQBI5F9UcM6QlX1JeMqV4A9GQ4Z3djGZILXzgBf9RaHiYQh5LVgr1oJrh6x/Ws/3ETt9x2JeIMt9GpcnygYSIkLGOHKpTYlCX9eLv7MdLxJiXgsJtzBALXuWsLhaLm5IQf3jRsxLSubLUlOz4gFgnI7FjU7/qdb795iXxEREZHd7fzzz2/tKYiIiIiIbJfVlR5GT/oJf/3a4X/9QzrHzryUdSE3VxS48JrNAIzqNopzepzTmlOV3UQBI5G9Xd0m+G1y8ywhf018u5xu8QEjwHB2IFQ5h6C7yTpCgVSCngQCNZFo2TiLhW535sat72MkuloMFhmAz27D78rBl51Jrjm+5MqvZSuYUZIf3fDXRn9aUL1hXdy2My+f4v6HRDODXNHsIGduHs4cFzaHo8UxREREREREREREZPvU+oJcNPEnNrmjD4Of0jOdy9fdjrt6FZcVuNhojYYSBhUO4oaDb2jNqcpupICRSFsVDkL16s1Kxi2H0mHxawp5q+CjLV+kG8rGGdVrSDSMaGYRYEQiLHzGTcSbt1mPCNCk3FsoTKgmgK1JRTpPu3w29eyKPz0NjyOBOpOBO+DH7a4h0lB/P+Kle3F8xk96Xn6Lc0zJzMKZm0d6fdm4wu49444npqQy4oaxWzxHERERkbbi66+/BuDII4+M296WhvYiIiIiIntaKBzhytd+YcH66MPdfdql8oj1SUJrf+Y6Vw6L7HYAemT14B9H/gOL2dKa05XdSAEjkbZixXfwyyuNAaKa1fHrCDWwJcYFjIzkPAI10TWEYusI+ZOjGUPVQSKeaCZQYq9udGoyjMlsxpKdS2TVqrjhI4A/yYE/Pw9/dibhtFRK69NNG/xUuY611hC4K8C95VOq2rCOvNIuse2CLt3pf8IpsRJyDVlC1vp/dERERET2doMGDcJkMuH1erHb7bHtLTEMA5PJRLjhoRsRERERkT3sng/m8dWCjQDkOx28WvQJ5p/e4+7sTL5Lii44kZ+cz/gh40myaU3wfZkCRiK7k7eyeam4ht8nPAxFTZ4krVoJM19uNoRhQMhrJlhnJeC2kOBYSuIpjcd9i5ex/MPczXr5m40TWL8pll0EEA6FqDiwNzXF7fEk2KgjQq3Pg7u2BiNSH6jyVWMOuhmckxM3ltOVx9qF8xp3mEykZmU3KRkX/d2ua4+4fq7iUlzFpVt/z0RERET2YhMmTMBkMmGz2QB44YUXWnlGIiIiIiJbNun75Uz8bjkASXYLkw9ZSPI343nOmcbk1GjJoRRbCv8a+i9yknK2MpLsCxQwEvk9gt5oxk9TXz8Ec9+BypXgr95y3/IlcQGjYCAV7yoHQY+NYMBJwOsgWAvBSj9GqPGJ08yuR9L0FW2Fhc2GjpjAl5FBID8Xf0Y6niQHabkuOtc/wQpgMpv43+olRMKhrZ5iJBymtnwTzlxXbF+PgYMp6NKd9FwXTlceaTm5WKy2rY4jIiIisj/485//HLd9/vnnt85ERERERES24asFG7jz3TlA9DnzlwfVUfDNrXycnMRjmekAWE1WHhn0CJ0zOrfiTGVPUcBIZGvCoWhpuMrlLWcK1W2Am1aCw9nYp3YdrJsdN0wkTH3JOGs0UyjgJKN/OU0LsdXO3cj6/2U29AA8LU4psGZN3HZdKMCaYwbjsZpwh0PUeuuoq6nGMAwgBLWboBby05I5okmGkdlswZmbS2XZWiBaoi4tOyeuXFzDmkLJGZlxr9mpT78deBNFRERERERERESkLVmwrpYrXv2FiBHdfuhIG/1+uIRf7BZuzc6KtRt72FgOKzislWYpe5oCRrJ/MwxwrwdfNeR0jT/29FHRwI+xjXryVSshr3ds073GhHd+DkF/KgGPlWB1mFC1F4z4bomWHnEBI3unorjjYZOJQL4LvysXf3oaXkcCdWaDLoccRvsm7Xy1tfy6fuU2T7Vq/bpm+wadNxqL1YbTlUdqVjYWqy4JIiIiIrtDOBxm/vz5VFZWEok0X6fyyCOPbKGXiIiIiMiut7HWz4UTp+P2RysPjemXzMj5l7My4uWqAhcBc/Sh89G9RzOi84jWnKrsYbo7LPs+b1XzzKDK5dE/V62EkA+yu8AV05v3bRIsCgdMBOssBNxWgt5EgkEnAW8C7cMGTZcxrl5komamDfBtdVqBVavjttebDBaeOAx3OEit201dTVX0QKQOKupi7bJ694nr58zNi/3ZbLHizM2NZgnVZwql5+aRluuKKynXoLjfwVudo4iIiIj8fv/4xz+4//77qamp2WKbcHgbDymJiIiIiOwCvmCYMS/9xJoqLwBDS1K4qfIOqmrX8n8FLqosFgD+WPRHrjjwitacqrQCBYxk7xf0RgM/lSsgOQva9W88VrEMxvXd9hhVK6PZRk1Ktm34LZ3Aqi4Ea00EKgNEPP7NOgWAAEEjKz5TqH1j/k/EbidQWIA/NwdvWjLeBDt1Rphan4dj/zAgbrRabx2LVy3b5lQ9NfHrIjlSUjj9jr/jzHWRkpmF2WzZ9vmKiIiIyB7x/PPPc/PNN3PUUUdxzDHHcOutt3Lttddis9l4/vnnKS4u5rLLLmvtaYqIiIjIfsAwDK5/axa/rKwCoEtOIk8lP01w0S9cnZfLSlt0jfJ+uf245/B7MJvMrThbaQ0KGMnewVMRLQ+3eaZQ1YpoSbkGfUbBiCYBI2chmMxgRDAMCHnN0TWE3BaCwXSC/hQCdRYsaWm0DwfAmhDrWrvQT2CFe6vTMkwmQuvKsBe2i+1bmGpn2R+Pora2Bk9t/VOk3oroTxM11ZW0a7LtdEUzhSw2G84cV2wdIWeuK5Yp5Mx1YU9MajaP9j16N9snIiIiIq3vySef5NBDD2Xq1KmUl5dz6623csIJJzBkyBCuvvpq+vbtq+wiEREREdkjHv18Ee/9Gl3PPDPZzluln2D55QNuzcniF4cDgI5pHXls8GPYLfatDSX7KAWMpPUZBrg3NAkCLYeux4OrZ2ObxV/A2xdve6iK5XHl4YIbyyn7pRfBCh/BSi9GcPMv49FSb5YsW1ywCMBWWEhgxQoiTifBdvn4srPwpibhtVlxR0LUety4q6u5om/fuH61Xg/r18aXm9tccnoG4WAwbl9h956MeXIiKemZmMyK3ouIiIjsC+bNm8c999wDgKk+m70hQJSfn8+YMWN47LHHuPDCC1ttjiIiIiKy75sycw2PfbEIALvFzH8OnkfaD08xLsPJRynJAGQkZPDE0CdId6S34kylNSlgJHvWqh9h1bTNsoRWQsgb3y4xIz5glNERgEgYgnUWgnVWgl4HgVAGQa+jvmycD+eQTuQ1GcacmEjd/E1bn5PNRtiZRiQQwGxvjJx/50qj4ogD8dbWAEGoXgfVzbvXlm8i3dX4qs7cPKw2e32GUJPsoIaMoRwXtvqIfdw0EhzYEprvFxEREZG9l8ViITk5+gW84Xd5eXnseKdOnVi0aFGrzE1ERERE9g8/r6jg+jdnxbYnDqyi47Q7mJySzLPpTgDsZjvjhoyjQ1qH1pqmtAEKGMmuEfRFAz9VK6ByeWMw6OTHITG9sd3st+DHp7c4jGFAyGfGXLaYpivx1M5ay7pPOhOq8kQbxQTqf+qn4W6aXwQWpxOz00k40UGgIA9/Zgae5CQ8VhN1oSA17lpqKyvILSqmuz0+zdLr9dQHi1pgMpGSmUV6bh6hQPzaRgef8icOPfXM2BOkIiIiIrL/6tChA8uWRdepTEhIoH379nzzzTeceeaZAEyfPp3MzMzWnKKIiIiI7MNWVXgYM+lnAuEIAHcNiPCHGX/luwQ7d2c3fg69d+C99M3t20qzlLZCAaO90cxXocdwsDdfywaAgAfmvgN9R+2+OXz3eHRNocrl0cCQe13L7QZeB4l9G7czOhEOmAjWWQi4rdF1hAJpBDw2gtVhghV1GMEwBQf1xtlkGFNaFqHKui3PJy2NUGEBwTxX3O6Ktav5vFcnfG43hN2w0Q0bm3evXt98/tkdi7BYrY2ZQU0yhdKyc7HaW67jaa1fHE5ERERE5Mgjj+SDDz7g73//OwCnnXYajz76KF6vl0gkwssvv6xydCIiIiKyW9T4glw4cTrlddEH7s/uYePcZdew0PBznctFuP6B92v6XcNxnY5rzalKG6GA0d5m5qvwzv/Br6/BWW80DxoFPPDaGbDs6+j29gaNDAPqNjYpFbe8MUvIZILzpsS3n/MOrPmpxaGalo1LLl+OqaBv7FjZu0uo+k/+Zj3czcYIltfGbdsKC6GoE/68XHwZ6XiTEvCYTLiDfmpqq6mtKMeIBDmwQx7FTfolp2dEg0UtMJnMpGZnR4NBuXlEwmHMlsa8ppOuubHFfiIiIiIi2+vqq6+mT58+eL1eEhMT+dvf/sbChQt58cUXATjmmGO4//77W3mWIiIiIrKvCYUjXP7KDBZtiN4bPbQwgbvr/sbGunVcXuDCXb+G+qmdT+XCXnqASaIUMNrb9BhO1UtPk7boG8yvnRENGpnqM1qCHnjzHCKLvqHG25f0HsPj+4ZDYGnyn9xbCW9fEg0MVa2M9m+JxQ6RCNRfRACClnyCG+0EvYnRdYR8SQTcZoKV/riycSW39qNpHo61XTEtMpkw5+YSKMwnkJVJuF1B3OEFSxfwWZoFPOXRny2o3hCfKZSQlExhj14kpqbhzM0jvUm2UFp2DharsoFEREREZPfp2rUrXbt2jW0nJyfz7rvvUl1djcViISUlpRVnJyIiIiL7IsMwuPO9OXyzKLq2e3unnRfTnsG3bDZX5LtYZ43eIz684HBuPfRWLa0hMQoY7WWqPviUsg82UN2xmPbUB41OexmCXixvnEdkyXes+qE9ntUbIP0S0rvQmCnU5VgY+UzjYPZUWPw5GOFmrxMrG1dnx7Bn4fRWQnJW7PjKf28gsDy7fit+HaGmAqtWYe/QuFCaqXMJ/kFH4ktPxeuwU4dBrc9LbU0VtRXlEKqF9bUcn55K0+Jyadm5LY5vMptJy8mtzxJykV/atVmbM+7QE5siIiIi0rY4nc5tNxIRERER2Qkv/G85L/+wEoCUBCtTunyEdfYnXOfKYV5C9PH+zhmdeeioh7CZ9UC9NFLAaC+TdtyxVE+ZgueHH1hFNGhknXA0WRVLMQIhVn2diWdDiKRcP2n+92Ce0di5ckXcWJFwBE9NAcHKIIFgGkFPAsGaCIEKL5E6X6ydJT0VZ5NgEYCtQycCy1fF7TPZbJjbtSOQn4cvKx1fciKdCuIzhX5aMp85lWugcuvnWbVZplBmu0K6HDaQ9FxXbE2hdFceqVk5cWXkRERERETaIo/Hw/LlyykvL8cwjGbHjzzyyFaYlYiIiIjsa76cv557PpgLgNkEbx80m4yfn+fvmRn8NykRgNzEXJ4Y+gQpdmW7SzwFjPYy5sREas4+nSTDwDNtGquMTrQ/ZCFAfbAoAXtugJo+VsJBE75KG0F/KoFQOnbSaPoco+H3s+qDMGAmuo5Qy2v9hKuqCLvdWJqUy/APOJhqVxYeu406I4zb66GmqgJ3RTl4NkZ/gJ4OO4lNxnK6XGzObLHizM0lLccVKxnXvkfvuDZp2blaU0hERERE9jp1dXVce+21TJo0iWAw2Oy4YRiYTCbC4eZZ/yIiIiIiO2JeWQ1XvvoLkfrnk54ZsIkuM+7lpbRUXnOmApBoTWT80PHkJee14kylrVLAaC8z579f8Olz/6J99170jxyEZ/pPLK/NIuw3E/JYMZLtfJvZnk0b7KxbmU5hZW19Ty/JCZlxASNLaioWp5NwdXVsnzk1FVP7dgRdLnwZTjyJCZjS0jDZ7XHz+HbODCrWxGcYtaR6/Toy8tvFtjv07EN4RAiny0V6/VpCKZlZmM3KEhIRERGRfc+ll17KK6+8wogRIxg4cCAZGRmtPSURERER2QdtqPFx0cTp1AWiDyLd0MfPsDk380ViAg9mpgNgNpl56KiH6J7VvRVnKm2ZAkZ7mS6HHs7cr79g5W+zINOgX3YQ/6ZoMCdsMvGTK4vyBDtZtR7yq+IzhoKr18Rv+31UnT6SuoAXdzhErcdNTfkm6qoqoWJ19AdwJKdwxGYBo3RXXlzAyGKz4cyJLxfnzM0jt7g0rl+7bj1o163HLns/RERERETasilTpnDRRRfx7LPPtvZURERERGQf5Q2EGT3pJ9ZWR5cZGV4M/7f2FmabgtyUk4thMgFw8yE3c2ShSiHLlilgtJexJTgYfs31vHPjuawsN+HL6US/8rUA/FSUR3lqEtleL4ek5RI57gA8zlQ8DjseImSXdokby4hE+ObHr7f5mr46Nz63G0eTknR9jzmBLocegbN+TaGU9ExMZvOuPVkRERERkb2czWbj4IMPbu1piIiIiMg+KhIxuO7Nmfy6OlpFqk+umYdD97DGu4ErCvLw1d+zPb/H+ZzZ7czWnKrsBRQw2tsEPNjePo+T07/jjXX92Wh38EPvUvxGGJ/ZhDUSwZ1s4cOIG9YuhrWNXTsZEfo3GcqemERimhNvTfRiYrUnxAJADeXiGrKF7ImJcdMoOvCgPXCyIiIiIiJ7tyFDhjBt2jTGjBnT2lMRERERkX3QI58t5MPZ6wBwJVt4PfMZ3CvncHl+HhWW6DIgwzoM4y8H/aU1pyl7CQWM9jZz3yGy6BvW/VxMv5VrmN63CxWRENSnFYbMZkIRe4tdq9eva7bvj5ddiz0pmXRXHknOdEz144iIiIiIyO/38MMPc+SRR/LYY49x2WWXYbPZWntKIiIiIrKPmPzzasZPXQyA3Wrivc7vY53/BVfk5bLUHv3ceUD2Adw38D7MJlWHkm1TwGgvE+k6glXzXsKzYjmJAwaQUJgJixfGjluAJK+f1AQ7eX88ifR2hbFsobTs3GbjKVNIRERERGT36dChA/fddx/nnXce119/Pfn5+Vjqn/RsYDKZWLJkSSvNUERERET2Rj8uq+Cmt2fFtt8+8FdyZr/EbdmZ/JjoAKBdSjvGDRlHojVxS8OIxFHAaC9T8/EneOYsxz5gAD8X5VE27zfySrvg83ioWruagu69/p+9+w5vqvrjOP5O90gXbWnZZU+hKMjeIEP2XgIyBGQLyFKW7D0LImhZskRApshGEWRvZJa9R0spdOb3RyU/K5TVlhT4vJ4njyTn3nM+Sdtrku895/LBuWtE7NpFqsa+uFesaunIIiIiIiLvrMDAQFq1aoWdnR3Zs2fHw8PD0pFERERE5A0XdOsBbefuITLaBMDk/FfIc3gk09xd+cUl9jr0LnYuBJQLwNPR05JR5Q2jgtEbxr1WTaKioth0dA8Xjx8hfZ68fNytLyEhwWz9bgoXjx2GnHkoW/Ub3GvVtHRcEREREZF32tChQ/H39+fXX3/Fy8vL0nFERERE5A0XHBZJy9m7uRsWCUDH7CFUPd2flc6OBHi4A2BjZcPEMhPJ5J7JgknlTaSFC98wkeGP2Hxin7lYVPPL/tja22NjZ0+17v1InycvF48fYfOJfUSGP7J0XBERERGRd9rly5dp1aqVikUiIiIikmCR0TG0n7+XszcfAFAxbSTdbw1gj3U0/b3/P5NocNHBFPQtaKmY8gZTwegNc3LnH1w4cuhfxSIHc5utvT01v+xP+jx5uXDkECd3/mHBpCIiIiIikj17du7cuWPpGCIiIiLyhjOZTPRfcYQdZ24DkM3dxFRGcC7iNl1SehNlMADweb7PqZa5miWjyhtMS9K9YXKXKgdAtsLF4hSLHrO1d6Dml/05ufMP87YiIiIiImIZffv2pUuXLrRo0YK0adNaOo6IiIiIvKFmbj/Hgr8uAuBhD8u8ZxB8+W8+T+3LfevYeSHVM1enXb52lowpbzgVjN5AzysE2do7qFgkIiIiIpIMHD9+nDRp0pAzZ05q1apFxowZsba2jrONwWDg66+/tlBCEREREUnu1h+9xrC1xwGwtoLVWVdifWYbnX1Tctk29iv+D30/ZGCRgRj+mWkk8ipUMBIREREREUkiAwcONP973rx5T91GBSMRERERic+Ry8F0WXgAkyn2/uL39uD79wJ6pPTikIM9AJncMjGu9DhsrW0tmFTeBioYiYiIiIiIJJFz585ZOoKIiIiIvKGuBT+i9ew9PIyMBmBkriA++Hs8Yz3c+c3ZCYAUDikIKB+Am72bJaPKW0IFIxERERERkSSSIUMGS0cQERERkTdQWEQUrefs5lrIIwBaZbxD/QvfsMjFmUB3VwAcrB2YUnYKaYxpLBlV3iJWlg4gIiIiIiLyLjh9+jR//PEHwcHBlo4iIiIiIslYTIyJbosOcORyCAClvMPoFzyI7bYmhnl6AGDAwIgSI3jP+z1LRpW3jApGIiIiIiIiSWjVqlVkzpyZ7NmzU7JkSfbu3QvAjRs3yJIlCz/99JOFE4qIiIhIcjLq17/59eh1APyco5hpO4q/o+7RI6UXMQYDAD0K9KBchnKWjClvIRWMREREREREksiWLVuoVasWKVKkYMCAAZgeX60YSJkyJZkzZ2bhwoUWTCgiIiIiycniPReZvvUMAM42Maz0mcHt4DN09PHmoVXs1/kNszfkk1yfWDKmvKVUMBIREREREUkigwcPJl++fOzatYsOHTo80V6kSBH27dtngWQiIiIiktz8eeY2fX8+/M89E+uyLMdw9Q86+Hhzw8YGgFJpS9Hrw14Y/plpJJKYVDASERERERFJIrt376ZJkyZYWT39o1fatGm5du3aa04lIiIiIsnN2ZuhtJu3l6iY2BnpP+bciW/QT/RI6cVJezsAcqbIyaiSo7CxsrFkVHmLqWAkIiIiIiKSRGJiYrC3t4+3/datW9jZ2b3GRCIiIiKS3NwLi6DV7D0EP4wEYFCWkxQ5N5lhnh784eQIgK+zL1PKTcHJ1smSUeUtp4KRiIiIiIhIEsmZMyfbt2+Pt33VqlXky5fvNSYSERERkeQkIiqGtnP3cu7WAwCapL5Gs6vD+cHNhZ9cXQBwtnVmarmppHRKacmo8g5QwUhERERERCSJtGrVip9++olZs2YRExMDgMFgICwsjM6dO/Pnn3/y2WefWTiliIiIiFiCyWSi37LD7Dp3B4DCHiF882gYvzpYMz6FBwDWBmvGlRpHNo9slowq7wgtdigiIiIiIpJE2rdvzx9//EGbNm3o3r07BoOBRo0acfv2baKjo/n0009p0qSJpWOKiIiIiAVM33qWJXsvAZDG4RFz7Mdw6GEI/Xx9zNt8XfhriqYpaqmI8o5J9jOMYmJiGD9+PDly5MDBwYF06dLRvXt3Hjx48EL7GwyGp96MRmMSJxcREREREYF58+axdOlSypUrR44cOUiRIgVVqlRhyZIlzJo1y9LxRERERMQC1h25ysh1JwBwtIpmtc8MroUG0dnHmwgrAwCt32tNnWx1LBlT3jHJfoZRt27dmDRpErVq1aJ79+4cP36cSZMmsX//fjZs2ICV1fNrXiVKlHhimQdbW9ukiiwiIiIiIsLDhw9ZsmQJ2bNnp1atWtSqVcvSkUREREQkGTh06R5dFx34556J1Rl/gqt/8XlqH+5aWwNQya8SnfJ3slhGeTcl64LR0aNHmTx5MrVr12bp0qXmxzNmzEjnzp1ZuHAhjRs3fm4/mTJlomnTpkkZVUREREREJA57e3vatGnDxIkTKVSokKXjiIiIiEgycDX4Ia1n7+FRZOz1LQMzbyPt5RW0SZWS8/9McsifMj9Dig/BypDsFwiTt0yy/o1bsGABJpOJrl27xnm8TZs2ODk5MW/evBfuKyIigtDQ0EROKCIiIiIi8nRWVlakS5eOkJAQS0cRERERkWTgQXgUrQL3cON+OAB90x2l5OVv+crbk30ODgCkd0nPxDITsbe2t2RUeUcl6xlGu3fvxsrKig8//DDO4w4ODvj7+7N79+4X6uenn35i3rx5REdH4+3tTYMGDRgyZAhubm4vtP/Fixe5dOlSnMcOHz4MQFRUFJGRkS/UT1KJjIxMFjlERN42Or6KiCSd5HKMjYqKStL+mzdvzty5c+nSpQv29vrQLyIiIvKuio4x0WXhAY5djT2ZqK73RdrcGc1kDzfWGp0BcLd3J6B8AB4OHpaMKu+wZF0wunLlCl5eXk/9YJUmTRp27NhBREQEdnZ28fbx4YcfUq9ePbJkyUJISAhr1qxhypQpbN26lR07dmA0Gp+bY9asWQwaNOipbaGhoQQHB7/4k0oCUVFRPHjwAAAbm2T9IxUReaPo+CoiknSSyzE2qVchKFq0KD///DP+/v58/vnnZM2aFScnpye2K1myZJLmEBERERHLGrH2OBuOXwfgA5c7jIocyXJHW75zj53UYGdlx6Syk8jgmsGSMeUdl6y//QoLC4v3LDyHf6bohYWFPbNgtGvXrjj3mzVrRt68eenXrx8TJ06kX79+z83RqlUrKlasGOexw4cP07ZtW4xG4wvPVEoqj8/KdHV1xfafdS5FRCThdHwVEUk6yeUY+yInkCVEhQoVzP/u0qULBoMhTrvJZMJgMBAdHZ2kOURERETEchb8dYHvtp8DIJVtGD86jmVXxAMGe6U0bzO0+FDyp8xvqYgiQDIvGDk5OXHjxo2ntj169Mi8zcvq2bMngwYNYvXq1S9UMEqXLh3p0qV7apuNjU2y+BLxcY7kkEVE5G2i46uISNJJDsfYpJ7d9MMPPyRp/yIiIiKSvP1x+hZfLz8CgL0hktU+33Ih+BJfpPYh6p+Tibq834VKGStZMqYIkMwLRqlTp+bYsWOEh4c/MdPo8uXLeHl5PXN2UXxsbW1JnTo1t27dSqyoIiIiIiIiT2jevLmlI4iIiIiIhZy+EUq7eXuJijEBJlamX0T0rb10SO1LqJUVAHWy1qFVnlaWDSryDytLB3iWggULEhMTw19//RXn8UePHnHgwAEKFCjwSv0+evSIS5cu4ePjkxgxRURERERERERERETM7jyIoGXgbu4/igJgRvoNpL2xlo4+3lz9Z5Z70dRF6Ve43xPLFotYSrIuGDVo0ACDwcCECRPiPP7dd98RFhZGkyZNzI+dOXOGEydOxNnu9u3bT+3366+/JioqimrVqiV6ZhERERERERERERF5d4VHRdN27h4u3AkDoIfvfsrd+IFe3p4c+2clraweWRlTagy2VloCX5KPZL0k3XvvvUeHDh2YMmUKtWvXpkqVKhw/fpxJkyZRqlQpGjdubN62XLlynD9/HpPJZH5syJAh7Ny5kzJlypA+fXpCQ0NZs2YNmzdvplChQnTq1MkST0tERERERERERERE3kImk4k+Sw+zO+guADU8gugQMoGRKTzY4uwEgLejNwHlAnCxc7FkVJEnJOuCEcCECRPw8/NjxowZrF69Gi8vLzp16sTgwYOxsnr2BKnSpUtz7NgxZs+eze3bt7G2tiZr1qwMHTqUL774AgcHh9f0LERERERERJJeWFgYefLk4dy5c+aT70RERETk9Zm6+TQ/778MQF6HG4yLGcV8owPz3WKLQ442jkwpNwVfZ19LxhR5qmRfMLK2tqZ79+507979mdsFBQU98ViNGjWoUaNGEiUTERERERFJXvr378/NmzctHUNERETknbTq0BXGrD8JQErr+yx2Gc/W6HBGpfACwMpgxeiSo8nlmcuSMUXilayvYSQiIiIiIiIvZt++fUyYMIFBgwZZOoqIiIjIO2f/hbt0X3wQAHsiWJ1yOqfDr9LL2xOTwQBA7w97UypdKUvGFHkmFYxERERERETecNHR0bRp04ZKlSpRu3ZtS8cREREReadcuhtGmzl7CY+KAUz8nOZHIu4foqOPN4/+uazKJ7k+oVGORpYNKvIcyX5JOhERERERkTfZhQsX+Pbbbzl16hS3b9/GZDLFaTcYDGzcuDFBY4wfP54TJ06wdOnSl9rv4sWLXLp0Kc5jhw8fBiAqKorIyMgE5UqoyMjIZJFDRORto+OrSOIJDY+iVeBuboWGAzDFdy3p7v5Gs1Q+3LaxBqBM2jJ0zttZf3PvgOR0fI2KinrpfVQwEhERERERSSJr166lVq1aREREYDQa8fT0TPQxzp07x4ABA+jfvz9+fn5Pvb5rfGbNmhXvEnahoaEEBwcnUspXExUVxYMHDwCwsdHHVxGRxKLjq0jiiI4x0X35Sf6+HgpAJ49dVLw3j/a+KTljZwdADvccfJnnS0Lvh1oyqrwmyen4Ghr68r9z+j+CiIiIiIhIEunTpw9eXl4sX76cAgUKJMkY7dq1I1OmTHzxxRcvvW+rVq2oWLFinMcOHz5M27ZtMRqNuLm5JVbMV/L4zExXV1dsbW0tmkVE5G2i46tI4hiy5gS/n70HQBXjSbqFB9DfKwW7HB0ASO2cmsllJuPpmPgnDUnylJyOr0aj8aX3UcFIREREREQkiZw4cYIhQ4YkWbFo3rx5/Pbbb2zbtu2VPpCmS5eOdOnSPbXNxsbG4h9y/50jOWQREXmb6PgqkjBzd55n9p8XAMhtd43JVuP4zujICpfYL+ld7FyYVn4avq6+lowpFpBcjq+vMsNJBSMREREREZEk4u3tjd0/y5EktvDwcL744guqVKmCr68vp0+fBuDy5csABAcHc/r0aby8vHB3d0+SDCIiIiLvoq0nbzLwl6MAeBpCWOIyjrVEMcXDCwAbKxsmlJ5AJvdMlowp8tKsLB1ARERERETkbfXJJ5+wdOnSJOn74cOH3Lx5k9WrV5M1a1bzrXTp0kDs7KOsWbMyc+bMJBlfRERE5F108vp9Os7fR3SMCXsiWJMygGPRN+nv/f9l5wYVHcSHqT60YEqRV6MZRiIiIiIiIkmkRYsWbN68mRo1atClSxcyZsyItbX1E9ulT5/+pft2dnZmyZIlTzx+8+ZNPv/8cypVqkSrVq3ImzfvK2UXERERkbhuhYbTMnA398OjMBDDEp85hIUdo0sqHyINBgDa52tP9czVLZxU5NWoYCQiIiIiIpJEcuTIgcFgwGQysWrVqni3i46Ofum+bW1tqVu37hOPBwUFAZA5c+antouIiIjIy3sUGc1nc/Zw6e5DACZ4ryTt/S00Se1DyD8nBFXLVI32+dpbMKVIwqhgJCIiIm+su3fvcvfu3Vf6olVELCsmJoaoqChu376NlVXSrZRtZWWFvb09vr6+r3TR14Tq378/hn/ONhURERGRN5PJZOLLnw6x78I9ADq47aBi6GJa+abkkq0tAAV9CzKo6CC995M3mgpGIiIi8kZ69OgR165dA8Da2lpvykXeMAaDARsbmyT92zWZTERGRhIREUFERATp06d/7UWjgQMHvtbxAPz8/DCZTK99XBEREZG31cSNp/jl4BUAKjke54vI6fT09uSQgz0AGd0yMr70eGytbS0ZUyTBVDASERGRN9KNGzcASJMmDa6urhZOIyIvKyYmhujoaKytrZN0hpHJZOLKlSuEhIRw7do10qZNm2RjiYiIiMjbZ8WBy0zYcAqAnNaXmGIzgYlGI785OwGQwiEFU8tNxc3ezZIxRRKFCkYiIiLyRoqIiMBgMODi4mLpKCKSjBkMBlKnTs39+/cJDw+3WI7o6GhOnDjB3bt3iYmJeaK9ZMmSFkglIiIiIs+y9/wdev50CAAvgvnJdQI/25j4wT32pEV7a3sml51MOpd0lowpkmhUMBIREZE3kslkwsrKSkvRichzGQwGrKysnlqoeR1GjhzJiBEjCAkJiXcbXYtNREREJHm5eCeMz+bsJSIqBgfCWek1hX3Rdxjm6Q2AAQMjSowgr3deCycVSTxJt/aDiIiISBJTsUhEXpSljhezZs2iT58++Pv7M2TIEEwmE127dqVnz56kSJGCAgUK8P3331skm4iIiIg8XcijSFrN3s3tBxEYiGGB5/eERJykR0ovov95X9m9QHfKZyhv4aQiiUszjEREROSdExwWyZK9F9lw/Dr3H0Xh4mBDhVy+1H0/LW5OukipiCSeadOmUbhwYTZv3szt27fp168fH3/8MWXLlqVLly74+/trdpGIiIhIMhIVHUPHH/dz8nooAGM9lpHq0Q6apPYh7J9rbzbI3oBmuZpZMqZIkkjwDKPTp0+zbt26OI/t2rWLatWqUaxYMWbMmJHQIUREREQSzeLdFyk0bANDVh9n59k7HL0Sws6zd/hm1TEKDdvA4t0XLR1RXlK7du3o3r37ax3TaDSyf//+1zZe6dKlGTNmzGsbTxLP8ePHqVevHvD/WU6PC0SpUqXis88+Y+LEiRbLJyIiIiJxDV51jG0nbwLwuXEbFR/9TEcfb27YxM69KJGmBL0/7K0VL+StlOCCUa9evRg5cqT5/q1bt6hcuTK//vorR44coX379ixfvjyhw4iIiIgk2OLdF/ly6SEeRT39OiaPomL4cumhJCkaff755xgMBv7888/nbvvf4sCFCxfIkSMHrVq1Ijo6mgcPHtCxY0dSp06Nm5sb9erV4/bt23H6OHHiBFWqVMHV1RUPDw/zF9Zvo+nTpzN27Ngk6TsoKAiDwcCtW7fiPB4aGkr+/PmTZEx5u1hbW+Ps7Axg/u+//179/Pw4deqURbKJiIiISFyBf5xjzp/nAfjI7jDdor+je0ov/ra3AyBnipyMKTUGGyst3CVvpwQXjPbs2UP58v9fq3HBggWEhISwb98+bt68SaFChXTGnIiIiFhccFgk/VcceaFt+/9yhOCwyEQb++HDh/z444+kSJGCmTNnvtS+x48fp1ixYlSvXp1Zs2ZhbW1Njx49OHr0KIcOHeLKlSvY29vTtGlT8z7Xrl2jdOnS1KhRg6tXr3L9+nV69eqVaM8nsUVGJt5r/TbS6/NmS58+PefOnQPA3t6edOnSsX37dnP77t27SZEihaXiiYiIiMg/Np+4weBVxwDIaXWBqXaTGJHClT+cHAHwcfJhSrkpONk6WTKmSJJKcMHo5s2bpE6d2nx/3bp1FCtWjDx58mBnZ0fDhg05duxYQocRERERSZCf9l2Kd2bRfz2KjGHpvkuJNvaSJUuwsrJiwoQJLF68mNDQ0Bfab9euXZQoUYJOnToxatQo8+NLly6ld+/eeHl54ezszJAhQ1i3bh3nz8eeCTd+/HhKly5N27ZtcXZ2xs7OjgIFCrxw3gsXLvDRRx/h5eWFh4cHH330UZwZEC1atKBFixbUq1cPFxcXcuXKxa+//vrS7Q0bNsTNzY2xY8cSGRlJ3759yZAhA15eXtSsWZNLl2J/BgEBAWTKlImQkBAADh06hKurK7t27TL317FjR+D/M4JmzZpFlixZMBqNdOjQgVu3blG9enVcXV354IMP4jyfcePGkS1bNoxGIxkzZowzu+vDDz8EYmeBGI1GZs2aBcQuLbZnzx4ATCYT48ePJ0uWLHh4eFC2bFmOHj1q7qN06dJ8+eWXVK5cGaPRSJ48edi5c2e8r3/p0qXp3r07lSpVwmg0snTpUgBu3LgRbx+hoaG0bduW1KlT4+vrS/Pmzbl79+7zf9iS5EqWLMnq1avN9+vVq8e3335Ly5YtadGiBTNnzqRKlSoWTCgiIiIiJ66F0GnBfmJM4M1dlrhMYJ6zFYtdXQBwtnVmarmppHRKaeGkIkkrwQUjZ2dn7t27B8Suxf37779TsmRJc7ujo6P5w72IiIhIUmoycyelR29+6m30uhMv1deodSfi7av06M00mRn/F/7/NXPmTBo2bEj9+vWxtbVl4cKFz91n69atVKxYkVGjRvHll1/GaTOZTJhMpjj3AQ4ePAjAli1bcHV1pUSJEnh6elK4cGE2b95s3v7333/H3d093rFjYmLo2rUrFy9e5OLFi6RKlSrODCaInVXeuHFj7t69S58+fahZsyaXL19+4faFCxfSrFkz7t69S+fOnRk+fDgrVqxgy5YtXLhwAW9vb2rWrInJZOLzzz/H39+fzz77jLCwMBo2bMhXX31FoUKF4n0OGzdu5MCBAxw6dIi5c+dSqVIlvv76a+7cuUPOnDnp2bOnedv06dPz22+/cf/+febPn8/gwYNZu3YtAH/99RcQW4gKDQ2lVatWT4w1d+5cRo0axU8//cT169cpVaoUlSpVIiwsLM42Q4YMITg4mEqVKtG6det4swPMmTOHgQMHcv/+fWrUqPHcPjp37syJEyc4ePAgJ06c4NatW0/NKq9fly5d6NChAw8fPgRg0KBBVKlShdmzZzN37lwqVKjAiBEjLJxSRERE5N114/4jWgXuITQ8CkcesdxjEjusgxmXwgMAa4M1Y0uNJXuK7BZOKpL0Elwwyp07N3PmzOH27dt89913hIaGUqFCBXP7+fPn8fb2TugwIiIiIs91+e5Dgm6HPfX2orOLHnsUFRNvX0G3w7h89+EL9XPy5Em2b99O8+bNsbe3p2HDhuZZKs+ydetWUqRIQbVq1Z5oq1q1KsOHD+f69euEhITQt29fDAaD+SSd27dv8+OPPzJ06FCuXbtG27ZtqVatmnnGTvHixc0n/DyNn58fVapUwdHREaPRyKBBg/jrr7948OCBeZtSpUpRq1YtbGxs+OSTT8iXLx9Llix54fby5ctTpUoVrKyscHJyYu7cufTt25eMGTPi5OTEuHHjOHDgAIcPHwZg1qxZ/PnnnxQpUoT06dPHKfg8Tb9+/TAajWTKlInChQtToEABChYsiI2NDfXr12fv3r3mbevWrUuGDBkwGAwULVqU2rVrxymwPc/cuXPp2LEj/v7+2NnZ8fXXXxMVFcWGDRvM2zRu3JgPPvgAa2trPv30U44ePcqjR4/i7bNBgwYULlwYg8GAo6PjM/uIiYlh/vz5DBs2DG9vb9zd3Rk9ejTLli3TiVvJQPbs2Wnbtq355+js7Mwvv/zCnTt3CA4OZu3atVqSTkRERMRCHkVG89mcvVy+9xArYpjn8R03Y87R19vTvM1Xhb+iWJpiFkwp8vokuGDUs2dPDh8+TMqUKenQoQP58+enRIkS5vb169fz/vvvJ3QYERERkedK4+GIn6fTU28ONi/3tsfBxirevvw8nUjj4fhC/cycOZMcOXKYZ8O0aNGCnTt3mpcsMxqN5tv8+fPN+/Xp0wd/f39Kly7N9evX4/Q5YcIEsmfPToECBciZMycffvghRqMRT8/YDzUuLi7UrFmTkiVLYmtry6effoqfnx+bNm16ocy3bt3ik08+IV26dLi6upIvXz7z449lyJAhzj5+fn5xZhA9rz19+vRx2i9duoSfn5/5vouLC15eXuYil4eHB40bN+bQoUP07t0bg8HwzOfg4+Nj/reTkxO+vr5x7t+/f998f8GCBXzwwQekSJECd3d3FixYwM2bN5/Z/7OyW1lZkSFDBnN2gFSpUpn/7ezsDPDMpQn/+/o8q4+bN28SERERJ0OmTJnM2SR5cnNzw2g0WjqGiIiIyDsrJsZE9yUHOXDxHgCjXJeQMnI3nX28CbeK/fzYMk9L6mara8GUIq+XTUI7+Pjjj9m0aRMrVqzAzc2Njh07mj/A3759m7Rp09KsWbMEBxURERF5nvmtC8fbNuv3c3yz6sWvq/hlpRy0LJ4xQXkiIyOZM2cOwcHBcQoWEDtjZty4cfEWDWxtbVm8eDGNGjWiVKlSbNq0yXzdSDc3N2bMmGHe9ujRo/Tt25fChWOff758+YiOjn7l3H369OHOnTvs2bMHHx8fgoKCyJgxY5xl8B5fL+mxoKAg8/gv0m5lFbeAlzZtWoKCgihWLPbMvdDQUG7dukXatGkB2Lt3L1OnTuXTTz+lU6dO7N69GwcHh1d+jo9dvHiRpk2bsnr1asqVK4etrS0tWrQwP9f/5nyax9kfi4mJ4cKFC+bsr+JFxn3M29sbOzs7goKCSJMmDYA5z+P7IiIiIiIS1/gNJ1l96CoA7Rw3USFqJU1T+XDX2hqAin4V6fJ+F0tGFHntEjzDCGIv5Dp27Fj69+8fZzkFT09Pfv75Z2rWrJkYw4iIiIi8srrvp33hWUYOtlbU+eDVv+x/bOXKldy9e5e9e/dy4MAB82306NHMnTuXiIiIZ+5vY2PDggULyJ8/P6VLlzbPFjl37hzXrl3DZDJx/PhxWrZsSY8ePfDwiF1j+7PPPmPFihXs2LGDmJgY5s6dy4ULFyhXrtwL5Q4JCcFoNOLh4cHdu3fp16/fE9ts3bqVX375haioKObPn8/BgwepU6fOC7f/V9OmTRk+fDhBQUE8fPiQ7t27ky9fPvLkycP9+/dp0KABQ4cOZebMmaRMmZJu3bq90HN5ntDQUEwmEz4+PtjY2LBx40aWLVtmbvf29sbKyoozZ848M3tAQACHDh0iIiKCoUOHYmVlRfny5RMl4/NYWVnRuHFjvvrqK27evMm9e/fo2bMnNWvWxM3N7bVkkP+zsrLCxsbG/PdtZWWFtbX1M282Ngk+j09EREREXsLP+y4xedNpACrYHqQb39MlpRdBdrYA+Hv7M7T4UKwMifL1ucgbQ59MRERE5J3g5mTL4Bp5+HLpoeduO7h6HtwcbRM85syZM2nUqBG5cuWK83i7du0YOnQoy5cvp379+s/sw8bGhnnz5tGiRQvzTKODBw/SuXNnbt26ha+vL+3bt6dHjx7mfYoWLcq0adP45JNPuHHjBjlz5mTVqlXm2Sbbt2+ncuXK8c5uGjx4MM2bNydFihSkSpWKr776ih9//DHONo0aNWLOnDk0adKEtGnTsnTpUtKlS/fC7f/Vp08fHj58SIkSJQgLC6NYsWIsW7YMKysr2rVrR+7cuenUqRMQe82gfPnyUaFCBWrXrv3M1+95cubMyYABA6hQoQJRUVFUqlSJevXqERUVBYCjoyMDBgygevXqhIeHM378eD799NM4fTRr1oybN29Sq1Ytbt++Tf78+Vm7di1OTk4JyvYyJk6cSPfu3XnvvfcwmUx89NFHTJgw4bWNL//XrFkzDAYD1v+cmfr4voiIiIgkD7uD7tB7aey1UnMazhNgP5mvPDzY6xi7gkE6l3RMKjsJe2t7S8YUsQiD6d9ri7yCsmXLPn8Qg4GNGzcmZJhk588//6Ro0aLs2LGDIkWKWDRLZGQkwcHBuLm5YWub8C+3REQklo6vydupU6cAyJo160vtt3j3RfqvOMKjqJgn2hxsrRhcPQ/1C8Zf2JDY6zAZjUamTJnySu0iELt0X3R0NNbW1i+1BN+riu+YkZze1ycXyek10f+LRUSSho6v8q46f/sBNaf+wd2wSHy4wwbXwcx2Cudbj9iZ+W52bsyrMg8/Nz/LBpU3VnI6vr7K+/oEzzA6e/bsE2fMRUVFcfXqVWJiYvDy8jJflFdERETE0uoXTEfF3L78tO8SG45dJ+RRJK4OtlTI5UOd99Pi5qQPzCIiIiIiIm+b4IeRtAzczd2wSJx4xE9uE9lg+4BvPTwBsLWyZVLZSSoWyTstwQWjf1/g99/Cw8MZN24cP/zwA1u3bk3oMCIiIiKJxs3JllbFM9KqeEZLRxGRd8SVK1dYuXIlZ8+eBSBTpkxUrVrVvFSkiIiIiCSdyOgYOszfx5mbD7Aihtmu07lsuMBgr5TmbYYUG8L7Pu9bMKWI5SXZNYzs7e3p06cPx44d44svvmDBggVJNZSIiIiIvEaBgYEJahd513zzzTcMGTKEqKgo/r0ieKdOnejXrx8DBgywYDoRERGRt5vJZGLAL0f5/fQtAEY6L8DDtJ9mKX2I+mflrM75O1MlUxVLxhRJFpJ8sfDixYvz66+/JvUwIiIiIiIiyc6UKVMYMGAA/v7+zJ8/nwMHDnDgwAHmz5+Pv78/gwcP1vW+RERERJLQrN/P8eOuCwC0tV9PGdbyua83961jvxqvnbU2rd9rbcmIIslGks0weuzcuXNEREQk9TAiIiIiIiLJzuTJk/nwww/5/fffsbH5/8evvHnzUrduXYoVK8bkyZPp2LGjBVOKiIiIvJ02HLvO0DXHAShvvZfO1nNp5ePN1X/elxVOVZivCn+F4Z+ZRiLvugQXjC5cuPDUx+/cucOGDRuYNGkSpUuXTugwIiIiIiIib5wLFy7w+eefxykWPWZra0uTJk3o3bu3BZKJiIiIvN2OXgmm88L9mEyQ23COKQ5T+dLTg6P29gBkcc/CuNLjsLWytXBSkeQjwQUjPz+/eCuwJpOJ7NmzM2nSpIQOIyIiIiIi8sZJnz499+/fj7f9/v37pE+f/jUmEhEREXn73Qh5ROvZewiLiCYVt1loHM9EFwc2OzsB4OXoRUC5AFzsXCycVCR5SXDBqH///k8UjAwGAylSpCBbtmyUL18eK6skv1SSiIiIyPMd+BFy1QQ7p6e3R4TBseXg3/h1phKRt1jHjh0ZNWoUrVq1IlWqVHHaLl++zPTp0zXDSERERCQRPYyIpvWcPVwNfoQzD1nkMp5fHB8xzy0FAI7WDkwpN4VUxlTP6Unk3ZPggtHAgQMTIYaIiIhIEjvwIyxvDwcXQKNFTxaNIsJgQQM4ty32/jtcNAoMDGTMmDEcOXLE0lFeS5akHqNdu3Y4OzszduzYJOk/MZQuXZqqVavSo0cPLly4QK5cuTh//jyenp6WjvbGmTNnTpz7bm5u+Pj4kCNHDpo2bUqOHDkAOH78OPPnzydbtmy4urpaIqqIiIjIWycmxkS3RQc4dCkYa6L5wRjAGdurjErhBYCVwYpRpUaT2zO3hZOKJE8JLhiJiIiIvBFy1YwtFp3bFlsY+nfR6N/FoowlY7dNZJ9//jnTpk1jx44dFClS5Jnb/vvLe4i9BspHH31EsWLFmDFjBo8ePaJXr178/PPPPHjwgI8++ojp06fH+XL/xIkTfPHFF/z+++9YW1tTvnx5lixZkujPS+L6788OYPr06RZM9PLSp09PaGiopWO8sVq0aIHBYMBkMj3RNm3atCce27t3L59++inNmjV7HfFERERE3mqj1//NuqPXABMjHOfhbHWIjt4+xPyzQtaXBb+kdLrSFs0okpy9dMFo27bYs25LliwZ5/7zPN5eRERExCLsnGKLRI8LQ4+LRhC3WPS02UcJ9PDhQ3788UdSpEjBzJkzn1sw+rfjx4/z0Ucf0ahRI0aNGgVAjx49OHHiBIcOHcLR0ZG2bdvStGlT1q5dC8C1a9coXbo0gwYNYsmSJdja2nLo0KFEfU7JkclkIjo6Ghubd/ecqMjISGxtddFeS9q8ebOlI4iIiIi8kxbvuci0LWcA+Mx2HcWsN9DEx5eH/1wupWnOpjTJ2cSSEUWSvZe+uFDp0qUpU6YMERERce7Hd3vcLiIiImJxj4tGGUvGFojm1Ii9JWGxCGDJkiVYWVkxYcIEFi9e/MKzN3bt2kWJEiXo1KmTuVgEsHTpUnr37o2XlxfOzs4MGTKEdevWcf78eQDGjx9P6dKladu2Lc7OztjZ2VGgQIGXzt2/f3+8vb1JkyYNI0eOjNM2f/58cuXKhZubG4UKFeL3338HYP/+/bi6urJ3714AQkNDyZ49O+PGjQPgwYMHdO7cmfTp0+Pt7U2DBg24deuWud/jx49TrFgxXFxcKF68OOfOnXtmRj8/P4YMGULRokVxdnZmz549CR7Dz8+Pn376yXx/y5YtGI1G8/2IiAj69+9P5syZcXFx4b333mPfvn10796d7du3069fP4xGI2XLlgViZ5x07NjRvP/evXspXrw47u7uZM+enRkzZpjbAgMDyZMnD8OGDcPX1xdvb28GDRoU7/N/vP2gQYNIlSoV1atXB+CTTz4hTZo0uLi44O/vz7p16+LsN336dDJkyECKFCn44osv4syGCQoKwmAwmF+z/+aH2GuW7tmzB4B9+/ZRpEgRXF1d8fLyonTp0vHmfReUKlXqlW4iIiIi8up2nr1Nv2WHAfjIajcdbebTwcebWzbWAJRJV4YeBXo8qwsR4RVmGH3//fcYDAbzmYs//PBDoocSEREReWU7p8Nf3z57G1MMuKWHS3/F3k/7IVQeDdOLPb9/l9Tw6eqXijRz5kwaNmxI/fr16dKlCwsXLqR169bP3Gfr1q0MGTKEcePG0bJly7jxTaY4X/A//vfBgwfJkCEDW7ZsIV++fJQoUYJjx46RNWtWhg8fbj6J5/fff6dq1arcu3cv3vFPnDhBeHg4ly5d4siRI1SsWJEsWbJQp04dtm3bRtu2bVm5ciUlSpRgzpw5VKpUiZMnT5I/f36++eYbGjZsyP79++nQoQNZsmShW7duALRq1Yro6Gj279+Pk5MTnTp1okWLFqxatYqoqCiqV69OnTp12LRpE0eOHOHjjz/Gy8vrma/V7Nmz+eWXX8iWLRtRUVE0b9480cf4t969e7N161bWrVtHlixZOHnyJA4ODowdO5a9e/c+sSTdv927d4+KFSvSr18/Nm/ezL59+6hcuTK+vr7mYs/ff/8NwMWLFzl48CBFixalSpUqFCxYMN6fVUxMDEFBQURHRwNQpkwZJk2ahIuLC9999x316tXj7NmzeHt7s3XrVnr27Mm6dev48MMPGT16NDt27KBatWov/Br8W8eOHalSpQq///470dHR7Nix45X6eVc8LsS9zO+ciIiIiMTv3K0HtJu3l8hoE+8ZzjLBIYAuKT05bWcHQB7PPIwoMQJrK2sLJxVJ/l66YNSiRYs495s3b55YWUREREQS7uFduHP2+du5pIp7PybqxfaLjnqpOCdPnmT79u2MHj0ae3t7GjZsyKxZs16oYOTl5fXUL/GrVq3K8OHDyZ8/P46OjvTt2xeDwUBISAgAt2/f5scff2TNmjUUKVKEefPmUa1aNU6cOEHatGkpXrz4M4tFAI6OjnzzzTfY2dnxwQcf0Lp1a+bOnUudOnWYO3cujRo1MhegWrZsyfTp0/npp5/o3LkzXbp0YePGjRQrVoxbt25x8OBBDAYDN2/eZPHixVy/ft18vaVhw4bh4+PD3bt3OXr0KNeuXWPw4MHmcR8Xep6lbdu25MyZE4A7d+4kyRiPmUwmvv32W1avXk3WrFkByJ49+wvtC7B69Wo8PDzMBbRChQrRunVrAgMDzQUjd3d3+vTpg8FgoECBAuTLl499+/bFWzAyGo30798fa+v/fwD+d5Gxffv2jBw5kt27d1OlShXzz69YsdgCae/evZkyZcoLP4f/srOz4+LFi1y+fJn06dO/8zOMnubKlSv06dOHFStWcP/+fQBcXV2pUaMGQ4cOJU2aNBZOKCIiIvJmuhcWQavA3dwLiyQNN5nnNI4RHs7sdHQEILVzaiaXm4yTbeKvJCHyNnrpJelEREREkjVHD0iRKf6bhx/YOMD9q7Ezi9J+GDvTaFW32LZn7ZsiE7inf6k4M2fOJEeOHBQqVAiIPflm586dHD16FIj9sv/xbf78+eb9+vTpg7+/P6VLl+b69etx+pwwYQLZs2enQIEC5MyZkw8//BCj0WgukLi4uFCzZk1KliyJra0tn376KX5+fmzatOmFc/v6+mL3zxl5ELtM2+XLlwG4dOkSfn5+cbbPlCkTly5dMt/v0KEDhw4dok2bNuaZFEFBQZhMJrJmzYq7uzvu7u5ky5YNe3t7Lly4wOXLl5867vOkT///n0lSjfHYzZs3CQsLI0uWLC+8z7+9yGvn4+OD4Z+L8gI4OzubiwxPkzp16jjFopiYGL7++muyZcuGq6sr7u7uXLx4kZs3bwJw+fJlMmTIYN7eysoqzmv4sn744QfCw8MpVKgQ2bJlY8yYMa/c19vowoULFChQgLlz55IpUyYaN25M48aNyZQpE3PmzOHDDz/k4sWLlo4pIiIi8saJiIqh/bx9nL31ABfCWGAczyJjFMtcYpeTdrE1ElA+AC9HzewWeVEvPcNo27ZtrzRQyZIlX2k/ERERkZdSuF3s7WkiwmBBA7gb9P9rFkHsY0lwHaPIyEjmzJlDcHAwvr6+cdpmzZrFuHHj4r2eka2tLYsXL6ZRo0aUKlWKTZs2kTp1agDc3NziXPfm6NGj9O3bl8KFCwOQL18+89Jkr+ratWtERESYCytBQUHmWRBp06YlKCgozvZBQUEULVoUgODgYNq3b0+bNm0YP348jRo1Inv27KRPnx6DwcCFCxdwdXV9Ysz79+8/ddznsbL6/zlQiTGG0WgkLCzMfP/KlSvmf3t7e+Pk5MTp06dJmzbtM7M8TXyv3dP6elH/HXPBggXMnj2b9evXky1bNqysrPDz8zMvXZgmTRrz9a4gtsD0rILFs14PgIwZMzJ79mwAdu/eTfny5Xn//ffN13B613399dfcvXuXVatWUaVKlThta9eupXbt2nz99dcEBgZaJqCIiIjIG8hkMvH18iP8efY2NkTxvdNkjtjfYHKK2OKQjcGa8WUmkNk9s4WTirxZXnqGUenSpSlTpsxL30REREQs6nGx6L+FITun2H9nLBnbtqBB7LaJYOXKldy9e5e9e/dy4MAB82306NHMnTuXiIiIZ+5vY2PDggULyJ8/P6VLlzbPQjl37hzXrl3DZDJx/PhxWrZsSY8ePfDw8ADgs88+Y8WKFezYsYOYmBjmzp3LhQsXKFeu3Atnf/jwIQMGDCAiIoL9+/cza9YsmjRpAkDTpk1ZsGABW7duJSoqisDAQI4cOULdunXN47///vvMmDGDbt260bBhQ8LDw/Hx8aFu3bp06NCBGzduAHDjxg1++uknIHZ5Nh8fHwYOHGge93Eh4kUlxhjvv/8+CxYs4OHDh1y4cIFx48aZ2wwGA23atOHLL7/k9OnTmEwm/v77b3MBxsfHhzNnzsSbr0qVKty5c4eJEycSGRnJ7t27+e677xJ1meeQkBDs7Ozw9vYmKiqKMWPGxJnB1LhxYxYuXMiff/5JZGQko0aNMr9WT/P++++zfv16Ll++TGhoKF999VWc9jlz5nDt2jUgtphpbW0dZ8bTu279+vV8/vnnTxSLACpXrkz79u1Zt26dBZKJiIiIvLlmbDvLoj0XARMjHGZjZXuCr7w9ze0Diw6iUKpClgso8oZ66YLRDz/8wPfff2++zZo1C39/f5ycnMxnkY4fP57WrVvj7OxM/vz5mTVrVlJkFxEREXlxx5bHP4vov0WjY8sTZciZM2fSqFEjcuXKha+vr/nWrl07oqKiWL78+ePY2Ngwb948ChUqRKlSpTh//jwHDx7kww8/xNnZmY8//pi6desyePBg8z5FixZl2rRpfPLJJ7i5uTF58mRWrVplniG0fft2jEbjM8fNkSMHtra2pE6dmqpVq/LFF19Qr149AEqVKkVAQADt2rXD09OTgIAA1qxZQ+rUqZkxYwa7du1i5syZQOzsChcXF3r27AnA999/j4+PD4UKFcLFxYUiRYrw+++/A7GzqlasWMHmzZtJkSIFnTp14rPPPnvp1z2hYwwZMoSwsDBSpkxJrVq1aNasWZz2kSNHUr58eSpUqICrqyv16tXjzp07AHTt2pWtW7fi7u5OhQoVnsjm4eHBunXrWLx4MV5eXjRp0oQhQ4ZQo0aNl36e8WnevDn+/v5kzJiRDBky8ODBA/LkyWNuL1u2LMOHD6d+/fqkTJmSGzdumGeHPU3Tpk0pV64cuXLl4r333qNixYpx2jds2IC/vz9Go5GPPvqIXr16UapUqUR7Pm+6u3fvmq939TRZs2Z97jXFREREROT/fj16jRHrTgDQzmY1BWy20sXHi8h/lnVum7ctNbIk3vtrkXeJwfR4bYpXNHHiRMaOHcvOnTvNy6Q8dunSJYoUKcKXX35Jp06dEhQ0ufnzzz8pWrQoO3bsoEiRIhbNEhkZSXBwMG5ubtja2lo0i4jI20TH1+Tt1KlTAM/8IvYJB36EXDXjX3IuIiy2WOTfOMH5ROTZYmJiiI6Oxtra+rlL+SWG+I4ZSf2+PkuWLBQoUICFCxc+tb1Ro0bs3r2b06dPJ/rYr0qfdURE3n46vsqb6sjlYOpN/5OHkdFUttrFUIfJNE3tw8V/fo+rZqrKsOLD4lwTVOR1Sk7H11d5X5/gT2ZTpkyhbdu2TxSLIHaN9rZt2zJ58uSEDiMiIiKScP6Nn319IjsnFYtEJFHVq1ePJUuW0KdPH4KDg82Ph4SE0LdvXxYvXkyDBg0smFBERETkzXAt+BGtZu/mYWQ0/obTjHKYRmcfb3Ox6AOfDxhUdJCKRSIJYJPQDi5evIiTU/xfvDg7Oz/zIroiIiIiIiJvq6+//prt27czcuRIxowZYz7R7sqVK0RHR1OsWLEnrgslIiIiInE9CI+i1ezdXA8JJ63hBoGO4xjo6cJBB3sA/Fz9mFhmInbWdhZOKvJmS/AMo4wZMzJv3jwePXr0RNujR4+YM2cOfn5+CR1GRERERETkjePk5MSWLVv49ttv+eijj3B2dsbZ2ZmKFSsyY8YMNm/ejKOjo6VjioiIiCRb0TEmui46wNErIbjygB8dx/KDm4n1RmcAUth7EFA+ADd7NwsnFXnzJXiG0RdffEHbtm0pWLAgHTp0IHv27ACcOHGCqVOncvz4caZPn57goCIiIiIiIm8iGxsb2rRpQ5s2bSwdRUREROSNM3LdCX47dh0bopjpOIldznf43t0TAHtreyaVm0w6l3QWTinydkjwDKM2bdowbtw4zp07x+eff0758uUpX748HTp04Ny5c4wePTpBH4xiYmIYP348OXLkwMHBgXTp0tG9e3cePHjw0n2FhYWRKVMmDAYDHTt2fOVMIiIiIiIiLyIqKoqQkJB420NCQoiKinqNiURERETeHAv/usCMbWcBE8PtfiDC4RRDPVOY24cVH0Y+73yWCyjylknwDCOArl278umnn7J+/XrOnj0LQKZMmahQoQLu7u4J6rtbt25MmjSJWrVq0b17d44fP86kSZPYv38/GzZswMrqxWte/fv35+bNmwnKIyIiIiIi8qK6d+/O2rVrOXny5FPbCxYsSNWqVRk7duxrTiYiIiKSvO04fYuvlh8BoL31SvI6/E7zlD5EGwwAdP+gOx/5fWTJiCJvnUQpGAG4ublRr169xOoOgKNHjzJ58mRq167N0qVLzY9nzJiRzp07s3DhQho3bvxCfe3bt48JEyYwatQounfvnqg5RUREREREnubXX3+lTp068bbXqVOH5cuXq2AkIiIi8i9nbobSbt5eomJMfGy1kxYOS2js48ODfyYP1M9Wn+a5m1s4pcjbJ8FL0iWlBQsWYDKZ6Nq1a5zH27Rpg5OTE/PmzXuhfqKjo2nTpg2VKlWidu3aSZBURERERETkSRcvXiRz5szxtmfKlImLFy++xkQiIiIiydvdBxG0DNxNyKMo3jecZKjDt3T08eaGTezch+JpitOnUB8M/8w0EpHEkygzjM6cOcP48ePZtWsXd+/eJSYmJk67wWDgzJkzL93v7t27sbKy4sMPP4zzuIODA/7+/uzevfuF+hk/fjwnTpyIM0tJRERE3l3B4cGsOL2CLZe2EBoRitHOSJl0ZaieuTpu9m6WjicibxE7OzuuXr0ab/u1a9deapltERERkbdZeFQ0beft5fztMNIZrjPTcTx9vV05YW8HQI4UORhTagw2Vom2cJaI/EuC/7IOHz5M8eLFCQ8PJ3v27Jw9e5bcuXNz+/Ztrl27RubMmUmbNu0r9X3lyhW8vLywt7d/oi1NmjTs2LGDiIgI7Ozs4u3j3LlzDBgwgP79++Pn50dQUNBL57h48SKXLl2K89jhw4eB2IvYRkZGvnSfiSkyMjJZ5BARedvo+Jq8xcTEYDAYnjhR5XmWnV7G8L+GEx4dHufx3dd2M3HfRPp82IdaWWolZlRJYvPnz2fy5Mns3LnztY1ZpUoVKleuTKdOnV7LeIMGDWLPnj2sXLnytYz3OphMJvN/X/bv+FXHM5lMTxzTo6KiknRcf39/Fi9eTK9evZ743BIZGcmiRYvImzdvkmYQEREReROYTCb6/nyEv87dwZVQ5jmMIcDDiu1OjgCkdErJlLJTcLZ1tnBSkbdXggtG/fv3x87Ojr/++gtPT09SpkzJxIkTKVu2LN999x19+/ZlxYoVr9R3WFjYU4tFEDvL6PE2zyoYtWvXjkyZMvHFF1+8UgaAWbNmMWjQoKe2hYaGEhwc/Mp9J4aoqCgePHgAgI2NqusiIolFx9fkLSoqChsbG6Kjo194n+VnljN41+B428Ojwxn450BiYmKomblmIqSE8uXLs3PnTmxtbYHYk16+/PJLmjVrFu8+gwcPZt++fSxfvjw2V3g4TZo04dKlS6xcuRJvb28CAgKYPHky165dI0+ePEyaNIn8+fOb+wgODqZPnz4sX76cR48ekTlzZtasWYO3t3eiPK/kpGHDhjRs2PClfhdeRtasWRkxYkSc69A8Ltwk1Zj/9big8rrGe11e5/MxmUxERUU98d49NDQ0Scft2LEj9erV4+OPP2b48OHkzZsXg8HAwYMH6du3L8eOHePHH39M0gwiIiIib4KALWdYuu8StkQxy2EiW1xCWOTqAYCTjRMB5QLwcfaxcEqRt1uCv/36/fff+eyzz8iePTu3b98G/n+2YJs2bdi+fTu9e/fml19+eem+nZycuHHjxlPbHj16ZN4mPvPmzeO3335j27Zt5i9pXkWrVq2oWLFinMcOHz5M27ZtMRqNuLlZdumax2dJurq6Juh5iohIXDq+Jm+3b9/GYDBgbW39QtuHhIcwcs/IF9p21J5RVMhQAVd714REBGKX5h0yZAjdu3fHZDKxefNmPv74YwoWLEiePHmeus/j5amsra0JCQmhVq1aGAwGNm3ahIuLC4sWLWLEiBH8+uuv5MyZk2+//ZZq1arx999/4+rqSkxMDHXq1CFr1qwcPXqUFClScPToUVxcXF749XqdIiMjk/3fmJWVlcVeu8jIyDi/E2+Lx58ZrKysXsv68waDARsbmyfeuxuNxiQdt06dOvTp04fhw4dTqFAhDAaDeXakyWSiV69eNGjQIEkziIiIiCR3aw5fZfSvfwMmRtjO5J7TWcam8ALA2mDF2NJjyZ4iu2VDirwDElwwun//vvkiro9n+jw+GxugWLFi9OnT55X6Tp06NceOHSM8PPyJmUaXL1/Gy8sr3tlF4eHhfPHFF1SpUgVfX19Onz5t3g9iz7o9ffo0Xl5euLu7PzNHunTpSJcu3VPbbGxsksUXHI9zJIcsIiJvEx1fk6/HX6C/6LU/Vp5b+cQydPF5FP2IVedW0TRX01fO928Gg8Gcs3z58qRPn54TJ07EuwzV4y+Ub926ReXKlfHz8+PHH380vx9atmwZLVq0MO/fsWNHxowZw4oVK2jevDm//vorZ8+eZcOGDebf3Xz58r1U5k8++YRNmzYREhJC5syZGTFiBJUqVQIgMDCQMWPGULt2baZNm4adnR2dO3emV69eL9Ver149pk+fjr+/P2vXrmX+/PkMHTqUy5cvkyNHDsaOHUvx4sXZv38/pUqVYvPmzXzwwQeEhobywQcf0LZtW7744gtzf0eOHAHAz8+PNm3a8Msvv3DkyBEKFSrE4sWLGTlyJN9//z3Ozs5MmTKF6tWrA7B+/Xr69u3LqVOncHR0pGrVqkyYMAGj0Ui9evW4cOECzZs3p2XLltStW5fAwEBKly5N1apV6dGjBwC//fYbvXv35vTp02TIkIHBgwdTs2ZNAAYOHMhff/1FlixZmDdvHo6OjgwcOJA2bdo89bUfOHAgu3btIlOmTCxatIh69erh4+ODyWSia9euT+3DZDIxYcIEpk6dyu3bt8mfPz+TJ08md+7cL/Vzf50ez5r6999HUnr8d/Xf4/nrmEE6dOhQatasybx588yfS7Jly0bjxo0pWLBgko8vIiIikpwdvHiPbosOANDRejlZnHbSyjslpn9OKupbqB/F0xS3YEKRd0eCPx35+Phw7do1AFxcXHB2dubkyZPm9rt3777yUhMFCxZk/fr1/PXXX5QoUcL8+KNHjzhw4AAlS5aMd9+HDx9y8+ZNVq9ezerVq59onzdvHvPmzWP06NHmD/oiIiLyZmu9vjVXQ59+cfnrYddfqq8J+yaw4MSCeNtTGVMx86OZL9VnTEwMGzZs4OrVq8/9kvjq1asUL16cUqVKMX369DgzSx5fi+XfTCYTBw8eBGDLli1kzZqVFi1asG7dOlKmTEmHDh3o2LGjeXt3d3dWrVpF8eJP/+BVpkwZJk2ahIuLC9999x316tXj7Nmz5iXtTpw4QXh4OJcuXeLIkSNUrFiRLFmymJdte5H2mJgYgoKCiI6OZtu2bbRt25aVK1dSokQJ5syZQ6VKlTh58iT58+fnm2++oWHDhuzfv58OHTqQJUsWunXrFu/rt3DhQlatWoWnpyfly5encOHC9OnThxs3bjBz5kxat27N5cuXsbW1xdHRkRkzZpAvXz4uX75MjRo1+Oabbxg5ciRLlizBz8+PMWPGULdu3aeOdfr0aapVq8acOXOoXbs269evp3bt2uzcuRN/f38gtqDUqFEjxo8fz6pVq8xLlKVOnfqpff72229Mnz6diRMnEhERwahRo57Zx9y5cxk1ahRr164lV65cDB8+nEqVKvH3338/c0a+vD4FCxZUcUhERETkPy7fe0jrOXsIj4qhutUfNHD8maY+voT/c0LRp3k+pX72+hZOKfLuSPCpfP7+/uzZs8d8v1SpUkycOJFt27axZcsWpkyZ8tJntD7WoEEDDAYDEyZMiPP4d999R1hYGE2aNDE/dubMGU6cOGG+7+zszJIlS564BQQEAFCpUiWWLFliPrNURERE3nxXQ69y4f6Fp95edHbRY+HR4fH2deH+hXgLU0/z9ddf4+7ujrOzM5UqVaJHjx5kzJjxmfucOnWKc+fO0apVqyeWIatatSqBgYEcOHCAyMhIJk6cyMWLFwkJCQFil+vbvHkzH374IVevXmXOnDn079+fZcuWmfu4d+9evMUigJYtW+Lh4YGNjQ3t27fH09OT3bt3m9sdHR355ptvsLe354MPPqB169bMnTv3hduNRiP9+/fH3t4eJycn5s6dS6NGjShTpgw2Nja0bNmSXLly8dNPPwHQpUsXcubMSbFixdiwYQOzZ89+5jJm7dq1I0OGDBiNRqpXr46NjY35tWzUqBE3b97kwoULAJQoUYL3338fa2tr0qdPT5cuXdi8efMzfz7/tmjRIkqUKEH9+vWxsbGhSpUqVK9enTlz5pi3yZs3L5988gnW1tbUqFEDNzc3Dh8+HG+fOXLkoHXr1tjY2JgLPs/qY+7cuXTs2BF/f3/s7Oz4+uuviYqKYsOGDS/8PEREREREXqfQ8ChaBe7m5v1wChhO0N/hOz73Scmdfz7/fJThI7q+39WyIUXeMQmeYdS4cWOmTp3Kw4cPzV8MlCpVijJlygCxXxYMGzbslfp+77336NChA1OmTKF27dpUqVKF48ePM2nSJEqVKkXjxo3N25YrV47z58+bz7a1tbV96lmgQUFBAGTOnDnes0RFRETkzZTKmCretuth11+qaGRvbY+PU/wXVH3WWP/1zTff0KNHD0wmE4cPH+ajjz7Cx8eH9u3bU7lyZbZv3w5A06ZNmT59OgAlS5akVKlSVKxYkTVr1lCsWDFzf82aNePatWs0aNCAW7duUbNmTcqXL4+npycQO+s7TZo0dOnSBYid2dC4cWNWrlxJrVq1nps3JiaGAQMGsGjRIq5du4aVlRX379/n5s2b5m18fX3jLA3s5+fHxo0bX7g9derUcQphly5deqKAlSlTJi5dumS+36FDBypVqsSAAQPw8vJ65nPw8fn/z87JyQlfX9849yF2aWWAvXv30qdPHw4ePMjDhw+Jjo4mZcqUz+z/3y5duoSfn98T2c+ePWu+nypV3N8XZ2dn8/hPkz59+icee1Yf/81gZWVFhgwZ4rx+YjlRUVEsX76cXbt2cffuXfNyfI8ZDAZmzZploXQiIiIir190jInOC/Zz4tp9/AxXmeYwgZ4p3Qmy+2dJbe98DC0+FCtD0i9dLCL/l+CCUYMGDeJcpDV//vwcPXqUZcuWYW1tTeXKlcmUKdMr9z9hwgT8/PyYMWMGq1evxsvLi06dOjF48ODXsta5iIiIvDmetUTc3GNzGbV71Av31fX9rol2DaPHDAYDefPmpUSJEqxZs4b27duzdu3aeLfv2bMntra2VKpUidWrV5uX4zUYDPTq1ct8TaCIiAgyZsxoXqItX7585pk5r2LBggXMnj2b9evXky1bNqysrPDz84uzDN61a9eIiIgwF4WCgoJIkybNC7f/931c2rRpzSf2PBYUFETRokWB2OtPtm/fnjZt2jB+/HgaNWpE9uyJc9Hbhg0b0qRJE37++WeMRiOBgYEMHDgw3qz/lTZtWrZs2fJE9rRp075yppd9n/vf1y8mJoYLFy4kKIMkjjt37lCmTBmOHDmCyWTCYDCY/5Ye/1sFIxEREXnXDF19nE0nbuDOfWbbj2Gcly17HB0ASGtMy6Syk3CwcbBwSpF3T4IqLuHh4Wzbto1Tp07FeTxdunR07tyZDh06JKhYBGBtbU337t35+++/CQ8P5/Lly4wbNw6j0Rhnu6CgoCfW8n+ax192TJkyJUG5RERE5M1SPXN17K3tX2hbB2sHqmdJmmVrjx07xvbt23nvvfdeaPuuXbsyfPhwqlSpwqZNm4DY5eROnDiByWTi5s2btGvXjixZslCxYkUAateuTUREBFOnTiU6Opr9+/ezYMGCF5pdBBASEoKdnR3e3t5ERUUxZsyYJ2aqPHz4kAEDBhAREcH+/fuZNWtWnOWCn9f+X02bNmXBggVs3bqVqKgoAgMDOXLkiHlG+Geffcb777/PjBkz6NatGw0bNiQ8/OWWGXzW8/Xw8MBoNHLq1CnGjx8fp93Hx4czZ87Eu3+DBg3Yvn07S5cuJTo6mnXr1rFixQo++eSTRMn3Ipo2bUpAQACHDh0iIiKCoUOHYmVlRfny5V9bBnm6r776ihMnTjBz5kzOnDmDyWTi119/5fjx4zRq1IiCBQty+/ZtS8cUEREReW3m7TzP93+cw45IZtqPZ5V7GKuMzgC42rkSUD6AFA4pLJxS5N2UoIKRtbU15cqVe+aZsSIiIiLJgZu9G/0K9XuhbfsW6ournWuijd2vXz+MRiPOzs589NFH1K9fn/79+7/w/h07dmTMmDFUq1aN9evXc+/ePerUqYPRaCRXrlzY2tqycuVK86wUV1dX1qxZw5w5c3B1daV+/foMGzaMatWqmfs0Go3mpfD+q3nz5vj7+5MxY0YyZMjAgwcPyJMnT5xtcuTIga2tLalTp6Zq1ap88cUX1KtX74Xb/6tUqVIEBATQrl07PD09CQgIYM2aNaROnZoZM2awa9cuZs6MnUH29ddf4+LiQs+ePV/4NXyWGTNmMGnSJIxGI82bN6dRo0Zx2vv168f06dNxd3enVatWT+yfJUsWli9fzpAhQ/Dw8KBnz57Mnz+f/PnzJ0q+F9GsWTO6detGrVq1SJkyJZs2bWLt2rXm5ffEclavXk2zZs349NNPcXWNPa5YW1uTPXt25s2bh6OjI3369LFwShEREZHXY/upmwz45ShgYpTtDC67XGS6hxsAtla2TCwzkYxuz77eq4gkHYPpRablPEO6dOno0aOHeY38d8Wff/5J0aJF2bFjB0WKFLFolsjISIKDg3Fzc8PW1taiWURE3iY6viZvj2c4Z82a9aX2W3ZqGUN3DX3q9YwcrB3oW6gvtbK+2Eycd1VgYCBjxozhyJEjr9QuArHL5kVHR2Ntbf1alpqO75iR1O/r7e3tmTx5Mp999hnBwcF4eHiwatUqqlSpAsQuwT169GguX76c6GO/Kn3WERF5++n4KpZw+sZ9agXs4P6jKLra/EQR42ra+aYkymAAYESJEXyc6WMLpxRJmOR0fH2V9/UJvoZRvXr1WLx4MZ06ddI1hURERCTZq5W1FmXTl+WXM7+w5eIW7kfcx8XOhTLpylAtczXc7N0sHVFE3iIpUqTgwYMHALi4uGBra8vFixfN7ba2tty9e9dS8URERERei9uh4XwauJv7j6KobbWNao6/8ElKX3OxqKN/RxWLRJKBBBeMWrduzebNm6lQoQJdu3Yla9asT136In369AkdSkRERCRRuNm78UmuT/gk1+u7xoyIvJuyZcvGsWPHALCysiJ//vwEBgbSokULoqOjmTNnToKv+yoiIiKSnIVHRdN27l4u3nlIIcNxejl8T3OflNy3jp18UDNLTT7L+5mFU4oIJELBKE+ePBgMBkwmE1u2bIl3u+jo6IQOJSIiIiLJQIsWLWjRosUrt4u8Sz766CPGjBnDlClTsLe354svvqBhw4akSJECg8HAw4cPmTFjhqVjioiIiCQJk8lE76WH2XP+LpkMV5jsMJ4uvh5cto39WrpQqkL0L9Ifwz8zjUTEshJcMOrfX3/QIiIiIiIiT9O3b1969OiBvb09APXr18fGxoZ58+ZhbW1N3bp1adCggYVTioiIiCSNyZtOs2z/ZVIQwvf2oxnq7cCRf94XZXbLzLjS47C10nW0RJKLBBeMBg4cmAgxRERERERE3j4Gg8FcLHqsdu3a1K5dO8F9//333wwePJh9+/Zx5coVIiMjSZ8+PVWqVKFnz56kSpUqwWOIiIiIvKqVB68w7reT2BPBd3bjWJwinE3OrgB4OXgSUD4AVztXC6cUkX9LcMFIREREREREXr9Lly5x9epVatWqRdq0abGxseHw4cPMmDGDhQsXcuDAAVKmTGnpmCIiIvIO2nfhLt2XHMRADGNsp/O32xXmuqUAwNHagSnlppLamNrCKUXkv1QwEhERERERSUQ///zzS21vbW2Nu7s7uXPnxsvL64X3K1euHOXKlXvi8ZIlS1K/fn0CAwP58ssvXyqLiIiISEJdvBPGZ3P2EBEVQ3ebn3A1HuArT28ADBgYUXIkub1yWziliDyNCkYiIiLyzri3bDmulSpi5ej41PaYhw8JWfcr7rVqvt5gIvJWqVu37itd59VgMFCzZk0CAwMxGo2vPH6GDBkAuHv37iv3ISIiIvIq7j+KpPXsPdwKjaCe9RbKOq3m05Q+xPzz3qjXh70om76sZUOKSLxUMBIREZF3wr1ly7napw/BK1aQblrAE0WjmIcPudj+c8J27gR4p4tGQUFBZMyYkZs3b77UbIc3NUtSjzF//nwmTpzIX3/9leh9J5aBAweyZ88eVq1aBUDu3LkZOnQoNWvWtGywN9QPP/zwUtubTCZCQkL4888/WbRoEV5eXkyfPv2F93/06BGhoaE8evSIY8eO0atXLwCqVKnyUjlEREREEiIqOoaOP+7n7+v3KWJ1lM4OgTTzSclDKysAmuRsQpOcTSycUkSeRQUjEREReSe4VqpI8IoVhO3cycX2n8cpGv27WORUuDCulSom2rilS5fmzz//xNbWFoC0adPSu3dvWrRoEe8+//3yPjw8nIYNG3Lx4kXWrl2Lt7c3U6ZMYcKECVy7do333nuPgIAA8ufPb+4jODiYL7/8kp9//plHjx6RJUsW1q9fj7e3d6I9N3nSf392AE2aNKFJkzfrg/HRo0ctHeGN1rx581far3PnzhiNRlasWPFSBaOZM2fSqVMn830/Pz/mzZtHiRIlnrvvxYsXuXTpUpzHDh8+DEBUVBSRkZEvnCMpREZGJoscIiJvGx1fJSkMXnWcrSdvktlwmbH2E+ngm4JbNtYAlEpTiq75uup3Tt56yen4GhUV9dL7qGAkIiIi7wQrR0fSTQswF4YeF42AOMWip80+SqihQ4fSo0cPTCYTmzdvpnLlyhQoUIA8efI8d9+QkBBq1KiBwWBg8+bNuLi4sGjRIoYOHcpvv/1Gzpw5mT59OpUqVeLUqVO4uroSExND9erVyZYtG8ePHydFihQcPXoUJyenRH1eyVFkZKS5OPcuetef/9ugaNGifP/99y+1T82aNcmRIwehoaHs37+fX375hVu3br3QvrNmzWLQoEFPbQsNDSU4OPilsiS2qKgoHjx4AICNjT6+iogkFh1fJbEt2neNubsu4kkwM+1GM9DHkdN2dgBkd8tOrzy9CL0fauGUIkkvOR1fQ0Nf/m9O/0cQERGRd8Z/i0YXPm0JwMMDB5KsWPRvBoOBsmXLkj59eo4dO/bcgtGNGzeoXLkyfn5+/Pjjj9jb2wOwdOlSWrRoYd6/Q4cOjB49mmXLltG8eXN+/fVXzp49y4YNG8zFg/fee++l8y5btozhw4dz7949qlevztSpU3F2dgZg7969dOnShSNHjuDj40P37t357LPPuH//Pvnz56dLly7mGQ9NmjQhNDSUFStWYDKZCAgIYMqUKVy9epU8efIwZcoU/P39gdiZUe3atTPPpOrZs+czMz6eqfXo0SPWrl1Lnz596NWrV4LGaNGiBUajkSlTppgfMxgM7N69mwIFCgCwYMECRowYwblz5/Dw8GDQoEG4u7szbNgwYmJizNefuX37NgsWLGDMmDEcOXLE/HPt1KkTmzdvxs7Ojrp16zJixAgcHBzMy+MFBgYyZMgQbty4QcWKFfnhhx/Mr/2/Pd5+5syZjBgxgmvXrnH//n3GjRvH9OnTuXLlCt7e3nTo0IEePXqY9/vjjz/4/PPPOXPmDKVKlSJTpkxx+vXz82PMmDHUrVuXwMDAOPkhduZc1apV6dGjB3fu3KF169Zs2bKFmJgY8+9rrly5nvmzk6f75JNPqF279kvtkzZtWtKmTQvEFo/q1KlDwYIFCQsLo0+fPs/ct1WrVlSsGHdW5eHDh2nbti1GoxE3N7eXewKJ7PGZma6uriqGiogkIh1fJTFtO3WLsZvPY08EM+zGEOgdxZ+Ose+HfZ18mVx2Ml6Oll3qWuR1SU7H11e5LmqiFIwePHjAjz/+yKlTp7h9+zYmkylOu8FgYNasWYkxlIiIiMgz3Zkzlzvz5z1zG1NMDDapU/PwwAEAHP398fnqK86+wPVabFP6kGHunFfKFhMTw4YNG7h69SoFCxZ85rZXr16lePHilCpViunTp2Ntbf3//CbTE++3TCYTBw8eBGDLli1kzZqVFi1asG7dOlKmTEmHDh3o2LGjeXt3d3dWrVpF8eLF482wZMkS9uzZQ2RkJDVq1KB3795MnjyZe/fuUbFiRfr168fmzZvZt28flStXxtfXl+rVq7Nw4ULKli1LyZIl2bdvH9u2bTNn+/bbb5k6dSrLli0ja9asBAYGUrlyZU6dOoXRaKRLly7cuHGDs2fPEhkZSZ06dZ77ui5cuJCff/6ZH3/8kUePHiXJGP+2cuVKOnbsyJIlSyhdujS3bt3i8uXL5M+fn759+z6xJN1/NWrUCC8vL86cOcP9+/epUaMGffv2Zdy4cXHG2LNnD+Hh4ZQoUYJp06bFKfj814oVK9i1axcODg4ApE+fnt9++4306dPz559/UqlSJXLnzk3lypW5d+8eVatWZciQIbRt25bNmzdTs2ZNypQp81Kvw2NjxowhIiKCixcv4ujoaJ7VJq/GxsYmwUWavHnzkj9/fgICAp5bMEqXLh3p0qWLN4ulP+T+O0dyyCIi8jbR8VUSw9/X7tNl0SFMphjG2k5jv8dNfnZxB8Bo68y08tNI5ZrKsiFFXrPkcnx9lRlOCS4Y/fXXX1StWvWZSx6oYCQiIiKvS3RwMJHnLzx3Oxsfn7gPREW+0H5EvvwawF9//TVDhgwhPDyc8PBw+vfvT8aMGZ+5z6lTp3j48CFz5syJUywCqFq1Kr169aJhw4bkzp2bgIAALl68SEhICBA7q2Xz5s1MmDCBH374gYMHD1KxYkXSpElDrVq1ALh3795zcw8cOND8xf/AgQNp2LAhkydPZvXq1Xh4eNCtWzcAChUqROvWrQkMDKR69eoUKFCAAQMGUKdOHW7evMmqVavM/UyePJnBgweTI0cOIHZ2w9ixY/ntt9+oXr06CxYsYPPmzebtv/rqKypXrvzMnOXLl6dKlSoAODk5JckY/xYQEECnTp0oW7YsAClTpiRlypQvtO/ly5fZtGkTly5dwsXFBRcXFwYNGkTTpk3jFIwGDBhgLhpUr16dvXv3PrPf/v37xynS1K1b1/zvokWLUrt2bfNyiKtWrTLPOgKoUKEC1apVe6XlCgDs7Oy4c+cOp06dIl++fOTOnfuV+pHE9fDhQ+7cuWPpGCIiIvIWu3k/nJaBuwkNj+JLm8VYux5mYorYmUQ2BmvGl5lAFo8sFk4pIi/DKqEdfPHFF0RERLB48WJu3bpFTEzME7fo6OjEyCoiIiLyXNZubthmSB/vzSZdWgwO9kRdv46jvz+O/v48PHCAa4MGY5Mu7TP3tc2QHts0aV460zfffMO9e/cICwvjwIEDTJ8+nWnTpgFQuXJljEYjRqORdu3amfcpWbIkw4YNo2LFivzxxx9x+mvWrBndunWjQYMG+Pr6cujQIcqXL4+npycALi4upEmThi5dumBnZ0fBggVp3LgxK1eufKncGTJkMP/bz8+P4OBgHjx4wKVLl/Dz84uzbaZMmbh06ZL5/qeffsrVq1fJnz8/JUqUMD8eFBTEp59+iru7u/kWFBTEpUuXuHXrFhEREU+M+zzp06ePcz8pxvi38+fPkyXLq33wvXTpEjY2NqT51+9RpkyZuHv3LmFhYebHUqX6/1mYzs7O3L9//5n9/vc1WLBgAR988AEpUqTA3d2dBQsWcPPmTSC2aPXv5w8v/xr8W8+ePSlVqhQNGzbEx8eHzz///Ll5JXFcu3btqY9v3ryZI0eOULhw4decSERERN4VjyKj+WzuHi7fe0gD680Udv6Vft6e5vb+RQZQOJXei4i8aRI8w2jv3r307ds3zlmMIiIiIpaSotknpGj2yVPbYh4+jL1+0cVL5msWAeZrGiX1dYwMBgN58+alRIkSrFmzhvbt27N27dp4t+/Zsye2trZUqlSJ1atXU7JkSXM/vXr1olevXgBERESQMWNG84yffPny8dNPPyU47/nz582FjaCgINzc3HB2diZt2rQEBQXF2TYoKMh8DRWANm3aUL58efbs2cO8efNo2rQpEFvYGD16NFWrVn1ivOjoaOzs7J4Y93msrOKeA5XQMYxGY5zizZUrV+K0Z8iQgdOnT79Qlv9KmzYtUVFRXL58Oc74Hh4eODk5PXPfZ/n3uBcvXqRp06asXr2acuXKYWtrS4sWLczLGKZJk4bz58/H2f9Zr/N/Xw+I+5o4OzszfPhwhg8fzsWLF6lbty6jR49m8ODBr/x85MW0b9+eq1evUrZsWTJkyMCjR4/Yu3cvCxcuxMXFhbFjx1o6ooiIiLyFTCYTPX86xP4L9yhudZi2jrNp5uNNpMEAwGd5P6NW1loWTikiryLBM4xcXV3NZ7OKiIiIJFfmYtF/CkNWjo6kmxaAU+HChO3cycX2nxPz8GGS5Th27Bjbt2/nvffee6Htu3btyvDhw6lSpQqbNm0CYpeTO3HiBCaTiZs3b9KuXTuyZMlivnB97dq1iYiIYOrUqURHR7N//34WLFhgXo7uRQ0ePJi7d+9y48YNBg0aRJMmTQCoUqUKd+7cYeLEiURGRrJ7926+++47mjdvDsQu2bZ//37mzJnD3Llz6dixo7nA0qlTJ/r378+xY8cwmUyEhoaydu1abt68ibW1NQ0aNGDAgAHmcYcNG/ZSmRNjjPfff5/169dz+fJlQkND+eqrr+K0t23blilTprB161ZiYmK4ceMG+/fvB8DHx4cLFy4QFfX0pQvTpElDmTJl6NGjB/fv3+fq1asMGDDA/NolhtDQUEwmEz4+PtjY2LBx40aWLVtmbv/444+5ceMG06ZNIyoqio0bNz7zmkv+/v5cuHCBzZs3ExUVxdSpUzl37py5feXKlfz999/ExMTg4uKCra3tE8soStJ4fD2suXPn0qVLF3r37s1ff/1F27ZtOXToEP7+/paOKCIiIm+h8RtOsfLgFbIaLjHcYRKdfD0J/uf9X5WMVejo3/E5PYhIcpXgGUa1a9fm119/5fPPP0+MPCIiIiJJImTdr/HOInpcNHpcUApZ9yvutWom2tj9+vVj4MCBmEwmPDw8qF+/Pv3793/h/Tt27IiNjQ3VqlVj2bJlZMuWjTp16hAUFISTkxO1a9dm5cqV5lkmrq6urFmzhs8//5wvv/yS1KlTM2zYMKpVq2bu02g0snbt2jjLxf1XnTp1+OCDD7h79y7VqlVj5MiRAHh4eLBu3Tq6du1K//798fHxYciQIdSoUYNDhw7Ru3dvNmzYgJubG2XLlqVDhw40bNiQHTt20L59e6ytralfvz4XLlzAaDRSpEgRChQoAMCkSZNo27YtGTNmxNvbm549e7J9+/aXer0TOkbTpk3ZunUruXLlIkWKFIwYMYIffvjB3F6zZk1CQkLo2LEjQUFBpEiRgm+++Yb8+fNTr149FixYgLe3NyaTievXrz+R78cff6RTp05kypQJOzs76tSpw9ChQ1/qOT5Lzpw5GTBgABUqVCAqKopKlSpRr149cxHLw8ODlStX0qFDB/Nycp9++mmcItC/ZcmShdGjR9OwYUOio6Np06YNxYoVM7efOXOGrl27cu3aNZydnalatSo9evRItOfzpnmVmVUGg4Gvv/76pferX78+9evXf+n9RERERF7V8v2XmbTxFN7c41v7UfT1MXLB1haA91O+zzfFvsHwz0wjEXnzGEyP16Z4RSEhIVSsWJECBQrQtWtXMmXK9E4cFP7880+KFi3Kjh07KFKkiEWzREZGEhwcjJubG7b/HKBFRCThdHxN3k6dOgVA1qxZX3ife8uW41qpYrxLzsU8fJjoxSIRebrH1zq1trZ+7lJ+iSG+Y0Ziv69/2nN5/Pnovx+9DAYDJpMJg8GQrK77qs86IiJvPx1f5VXsCbpD4+92YRX9kB/tBrPAJ4R1RmcA/FwyMLfKPNwd3C0bUsTCktPx9VXe1yd4hpG7uzsGg4G//vqLgICAp25jMBjiXZZDRERE5HV5XiHIytFRxSIRSZD/ztQKDQ2lWbNm2NjY0K1bN3LlygXA0aNHGT9+PDExMcyZM8cSUUVERERe2IXbYXw2dy+R0VFMsw1gm+cd1hndAPCwdyeg/DQVi0TeAgkuGDVr1uydmFEkIiIiIiLyPBkyZIhzv3Pnztjb27Nt2zZsbP7/8Stv3rzUrVuXkiVLMn36dCZNmvS6o4qIiIi8kOCHkbScvZs7DyLoY7OAUPdjzHKPvaa9nZUtk8pOJp1rOgunFJHEkOCCUWBgYCLEEBERERERefssXryYvn37xikWPWZra0vDhg0ZMWKECkYiIiKSLEVGx9Dxx32cvhFKE+sNvOeykc89vc3tw0oMxz+lv+UCikiiSnDBSERERERERJ4uJCSE4ODgeNvv3bv3zHYRERERSzGZTAz85SjbT92ilNVBmjjNp2VKb6L/WW2q2wfdqOhX0cIpRSQxJWrBKDQ0lHv37hETE/NEW/r06RNzKBERERERkWQvf/78TJkyhcaNG5M5c+Y4badPn2bq1Km8//77FkonIiIiEr8f/ghi/q4L5DBcYKDDFNr4evLAygqAetnq8WnuTy2cUEQSW6IUjBYuXMiQIUM4fvx4vNtER0cnxlAiIiIiIiJvjJEjR1KhQgVy585NzZo1yZ49OwAnTpxgxYoVGAwGRowYYeGUIiIiInFtOnGdIauPkZK7TLUfRS9fF67/s8RusdTF6Fuor65rL/IWSnDBaPny5TRu3Jhs2bLRtm1bpk+fTuPGjYmKimL58uXkzZuXjz/+ODGyioiIiIiIvFGKFy/Oli1b6NatG4sXL47TVrhwYcaNG0fhwoUtlE5ERETkScevhtDpx/3Ymx4xw24U43ysOW5vB0A296yMKTUGGytd6UTkbZTgv+wxY8aQM2dO9u7dS2hoKNOnT6dly5aULVuWI0eOUKxYMfr165cYWUVEREQSxaMHkZz48ypBh24R/jAKe0cbMubzJnthXxycbS0dT0TeMoUKFWLHjh3cvHmTs2fPApAxY0ZSpkxp4WQiIiIicd0IeUSrwN08jIhkuu0UVnuHsM3JBYCUjl5MLR+A0c5o4ZQiklSsEtrBoUOHaN68OQ4ODlj9s4bl4+Xn8uTJw2effcbw4cMTOoyIiIhIojj2xxUCe//BHz+d5vLJe9y6GMrlk/f4fckpAnv/wbE/rlg6orxBtm/fjq+v72sds127dnTv3v21jRcYGEiePHle23hvM29vbwoVKkShQoVULBIREZFk52FENG3m7OFK8CP62cznmscpFrrGFoucbByZWn4avs6v972viLxeCS4YRUdH4+npCYCjoyMAwcHB5vbs2bNz5MiRhA4jIiIikmDH/rjC5rkniI6MeWp7dGQMm+eeSPSi0Y4dO6hSpQoeHh64ubmRJ08evvnmG8LCwp66/ZYtWzAa4561N3LkSLy9vdm9ezcAmzZtomDBgri4uJA1a1YWLFgQZ3uTycSoUaPIlCkTzs7OZM6cmY0bNz4366pVqyhTpgyurq64u7uTM2dOevTowZUrsa/J+fPnKVKkCJ6enri5ueHv78+yZcvM++/cuZNKlSrh7e1NihQpKFu2LPv27Xup1+tNUqJECa5du5Zk/ZcuXZoxY8bEeWz69OmMHTs2ycaUxBcdHc2cOXNo2rQpFSpUYP/+/QDcvXuXOXPmcPnyZQsnFBERkXddTIyJ7ksOcPBSMM2sf8XPdQtjUrgDYIWB0aXGkCNFDsuGFJEkl+CCUdq0aTl//jwQWzBKmTIle/fuNbf//fffODs7J3QYERERkQR59CCSbQtPvtC22xee5NGDyEQZd82aNZQtW5aCBQty7NgxgoOD+eWXX7h69SqnTp167v4mk4kePXowdepUtm/fTsGCBTl37hzVq1end+/eBAcHM3v2bNq2bcuff/5p3q9///4sW7aMtWvXEhoaytatW8maNeszx5o2bRoNGjSgdu3anD59mnv37rFhwwZSpkzJH3/8AYCnpyezZ8/m5s2bBAcHExAQQNOmTTlz5gwQ+wV4s2bNOHnyJDdu3KBy5cpUqlSJBw8eJOBVTDqRkYnzc35b6fVJuLCwMEqVKkWLFi1YsWIFmzZt4u7duwC4urrSu3dvpk2bZuGUIiIi8q4b+9vfrDl8jTJW+6ltXERvb09MBgMA/Qp/Rcm0JS2cUERehwQXjIoWLcqGDRvM96tXr86ECRMYPHgwAwcOZOrUqZQuXTqhw4iIiIgkyN87r8U7s+i/oiJj+HtnwmeNmEwmOnbsSLNmzRg0aBCpUqUCIFOmTAQEBJAvX75n7h8dHU3Lli1ZvXo1f/zxBzlyxJ7Rt3btWt577z3q1KmDlZUVRYsWpXbt2syYMQOILdqMHj2aH374gezZs2MwGEibNi3p06ePd6z79+/Tq1cv+vfvT6dOnczLZaVJk4Yvv/ySevXqAWA0GsmWLRtWVlaYTCYMBgPR0dHm67JUrlyZxo0b4+HhgY2NDT169ODevXv8/fffL/y6ffnll/j5+eHi4kKOHDmYO3euue3x7Ktp06aRJk0avL296d69O1FRUS/VPmPGDDJkyIC/vz8Av/32Gx988AFubm7kzZuX5cuXA3Dx4kW8vLz45ZdfzD+TkiVL0rlz5zj9PVa6dGl69OhBhQoVMBqNfPDBB5w7d46xY8eSOnVqfHx8+Pbbb83b79+/nxIlSuDp6Ymnpyd169bl+vXrAHTv3p3t27fTr18/jEYjZcuWBaBFixZ07NjR3MfevXspXrw47u7uZM+e3fx7AP9fTm7YsGH4+vri7e3NoEGD4n3tH2//+Pe1evXq5rZn9TF//nxy5cqFm5sbhQoV4vfff3/Wj/idMnDgQPbs2cOyZcs4e/YsJpPJ3GZtbU3t2rX59ddfLZhQRERE3nU/7b3E1M1nyGUIoq/jFLr4ePLon0uPtMjdgvrZ61s4oYi8LjYJ7eDzzz9n2bJlPHz4EEdHR4YOHcpff/3FwIEDAcidO/cTy2iIiIiIJIUVE/Zz//ajp7aF3gt/qb7+XH6Gw1suxdvu4ulAja75n9nHqVOnOHfuHA0bNnypsQFiYmKoXbs2V69eZfv27Xh5eZnbTCZTnC+dHz926NAhIHZZOHt7exYvXsy3336LjY0N1atXZ8SIEfHO/N6xYwf37983F4aeJ2/evJw4cYLIyEjKli0b7wlCv//+OzY2NnFmN+XNm5fevXvTuHHjePvu3r073t7erFy5kvr165M/f37zdXQePnzIjh07OHnyJNeuXaNixYqkTp3afF2f57WHhYWxe/dujh07hsFg4PTp01SrVo05c+ZQu3Zt1q9fT+3atdm5cyf+/v7MmjWLVq1aceDAAWbMmEFoaCijR4+O97VZsGAB69atI3v27DRs2JAKFSrQtGlTzp8/z8aNG6lRowY1a9bEx8cHKysrhg0bRqFChQgJCaFJkyZ07tyZRYsWMXbsWPbu3UvVqlXp0aPHU8e6d+8eFStWpF+/fmzevJl9+/ZRuXJlfH19zcWex8W6ixcvcvDgQYoWLUqVKlUoWLDgU/s8ceIEMTExBAUFER0dzeLFi5/Zx7Zt22jbti0rV66kRIkSzJkzh0qVKnHy5ElSp04d7+v0rliyZAmfffYZNWrU4Pbt20+0Z8mShUWLFlkgmYiIiAjsOnubPj8fwpfbTHQYTXdfd+5YWwNQIX15un3QzcIJReR1SvAMo4IFCzJs2DDz9Yu8vb05cOAABw4c4PDhwxw8eJB06dIlOKiIiIjI89y//Yjgmw+fenvR2UWPRUfGxNtX8M2H8Ram/u3mzZsAr/SleUREBBs2bKB+/fpxikUAFSpU4MCBAyxatIioqCi2bdvG0qVLCQkJAeD27duEhIRw/PhxTpw4wY4dO9i1axd9+/aNd7xbt249kfXLL7/E3d0do9FImzZt4mx/6NAhQkNDWblyJZUrV8b6nw+V/3b16lU++eQThg4diouLS5x94ysWATRt2tRcTKlRowZFihRh27Zt5vaYmBhGjhxpvjZTjx494sxCel67yWRi+PDhODs74+TkxKJFiyhRogT169fHxsaGKlWqUL16debMmQNAjRo1aNiwIRUrVmTixIksXLgQe3v7ePM3btyY9957Dzs7O+rWrcvVq1f5+uuvsbW1pVKlSjg7O5uv8ZkvXz5KlCiBnZ0dXl5e9OnTh82bN8fb93+tXr0aDw8PunXrhq2tLYUKFaJ169YEBgaat3F3d6dPnz7Y2tpSoEAB8uXL98zrShmNRvr374+9vT1OTk7P7WPu3Lk0atSIMmXKYGNjQ8uWLcmVKxc//fTTCz+Pt9mVK1eeOZvQycmJ+/fvv8ZEIiIiIrGCbj2g7by92EWH8a3daIb72HLOzhaAvF7vMazEcKwMCf76WETeIEn2F583b15y586NlZUOKiIiIvJ6uHg64Obt+NSbte3LvSextrWKty83b0dcPB2e28fjQs+VK1fi3WbYsGEYjUaMRiO5c+c2P+7g4MDq1asZNGgQkyZNirNPtmzZWLp0KaNHjyZlypR8/fXXtGrVCk9Pz9jX4Z/izDfffIOLi4t5WbmVK1cC0K5dO/OYlStXjjfrqFGjuHfvHq1bt37qtWzs7OyoWrUqmzdvjlOQAbh27Rply5alUaNGdOv2cmclTpo0iTx58uDu7o67uzt//PGHufgGYGNjE6ew5efnx+XLl1+43dHRMU4R7tKlS/j5+cXJkClTJi5d+v8Msw4dOnD06FFq1qxJtmzZnpnfx8fH/G8nJye8vb3jFNT+XSA4c+YMtWrVInXq1Li6ulKtWrU4z/V5XiS7j48Phn/WnwdwdnZ+ZoEiderUTxQAn9XHi2R4l3l6esb5/fuvo0ePaiaWiIiIvHbBYZG0DNzN/bBHTLSdzKKUD9jtGPsZJ41zaiaVnYyDzfM/84jI2yXBS9I9tm3bNtavX8/169fp3r07OXLkIDQ0lH379pE3b17c3d0TaygRERGRp3rWEnEHN17k9yWnXrivIjUzk69cwmZJZ8uWjYwZM7Jo0SLz9Wf+q2/fvvHO/CldujRr1qzh448/Jioqii+++MLc9vHHH/Pxxx+b79erV8+8LNzj2Qz//oL/36ZPn8706dPjPFakSBGMRiNLly6lZ8+eL/wcASIjIzl58qT5/tWrVylbtiw1atRg+PDhL9XXH3/8Qb9+/di4cSMFChTAysqK0qVLx1mCLyoqiitXrpi/ZA8KCiJNmjQv3P7fE5rSpk3Lli1b4jwWFBRE2rRpzc+vRYsWNG3alOXLl7N161ZKlSr1Us8rPu3atcPPz4+jR4/i4eHBli1bKFOmTLxZ/ytt2rQEBQXFm/1VvOwJX/FlKFq06CtneJuUK1eOH3744anLCp47d47vv/+eTz75xALJRERE5F0VGR1D+/l7OXvrAQNs5nHK8ywrXdwBcLU1ElBhGp6OnpYNKSIWkeDpP9HR0TRo0IAyZcowbNgwvv/+e/OZqTY2NtSsWZOAgIAEBxURERFJiOyFfV94lpGNrRU5ivgmeEyDwcCUKVOYPXs2gwYN4vr16wCcP3+ezp07m6859CwlSpRg3bp1DBo0iJEjR5of3717N5GRkTx48ICAgAC2b99unsnj5+dHxYoVGThwIA8ePOD69euMHTuWWrVqxTuOq6srw4YNY9CgQUydOtW8RN3Vq1fN168B2Lx5Mzt37iQiIoLw8HB++OEHNm3aRIUKFYDYGUqlS5emWrVqjBgx4qVfs5CQEKytrfHx8cFkMrFgwQJ27NgRZxsrKyv69OlDWFgYZ8+eZezYsTRp0uSF2/+rQYMGbN++naVLlxIdHc26detYsWKF+Uv8fv36AfDDDz8wYcIEmjRp8tRr0byKkJAQXF1dcXNz48qVKwwZMiROu4+PD2fOnIl3/ypVqnDnzh0mTpxIZGQku3fv5rvvvqN58+aJku9FNG3alAULFrB161aioqIIDAzkyJEj1K1b97VlSM4GDBjA3bt3KViwINOmTcNgMLBu3Tr69OnD+++/j729PX369LF0TBEREXlHmEwmvl5+hB1nbtPCeh2e7r8T4OEOgI3BmgllJ5HJLZNlQ4qIxSS4YDRy5EiWLl3KuHHjOH78eJyzPx0cHKhVqxZr1qxJ6DAiIiIiCeLgbEvJhs9eSuyxEg2zYe9kmyjjVqlShU2bNrFr1y6yZ8+Om5sbH3/8MT4+PmTJkuWF+ihatCi//fYbI0aM4JtvvgFiZyZ5enri6+vLqlWr2LJlC76+/y9yzZs3j4cPH+Lr60v+/Pn58MMPnyhG/FenTp2YP38+S5YsIWPGjLi7u1OuXDly585t3vfBgwe0adOGFClS4Ovry/Tp01mwYIF5dtN3333HyZMnCQgIMC97ZzQamT9/vnmc3Llzx7n/bxUrVqRRo0bky5cPHx8ffv/9dz766KM42zg6OlK4cGGyZs1KoUKFqFq1Kl27dn3h9v/KkiULy5cvZ8iQIXh4eNCzZ0/mz59P/vz5+fXXX/nuu+9YuHAhNjY2tGjRglKlStGyZctnvpYvavz48axfvx5XV1cqVapE7dq147R37dqVrVu34u7ubi7K/ZuHhwfr1q1j8eLFeHl50aRJE4YMGUKNGjUSJd+LKFWqFAEBAbRr1w5PT08CAgJYs2aNlln7R5YsWdi4cSM2Njb0798fk8nEmDFjGDlyJOnSpWPjxo265quIiIi8NjO3n2Ph7ouUt9pLZeNiBnilMLcNLvYNBX0LWjCdiFiawfTvCs8ryJEjB0WLFuX777/n9u3beHt7s2HDBvOyK2PGjGHs2LFcvXo1UQInF3/++SdFixZlx44dFClSxKJZIiMjCQ4Oxs3NDVvbxPlyS0REdHxN7k6dil1eLmvWrC+137E/rrBt4UmiI2OeaLOxtaJEw2zkKqYvupOrLVu2ULVqVUJDQ1+pXZKPmJgYoqOjsba2fi3XPY3vmPE639cfOXLEfJJd1qxZyZ8//mU0LUmfdURE3n46vr6b1h+9Rtt5e8nNWUY6D6NN6hTct459H9bBvwPt8rWzcEKRN19yOr6+yvv6BF/DKCgoiO7du8fb7u7uzt27dxM6jIiIiEiiyFUsNZn8vfl75zXOHbpJeFgU9k42ZMzrTfbCvjg46wOziCSebdu2kTNnTry9vcmTJw958uSJ037r1i2OHTtGyZIlLZRQRERE3gVHLgfTZeEBfE23GeM0lq6+7uZiUY3MNWibt62FE4pIcpDggpGLiwt37tyJt/306dN4e3sndBgRERGRROPgbEu+cunIV07LQIlI0ipTpgxz586lcePGT23fuHEjjRs3Jjo6+jUnExERkXfFteBHtJ69B+vI+0yzH8VgHzsu28Z+LVzI90MGFBmAwWCwcEoRSQ4SvPZD8eLFmTdvHk9b2e7u3bt8//33lClTJqHDiIiIiIhQunTpZy4397x2kdfteSuAR0dHv5Yl+UREROTdFBYRRes5u7kZ8oBJtpMI9HnIYQd7ADK7ZmRcmfHYWmuVBRGJleBPJv369ePUqVOULVuWVatWAXDw4EG+/fZb3n//fR48eEDv3r0THFRERERERORN9Kwzdnfs2IGXl9drTCMiIiLvipgYE10XHuDI5WAG2QSy1/sCG52dAPC092BqhWm42rlaOKWIJCcJXpKuQIECLF26lNatW/Ppp58C0KNHD0wmEylTpmTZsmXkypUrwUFFRERERETeBBMnTmTixInm+127dqVfv35PbHf37l1CQkJo2bLl64wnIiIi74iRv55g/bHrtLJeg43HLua4pQDAwcqOKeUDSGNMY+GEIpLcJLhgBPDxxx8TFBTEb7/9xvHjxzGZTGTNmpWKFSvi5OSUGEOIiIiIiIi8Edzd3cmQIQMAQUFBeHp64uPjE2cbg8FAnjx5KFy4MN26dbNETBEREXmLLdp9gW+3nqWi1W5KuPxMF8/YGc0GDIwsNZo8XnksnFBEkqNEKRgB2NvbU7VqVapWrZpYXYqIiIiIiLxxmjdvTvPmzQHImDEjI0aMoHr16hZOJSIiIu+KHWdu0W/ZEfIaztDO+Vva+ngS888SuT0L9qRs+rIWTigiyVWiFYxEREREREQkrnPnzlk6goiIiLxDzt4Mpf28ffjE3GC401g6+brz0Cr2MvaNczSmac6mFk4oIsmZVWJ08uOPP1KsWDFSpkyJtbX1EzcbG9WlRERExPKObt1IZPijeNsjwx9xdOvG15hIREREREQkcdx9EEHLwN3EPAxmiv1ovk7lwM1/vpctnbYUXxb8EsM/M41ERJ4mwZWcIUOGMGDAAHx8fChatCgeHh6JkUtEREQkUR3dupF1AeM5tm0jNb/sj629Q5z2yPBHLB81mAtHDgGQu1Q5S8QUYq/tsnv3bgoUKGDpKK8lS1KOsX37durVq8e1a9cSve/EEhgYyLhx4zhy5AgAlStXpnLlynTu3NnCyRLPH3/8wfDhw9m1axd3797FZDLFaTcYDERFRVkonYiIiLwNIqJiaDdvL5duh/Cd7QSm+4Zzys4RgJwe2RlZchTWVtYWTikiyV2CZxgFBARQunRpzp8/z/Lly/nhhx+eehMRERGxpGyFi5E+T14uHDnE8lGD48w0+nexKH2evGQrXCxRx96xYwdVqlTBw8MDNzc38uTJwzfffENYWNhTt9+yZQtGozHOYyNHjsTb25vdu3cDsGnTJgoWLIiLiwtZs2ZlwYIFcbY3mUyMGjWKTJky4ezsTObMmdm48fmzp1atWkWZMmVwdXXF3d2dnDlz0qNHD65cuQLA+fPnKVKkCJ6enri5ueHv78+yZcvM++/cuZNKlSrh7e1NihQpKFu2LPv27Xup10teTWBgIHnyxL14cYkSJZJ1sehp1q5d+1YVi7Zt20aZMmXYtWsXhQoVIiYmhjJlylCwYEFMJhN58uThk08+sXRMEREReYOZTCb6LTvMrnO3GWzzPVtTXmGHU2yxyNfRm6nlp+Fk62ThlCLyJkhwwSgkJIT69etja2ubGHlEREREkoStvQM1v+z/RNHov8Wip80+Sog1a9ZQtmxZChYsyLFjxwgODuaXX37h6tWrnDp16rn7m0wmevTowdSpU9m+fTsFCxbk3LlzVK9end69exMcHMzs2bNp27Ytf/75p3m//v37s2zZMtauXUtoaChbt24la9aszxxr2rRpNGjQgNq1a3P69Gnu3bvHhg0bSJkyJX/88QcAnp6ezJ49m5s3bxIcHExAQABNmzblzJkzANy9e5dmzZpx8uRJbty4QeXKlalUqRIPHjxIwKv4ZoiMjLR0BIuKjo4mJibG0jGSnaFDh5IqVSqOHTtGYGAgAH379mXnzp2sW7eOc+fO0bp1a8uGFBERkTfa9K1nWbL3Em2tV/HQcy9LXWNPPjPaOBJQ4Vu8nbwtnFBE3hQJLhjlz5+fi/9r777jo6jzP46/t2/qJiGNEqo0BUQFRU7FQ9DDH6KiiB0VBRuIYheVaj9FBE5RFAQV+53lzoLinXcWEEFQmoL0UEPKJtk+vz82WQkJECDLpryej8c+zszMznx2w0125r2f73fTppqopUqhUEjPPPOMOnToIKfTqZycHI0ePbpaNx1Wr16tK664Qh07dpTL5VJ8fLw6dOigO+64Q7m5uVGrGQAA1E77hkZvTxyjtyeOiVpYZBiGbr31Vl199dUaN26cGjduLElq3bq1pk+fruOPP/6Azw8Gg7ruuuv08ccf63//+586dOggKdyB0blzZ1100UUym83q2bOnBg4cqBkzZkgKhzZPPvmkXnnlFbVv314mk0nNmjVT8+bN93usoqIi3XPPPXrooYc0YsQIZWZmSpKaNm2qu+++W4MGDZIkJSYmql27djKbzTIMQyaTScFgUOvWrZMUHk7s8ssvV2pqqqxWq+68807l5+dr9erVh/Te/e9//9Oxxx4rl8ul888/Xzt37oysW7t2rfr166e0tDS1atVKEydOVDAYVCAQUM+ePTV69OjItvfff7+6desmn88nSXr77bd1/PHHy+Vy6YQTTqjQdeXz+XTLLbeoUaNGat68uebMmXPAGseOHat+/frplltuUXp6eqQr5kiOMXbsWPXv37/CspYtW+qdd96J/Pz555/rlFNOUUpKiho3bqxHH31US5Ys0Y033qiVK1cqMTFRiYmJWrNmTaVuNbfbreHDh6tJkybKzs7WkCFDtGfPnsh6k8mk6dOnq0uXLkpKStLZZ5+t7du37/c9MJlMmjJlirp06aL4+Hjt2LFDr732mjp37qzk5GQ1a9ZMd911l4LBYOQ5K1eu1J/+9Ce5XC6deeaZWr9+fYV9nnnmmXrqqackVd1td8011+jWW2+VJHm9Xg0dOlQZGRlKTk7Wscceq6+++mq/9cbCwoULdf311ysjI0Pmsgmny4O1s88+W1dddZUefPDBWJYIAADqsE9+ztXjn6xSP/P3OiHlH5qcFp4uxGoy669/nqy2qQf+0hgA7K1G5jC66KKLdNFFF+mEE06oiZoquP322zVlyhRdeOGFGj16tFauXKkpU6ZoyZIlmj9/fuSiqyqbN29Wbm6uLrzwQjVr1kxWq1XLly/XjBkzNG/ePC1dujRyMwQAANQPP/7rAy355MMDbmMYhpIzMpW7ZpUkqXG7Dup97Y169e4RB91/YlojDX74sWrV8uuvv+r333/XpZdeWq3t9xYKhTRw4EDl5ubq66+/Vnp6eoX6950DxTAMLVsWnn/pu+++k8Ph0FtvvaUXXnhBVoBDRYIAAHDiSURBVKtVAwYM0GOPPaaEhIQqj/fNN9+oqKgoEgwdTJcuXbRq1Sr5/X717t1bZ555ZpXb/fe//5XVaq3Q3dSlSxfde++9uvzyy/e7/1mzZunTTz+Vy+XSVVddpeuuu04ffvihAoGA/u///k/nnnuu3n//fW3YsEH9+vVTcnKyRo4cqTfeeEMnnHCC+vTpI7vdrunTp2vx4sWy2+369NNPNXLkSP3jH/9Qt27d9Omnn+rCCy/UL7/8opycHD3yyCP6z3/+o59++knJycm69tprD/o+fP7553r++ef17LPPyufzReUYe1uyZInOP/98zZkzRwMGDFBJSYlWrlypE044Qc8//7yeeuqpyFxAkiJDCZYbOXKk1q5dq59++kk2m01XXHGFhg4dqvfeey+yzbx58/TZZ58pMTFR/fr104QJEzR16tT91vT666/rn//8pzIzM2WxWJSWlqa3335b7dq108qVK9WvXz+1bt1aN910kwKBgAYMGKCLLrpI8+fP17Jly3T++edX+Pd9KGbPnq0lS5ZozZo1SklJ0bp162rdRM5er1dNmzaVJDkcDknhgLZc165dNXfu3JjUBgAA6rZlm/M16s2lOsH0q65JfFE3pzeKrHvw1IfVs0nPGFYHoC464sCoV69emjlzpnr06KEePXqoZcuWslgqTqBmMpk0c+bMQ973L7/8oueee04DBw7Uu+++G1neqlUrjRw5UvPmzTvgjYazzjpLZ51VecLqM844Q5dccolmzZqlu++++5DrAgAAtZfHXaT8bQfvJE5Ma1Th51AwWK3nhfbqlDiY8q6YJk2aVPs55Xw+n+bPn69x48ZVupnet29fjR49Wm+++aYuuugiffPNN3r33XeVnZ0tSdq9e7cKCwu1cuVKrVq1SoWFhbrwwgt1//3369lnn63yeLt27apU6913360ZM2YoEAjosssu04svvhhZt2zZMvl8Pn322WdatWpVpc9/kpSbm6urrrpKkyZNUlJSUoXnHsxdd92lnJwcSeH5mzp27Kj8/Hz98ssv2rx5sx599FE5HA61b99ed955p1566SWNHDlSLVq00IsvvqghQ4bIarVq+vTpatOmjSRpypQpGj16tE4++WRJ4W6o0047Te+8845uv/12zZ07V+PHj1ezZs0khYcS2ztIqUqHDh0iw4lZrdaoHGNvM2bM0CWXXKKLLrpIkuRyudSjR49qPTcUCum1117Tl19+qYyM8LAkTz75pI477jgVFhYqOTlZUvi9L/+3dOmllx40zLjzzjsjr6f8NZc77rjjdN1112nBggW66aab9N1332nbtm0aP368rFarTjzxRA0ZMkQff/xxtd+Dvdntdrndbq1cuVKnnHJK5HddmzRu3FibN2+WJCUkJCglJUU///yzLrzwQknhL7lZrUd8WQYAABqY3IJSXT/7B6UHtmls/DO6NTtVPnP4izM3dL5BA9sOjHGFAOqiIx6S7vvvv9eQIUPk9/v19ddfa86cOZo1a1alx+F44403ZBiGRo0aVWH5DTfcoPj4+MP+Jl6LFi0kqcLwGwAAoH5wJiYpJbvxfh+urGxZ7Xa583arcbsOatyug3LXrNL8l6bJlZV9wOemZDdWckb1u5PLg559uzz29sgjj0SGEDvuuOP+eB1Opz7++GONGzdOU6ZMqfCcdu3a6d1339WTTz6pzMxMPfjggxo6dKgaNQqHYOXhzIQJE5SUlBQZVu7DD8OdVzfeeGPkmOU396uq9YknnlB+fr6uv/76Kufnsdvt6t+/vxYsWFBpaLVt27apd+/euuyyy3T77bdX7w3bS/nnNSk8JJskbdmyRZs3b1bjxo0jnRpSeIi/8hvyknTeeecpLi5OcXFxFbq71q9fr4cfflgpKSmRx7///W9t2bIlsv+qjnsg+w7zF41j7G3Dhg065phjDuk55Xbu3Cmfz1fhmK1bt5akCu9f+dCJUjjg2Lsbpir7vgeff/65TjvtNKWnp8vlcunxxx+PhKdbtmxRdna27HZ7ZPu9349DdeWVV+q6666LDAt45ZVXatu2bYe9v2jo3r17ZA4wKTwM3TPPPKNXX31Vs2bN0tSpU3XKKafEsEIAAFDXFHsDGjrrB3mKdmuK4wk90DhO+WVf4OrXsp9uPeHWGFcIoK464q+y3XbbbbLb7frHP/6h008/XSkpKTVQVtiiRYtkNpsj39As53Q61bVrVy1atKha+/F4PHK73fJ4PFqxYoXuueceSdK5555bredv2rSpwkW0JC1fvlySFAgEYj7Bsd/vrxV1AEB9w/m1dguFQjKZTJG5QMp1Pae/up7Tv8rn+L1e/ePJCSrYvk05x3XR+XeF5w35x5MTtOmXZZFltr3CiP0duzqOOeYYtWrVSvPmzdvvkG333nuv7r333gr7Lt//GWecoY8++kjnnXee/H5/heClX79+FTo5LrnkEvXq1UuhUEidO3eWFB6mrnxfe//v9OnTNX369ArHPOWUU5SYmKh33nlHd955Z4Uay4fA29/r9vl8Wr16dWR9bm6u+vTpowEDBmjSpEnVfr/29vvvv+vUU0+VpMj8SI0bN1ZeXp5yc3NVWloaCY3WrVunZs2aRY5zzz33qFmzZnK73Zo0aZIeeOABSVJOTo5uvPFG3XLLLZWOFwqF1LRp0yqPu/fvZN/3Zd9/g0d6jISEBJWUlET2GQgEtGPHjsj65s2b69dffz3ge7r3ur1/740aNZLdbte6desiodDe7+3e21b17+ZAytf7fD5dcMEFeuaZZ3TllVfK6XRq3Lhx+ve//61QKKTGjRtr27Zt8ng8stlskhSZw2jvY5T/e4uPj5fH41EgEIgMRb1lyxa1a9dOoVBIZrNZd999t+6++27t2rVLQ4cO1V133aXZs2dXqrH83/G+5/RAIHDA13akhg4dqlmzZqm0tFRxcXF65JFH9PXXX+uaa66RJGVnZ+uJJ56Iag0AAKD+CIYM3TZviX7NzdOL9sl6tnFAG2zheVhPzDheE0+bKLPpiHsEADRQRxwYLVu2TGPHjtV5551XE/VUsHXrVqWnp1f4Bmm5pk2b6ptvvpHP56vwDcWqvPTSSxox4o85CVq2bKm5c+fq9NNPr1YdM2fO1Lhx46pc53a7VVBQUK39REsgEFBxcbEkMZwFANQgzq+1WyAQkNVqVbCaQ8T5vV59+NdJ2rxiuZod21n977hf5rLfa/877teHf52kTb8s09+fGK/zRj9w0NCouiZPnqxLLrlE2dnZGjZsmLKysrRhwwZNnjxZ1157rbp06VLpOeU3zoPBoHr27BkJjbxer+666y5J0g8//KDjjz9ePp9Pc+bM0X//+18988wzCgaDysnJ0dlnn62xY8dq2rRpcrvdevrpp3X++efv9/1KSEjQhAkTNGbMGDmdTg0aNEjp6enKzc3V6tWrlZmZqWAwqK+++kpxcXE64YQTZBiG5s2bpwULFuiee+5RMBjU1q1b1bdvX5133nmaOHFitX8/+3r66af1pz/9SS6XS/fdd5/69eunpKQknXTSSWrcuLEeeOABjRs3Ths3btRf//pX3XLLLQoGg/rXv/6lV199VQsXLpTb7dbpp5+uM844Qz179tTNN9+skSNH6qSTTtJJJ50kn8+nhQsXqmnTpmrTpo0GDx6sJ554QqeddpqSk5P14IMPRn4fVb2OvX9P5Y70GMcff7zGjh2r5cuXq02bNho/frz8fn9k/XXXXaczzzxT/fv3V//+/VVcXKxVq1bplFNOUUZGhrZt26aioiLFx8dXWeOll16qBx98UK+//rpsNpvuuusuDRgwQImJiZFt9n695SHLgX6Pe29fWloqr9ertLQ02Ww2/fDDD3r55ZfVunVrBYNBdevWTVlZWXr44Yf10EMPafny5ZozZ47S0tIqHLN8n23atJHT6dTs2bN15ZVX6sMPP9TXX3+ttm3bKhgMasGCBUpNTVWnTp3kcDjkcDhkNpurrNcwDAUCgUqf3d1u9wH/LR6pvn37qm/fvpGfW7durTVr1uiLL76QxWLRaaedJpfLFdUaAABA/fHoP1dq/srtetz6kv6ZtV0/OsNzlLZIbKZne0+V3XLg+6QAcCBHfPcrMzPzoIHN4SopKakyLJLCXUbl2xzs+BdccIE6dOggt9utJUuW6IMPPoiM018dQ4cO1TnnnFNh2fLlyzV8+HAlJibG/AKv/FuSycnJkW9qAgCOHOfX2m337t0ymUxVzp1TlVWLvtXmFcur7CKyxMfrgrsfinQarV30rY7rVXkexMPRv39/zZ8/X5MmTdKUKVNkGIZycnJ06aWXqn379lXWX95JUb7utNNO06effqp+/fopEAhozJgxevDBB7Vw4UIZhqHTTjtNX375pZo2bRrZx5w5c3TTTTcpJydHSUlJuvjiizVx4sQDvl/lcwA9++yzeuCBB2SxWNSkSRP169dPt912mywWi0pLS3XHHXfo999/l81mU7t27fTaa6+pd+/ekqRXXnlFv/76q1544QW98MILkX3/7W9/0xVXXCFJ6ty5s+69997Iz1W5+uqr9X//93/avHmzzjjjDM2cOVMWi0UWi0UfffSRRowYoRYtWig5OVnXXHONRowYoe3bt2vo0KF64YUX1KpVK0nhOXquvvpq/fjjj+rfv798Pp9Gjhyp3377TXa7XSeeeKKeeeYZWSwWjRkzRrt379ZJJ50UCdDee+89mc3mav2eyn/fR3KMPn366KabblLv3r3lcDh09913q2nTppH13bp109tvv62xY8dq6NChSkxM1MiRI9WzZ0/16dNHf/rTn9SqVSuFQiF9//33lWp89tlndeedd+rEE0+UYRjq27dvpLa9X1f5zyaT6aD/P9t7e5fLpalTp+q2227Ttddeq549e+qyyy7TwoULI7+/999/PxKeHn/88br++uv1/vvvVzhm+T5TU1M1Y8YM3Xvvvbrjjjt00UUX6cILL4zUtHPnTt12223atGmTHA6HzjjjDD322GNV1msymWS1Wit9dk9MTNzva4uWhIQEDRgw4KgfFwAA1G2vf79RL/33d91s+Ye2Z/ykfyWGP9ek2JI0ve8LSnGmxLZAAHWeyTAM40h2MH78eH3wwQf67rvvavzb1507d9aOHTu0ffv2SusuueQSvf322/J6vYccWC1btkzdu3fX2LFjdd999x1Wbd9++6169uypb775JjKkSKz4/X4VFBTI5XJxQxMAahDn19rt119/lSS1bdu22s/55d9fqF2PP8nmcFa53u/1aM13/6uxsAjA/pV3EVkslkiwFU37O2fUps/1tUVtek/4WwwA0cH5te7576+7NOSVhTpX/9Of017Rwxnh+UvtJqtm/uUVdc3sGtsCAUiqXefXw/lcf8QJz2mnnaaPPvpIPXr00M0336xWrVpV+Y2+M84445D33aRJE61YsUJer7dSp9GWLVuUnp5+WN1NXbp00QknnKDp06cfdmAEAADqnoMFQTaHk7AIQI1q3br1AdebTCbFxcWpefPmOvvss3XDDTcoISHhKFUHAADqgt92FOmm1xbreGO1Lna9otvS0yLrJp3xKGERgBpzxIFRnz59Iv99/fXXy2QyVVhfPhnw4Yxf3717d3322WdauHBhhfmGPB6Pli5delghVLnS0lLl5eUd9vMBAAAA4GCaN2+urVu36rfffpPL5YoMl/j777+roKBAbdu2VVxcnL777jt98sknev755/X1118rIyMjxpUDAIDaIK/Yp+tm/aBU7xbdl/CMRmalKlh2/3XUiaP0l5Z/iXGFAOqTIw6MXnnllZqoo0qDBw/WI488osmTJ1cIjF588UWVlJRUGPd+7dq18vv96tChQ2TZtm3blJ2dXWm/CxYs0M8//6wzzzwzarUDAAAAwOTJk9WnTx9Nnz5dQ4cOjQxLEQgENGPGDD388MN6++231alTJ/3tb3/TyJEj9dBDD+lvf/tbjCsHAACx5g0ENXzODyrI26GX4p7Q/Y0T5C4byveiYy7SdZ2ui3GFAOqbIw6MhgwZUhN1VKlz58665ZZbNHXqVA0cOFDnnnuuVq5cqSlTpqhXr166/PLLI9ueddZZ2rBhg/aekummm25Sbm6uevfurRYtWsjj8Wjx4sWaN2+ekpKS9Ne//jVqtQMAgOg7wqkYATQg5SMfHG133nmnBg8erBtvvLHCcqvVqptvvlk///yzRo8erc8//1y33HKLvv32W3388cdHvU4AAFC7GIah+95drqXrd+pFx9N6srGhbdbwF096ZvfQA6c+EJPPNgDqtyMOjKJt8uTJatmypWbMmKGPP/5Y6enpGjFihMaPH3/QyXEvu+wyvfrqq5ozZ4527twpk8mkFi1aaPjw4brrrrvUvHnzo/QqAABATTObzfL5fPL5fIc1pyGAhsEwDBUWFioYDFaaF/Vo+P777zVo0KD9ru/SpYvmzJkT+blnz556++23j0ZpAACgFpu24De9t2SznrTN0DvZu7XSESdJautqrb/++RnZzLYYVwigPqr1gZHFYtHo0aM1evToA263fv36SssuueQSXXLJJVGqDAAAxFJycrJ27dqltWvXymKx8O06oI4xDCPS9ROt//8ahqFQKBTpRkxNTY3KcQ7E4XBo0aJFGj58eJXrFy5cWCHI8nq9SkxMPFrlAQCAWuijZVv11GdrdKvlPf2a8Yv+HZ8kScp0pGl63xeUaOezAoDoOHCLDgAAQC2Vnp6ujIwM2e12wiKgDjIMQ4FAIKpDS5pMJlmtViUkJKhVq1YxCYwGDBigV155RY899phKSkoiy0tKSvToo49q9uzZGjBgQGT5N998o3bt2h31OgEAQO2wZOMejX7rJ51v/q+yGn2mN1zhsCjO7NDUvs8rO6HyfO0AUFNqfYcRAABAVUwmk9LT05Wenh7rUgAcBr/fr4KCArlcLtls9XdIlaeeekpLlizR/fffr4ceekhNmjSRJG3dulWBQECdO3fWk08+KUnyeDxyOp265ZZbYlkyAACIkc17SnTDq4vVJbhC56a8qrvSwl92Mcukp/78tDo26hjjCgHUdwRGAAAAABAlaWlp+v777/XSSy/po48+0u+//y5JOuuss3Teeefp+uuvj8zD5nQ6K8xnBAAAGo4ij1/Xz/5BicXrNSrpWY3KTJFRNpLCfafcrzOanRHjCgE0BARGAAAAABBFdrtdN998s26++eZYlwIAAGqhQDCkkW8s0bZtW/V8/BO6JztRHnN4JpEhx16tSztcGuMKATQUBEYAAAAAAAAAECMTP16p/63eqhnOv+rRxmblWSySpD45f9Yd3UbHuDoADQmBEQAAAABE2Q8//KDvv/9ee/bsUSgUqrDOZDLpwQcfjFFlAAAgluZ8u16zvvldT9me15zsfK2zOyVJndM66pEzHpfZZI5xhQAaEgIjAAAAAIiS0tJSDRw4UJ999pkMw5DJZJJhGJIU+W8CIwAAGqZ/r9mpsR+u0CjrO1qStVoL4xIlSU3jszSlz3TFWeNiXCGAhoaIGgAAAACiZPz48frss8/0wAMPaMGCBTIMQ7Nnz9a//vUvnX766erevbtWrFgR6zIBAMBRtmZ7kW597Uedr3/Lmf6FPkgKh0VJ1nhN7ztD6XHpMa4QQENEYAQAAAAAUfLOO+9o0KBBGj9+vDp16iRJatq0qc455xzNnz9fPp9Ps2bNim2RAADgqNrl9uq6WYt0nH+Zzkidq2mpKZIkq8msZ8+aqtYprWNbIIAGi8AIAAAAAKJk06ZN6tWrlyTJUjaBtc/nkyRZrVZddtllmjdvXszqAwAAR5fHH9SwV3+QI/833Zg0ReMyUiPrxv9porpnd49hdQAaOuYwAgAAAIAoSUpKUiAQiPy32WzW1q1bI+tdLpe2bdsWq/IAAMBRZBiG7n5nmdZv3KgpCU/p7qxkBUwmSdLNx9+k89qcF+MKATR0dBgBAAAAQJS0adNGa9askRTuMDruuOP0zjvvSArfNHrvvfeUk5MTyxIBAMBR8uwXv+rTn9brr3FPaWJji4os4VuzA1r9n248/qYYVwcABEYAAAAAEDV9+vTRu+++q2AwKEkaPny4PvnkE7Vp00Zt27bV/PnzNXTo0BhXCQAAou0fS7fo2fmr9aj9b3opu0hbbOGBn07OOEFj/zRBprJOIwCIJYakAwAAAIAouffee3XVVVfJMAxJ0s033yyPx6O5c+fKYrHohhtu0N133x3jKgEAQDQt3pCnu95Zpjusb+m/2Wu1zBkvSWqV2ExPn/WcbBZbjCsEgDACIwAAAACIksTERLVv377CsjvuuEN33HHHEe97zZo1mjt3rj777DOtXbtWHo9Hbdq00aBBgzRq1CglJCQc8TEAAMCR2ZRXomGvLtb5xpfyZf5HnyckS5LSbEmafvaLcjlcMa4QAP7AkHQAAAAAUAe9/PLLeuaZZ9SmTRs99NBDevLJJ9W+fXuNGTNGPXv2VGlpaaxLBACgQSv0+HXdrEVqX/qjTkidp1kp4bDIabJqat8X1CypWYwrBICK6DACAAAAgCjyeDyaMmWK3n//fa1bt06S1Lp1a1144YUaMWKE4uLiDmu/F198se677z65XH98M/nGG29U27ZtNWnSJM2cOVO33nprjbwGAABwaALBkG557UcZO1fpatc03ZMe/nttkvRYr6fUOaNzbAsEgCrQYQQAAAAAUbJz5051795d9957r1auXKmmTZuqadOmWrlype699151795dO3fuPKx9d+vWrUJYVG7w4MGSpJ9//vmIagcAAIfHMAyN+3CFVv66VmMSntLDWUkKmUySpDu73aWzWpwV4woBoGp0GAEAAABAlNx1111asWKFnn76ad18882y2+2SJJ/Pp2nTpunOO+/UXXfdpVmzZtXYMTdv3ixJysrKOui2mzZtimxfbvny5ZKkQCAgv99fY3UdDr/fXyvqAID6hvNrdM3+doPe/m6Npsc9qQmNrSoxh7+zf8kxF+vStpfyvgP1WG06vwYCgUN+DoERAAAAAETJhx9+qKFDh2rUqFEVltvtdt1+++365Zdf9P7779fY8YLBoCZMmCCr1arLL7/8oNvPnDlT48aNq3Kd2+1WQUFBjdV2OAKBgIqLiyVJViuXrwBQUzi/Rs9/1+7Ro/9apSft0zStcYl2WMNfFumR3k3D2t6owsLCGFcIIJpq0/nV7XYf8nP4iwAAAAAAUeLz+XTiiSfud323bt305ptv1tjxRo0apW+//VaPPPKI2rdvf9Dthw4dqnPOOafCsuXLl2v48OFKTEyscsi7o6n8m5nJycmy2WwxrQUA6hPOr9GxaluRHvh4rW63zNNnjTdojSM8T2GH5NZ66s+TFW+Lj3GFAKKtNp1fExMTD/k5BEYAAAAAECXdu3fXjz/+uN/1ixcv1sknn1wjx3rwwQc1depUDRs2TPfdd1+1npOTk6OcnJwq11mt1phf5O5dR22oBQDqE86vNWtHkUc3vrZU/YOfaXf2//S/+CRJUrYjTdPOeVGu+Nh+CQPA0VNbzq+H0+FEYAQAAAAAUfLXv/5VZ511ljp37qybbropctEWCAQ0bdo0vffee/riiy+O+Dhjx47VxIkTde211+r5558/4v0BAIDq8/iDGvbqYrUqXKi26W/r2eQUSVKC2aFp57yozPjM2BYIANVEYAQAAAAAUTJ69Gg1atRIo0aN0kMPPaTWrVtLktatW6fCwkK1adNGd9xxR4XnmEymQwqRxo4dq3HjxmnIkCF66aWXZDKZavQ1AACA/QuFDI1++yeVbF6ua1Kn68FGKZIki0x6uvcUtUttF9sCAeAQEBgBAAAAQJSsW7dOJpNJzZs3lyTl5eVJklJSUpSSkiK/36/ff//9sPc/fvx4jRs3TldddZVefvllmc3mGqkbAABUzzPz12jhspWalPxX3ZeRHFn+4KkPqWfTnjGsDAAOHYERAAAAAETJ+vXro7bvadOm6eGHH1bz5s3Vp08fvf766xXWZ2VlqW/fvlE7PgAADd17P27Wi1/+oinxT2pitk0+c7jL9/rjrtVF7S6OcXUAcOgIjAAAAACgDlq0aJEkaePGjRoyZEil9b169SIwAgAgShatz9P97/6kSY6peq6xR3ss4cnt/5JzlkacNCq2xQHAYSIwAgAAAIAoKyws1Pz587Vu3TpJUuvWrdW3b18lJSUd9j5nzZqlWbNm1VCFAACgujbsLtawV3/QbeY5+qDJJq23OyVJJ6Qdq4m9HpfZxBCxAOomAiMAAAAAiKKXXnpJo0ePltvtlmEYkiSTyaTExEQ9/fTTGjp0aIwrBAAA1VVQ6td1sxbpXO/HWt9koX50JkiSmsdl6dm+z8thccS4QgA4fARGAAAAABAlH3zwgYYNG6bWrVtrwoQJOu644yRJv/zyi5577jkNGzZMmZmZOu+882JcKQAAOBh/MKSbX1usnN3/U1bm+3ox0SVJSrHGa/pfZirVmRrjCgHgyBAYAQAAAECUPPHEE+rYsaO+//57JSYmRpafddZZuvbaa9WjRw89/vjjBEYAANRyhmHooX/8ory1P+rqRjP0SGo4LLKZzHq2z9/UIrlFjCsEgCPHgJoAAAAAECU//fSTrrnmmgphUbmkpCQNGTJEP/30UwwqAwAAh2Lmf3/XFwt/0i2up/VE+h9zEE467VGdmHViDCsDgJpDhxEAAAAAREn5nEX7YzKZjlIlAADgcH2+Yrue+ecSPZX4hMZnORQo+/t9W9db1a/1uTGuDgBqDh1GAAAAABAlxx9/vGbNmqXi4uJK69xut2bNmqXjjz8+BpUBAIDq+GVrgW6ft1jjHVP0TGO/3Obw7dSBrfpraJdhMa4OAGoWHUYAAAAAECV33XWXBg4cqBNPPFEjR47UscceK0n65Zdf9Nxzz+m3337Te++9F+MqAQBAVbYXejR01g8aYbyit5rkKtfqkCSdmnGCxpw2nk5hAPUOgREAAAAARMkFF1ygqVOn6p577tGIESMiN5YMw1BCQoKmTp2q888/P8ZVAgCAfZX6grrh1R90dvHf9XPTH7XCES9JOiaxmf7aZ5psZluMKwSAmkdgBAAAAABRdPPNN+vyyy/X559/rt9//12S1Lp1a/Xt21culyvG1QEAgH2FQoZuf3Op0rcukDP7Q32VkCRJyrAlafo5LyvJnhTjCgEgOgiMAAAAACAK3G63BgwYoCuuuEJDhw7VoEGDYl0SAACohic/W61NK77TRekv6RlXsiQpzmTV1HNeUuPExjGuDgCixxzrAgAAAACgPkpMTNSiRYtiXQYAADgEb/2wSe9/tVBDU57R5EbhTiKzpCf/PFnHNjo2tsUBQJQRGAEAAABAlHTt2lUrV66MdRkAAKAavlu3W4+8v1APJD2pxzIdMsrmHrz35HvVK6dXjKsDgOgjMAIAAACAKBk3bpxefPFFLViwINalAACAA/h9V7FumbNQY+zP6K/ZQZWaw7dNr2p/qS7reEWMqwOAo4M5jAAAAAAgSubOnavmzZurT58+Ov7449WuXTvFx8dX2MZkMmnmzJkxqhAAAOSX+HTdKws1PDhDc3N2arfVLkk6q/GfNPrke2NcHQAcPQRGAAAAABAls2bNivz30qVLtXTp0krbEBgBABA7vkBIN85drF75b2thzs9aa3dKkjolt9ajvZ+RxWyJcYUAcPQQGAEAAABAlIRCoViXAAAA9sMwDI35+3Ilrv9U/iaf6Pu4BElSU0eanvvLTMVZ42JcIQAcXcxhBAAAAAAAAKDBeeE/67Ri8X/UNWO2PkgKh0VJZrum/eVlpcelx7g6ADj66DACAAAAgKNg9erVWrdunSSpdevWat++fYwrAgCg4frk522a88l/dUvaZD2alihJssqkZ/pMV5uUNjGuDgBig8AIAAAAAKLoyy+/1IgRI7Rq1aoKyzt06KApU6borLPOilFlAAA0TMs3F+iBN/+n+5Oe1KSMP4adG9tznE5pfEoMKwOA2CIwAgAAAIAo+fLLL/WXv/xFDodDN9xwg4499lhJ0i+//KI33nhD/fr10yeffKLevXvHuFIAABqGbQUeDZ/9ne6x/VVPZ4fkN1kkSTd1Gqrz214Y4+oAILYIjAAAAAAgSu6//35lZWXpu+++U9OmTSuse/DBB9WjRw898MAD+vbbb2NUIQAADUexN6ChsxbqWt80vZKTp0KLTZJ0Xk4f3XTibTGuDgBizxzrAgAAAACgvlq2bJmGDx9eKSySpGbNmmn48OH66aefYlAZAAANSzBkaNSbS9V952v6qskqbbaFw6JuqR01ttfjMplMMa4QAGKPwAgAAAAAosTlcikpKWm/65OTk5WSknL0CgIAoIF6/JNVMq36QIWN52uZ0yFJahWXqcnnvCi7xR7j6gCgdiAwAgAAAIAoGTRokN544w0FAoFK6/x+v9544w0NGjQoBpUBANBwzFu4Ud9//ZmOyZqjzxPjJUlpljhN6zdLLocrxtUBQO1R6wOjUCikZ555Rh06dJDT6VROTo5Gjx6t4uLigz53zZo1euihh9SjRw9lZGQoKSlJXbt21aRJk6r1fAAAAAA4EjfeeKMCgYDOOOMMvf3221q+fLmWL1+ut956S2eccYaCwaBuvPFGbdy4scIDAADUjP/9tkt/+/sCnd9oiuakJEqSHDLruXNeUk5SToyrA4DaxRrrAg7m9ttv15QpU3ThhRdq9OjRWrlypaZMmaIlS5Zo/vz5Mpv3n3m9/PLLmjZtmgYMGKArrrhCNptNCxYs0JgxY/TWW2/pu+++U1xc3FF8NQAAAAAakk6dOslkMskwDF166aUV1hmGEdlmX8Fg8KjUBwBAffbbDrfunvsfjUh+XI+lh+8BmiQ9esaT6pLRJbbFAUAtVKsDo19++UXPPfecBg4cqHfffTeyvFWrVho5cqTmzZunyy+/fL/Pv/jii3XffffJ5fqjtfTGG29U27ZtNWnSJM2cOVO33nprVF8DAAAAgIbroYceYhJtAABiIK/Yp2GzvtWtlif0TJZJwbK/x6O7jlTfVmfHuDoAqJ1qdWD0xhtvyDAMjRo1qsLyG264Qffee6/mzp17wMCoW7duVS4fPHiwJk2apJ9//rkmywUAAACACsaOHRvrEgAAaHC8gaBufPUHXVoyWS/lFKjEHL4FOrjVebq6y/Uxrg4Aaq9aHRgtWrRIZrNZJ598coXlTqdTXbt21aJFiw5rv5s3b5YkZWVlVWv7TZs2RZ5Tbvny5ZKkQCAgv99/WHXUFL/fXyvqAID6hvMrAERPbTnHBgKBmB4fAADULMMwdN97y9V5y8v6rPlv2mG1S5JOzzhB9542ns5fADiAWh0Ybd26Venp6XI4HJXWNW3aVN988418Pp/sdnu19xkMBjVhwgRZrdYDdiftbebMmRo3blyV69xutwoKCqp9/GgIBAIqLi6WJFmttfpXCgB1CudXAIie2nKOdbvdMTs2AACoedO/WqvSpW+rJOcrrXaE5y3qkNBUT/V9XlYz13UAcCC1+ixZUlJSZVgkhbuMyrc5lMBo1KhR+vbbb/XII4+offv21XrO0KFDdc4551RYtnz5cg0fPlyJiYkV5kiKhfJvZSYnJ8tms8W0FgCoTzi/AkD01JZzbGJiYsyODQAAatbHy3I1/7MPdHyTN/RufIIkKcuapKn9ZiveFh/j6gCg9qvVgVF8fLx27NhR5TqPxxPZproefPBBTZ06VcOGDdN9991X7efl5OQoJyenynVWq7VW3EQsr6M21AIA9QnnVwCIntpwjqWDFACA+uGnTfl6+q1PNTBjup5PDodFCSarpvV7RVkJ1ZuWAgAaOnOsCziQJk2aaNeuXfJ6vZXWbdmyRenp6dXuLho7dqwmTpyoa6+9Vs8//3xNlwoAAAAAAAAgBrbkl+r22V9pSNITer5ReBg6i6S/9p6i9mnVG2EIAFDLA6Pu3bsrFApp4cKFFZZ7PB4tXbpU3bp1q9Z+xo4dq3HjxmnIkCF66aWXmNwOAAAAAAAAqAfc3oCGv/KNrtMjmpL5xz2/B06+T39qdnoMKwOAuqdWB0aDBw+WyWTS5MmTKyx/8cUXVVJSoiuuuCKybO3atVq1alWlfYwfP17jxo3TVVddpZdffllmc61+yQAAAAAAAACqIRgyNPL1H3Ve4ZN6sYlb3rL7fte1u1SDOl4e4+oAoO6p1QN2d+7cWbfccoumTp2qgQMH6txzz9XKlSs1ZcoU9erVS5df/seJ/6yzztKGDRtkGEZk2bRp0/Twww+refPm6tOnj15//fUK+8/KylLfvn2P2usBAAAA0PAUFxfr9ddf16+//qrdu3dXuGaRJJPJpJkzZ8aoOgAA6q5JH69U63V/00fN1yvPEp4T8ZzGPXVbj+rPXQ4A+EOtDowkafLkyWrZsqVmzJihjz/+WOnp6RoxYoTGjx9/0G6hRYsWSZI2btyoIUOGVFrfq1cvAiMAAAAAUbNw4UL1799fu3bt2u82BEYAABy6ud9t0I7vXpU7579ab3dKkromt9bE3s/KbGKEIQA4HLX+7GmxWDR69GitXr1aXq9XW7Zs0dNPP63ExMQK261fv77SN/VmzZolwzD2+/jqq6+O4isBAAAA0NDccccd8vl8euutt7Rr1y6FQqFKj2AwGOsyAQCoU/6zZqf+8eE7im/8thbHhcOiHHuKpvSbJafVGePqAKDuqvUdRgAAAABQVy1evFj333+/Lr744liXAgBAvfDr9iI99to/9eeMv+nVpHhJksts1/Rz5yjVmRrj6gCgbqv1HUYAAAAAUFclJyerUaNGsS4DAIB6Ybfbq1GzvtCFiY/p1dQ4SZJNJj3b9wW1dLWMbXEAUA8QGAEAAABAlAwcOFCffvpprMsAAKDO8/iDuvnVbzUoNFHTMyyR5RN7TtBJ2d1iWBkA1B8ERgAAAAAQJY8//rh27NihESNGaO3atZXmXQUAAAdnGIbueecn/XnXI5qRVaqAySRJGnnc9Tq37fkxrg4A6g/mMAIAAACAKElJSZHJZNLChQs1ffr0KrcxmUwKBAJHuTIAAOqOKV/8poyVz+j95ptUZAnfzhyY00fXnzQyxpUBQP1CYAQAAAAAUXL11VfLVPYtaAAAcOg++GmrflswQztbfK+tNockqUdKB4058wn+xgJADSMwAgAAAIAomTVrVqxLAACgzvpx4x698fbratT0ff3iiJMkHePM0NP9XpbNbItxdQBQ/zCHEQAAAAAAAIBaZVNeiSbO/kAdM1/QvxPCYVG62anp//eakuxJMa4OAOonOowAAAAA4Chwu93Kz89XKBSqtK558+YxqAgAgNqpyOPXHa98oT/HPaoZLqckySmzpvabpcaJjWNcHQDUXwRGAAAAABBF8+bN08SJE7Vy5cr9bhMMBo9iRQAA1F6BYEi3v/a9zvY/rKnZ4VuXZkN68syndFz6cTGuDgDqN4akAwAAAIAo+fvf/67LL79cgUBAw4cPl2EYuuyyyzRo0CDZbDaddNJJeuihh2JdJgAAtcaED3/RibkP68Usr0ImkyTp7hNG6syWfWNcGQDUfwRGAAAAABAlTz31lDp27KilS5dq/PjxkqTrrrtO8+bN0w8//KDVq1era9eusS0SAIBaYvY362Vb+ojebpyrUnP4tuWVrfrriuNviHFlANAwEBgBAAAAQJQsW7ZMQ4YMkdPplLnsxlf58HOdOnXSsGHD9Oijj8ayRAAAaoUFq3fox39N0eKmP2qX1SJJ+nOj43XnaRNjXBkANBwERgAAAAAQJcFgUI0aNZIkxcXFSZIKCgoi69u3b6+ff/45JrUBAFBbrN5WpFmvz1Zxkw/1m90uSTouvokeO2eGLGZLjKsDgIaDwAgAAAAAoqRZs2basGGDpHBglJmZqcWLF0fWr169WgkJCbEqDwCAmNtZ5NXDL7+nxukz9H28U5LUxJKgqf1fU7wtPsbVAUDDYo11AQAAAABQX/Xs2VPz58+PzF80YMAATZ48WXFxcQqFQpo2bZrOO++8GFcJAEBsePxBjZ71hU5yPqpXk8NhUZIsmv5/c5Uelx7j6gCg4SEwAgAAAIAoufnmm/X++++rtLRUcXFxmjRpkhYuXKixY8dKko477jg99dRTsS0SAIAYCIUM3ffmQp1S+oCez7JJkqyG9HSfqWqTekyMqwOAhonACAAAAACipHv37urevXvk54yMDC1dulTLli2TxWJRx44dZTYzUjgAoOGZ/Pkqtd5wn2Y29UsySZIePvk+9Wh2WmwLA4AGjMAIAAAAAI6yLl26xLoEAABi5u9Ltqj0u4f0Xs4O+U0WSdLwtoN1wbGXx7gyAGjY+CobAAAAAETZf/7zH40ZM0Y33HCDVq1aJUlyu936z3/+o/z8/NgWBwDAUfTD+jx9/fcn9L+my1VgCYdF/5fVQ7ec+kCMKwMAEBgBAAAAQJQEg0ENHjxYf/7zn/XII4/o5Zdf1tatWyVJVqtVF1xwgaZPnx7jKgEAODo27i7RjFdf1PYm/9ImW3jeopMSW2h832kymUwxrg4AQGAEAAAAAFHy+OOP691339XTTz+tlStXyjCMyDqn06kLL7xQ//znP2NYIQAAR0dBqV9jX35HcY1e0k9OhySppc2lZ/u/JrvFHuPqAAASgREAAAAARM2rr76qq6++WrfddpvS09Mrre/YsaPWrl172Pt/9NFHNWjQILVu3Vomk0ktW7Y8gmoBAIgOfzCk++d8oZb2R/RlolOSlGKyaXr/1+VyuGJcHQCgHIERAAAAAETJ+vXrdeqpp+53fUpKivbs2XPY+7///vv15Zdfqk2bNkpNTT3s/QAAEC2GYWjS3xerTcG9ejslPAydwzDpubNnKCe5eYyrAwDszRrrAgAAAACgvkpKSlJeXt5+1//222/KyMg47P2vXbtWrVu3liR16tRJbrf7sPcFAEA0vPzfdUpZc4dmNQ5ICs9T9EjPceqa3S22hQEAKqHDCAAAAACi5LTTTtPcuXMrzF1Ubs+ePXr55Zf15z//+bD3Xx4WAQBQG32xcru2LbhL87LzFDSFw6LRx16ns9tdGOPKAABVocMIAAAAAKLkgQce0GmnnabevXvrmmuukST99NNP+vXXX/XYY4+puLhY9957b8zq27RpkzZv3lxh2fLlyyVJgUBAfr8/FmVF+P3+WlEHANQ3R+P8ujK3SJ+9NV6Lm61UsTl8C/Lixr10+fG3cF4HUG/Vps+vgUDgkJ9DYAQAAAAAUdKtWze9++67uv7663XttddKku68804ZhqHMzEy9//77OvbYY2NW38yZMzVu3Lgq17ndbhUUFBzliioKBAIqLi6WJFmtXL4CQE2J9vl1l9un5+fO0o7G87Xdapck9UhorRu7PqjCwsIaPx4A1Ba16fPr4QxXzSduAAAAAIii//u//9P69ev1+eefa+XKlTIMQ23bttU555yj+Pj4mNY2dOhQnXPOORWWLV++XMOHD1diYqJcLleMKgsr/2ZmcnKybDZbTGsBgPokmufXUl9Qz815U4FGs7Ta4ZAktbOl6clzZyvBllCjxwKA2qY2fX5NTEw85OcQGAEAAABAlDkcDvXv31/9+/ePdSkV5OTkKCcnp8p1Vqs15he5e9dRG2oBgPokGufXUMjQxLe/VJLlUX0cHw6LMk0OTT//TaXEp9TYcQCgNqstn18Pp8PJHIU6AAAAAAAAADQwUz5ZqtTdd+tjV/gmabxh0vRzZysrITvGlQEAqoPACAAAAACi6PXXX9ef/vQnZWZmymKxVHrEemxzAABqwjs/bFBg2a16o1FQkmQxpKd6Pan26cfFuDIAQHVxZQIAAAAAUTJx4kQ9/PDDysrKUs+ePZWamhrrkgAAqHHfr9utVZ+M1N+b5sswhb+ffn/XW3V6q3MO8kwAQG1CYAQAAAAAUTJ9+nSdeeaZ+uSTT6IyhvmcOXO0YcMGSdLOnTvl8/k0ceJESVKLFi101VVX1fgxAQDY2/pdxfrHG2P0vya/ymu2SJKubXGuLuk6PMaVAQAOFYERAAAAAERJYWGhLrnkkqhNeDtz5kz9+9//rrDswQcflCT16tWLwAgAEFUFJX49N3Oy1mQuUJ4l/Leub2onjer1aIwrAwAcDgIjAAAAAIiSE044QZs2bYra/r/66quo7RsAgAPxB0OaOOt17XS9qvV2uySpizNLj5z7iswmpk0HgLqIszcAAAAARMnEiRP1/PPPa8mSJbEuBQCAGmMYhp5460sFQo9qSVw4LGpqjtNzA96U0+qMcXUAgMNFhxEAAAAAREmvXr00c+ZM9ejRQz169FDLli1lsVgqbGMymTRz5swYVQgAwKF7ZcHP8m+7XfPTwsPQJRtm/e3/XldaXKMYVwYAOBIERgAAAAAQJd9//72GDBkiv9+vr7/+Wl9//XWlbQiMAAB1yac/b9H2H27Qu1nhn22G9OxZz6lV2jGxLQwAcMQYkg4AAAAAouS2226T3W7XP/7xD+Xl5SkUClV6BIPBWJcJAEC1/LylQIs+uFFvZbojyyZ0u0fdcs6IYVUAgJpChxEAAAAARMmyZcs0duxYnXfeebEuBQCAI7KtwKM3X71TX2SvU8AU/g76rW0u0v91ujLGlQEAagqBEQAAAABESWZmpux2e6zLAADgiJT4Anr2pSf1Y8Z/VWQJ304ckN5Nw/70cIwrAwDUJIakAwAAAIAoue666zR37lwFAoFYlwIAwGEJhQw9Mfs1rUt8TVtt4bDo5LimGvuXGTKZTDGuDgBQk+gwAgAAAIAoOe200/TRRx+pR48euvnmm9WqVStZLJZK251xBnM/AABqp6n/WKDt/ke1IsEmSWplTtAzF7wlm8UW48oAADWNwAgAAAAAoqRPnz6R/77++usrfRPbMAyZTCYFg8GjXRoAAAf17rcrtXXjbfpvSjgcSjMsev78t5VsT45xZQCAaCAwAgAAAIAoeeWVV2JdAgAAh+WbX7fp5/9do48zwj87DWn6OS+qSXJObAsDAEQNgREAAAAARMmQIUNiXQIAAIds3Y4iffnetXovq1iSSSbD0OM9xum4xt1jXRoAIIoIjAAAAAAAAABIkvYU+zR79ij9K3OjQiazJOnuDlerd4eLYlwZACDazLEuAAAAAAAAAEDs+QIhTZ75qL5O+1al5vBtw8uyTtOVPe6OcWUAgKOBwAgAAAAAAABo4AzD0OTX5uinuDe002qRJJ0e30r3nD01xpUBAI4WAiMAAAAAAACggZv1yb+1ouQxrbXbJEntzcl66oJ5spgtMa4MAHC01PrAKBQK6ZlnnlGHDh3kdDqVk5Oj0aNHq7i4uFrPf/TRRzVo0CC1bt1aJpNJLVu2jG7BAAAAAAAAQB3y2eLV+um3W7U4PjzdeZZh1fMD31O8LT7GlQEAjiZrrAs4mNtvv11TpkzRhRdeqNGjR2vlypWaMmWKlixZovnz58tsPnDmdf/99ystLU0nnnii8vPzj07RAAAAAAAAQC1TUOLX24s36fMV25Rf7FVKgkNdGieq9Lfr9EUjkyQpISQ93/9VpSdkxbhaAMDRVqsDo19++UXPPfecBg4cqHfffTeyvFWrVho5cqTmzZunyy+//ID7WLt2rVq3bi1J6tSpk9xud1RrBgAAAAAAAGqbtxZt0pMfLlD7pHcVSNwge2JQgaBZGzcY+qaRIUmyGIaeOeMJHZPROcbVAgBioVYPSffGG2/IMAyNGjWqwvIbbrhB8fHxmjt37kH3UR4WAQAAAAAAAA3RW4s26f0v75bR6gn9lLlOa+KDWu+Q1sSH9E2yEdluYNrlOrXNuTGsFAAQS7U6MFq0aJHMZrNOPvnkCsudTqe6du2qRYsWxagyAAAAAAAAoPYrKPHrgwV36+fsn+U1HXjbNauXq6DEf3QKAwDUOrV6SLqtW7cqPT1dDoej0rqmTZvqm2++kc/nk91uj2odmzZt0ubNmyssW758uSQpEAjI74/tH1K/318r6gCA+obzKwBET205xwYCgZgeHwCAaHvr20ValbFcMiSZDpAYGYZWpS/X298u0vVn9Txq9QEAao9aHRiVlJRUGRZJ4S6j8m2iHRjNnDlT48aNq3Kd2+1WQUFBVI9/MIFAQMXFxZIkq7VW/0oBoE7h/AoA0VNbzrHMcQoAqC/8wZB2FHmVm1+qrQUe7dydp5Jdm/T75iflbXSQ1iJJMpnkNUnLVk+WCIwAoEGq1Xe/4uPjtWPHjirXeTyeyDbRNnToUJ1zzjkVli1fvlzDhw9XYmKiXC5X1Gs4kPJvZSYnJ8tms8W0FgCoTzi/AkD01JZzbGJiYsyODQBAdQVDhnYWebW1oFS5+R7tyMtT8c7N8uWvl8f9u3y+TQoZO2RYCxWwlajE6lWeVdputWhbmkVSNQKjMtstv0bvhQAAarVaHRg1adJEK1askNfrrdRptGXLFqWnp0e9u0iScnJylJOTU+U6q9VaK24iltdRG2oBgPqE8ysARE9tOMfSQQoAiLVQyNCuYq9y8z3KLSjV9t35Ktm9Ud68DSpxr5Pfu0kBY6dkLZB/nzBop82iYFpVYVDVI/ZUh8ccPPwXAwCo02r11VH37t312WefaeHChTr99NMjyz0ej5YuXaozzjgjhtUBAAAAAAAA+2cYhvKKfcot8Ghrfql27slX0c5N8uZtUKl7rby+TQqEdsiwFihgLVHxXmHQLqtFwdTDD4MchkmZ5jjtCRbLba5+h1GSqVbfLgQARFGt/gswePBgPfLII5o8eXKFwOjFF19USUmJrrjiisiytWvXyu/3q0OHDrEoFQAAAAAAAA2IYRgqKPVra75H2wrDnUHunRvlyVuv4qK18vk2y1cWBvmtJSq2+pRvNbTNatEui0WhlMMPg5xlYVC2w6XG8RnKSmyq7JRWykptp6zkZspOyFayPVkmk0kvfzhMz+R9W+3XdVZ292pvCwCoX2p1YNS5c2fdcsstmjp1qgYOHKhzzz1XK1eu1JQpU9SrVy9dfvnlkW3POussbdiwQYZhVNjHnDlztGHDBknSzp075fP5NHHiRElSixYtdNVVVx29FwQAAAA0cGu3btestz+TNhuyBm0KWPxSjlnXXNxXbZpkxbo8AAAiCj1+bSvrDNqeVyD3zo0qzVuvksK1KvVtkj+0XSFLWWeQzac9FiPcGbTfMKh60yqUh0GN7S41TigPg1orK62tspJzlJ2QrSRbkkym6nUNXXTGGE1/71x5TZIO9BzDkNOQLjp9TLX2CwCof2p1YCRJkydPVsuWLTVjxgx9/PHHSk9P14gRIzR+/HiZzeaDPn/mzJn697//XWHZgw8+KEnq1asXgREAAABwlEyc8aaSlriUbTStuGKF9NGEJSo6oUBjhg2OTXEAgAalxBfQ1sicQQUqKusMKir6TR7vRvmCOxW0FChgLQ6HQVZD2y3hMMhwHX4YFF8WBmXZXWqSkKnsxKbKSmmlrLR2kTAo0ZZY7TCoOlyu5nogp58e2vKJZBhVh0Zly+9v9hclu6qexxsAUP/V+sDIYrFo9OjRGj169AG3W79+fZXLv/rqq5ovCgAAAMAhmTjjTaX+mCFDRpXrLYZNqT9maOKMNwmNAABHxOMPKrfAo9z8Um3LK5R75wYV7/5d7sLfVOrbJF9wu4LmQvlsxSqxhsOgbRardlstUpLCjwqqFwYlGCZl7BUGNU5qqixXOAzKdrVQVnyWEu2JNf1yq+XCPk9K86VJm/4V7jTah9OQ7m/2l/B2AIAGq9YHRgAAAADqtrVbtytpiUuGDJlU9TemTTLJkKHEpS6t3bqd4ekAAFXyBoLaXuDV1oJS7cgrVOHODSre9buKin5TiWejfKGdCpjyy8Igv/ZYQ9pmtSrPYqkiCJKqHwaZlWl2/tEZlNRM2a5WymrUTtnJLZQZnxmzMKi6LuzzpHoXjNAH/5ukBTsWqyjkV5LZpt5Z3XRez/vlcjWPdYkAgBgjMAIAAAAQVbPe+VzZRpODbmeSSbaQXbPe+VwTRl55FCoDANQm/mBI2ws92lbg0ba8QhXu2KjiXetUUPirSr2b5A1ul99UIL+tWMVWf3iYOKslHAYlKPyowFat4yYZZqWbncqyudQ0MUvZSU2VldJa2WntlOVqrqz4LCXYKu28TnK5muuqc1/QpX6/CgoK5HK5ZLNV730CANR/BEYAAAAAqqXE49XWvDxt212gHfkFKigsVqG7RCUlHnlLffKVBhT0BSW/IZNPMgfMsgYtSitOP7QDbQ5F5wUAAGImGDK0syjcGbR9d6EKdm5U0c61KizrDPIGd8hnypffWqLismHitlst2lMDYVCGyalMe4qaJmSqcXIzZblaKbtR+0gYFG+Lr+mXCwBAnURgBAAAANRzwWBQeUVubc3box15BdpVUKTCohK53SUqLfHKW+pTwBdUyBuS/JI5YJLFb5Y1aJU1aJM9aJc96JQtVHnYHqvilayavdFmDfBNZwCoS0IhQ7uKvcrN92jbngIVbN8k9+51yi9Yo2LPJnkD2+Uz5cu3TxiUb7FIcQo/Kqje34Fkw6x0U9mcQYmZapLUTFkprZSd1l5ZrvAwcYRBAABUH4ERAAAAUIt5fH7l5u1R7u492lVQpD2FRSosKlZxsVeeUq/8pQEFvUEZfkMmf7irxxKwyBa0yha0y1YW9lgMyz57tsqpZDmjXL/f7JPZMMtiVP/SI2D1R7EiAMChMAxDecU+5RZ4lJtXoMKdm1Sw4zcVFP2m4pKNKg3ukE97wmGQzac9lnAYVGCxSE6p8h+a6oVBrr3DoIRMNUlupuyU1spq9EcYFGetlDQBAIAjQGAEAAAARElekVtbd+/Rjj352pVfqIKiEhW5S1RS7JXP45PfG4h09Zj8JlkCZlkD1kjYYw86ZQ85Ku3XrHglKb7qubtrSEgh+Swe+S0++Sw+Bax+BSxBhawhGXbJZDPJbLfIHmeVM86uuHinkhPjleJKVIYrWdmNXGqSlqZ4p0MPTpmr7BUHn8Moopk5ei8MABBhGIYKSv3amu/R9vxC5W/fpIKdv2pPwa8q9myUJ7BdXuXLa6kYBhVaLJJd4UcF1QyDQmZlmOOUWRYGNU1upqyUNuE5g1JaKjM+U05rtL/SAAAA9kVgBAAAAOzD7w9oW36+tu7O1678AuUVuMNdPSUelZZ45ff4FfAGZfiM8BBufrOswfKwxx4Zwq1yV41FDiXJEdWoRwqY/PJZPfJZfPJb/ApYAgpagwpZDckmmR1mWR0W2Zw2xcU7lJgYp+TEeDVKSVJmSoqy01KUlZIsi2XfrqTDc83FffXRhCWyGDaZZNrvdoYMBcx+XXfJ2TVyXABo6Ao9fm0r8Cg3r1D5OzYpf8ca7Sn4TSWlG1VcHgZZi1Vs9WuPNaTtFquKLOYjCoNSQmalm+OUaSsLg1w5ZZ1B7ZTtaqXMhEw5LJW/DAEAAGKPwAgAAAD1ittTqs0787Q9L9zVk19YrCJ3iYpLyrp6POGunvAQbuGunvAQbjbZgrZwV0/QLpP27XJxKkHOynNu1yBDIfksXvksXvktfvktfgWtQQWtIRk2Q7JLFrtFNmdZV0+cQ0mJ8UpJTlR6SpKy0lLUJC1NyQm1a4ieNk2yVHRCgVJ/zJAho8rQqHy5u2uBWmVnxqBKAKhbSnwBbc33aNueQuVv36y8nWuUn79GRaUbVRrYLo+RL08kDAp3BrnN5nDuUyn7qd7todS9wqDGCZlq5mpeFga1VVZKK2XFZ8luqTzfHQAAqBsIjAAAAFArBINBbc8v1La8fO3Iz9fu/CIVFJWouLhUpSVe+Tx+Bb3ByBBu5oBJloBF1qC1LOixyx5wymrsexfMLJsSlaLE6NZvCshnKe/q8SlgDShgCSlkC0k2yWQ3yeqwyua0Ki7eoYQEp5ITE5TmSlRmqkvZaSnKTkmRzVY/P6KPGTZYE2e8qaQlLlmNyjcTA2a/3F0LNGbY4BhUBwC1i8cfVG6BR9vyCrV720bl71qjvPzf5C7ZoJLADpVqjzyWimFQsdkcvstTqYn14H9XTIaUYoSHicuwudQ4PlM5Kc2VndpaWWntlJ3aWplxmbJZqtdlBAAA6qb6eTUKAACAo6rE49XWvDxt212gnQWFyi9wq9BdopISj7ylPvlKAwr6gpLfkMknmQNmWYMWWQPhoCc8jJtDZu07BJpd8bIrPsr1+8yesq4en/xlc/VEunrKhnCzOaxyxNkVF1/e1ZOgRq4kZaamqFlGmlISotl7VD+MGTZYa7du16x3Ppc2hWQN2hSw+KUcs665uK/aNMmKdYkAEHXeQFDbC7zauqdI+ds3aveO1crL/1VFpRtV4t+uUiMcBrmtfuVbDW2zWlRiNksWHXYYlGqYlW6KU4b9jzCocWprZTVqr+yU1sqIyyAMAgAABEYAAAANWTAYVF6RW1vz9mhHXoF2FRSpsKhEbneJSku88pb6FPDt09XjN8satMpa3tUTdMoWqtwxYlW8kqMc9QRNQfksHvkjQ7gFFLAGZVhDMsq6eix2i2xxVjnjwnP1JCXGKTU5SZmpLmWlutQ4LVVOOzfJjpY2TbI0YeSV8vv9KigokMvlks3G+w+gfvAHQ9pe6NG2PW7t3rZReTtWa3fBGhUUb1Spf7tKlC+PuVhuqy/SGVRqNktmqfKYpwe/ZWPeOwwq6wxqltpcTVJbKyutvbJT2yg9Pl02M+dZAABwcARGAAAAdZTH51du3h7l7t6jXQVF2lNYpMKiYhUXe+Up9cpf1tVj+AyZ/OGunvBcPVbZIl09TlmMfbt6rHIqWc4o1+83e8vm6/GF5+qxBBS0hRSy/tHVY3VY5XDaFBfvUGJivFzJCWqUnKjMNJeaNmqk1MR4WSz71g8AQM0LhgztLAp3BuXlbtKunSu1c8+vKirZoBL/dpUY+So1F6vY6lOe1dCO8jBIOuwwKM0wq1EVYVB2o47KTm2tRnGNCIMAAECNITACAACIgbwit7bu3qMde/K1K79QBUUlKnKXqKTYK5/HJ783EOnqMflNsgTMsgaskbDHHnTKHnJU2q9Z8UpSfOURa2pQSKGyuXrCXT2BsiHcQtaQDLtksplktltkj7PKGWdXfIJTSQnxSnElKjMlWVlpLjVJS1O8s3L9AADEQihkaFexV7l54c6gndtXaVdZZ1A4DNqjkvIwyBIOgzzlYVClZtrqhUGNysKg9L3mDGqS1kbZjdorK7WN0uPSZTVz2wYAABw9fPIAAAA4BH5/QNvy87V1d7525Rcor8Ad7uop8ai0xCu/x6+AN9zVI79k9ptlDZaHPfbIEG4WY9+PYRY5lCRHVKMeyW/2yb9XV0/AElDQGtynq8ciW6SrJ04piQlKTUlUZkqKmqanKj05ia4eAECdYRiG8op9yt1TrF3bNmrHjpXalbdGBSUb5PZti3QGFZUNE7fDYpG3PAyKK3tEVDMMClnUyORUht2l7PhMNXM1V06jY5SV3l5ZKa2VHpchi5m/pQAAoHYhMAIAAA2G21OqzTvztD0v3NWTX1isIneJikvKuno84a4ew29EunrCQ7jZZAvaZA86ZA86ZJJ5nz07lSBn5dFmapChUNnwbeVz9fgVsAYUshoybIZkV3iuHme4qycuzqGkxHilJCcqI8WlzLRkNUlLU3JC3MEPBgBAHWEYhgpK/dqaV6zd2zdo+/YV2rlnjfKLN4XDoFC4M6jI6le+1dB2i0U+syn8ZGfZI+Lgt0gshtQo9McwcVnxmWqW0lzNGx2jrLLOoEZx6YRBAACgTiIwAgAAtV4wGNT2/EJty8vXjvx87c4vUkFRiYqLS1Va4pXP41fQG4wM4WYOmGQJWGQNWsuCHrvsAaesxr5j/JtlU6JSlBjd+k2BsiHcfPJbfApYAwpYQgrZQpJNMtlNsjqssjmtiot3KCHBqeTEBKW5EpWZ6lLjRmnKciXLZuOjGwCgYSn0+JWbV6yd28rCoLw12lO8QcW+7So29qjEFA6D9lgNbbda5DeVhUGOskfEwQMcqyGlhSxKNzmVbktRVlyGmqWGw6Ds9A7KSmmjRvHpMpv2/eIIAABA/cBdBwAAEFUlHq+25uVp2+4C7SwoVH6BW4XuEpWWeOQp9clXGlDQF5T8hkw+yRwwyxq0yBoIBz3hYdwcMle60WNXvOyVpw2oYT5z+Vw9PvnL5uoJWkORrh6z3SybwypHnF1x8eVdPQlq5EpSZmqKmmWkKSUhmr1HAADUTSW+gLaWhUG5uSu0Y88a7XFvkNu/TcWhfBWbi+W2lA0Td8RhkKFGIasamZxKt/7RGdQivSwMSm2rtLg0wiAAANCgERgBALCPtVu3a9bbn0mbDVmDNgUsfinHrGsu7qs2TbJiXd5REwwGlVfk1ta8PdqRV6BdBUUqLCqR212i0hKvvB6/At5Axa4ev1nWoFXW8q6eoFO2kL3Svq2KV5LiozpbT9AUlM/iKZuvJzxXT8AalGENySjr6rE4yodwC8/Vk5QYp9TkJGWmupSV6lLjtFQ57ft2JQEAgIPx+IPK3VOsHbkbtWXbz9pZFgYV+barOLRHxaZiFe0VBgWOIAyyGUbZnEFxalQeBrmaq0XGMcpO76js1GOUShgEAABwUARGAADsZeKMN5W0xKVso2nFFSukjyYsUdEJBRozbHBsijsEHp9fuXl7lLt7j3YVFGlPYZEKi4pVXOyVp9Qrf1lXj+EzZPJLpoBZ1oBFtqBVtkhXj1MWY9+bNFY5lVxxuP8o8Ju9ZfP1+PaaqyekkPWPrh6rwypHnE1xcQ4lJsbLlZygRsmJykxzqWmjRkpNjJfFwvwBAADUNG8gqO35pdqeu0Fbty3X9t1rlOfeqCLfNrnLwyCrT3sshnbuHQbZyh4R1QuD0kNWpcmpdJtLmXGZaurKUYuMtmqS0VHZaW2V6kyTqfwYAAAAOGwERgAAlJk4402l/pghQ0aV6y2GTak/ZmjijDejGhrlFbm1dfce7diTr135hSooKlGRu0QlxV75PD759+rqMflNspQN4WYL2CJBjz3kqLRf81Ho6gkpVDZXj7cs6AkP4RaylXX12EyyOMyyOW1yxtkVn+BUUkK8UlyJykxJVlaaS03S0hTvrFw/AACIPn8wpO0FJdqRu1Gbc3/Wtt2rtMe9UYXeimFQntXQTotFwSMIg+x7hUGNImFQuDOoWeaxykptqxRnKmEQAADAUUJgBACAwsPQJS1xyZAhk6q+KWGSSYYMJS51ae3W7ZWGp/P7A9qWn6+tu/O1K79AeQXucFdPiUelJV75PX4FvOGuHvkls98sa9Asa8BaFvSUd/Xs++fZIoeS5Ihq1CP5zb6y4dvKunosAQWtwX26eiyyx9kiQ7ilJCYoNSVRmSkpapqeqvTkJLp6AACopYIhQzsKS7Qtd4M2b/lZ2/JWK8+9UYXeXLlDe+QuGyYuz2po195hkFX73D04+N96Ryg8Z1CayalG1r3CoMy2yikLg1zOFMIgAACAWoTACADQIASDQZX6fSou9aqotFTFHq+KS70q8Xjl8fn09Zc/qbnR6qD7MckkW8iuN574Sj6rX7agTbagTfagQ/agQybtOza+UwlyKiE6L0uSZChUNnxbuKvnjyHcDBm2cNhjsZfP1WNXXLxTSQlxSklOVEaKS5lpyWqSlqbkhLgoVgkAAKIpFDK0y12qbbkbtXHLT8rdvUZ57g0q9G6XO5inIlOxiiz+sjDIrFB5UGORFL/3ng4eBjn3CoPSysKgJq7mapXRVs2yjlN2WlslO1yEQQAAAHUMgREAoEaVeLwq9nhUWOJRsTccyJR4PCr1+uTx+uTx+eX1+uX1++X3B+T3BeQPBBXwBxUMBBUMhhQKhBQKhmQEjchDIUkhyRSUFDLJFDLJbJT9b8gss2GS2bDIHDLLYphlDllkMSwyG1ZZyv67cpjzh+Y6eFi0t1RPxpG9UWUCJv9eXT0+BawBBSwhhWwhySaZ7CZZHVbZnTY54+1KSHDKlZSg1OREZaa61LhRmrJcybLZ+JMOAEB9ZRiG8twebd26IRwG5a3R7qLwMHFFoT1yy60Ci197rIZ27x0GmSVV+D7I/j8LlXOGwsPEpSrcGZQRl6kmrhy1ymynnKzjlJXWTsmOZMIgAACAeoi7SwBQB3l8frlLPX90yni8Kin1yOPzqcTjk8/nDwczfl9ZKBMsC2UCCgSCCgVDCgZCCgUMGeXBTMiQgvojmAmZZCr73/JAxrRXGGM2zOFApiyMsYSs4cCmGjciJJMkuyyyyyLJGdV3K3p8Zo98Vm94KLeyuXqC1lCkq8dsN8vmsMpRNldPYkKcUpITlO5KVkaqS80y0pSSEM3eIwAAUNsZhqGCEq82l4dBu9dod9EGFXi3qSj4RxiUVxYGGUcQBsWVdwbJqTRrijLiMtQ4OUctM9uqZXYnZTVqpyQ7YRAAAEBDRWAEAPvh9wdU5PGoqKRUJV6v3KUelXi8Ki0LZbw+v7w+n7zegPyBgHw+vwKBoPz+oIL+8k6ZcDgTChoyAlWFMuXBjEnmkEkmw1wWzphlCZnDAUxZMFMeyFhCFpmrMVRImF1m2eWQ5IjeWxVTQVNAQVNQQXNAIVNQQVNQIXNQIVNIQXNIIVNIhjmkkMlQyGzIMBuS2ZBhVnjEFZMkq5Sam6IkX0q1j7staYsmPHlVlF4VAACoLwpLfdqydb3Wb1mm3F2rtKssDCoM7pFbxSowh+cMyts7DDJpn2/UHDwMii8Lg8o7g9KdmX+EQY2PU3ajDkpyJEfjJQIAAKCeIDACEFPBYDASyoQDmfAcM6U+r0q8e4cy4eHLfP6AAoGAAv5wMBMKhhT0BxUKGuFQJhiqGMoEK4YyJmOv4cvKA5lQWadMWRhjMcqCGaO6oYxVJlnlkLNehzLlYUw4mAkpWBbKhExBhcqCmfJAxjAbMkx7hTJmyWQxSWaTTFbJbDHLbDHLYrXIYjXJarPKarXIZrXIZrfKbrPJYbfJbrcpzmFXvMOuOIdD8U6H4p12JcbFKTkuTglOuyyW6v6eDuzBKXOVtCKl+k/I4Zu3AAA0dMUevzbnbtDvm5cod9ca7Sz8Xfne7SoK7lGRilVg8WmPxdBu6z6fVyp8aDx4GJQQMpQWtCjNFKe0vcKgFhlt1bpJJ2U36qBER1KNvjYAAAA0PARGddjards16+3PpM2GrEGbAha/lGPWNRf3VZsmWbEuD7VIMBhUscenwtJSuUtLVeLxhTtlvOWhjE8er19en18+v/+POWXKhjALBgwFI50yIRkBSaG95pUJSiZjr1Bmr26ZCl0y5cOYhax/BDRGdU9DVklW2SXZo/ZOxVbQFCwLZQIKmcvCmUgYEyzrkAkpZA6VhTHlD0VCGVkkk9kkk8Uss8Uks8Uki80is8Usm80ii9Uim80qu80qm80qh8Mmu80mZ3koY3coIa48lHEqKT5OCXZHg5gf55qL++qjCUtkMWwyaf9hkCFDAbNf111y9lGsDgCAhmXTtnWa9flD+rn0F5UqoDhZ1Tmuk4b0Haec7NZHpQaPL6BNW9fr981LtWXXau0qXK893m0qCuxR4V5hUN4RhkGJZWFQqiqGQc0z2qpNk05qnNFRCfbEGn1tAAAAQFXq/x3AemrijDeVtMSlbKNpxRUrpI8mLFHRCQUaM2xwbIproILBoEr9PhUWe8KhjNen4lKPSr2+fUIZn3z+YNnwZeFOmYA/qEAwqKA/JCMUnlcmVN4pE5BkKNIpo70CmfI5Zf4YvswcmU/GXDaEmeWQQhmLJItskmzReqNiLKRgWWdMxW6ZkCkYGb4sHMjs3S2jsmHMVDGUsZpkMptlsZpktlpksZhlsVnCnTI2i+x2m2xWqxwOqxxWmxxOu+KddsXZ7Yp3OhTncCipLJSJdzjktNfXd71uaNMkS0UnFCj1xwwZMqoMjcqXu7sWqFV2ZgyqBACg/nv6zZv1esl/5DWb9gpfAloRWqq//2uALo8/Q3cMnn5Ex/D6A9q8db3Wbl6qrbtWa2fheu3xbFdRIE+Fpj+GiduzbydzhW8vHTwMSgoaSg1ZlKY4pZaFQdlJOWqecYzaNO2spukdFe8gDAIAAEDtQGBUB02c8WbkhmZVLIZNqT9maOKMN+tdaBQMBuX1B1RUWqqi0lIVe7wqLvXK4wt3zHh9fpV6w8OX+QIB+XwBBfxl3TL+gIKBUFmnTFkgEzAUChnS3p0ye4cyRnk4Ux7GmCPzyYSHMbOWdc2EQxpTNS4a/whl7PU2lDEUqjinTFm3TDiMCUZCmfKOmfDwZZJhqSKUsYQfZqtZFmt4GDOrNdwpY7dZZbOHO2YcNrvsdqucdrucDpviHQ7FxzkV57ArKc6pRKdDSfHxhDI4oDHDBkcCeatRuZctYPbL3ZVAHgCAaHn6zZv1iudr7a/Z12tSeP2bN+83NPIHgtqUu05rNy3Xll0rtbNwg/JLt6kgmBfuDDL7tNsqFVj2+ex+GGFQo5BFKYpTmsWlRs5MZSfnKCf9GLVt1kVN0jsQBgEAAKBOITCqY9Zu3a6kJa79fvtdkkwyyZChxKUurd26/bCGp/P4/CoqKVFhiUfFXm94+DKPVyVerzxevzy+8lDGXxbKBMNzy/iDCgaDCgbCQ5eFAiEZQSPyUEhSSDIFFQllTIb2M6eMOTKnjLksmKl+KGOWZJdVdlm1z3yx9UjA5N+rWyagkDm01xBme3XLmA2FyoYwU6UhzMLDlpksprJ5ZUxl88pYZLWF55Sx2qyy261y2Gyy262Kc9jlsNvK5pNxKs5hU6IzTolxDiXFxSneWV9n8kFDMGbY4PCQn+98Lm0KMeQnAABHyaZt6/R6yX/CYZFpP4mRySQZhl4r+Y+cb9+tEm++8jzbVBAIh0GFBwqDIg5+PZFcHgYZcUop6wzKSspR8/RjdEyzLmqW2VFx9oTDfakAAABArURgVMfMeudzZRtNDrqdSSbZQnbNfeoLeZzeSJeMqSyUsUS6ZaqYUyZklblaoYxJkl0W2WXRPkN11yNBUyD82GsYs/D8MqFIMGNEumWqCGVMkqyS2Wwu65QpG8bMFu6UsZYFM1arVXa7TQ5b+H+dDpsc9vC8MuFgxqHEOIfiHU4lJzgVZ7PLsu8QGQBqRJsmWZow8kr5/X4VFBTI5XLJZqM7DQCAaJr9+cPhYegOxmSSzyT9reRf4Z8rjKd88OuYlKCh1GC4Myi1rDMoK6mZctLb6phmndU86zg57fGH+zIAAACAOovAqK7ZFDqkzdNLsqWSKNVSg8KhTFkgUzanTLB86LKyuWUqziljRIYxK++UMVlM4WHMIp0y5nCXjHWveWWsFtnsVtltNjn2CWXiHHuFMk6nkuPilOAklAEAAEDtFQqF9Oyzz+qFF17Q+vXrlZGRoUsuuUTjx49XQkLd6oBZXvrzEX8LLSVoKK0sDEqxuNTIkaGs5BzlNAp3BrVo3EkOW1zNFAwAAADUMwRGdYw1ePjfcI8EMpW6ZcKhTIU5ZUx7BzPaa/gyyVTeKVP2sNgsMlvMstn26pSxWcPzypQFMg67LRzK2B1KiCt7OB1Kio9Tgt0hm41/igAAAMChuv322zVlyhRdeOGFGj16tFauXKkpU6ZoyZIlmj9/vszm6owcUDt4TIFD2r5RIKTBGeepeaO2apPTWa0ad5LDRmcQAAAAcLi4S1/HBCz+Q9p+W+IW3f7whUpyOgllAAAAgHrkl19+0XPPPaeBAwfq3XffjSxv1aqVRo4cqXnz5unyyy+PYYWHxmlYJVU/NMoO2nXTgMeiVxAAAADQwNSdr5shLOcQf2XNTUpLSiQsAgAAAOqZN954Q4ZhaNSoURWW33DDDYqPj9fcuXNjU9hh6hzX6ZC27xTXOUqVAAAAAA0TKUIdc83FffXRhCWyGDaZtP8JYQ0ZCpj9uu6Ss49idQAAAACOlkWLFslsNuvkk0+usNzpdKpr165atGjRQfexadMmbd68ucKy5cuXS5ICgYD8/kMb4eBIXNl7jP7+2UXymiSZ9n+tI8OQw5Cu6vPgUa0PAOoTv99/1M/zANAQ1KbzayBwaEM+SwRGdU6bJlkqOqFAqT9myJBRZWhUvtzdtUCtsjNjUCUAAACAaNu6davS09PlcDgqrWvatKm++eYb+Xw+2e32/e5j5syZGjduXJXr3G63CgoKaqzeg0l0NNLFtlP1WvA7yTCqDo3Kll9s7aEER+pRrQ8A6pNAIKDi4mJJktXK7UEAqCm16fzqdrsP+Tn8RaiDxgwbrIkz3lTSEpesRuWLv4DZL3fXAo0ZNjgG1QEAAAA4GkpKSqoMi6Rwl1H5NgcKjIYOHapzzjmnwrLly5dr+PDhSkxMlMvlqrmCq2H04OmyvjNS8zxfhzuN9uEwpEsdp+m2i6cc1boAoL4p/+Z7cnKybDZbjKsBgPqjNp1fExMTD/k5BEZ11Jhhg7V263bNeudzaVNI1qBNAYtfyjHrmov7qk2TrFiXCAAAACCK4uPjtWPHjirXeTyeyDYHkpOTo5ycnCrXWa3WmFzk3nnZ3zR42zrN/vxhLS/9WaUKKE5WdY7rrCHnjFVOduujXhMA1Efl5/lY39AEgPqmtpxfD6fDicCoDmvTJEsTRl4pv9+vgoICuVyumP8jBAAAAHB0NGnSRCtWrJDX663UabRlyxalp6cfsLuoNsvJbq0xV83hWgcAAAA4isyxLgAAAAAAcOi6d++uUCikhQsXVlju8Xi0dOlSdevWLUaVAQAAAKiLCIwAAAAAoA4aPHiwTCaTJk+eXGH5iy++qJKSEl1xxRWxKQwAAABAncSQdAAAAABQB3Xu3Fm33HKLpk6dqoEDB+rcc8/VypUrNWXKFPXq1UuXX355rEsEAAAAUIcQGAEAAABAHTV58mS1bNlSM2bM0Mcff6z09HSNGDFC48ePl9nMgBIAAAAAqo/ACAAAAADqKIvFotGjR2v06NGxLgUAAABAHcdXzgAAAAAAAAAAABo4AiMAAAAAAAAAAIAGjsAIAAAAAAAAAACggSMwAgAAAAAAAAAAaOAIjAAAAAAAAAAAABo4AiMAAAAAAAAAAIAGjsAIAAAAAAAAAACggSMwAgAAAAAAAAAAaOCssS6griouLpYkLV++PMaVSIFAQG63W4mJibJa+ZUCQE3h/AoA0VNbzrHln+fLP9+Dax0AaAg4vwJAdNSm8+vhXOvwF+EwrVu3TpI0fPjwGFcCAAAA4EiVf74H1zoAAABAfXIo1zomwzCMKNZSb23dulUfffSRWrdurYSEhJjWsnz5cg0fPlwvvPCCOnfuHNNaAKA+4fwKANFTW86xxcXFWrdunfr3768mTZrErI7ahGsdAKj/OL8CQHTUpvPr4Vzr0GF0mJo0aaJhw4bFuowKOnfurFNPPTXWZQBAvcP5FQCih3Ns7cO1DgA0HJxfASA66ur51RzrAgAAAAAAAAAAABBbBEYAAAAAAAAAAAANHIERAAAAAAAAAABAA0dgVA80a9ZMDz/8sJo1axbrUgCgXuH8CgDRwzkW1cG/EwCIDs6vABAddf38ajIMw4h1EQAAAAAAAAAAAIgdOowAAAAAAAAAAAAaOAIjAAAAAAAAAACABo7ACAAAAAAAAAAAoIEjMAIAAAAAAAAAAGjgCIwAAAAAAAAAAAAaOAIjAAAAAAAAAACABo7AqI569NFHNWjQILVu3Vomk0ktW7aMdUkAUC+sWbNGDz30kHr06KGMjAwlJSWpa9eumjRpkoqLi2NdHgDUWatXr9YVV1yhjh07yuVyKT4+Xh06dNAdd9yh3NzcWJeHWoRrHQCIDq51ACA66tO1jskwDCPWReDQmUwmpaWl6cQTT9TixYuVnJys9evXx7osAKjz7r33Xk2bNk0DBgxQjx49ZLPZtGDBAr311lvq0qWLvvvuO8XFxcW6TACoc7744gtNmjRJPXr0ULNmzWS1WrV8+XK98sorSk5O1tKlS5WZmRnrMlELcK0DANHBtQ4AREd9utYhMKqj1q1bp9atW0uSOnXqJLfbzUUUANSAH374QW3btpXL5aqwfMyYMZo0aZKee+453XrrrTGqDgDqn7fffluXXHKJHn/8cd19992xLge1ANc6ABAdXOsAwNFVF691GJKujiq/gAIA1Kxu3bpVuoCSpMGDB0uSfv7556NdEgDUay1atJAk7dmzJ8aVoLbgWgcAooNrHQA4uuritY411gUAAFAXbN68WZKUlZUV40oAoG7zeDxyu93yeDxasWKF7rnnHknSueeeG+PKAABomLjWAYCaUR+udegwAgDgIILBoCZMmCCr1arLL7881uUAQJ320ksvKSMjQzk5OTrnnHOUn5+vuXPn6vTTT491aQAANDhc6wBAzakP1zp0GAEAcBCjRo3St99+q0ceeUTt27ePdTkAUKddcMEF6tChg9xut5YsWaIPPvhAu3btinVZAAA0SFzrAEDNqQ/XOgRGAAAcwIMPPqipU6dq2LBhuu+++2JdDgDUec2aNVOzZs0khS+oLrroInXv3l0lJSWcZwEAOIq41gGAmlUfrnVMhmEYsS4CR6ZTp05yu91av359rEsBgHpl7NixGjdunK699lrNnDlTJpMp1iUBQL3Uo0cPbdmyRZs2bYp1KahluNYBgOjgWgcAjo66dq3DHEYAAFSh/AJqyJAheumll7iAAoAoKi0tVV5eXqzLAACgQeBaBwCOnrp2rUNgBADAPsaPH69x48bpqquu0ssvvyyzmT+XAHCktm3bVuXyBQsW6Oeff1aPHj2OckUAADQ8XOsAQM2rT9c6DElXR82ZM0cbNmyQJD333HPy+XwaPXq0JKlFixa66qqrYlkeANRZ06ZN06233qrmzZtrwoQJlS6gsrKy1Ldv3xhVBwB114UXXqjc3Fz17t1bLVq0kMfj0eLFizVv3jzFx8frq6++UteuXWNdJmoBrnUAIDq41gGA6KhP1zoERnXUmWeeqX//+99VruvVq5e++uqro1sQANQT11xzjWbPnr3f9ZxjAeDwvPXWW3r11Vf1008/aefOnTKZTGrRooX69u2ru+66S82bN491iagluNYBgOjgWgcAoqM+XesQGAEAAAAAAAAAADRwDFQKAAAAAAAAAADQwBEYAQAAAAAAAAAANHAERgAAAAAAAAAAAA0cgREAAAAAAAAAAEADR2AEAAAAAAAAAADQwBEYAQAAAAAAAAAANHAERgAAAAAAAAAAAA0cgREAAAAAAAAAAEADR2AEAAAAAAAAAADQwBEYAQAAAAAAAAAANHAERgAAAAAAAAAAAA0cgREA4IjNmjVLJpNJX331VbW2b9mypc4888yo7NtkMumaa66p1rbRcuaZZ6ply5YxrQEAAADAkeE6pyKucwCg/iMwAgAAAAAAAAAAaOCssS4AANDwrF69WiaTKdZlAAAAAECN4ToHAFDX0WEEADjqHA6H7HZ7rMuApKKioliXAAAAANQLXOfUHlznAMDhITACANSYUCikp556Sm3atJHD4VC7du00e/bsStvtb2zvF198UR06dJDD4dAxxxyjyZMnyzCMKo/1yy+/6C9/+YsSEhKUlpamK664Qjt27NhvbW+++aZOO+00JSUlKT4+XqeccoreeeedStuVjw3+7bffqlevXkpISFCjRo10/fXXy+12V//N2MfChQt1zTXXqF27doqPj1dSUpL+9Kc/6f3336+w3W233SaTyaRff/210j5yc3NltVp13XXXVVg+f/58nX322UpJSZHT6VSXLl30/PPPV3p++fu+ZMkSnXPOOXK5XOrSpcthvyYAAACgIeA6Z/+4zgGA+oXACABQY+6//37NmTNHw4cP1xNPPCGz2axrrrlG//vf/w763MmTJ2vYsGFyOp165JFHNGTIED311FN67rnnKm37+++/6/TTT9fXX3+tW2+9VePHj9euXbv0l7/8pcp9jxkzRpdeeqmSkpI0YcIEPfbYY4qPj9egQYM0bdq0StsvXbpU/fv3V/fu3fX000/r7LPP1syZM3XHHXcc+ptS5v3339eqVat0ySWX6Nlnn9UDDzygvLw8DRw4UK+//npkuxtuuEGS9PLLL1fax+zZsxUMBnX99ddHls2YMUNnn3223G63HnjgAT399NNq06aNbrrpJt11112V9rFx40b17t1bLVq00JNPPqkRI0Yc9msCAAAAGgKuc/aP6xwAqGcMAACO0CuvvGJIMrp27Wp4vd7I8s2bNxt2u9249NJLK2zfokULo1evXpGf9+zZY8THxxsdO3Y0iouLI8s3bdpkJCQkGJKMBQsWRJZfdtllhiTjyy+/jCwLhULGBRdcYEgyhgwZElm+ePFiQ5Jx3333Var7/PPPN5KSkozCwsLIMkmGyWQyvvvuuwrbnnvuuYbVajWKiooO+n706tXLaNGiRYVlbre70nbFxcVGu3btjI4dO1ZYfuqppxqNGzc2AoFAheVt27atsO3WrVsNh8NhXHbZZZX2PXLkSMNsNhtr166NLGvRooUhyXjxxRcP+hoAAACAho7rnIq4zgGA+o8OIwBAjbn55psrjNndtGlTtWvXrsphB/b22WefqaSkRLfccovi4+Mjy5s1a6YrrriiwrahUEgffvihunXrpj//+c+R5SaTSXfffXelfb/22msymUwaMmSIdu3aVeExYMAAFRUV6dtvv63wnFNPPVWnnHJKhWW9e/dWIBDQ+vXrD/o+VCUhISHy3yUlJdq9e7dKSkrUu3dvrVy5UoWFhZH1w4YNU25urv75z39Glv3nP//Rr7/+qqFDh0aWvfPOO/J6vRo6dGil13beeecpFApp/vz5FepIS0vTtddee1ivAQAAAGiIuM7ZP65zAKB+sca6AABA/dG6detKyxo1aqQNGzYc8Hnr1q2TJHXo0KHSumOPPbbCzzt27JDb7a7WtpK0cuVKGYZR5fbltm/fXuHn/b0OSdq9e/d+93MgO3bs0JgxY/SPf/yjyjHI8/PzlZycLEkaPHiwRo0apZkzZ+q8886TJM2cOVN2u11XX3115DkrV66UJPXp02e/x933tbVp00YWi+WwXgMAAADQEHGds39c5wBA/UJgBACoMfv7gG7sZ0LXo8EwDJlMJv3rX//ab33HHXdchZ8PdKFxOK/FMAydffbZWrlypW677TZ169ZNLpdLFotFr7zyil5//XWFQqHI9nFxcbryyiv1wgsvaPv27YqLi9M777yjAQMGKCMjo1Itr776qho3blzlsfe9KNz7m40AAAAADo7rnP0/h+scAKhfCIwAADFX/mF/1apVOuussyqsW7FiRYWfMzIylJiYqFWrVlXaz77bSlLbtm31ySefqHnz5urYsWMNVl19y5Yt008//aSHHnpI48aNq7DupZdeqvI5w4YN07Rp0zR79my5XC6VlJRUGKZBCr82SUpPTz/gt+8AAAAAHH1c51TGdQ4A1G7MYQQAiLm+ffsqLi5O06ZNU0lJSWT55s2b9frrr1fY1mKxqH///vrhhx+0YMGCyHLDMPTEE09U2vdVV10lSbr//vsVDAYrrd93KINoKP8m377f2vv555/1/vvvV/mcLl266OSTT9bLL7+smTNnqnnz5jr77LMrbHPJJZfI4XDo4YcfVmlpaaV9FBQUyOv11tCrAAAAAHAouM6pjOscAKjd6DACAMRcamqqJkyYoDvvvFM9e/bU1VdfrZKSEj3//PNq27atlixZUmH7iRMn6l//+pf69++vESNGqFmzZvrwww+1c+fOSvvu3r27xo4dq7Fjx6pr164aNGiQmjRpotzcXC1evFj//Oc/5fP5ovr6OnbsqOOOO05PPPGESkpK1L59e61Zs0YvvPCCOnfurMWLF1f5vGHDhun666+XJD388MMymyt+z6NZs2b629/+puuvv14dO3bUVVddpRYtWmjnzp1avny5/v73v2vFihVq2bJlVF8fAAAAgMq4zuE6BwDqGgIjAECtMHr0aCUmJurpp5/Wfffdp5ycHN15551yuVy67rrrKmzbpk0bff311xo9erSee+45ORwO9evXT3PmzFFWVlalfT/88MPq1q2bpkyZosmTJ6u4uFiZmZnq1KmTpkyZEvXXZrFY9PHHH+vOO+/U7NmzVVxcrE6dOmn27Nn66aef9nshdemll+qOO+6Q2+3WtddeW+U21157rdq1a6ennnpKL7zwgvLz85Wenq727dtrwoQJys7OjuZLAwAAAHAAXOdUxnUOANReJiOWM/QBAID98nq9aty4sbp3765PP/001uUAAAAAwBHjOgcAai/mMAIAoJZ67bXXtGfPHg0bNizWpQAAAABAjeA6BwBqLzqMAACoZT788ENt2LBBY8eOVVZWlpYtWxaZUBYAAAAA6iKucwCg9iMwAgCglmnZsqW2bt2qk046SS+99JKOO+64WJcEAAAAAEeE6xwAqP0IjAAAAAAAAAAAABo45jACAAAAAAAAAABo4AiMAAAAAAAAAAAAGjgCIwAAAAAAAAAAgAaOwAgAAAAAAAAAAKCBIzACAAAAAAAAAABo4AiMAAAAAAAAAAAAGjgCIwAAAAAAAAAAgAaOwAgAAAAAAAAAAKCBIzACAAAAAAAAAABo4AiMAAAAAAAAAAAAGjgCIwAAAAAAAAAAgAbu/wFZtdEGvyznaQAAAABJRU5ErkJggg==", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "primary = layer_summary.loc[[\"A-K96\", \"B-K96\", \"C-K96-G32\"]].reset_index()\n", + "fig, axes = plt.subplots(1, 2, figsize=(13, 4.5))\n", + "for method, group in primary.groupby(\"method\", sort=False):\n", + " axes[0].plot(group[\"layer\"], group[\"mean_tanh_prime_selected_rho\"], marker=\"o\", label=f\"{method}: approximation rho\")\n", + " axes[0].plot(group[\"layer\"], group[\"mean_quadratic_core_box_radius\"], marker=\"x\", linestyle=\"--\", label=f\"{method}: boxed reduction radius\")\n", + " axes[1].plot(group[\"layer\"], group[\"mean_propagated_J_remainder\"], marker=\"o\", label=method)\n", + "axes[0].set_xlabel(\"hidden layer\")\n", + "axes[0].set_ylabel(\"mean radius\")\n", + "axes[0].set_xticks([1, 2, 3])\n", + "axes[0].legend(fontsize=7)\n", + "axes[0].grid(alpha=0.25)\n", + "axes[1].set_xlabel(\"hidden layer\")\n", + "axes[1].set_ylabel(\"mean propagated Jacobian remainder\")\n", + "axes[1].set_xticks([1, 2, 3])\n", + "axes[1].legend()\n", + "axes[1].grid(alpha=0.25)\n", + "fig.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "80b06901", + "metadata": {}, + "source": [ + "## Programmatic conclusion\n", + "\n", + "The following cells rank the variants by final normalized $W^{1,2}$ width\n", + "and report how much B and C improve over A at the same retained support." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "c928e1c0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " W12_normalized_lower W12_normalized_upper \\\n", + "method \n", + "affine-A-K96 0.0 85.167723 \n", + "interval 0.0 88.846805 \n", + "C-K192-G32 0.0 462.682211 \n", + "B-K192 0.0 465.534292 \n", + "C-K96-G32 0.0 471.174670 \n", + "A-K192 0.0 471.885901 \n", + "C-K96-G8 0.0 473.419414 \n", + "B-K96 0.0 474.166433 \n", + "C-K96-G0 0.0 474.166433 \n", + "A-K96 0.0 480.616605 \n", + "\n", + " W12_normalized_width W12_relative_width L2_normalized_width \\\n", + "method \n", + "affine-A-K96 85.167723 1.0 2.676831 \n", + "interval 88.846805 1.0 7.979523 \n", + "C-K192-G32 462.682211 1.0 2.676831 \n", + "B-K192 465.534292 1.0 2.676831 \n", + "C-K96-G32 471.174670 1.0 2.676831 \n", + "A-K192 471.885901 1.0 2.676831 \n", + "C-K96-G8 473.419414 1.0 2.676831 \n", + "B-K96 474.166433 1.0 2.676831 \n", + "C-K96-G0 474.166433 1.0 2.676831 \n", + "A-K96 480.616605 1.0 2.676831 \n", + "\n", + " J_mean_width total_s \n", + "method \n", + "affine-A-K96 16.809482 2.794268 \n", + "interval 17.548815 0.214725 \n", + "C-K192-G32 91.812086 31.528201 \n", + "B-K192 92.380192 16.254778 \n", + "C-K96-G32 93.503561 14.119499 \n", + "A-K192 93.641813 21.279968 \n", + "C-K96-G8 93.950519 10.287620 \n", + "B-K96 94.099304 8.400954 \n", + "C-K96-G0 94.099304 10.784935 \n", + "A-K96 95.380557 10.921846 " + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + " W12_width_change_percent runtime_change_percent\n", + "comparison \n", + "B-K96 versus A-K96 -1.342062 -23.081191\n", + "C-K96-G32 versus B-K96 -0.630952 68.070199\n", + "B-K192 versus B-K96 -1.820488 93.487296" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ranking = benchmark_table.sort_values(\"W12_normalized_width\")[[\n", + " \"W12_normalized_lower\", \"W12_normalized_upper\", \"W12_normalized_width\",\n", + " \"W12_relative_width\", \"L2_normalized_width\", \"J_mean_width\", \"total_s\",\n", + "]]\n", + "display(ranking)\n", + "\n", + "comparison = pd.DataFrame([\n", + " {\n", + " \"comparison\": \"B-K96 versus A-K96\",\n", + " \"W12_width_change_percent\": 100 * (benchmark_table.loc[\"B-K96\", \"W12_normalized_width\"] / benchmark_table.loc[\"A-K96\", \"W12_normalized_width\"] - 1),\n", + " \"runtime_change_percent\": 100 * (benchmark_table.loc[\"B-K96\", \"total_s\"] / benchmark_table.loc[\"A-K96\", \"total_s\"] - 1),\n", + " },\n", + " {\n", + " \"comparison\": \"C-K96-G32 versus B-K96\",\n", + " \"W12_width_change_percent\": 100 * (benchmark_table.loc[\"C-K96-G32\", \"W12_normalized_width\"] / benchmark_table.loc[\"B-K96\", \"W12_normalized_width\"] - 1),\n", + " \"runtime_change_percent\": 100 * (benchmark_table.loc[\"C-K96-G32\", \"total_s\"] / benchmark_table.loc[\"B-K96\", \"total_s\"] - 1),\n", + " },\n", + " {\n", + " \"comparison\": \"B-K192 versus B-K96\",\n", + " \"W12_width_change_percent\": 100 * (benchmark_table.loc[\"B-K192\", \"W12_normalized_width\"] / benchmark_table.loc[\"B-K96\", \"W12_normalized_width\"] - 1),\n", + " \"runtime_change_percent\": 100 * (benchmark_table.loc[\"B-K192\", \"total_s\"] / benchmark_table.loc[\"B-K96\", \"total_s\"] - 1),\n", + " },\n", + "]).set_index(\"comparison\")\n", + "display(comparison)" + ] + }, + { + "cell_type": "markdown", + "id": "eea2d501", + "metadata": {}, + "source": [ + "## Interpretation of the executed 100D experiment\n", + "\n", + "The three new reduction operators behave correctly and improve monotonically in the expected local sense, but they do **not** rescue the expanded quadratic one-jet on this network. Variant B is the correct cheap default: at Top-96 it lowers the normalized $W^{1,2}$ width from $480.6166$ (A) to $474.1664$, a $1.34\\%$ improvement, and its parity test has negligible cost. C with 32 retained directions lowers this further to $471.1747$, but its runtime is about $63\\%$ above B in this run. Increasing B's polynomial budget from 96 to 192 terms reaches $465.5343$, at roughly $2.49\\times$ the runtime. The best tested quadratic configuration is C-K192-G32 at $462.6822$.\n", + "\n", + "These gains are real, but the affine derivative control remains far tighter: affine-A-K96 has normalized $W^{1,2}$ width $85.1677$, and interval arithmetic has $88.8468$. The quadratic approximation radii themselves fall from layerwise affine means $(0.2751,0.3059,0.3594)$ to $(0.0385,0.0520,0.0821)$, yet the mean quadratic-core box radii under B-K96 are $(0.5412,0.6096,0.7139)$. Thus the new reductions recover only a small part of the dependence lost when the expanded square support is collapsed.\n", + "\n", + "The small-network reference is important: there, B improves the mean Jacobian width from $0.07346$ to $0.06827$, and C with eight retained directions reaches $0.06458$, close to the unreduced value $0.06258$, with zero violations among the sampled Jacobians. The variants are therefore working as designed; the remaining failure on the 100D PINN is a scale/representation issue, not a failed quadratic certificate. A grouped or factored representation of $aZ^2+bZ+c$ is still the most promising next step.\n" + ] + } + ], + "metadata": { + "jupytext": { + "cell_metadata_filter": "-all", + "main_language": "python", + "notebook_metadata_filter": "-all" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/checkpoints/pinn_100d_poisson.pt b/notebooks/checkpoints/pinn_100d_poisson.pt new file mode 100644 index 0000000..b4bbf34 Binary files /dev/null and b/notebooks/checkpoints/pinn_100d_poisson.pt differ diff --git a/notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt b/notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt new file mode 100644 index 0000000..0108ff3 Binary files /dev/null and b/notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt differ diff --git a/notebooks/neural_set_propagation_reproducible.ipynb b/notebooks/neural_set_propagation_reproducible.ipynb new file mode 100644 index 0000000..de461d1 --- /dev/null +++ b/notebooks/neural_set_propagation_reproducible.ipynb @@ -0,0 +1,550 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f909dcf3", + "metadata": {}, + "source": [ + "# Reproducing the neural set-propagation experiment\n", + "\n", + "This notebook reproduces computations for a fixed random feedforward network acting on the input set\n", + "\\[\n", + "X_0=[0,1]^2 \\subset \\mathbb{R}^2.\n", + "\\]\n", + "\n", + "The network has:\n", + "- input dimension $2$,\n", + "- hidden width $30$,\n", + "- output dimension $2$,\n", + "- five affine layers in total (linear maps plus bias),\n", + "- componentwise $\\tanh$ activations after every affine layer except the last.\n", + "\n", + "So the map is\n", + "\\[\n", + "F = T_5 \\circ \\tanh \\circ T_4 \\circ \\tanh \\circ T_3 \\circ \\tanh \\circ T_2 \\circ \\tanh \\circ T_1.\n", + "\\]\n", + "\n", + "This notebook generates interval-arithmetic (IA) figures:\n", + "1. the input square and a dense-sampling approximation of the image $F(X_0)$,\n", + "2. the interval-arithmetic enclosure overlaid with the sampled image,\n", + "3. the corresponding intervalNets interval propagation enclosure.\n", + "\n", + "This notebook now documents interval-arithmetic propagation only. The word \"affine\" above refers only to neural-network linear layers plus bias." + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "id": "d087f495", + "metadata": {}, + "source": [ + "## Imports and configuration\n", + "\n", + "We use the same fixed random seed and network-generation procedure as in the previous computation.\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "71a7a84d", + "metadata": {}, + "outputs": [], + "source": [ + "import math\n", + "from pathlib import Path\n", + "\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "\n", + "# Fixed random seed and network width\n", + "seed = 3\n", + "width = 30\n", + "rng = np.random.default_rng(seed)\n", + "\n", + "# Directory for saved figures\n", + "outdir = Path(\"neural_set_propagation_outputs\")\n", + "outdir.mkdir(exist_ok=True)\n", + "\n", + "print(f\"seed = {seed}\")\n", + "print(f\"hidden width = {width}\")\n", + "print(f\"figure output directory = {outdir.resolve()}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "bc61ad90", + "metadata": {}, + "source": [ + "## Build the random network\n", + "\n", + "The dimensions are\n", + "\\[\n", + "2 \\to 30 \\to 30 \\to 30 \\to 30 \\to 2.\n", + "\\]\n", + "\n", + "Each affine layer has the form $T_i(x)=W_i x + b_i$. \n", + "The same random network is used in all four plots.\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04feb7f0", + "metadata": {}, + "outputs": [], + "source": [ + "dims = [2, width, width, width, width, 2]\n", + "\n", + "Ws = []\n", + "bs = []\n", + "for din, dout in zip(dims[:-1], dims[1:]):\n", + " scale = 0.7 / math.sqrt(din)\n", + " W = rng.normal(0.0, scale, size=(dout, din))\n", + " b = rng.normal(0.0, 0.15, size=(dout,))\n", + " Ws.append(W)\n", + " bs.append(b)\n", + "\n", + "print(\"Network architecture:\")\n", + "for i, (W, b) in enumerate(zip(Ws, bs), start=1):\n", + " print(f\"T{i}: W shape = {W.shape}, b shape = {b.shape}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "2cbcec2b", + "metadata": {}, + "source": [ + "## Forward map on point samples\n", + "\n", + "This is the standard pointwise forward evaluation of the network. \n", + "We use it to generate a dense-sampling approximation of the image $F(X_0)$.\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e2c3c51", + "metadata": {}, + "outputs": [], + "source": [ + "def forward_points(X: np.ndarray) -> np.ndarray:\n", + " \"\"\"Forward evaluation on a batch of input points of shape (N, 2).\"\"\"\n", + " A = X\n", + " for i, (W, b) in enumerate(zip(Ws, bs)):\n", + " A = A @ W.T + b\n", + " if i < len(Ws) - 1:\n", + " A = np.tanh(A)\n", + " return A\n" + ] + }, + { + "cell_type": "markdown", + "id": "2c539373", + "metadata": {}, + "source": [ + "## Interval arithmetic (IA) propagation\n", + "\n", + "For an interval box $[\\ell, u]$, an affine map is propagated by splitting each matrix into positive and negative parts:\n", + "\\[\n", + "W = W_+ + W_-,\n", + "\\qquad\n", + "W_+ = \\max(W,0),\\quad W_- = \\min(W,0).\n", + "\\]\n", + "Then\n", + "\\[\n", + "[\\ell',u'] = [W_+\\ell + W_- u + b,\\; W_+ u + W_- \\ell + b].\n", + "\\]\n", + "\n", + "Since $\\tanh$ is monotone increasing, it is applied coordinatewise to the endpoints.\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c34e6e37", + "metadata": {}, + "outputs": [], + "source": [ + "def ia_propagate(lo: np.ndarray, hi: np.ndarray) -> tuple[np.ndarray, np.ndarray]:\n", + " \"\"\"Propagate an interval box through the network using interval arithmetic.\"\"\"\n", + " for i, (W, b) in enumerate(zip(Ws, bs)):\n", + " W_pos = np.maximum(W, 0.0)\n", + " W_neg = np.minimum(W, 0.0)\n", + "\n", + " new_lo = W_pos @ lo + W_neg @ hi + b\n", + " new_hi = W_pos @ hi + W_neg @ lo + b\n", + " lo, hi = new_lo, new_hi\n", + "\n", + " if i < len(Ws) - 1:\n", + " lo = np.tanh(lo)\n", + " hi = np.tanh(hi)\n", + "\n", + " return lo, hi\n" + ] + }, + { + "cell_type": "markdown", + "id": "67a8f2b6", + "metadata": {}, + "source": [ + "## Approximate the exact image by dense sampling\n", + "\n", + "We sample the input square $X_0=[0,1]^2$ on a dense $350 \\times 350$ grid and push those points through the network. \n", + "This gives a high-resolution approximation of the true image $F(X_0)$.\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96b2b2b1", + "metadata": {}, + "outputs": [], + "source": [ + "# Dense grid on the input square\n", + "n_grid = 350\n", + "xs = np.linspace(0.0, 1.0, n_grid)\n", + "ys = np.linspace(0.0, 1.0, n_grid)\n", + "X0_grid = np.stack(np.meshgrid(xs, ys), axis=-1).reshape(-1, 2)\n", + "\n", + "# Approximate exact image\n", + "Y_exact = forward_points(X0_grid)\n", + "\n", + "# Input square polygon\n", + "X0_square = np.array(\n", + " [\n", + " [0.0, 0.0],\n", + " [1.0, 0.0],\n", + " [1.0, 1.0],\n", + " [0.0, 1.0],\n", + " ],\n", + " dtype=float,\n", + ")\n", + "\n", + "print(\"Approximate exact image bounds from dense sampling:\")\n", + "print(f\"x in [{Y_exact[:,0].min():.6f}, {Y_exact[:,0].max():.6f}]\")\n", + "print(f\"y in [{Y_exact[:,1].min():.6f}, {Y_exact[:,1].max():.6f}]\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "e90bc84e", + "metadata": {}, + "source": [ + "## Compute the IA propagated set" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76aa72bd", + "metadata": {}, + "outputs": [], + "source": [ + "# IA output box\n", + "ia_lo, ia_hi = ia_propagate(np.array([0.0, 0.0]), np.array([1.0, 1.0]))\n", + "IA_box = np.array(\n", + " [\n", + " [ia_lo[0], ia_lo[1]],\n", + " [ia_hi[0], ia_lo[1]],\n", + " [ia_hi[0], ia_hi[1]],\n", + " [ia_lo[0], ia_hi[1]],\n", + " ],\n", + " dtype=float,\n", + ")\n", + "\n", + "print(\"IA output box:\")\n", + "print(f\"x in [{ia_lo[0]:.6f}, {ia_hi[0]:.6f}]\")\n", + "print(f\"y in [{ia_lo[1]:.6f}, {ia_hi[1]:.6f}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "32f2dcfd", + "metadata": {}, + "source": [ + "## Plot 1: input square and approximate exact image" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b58dbba1", + "metadata": {}, + "outputs": [], + "source": [ + "fig1, ax1 = plt.subplots(figsize=(7, 7))\n", + "\n", + "sq_closed = close_poly(X0_square)\n", + "ax1.fill(X0_square[:, 0], X0_square[:, 1], alpha=0.20, label=r\"input $X_0=[0,1]^2$\")\n", + "ax1.plot(sq_closed[:, 0], sq_closed[:, 1])\n", + "\n", + "ax1.scatter(\n", + " Y_exact[:, 0],\n", + " Y_exact[:, 1],\n", + " s=1,\n", + " alpha=0.25,\n", + " label=r\"approximate exact image $F(X_0)$\",\n", + ")\n", + "\n", + "xlim1, ylim1 = combined_limits([X0_square, Y_exact])\n", + "style_axes(ax1, xlim1, ylim1, \"Input square and approximate exact image of the random network\")\n", + "ax1.legend()\n", + "\n", + "path1 = outdir / \"plot_1_exact_image.png\"\n", + "fig1.savefig(path1, dpi=220, bbox_inches=\"tight\")\n", + "plt.show()\n", + "\n", + "print(path1.resolve())\n" + ] + }, + { + "cell_type": "markdown", + "id": "78ae4330", + "metadata": {}, + "source": [ + "## Plot 2: IA enclosure and approximate exact image" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb10141e", + "metadata": {}, + "outputs": [], + "source": [ + "fig2, ax2 = plt.subplots(figsize=(7, 7))\n", + "\n", + "ia_closed = close_poly(IA_box)\n", + "ax2.fill(IA_box[:, 0], IA_box[:, 1], alpha=0.25, label=\"IA enclosure\")\n", + "ax2.plot(ia_closed[:, 0], ia_closed[:, 1])\n", + "\n", + "ax2.scatter(\n", + " Y_exact[:, 0],\n", + " Y_exact[:, 1],\n", + " s=1,\n", + " alpha=0.25,\n", + " label=r\"approximate exact image $F(X_0)$\",\n", + ")\n", + "\n", + "xlim2, ylim2 = combined_limits([IA_box, Y_exact])\n", + "style_axes(ax2, xlim2, ylim2, \"Interval arithmetic propagation vs. approximate exact image\")\n", + "ax2.legend()\n", + "\n", + "path2 = outdir / \"plot_2_ia.png\"\n", + "fig2.savefig(path2, dpi=220, bbox_inches=\"tight\")\n", + "plt.show()\n", + "\n", + "print(path2.resolve())\n" + ] + }, + { + "cell_type": "markdown", + "id": "999f973f", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "Running this notebook from top to bottom reproduces:\n", + "- the same fixed random network,\n", + "- the same dense-sampling approximation of the image of $X_0=[0,1]^2$,\n", + "- the same IA propagation,\n", + "- and the same intervalNets IA comparison figures." + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17f5e529-7ab6-4ab4-9084-f22dd9719315", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Repeat the enclosure with intervalNets IA code\n", + "\n", + "This section reproduces the same interval enclosure for the **same random network** using `intervalnets` (`IntervalTensor`)." + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import torch\n", + "from torch import nn\n", + "\n", + "from intervalnets import IntervalTensor, interval_forward" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Build the identical network in torch using the same sampled Ws, bs\n", + "torch.set_default_dtype(torch.float64)\n", + "\n", + "layers = []\n", + "for i, (W_np, b_np) in enumerate(zip(Ws, bs)):\n", + " W_t = torch.tensor(W_np, dtype=torch.float64)\n", + " b_t = torch.tensor(b_np, dtype=torch.float64)\n", + " lin = nn.Linear(W_t.shape[1], W_t.shape[0], bias=True, dtype=torch.float64)\n", + " with torch.no_grad():\n", + " lin.weight.copy_(W_t)\n", + " lin.bias.copy_(b_t)\n", + " layers.append(lin)\n", + " if i < len(Ws) - 1:\n", + " layers.append(nn.Tanh())\n", + "\n", + "model_torch = nn.Sequential(*layers).eval()\n", + "\n", + "# intervalNets IA propagation on X0 = [0,1]^2\n", + "X0_interval = IntervalTensor.from_bounds([0.0, 0.0], [1.0, 1.0])\n", + "Y_ia_intervalnets = interval_forward(model_torch, X0_interval, enclosure_mode=\"box\")\n", + "ia2_lo = np.array(Y_ia_intervalnets.lower, dtype=float)\n", + "ia2_hi = np.array(Y_ia_intervalnets.upper, dtype=float)\n", + "\n", + "IA2_box = np.array(\n", + " [\n", + " [ia2_lo[0], ia2_lo[1]],\n", + " [ia2_hi[0], ia2_lo[1]],\n", + " [ia2_hi[0], ia2_hi[1]],\n", + " [ia2_lo[0], ia2_hi[1]],\n", + " ],\n", + " dtype=float,\n", + ")\n", + "\n", + "print(\"intervalNets IA output box:\")\n", + "print(f\"x in [{ia2_lo[0]:.6f}, {ia2_hi[0]:.6f}]\")\n", + "print(f\"y in [{ia2_lo[1]:.6f}, {ia2_hi[1]:.6f}]\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Plot 5: (intervalNets) input square and approximate exact image\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "fig5, ax5 = plt.subplots(figsize=(7, 7))\n", + "\n", + "sq_closed = close_poly(X0_square)\n", + "ax5.fill(X0_square[:, 0], X0_square[:, 1], alpha=0.20, label=r\"input $X_0=[0,1]^2$\")\n", + "ax5.plot(sq_closed[:, 0], sq_closed[:, 1])\n", + "\n", + "ax5.scatter(\n", + " Y_exact[:, 0],\n", + " Y_exact[:, 1],\n", + " s=1,\n", + " alpha=0.25,\n", + " label=r\"approximate exact image $F(X_0)$\",\n", + ")\n", + "\n", + "xlim5, ylim5 = combined_limits([X0_square, Y_exact])\n", + "style_axes(ax5, xlim5, ylim5, \"(intervalNets run) Input square and approximate exact image\")\n", + "ax5.legend()\n", + "\n", + "path5 = outdir / \"plot_5_intervalnets_exact_image.png\"\n", + "fig5.savefig(path5, dpi=220, bbox_inches=\"tight\")\n", + "plt.show()\n", + "\n", + "print(path5.resolve())\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Plot 6: (intervalNets IA) enclosure and approximate exact image\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "fig6, ax6 = plt.subplots(figsize=(7, 7))\n", + "\n", + "ia2_closed = close_poly(IA2_box)\n", + "ax6.fill(IA2_box[:, 0], IA2_box[:, 1], alpha=0.25, label=\"intervalNets IA enclosure\")\n", + "ax6.plot(ia2_closed[:, 0], ia2_closed[:, 1])\n", + "\n", + "ax6.scatter(\n", + " Y_exact[:, 0],\n", + " Y_exact[:, 1],\n", + " s=1,\n", + " alpha=0.25,\n", + " label=r\"approximate exact image $F(X_0)$\",\n", + ")\n", + "\n", + "xlim6, ylim6 = combined_limits([IA2_box, Y_exact])\n", + "style_axes(ax6, xlim6, ylim6, \"intervalNets interval propagation vs. approximate exact image\")\n", + "ax6.legend()\n", + "\n", + "path6 = outdir / \"plot_6_intervalnets_ia.png\"\n", + "fig6.savefig(path6, dpi=220, bbox_inches=\"tight\")\n", + "plt.show()\n", + "\n", + "print(path6.resolve())\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/notebooks/notebooks/artifacts/figure_pinn_results.png b/notebooks/notebooks/artifacts/figure_pinn_results.png new file mode 100644 index 0000000..40bdbf7 Binary files /dev/null and b/notebooks/notebooks/artifacts/figure_pinn_results.png differ diff --git a/notebooks/pinn_100d_poisson_deep_hybrid_benchmark.py b/notebooks/pinn_100d_poisson_deep_hybrid_benchmark.py new file mode 100644 index 0000000..c3af19c --- /dev/null +++ b/notebooks/pinn_100d_poisson_deep_hybrid_benchmark.py @@ -0,0 +1,159 @@ +"""Benchmark the fast factored deep hybrid certificate on the saved 100D PINN.""" + +from __future__ import annotations + +from math import sqrt +from pathlib import Path +from statistics import median +from time import perf_counter + +import torch + +from intervalnets import ( + IntervalTensor, + PZIntegrationCell, + DeepHybridOneJetResult, + integrate_deep_hybrid_onejet_squared, + load_tanh_mlp_checkpoint, + scalar_hybrid_onejet_reverse, +) + + +SEED = 20260806 +REPEATS = 7 +VALIDATION_SAMPLES = 16_384 + + +def main() -> None: + torch.set_num_threads(1) + torch.manual_seed(SEED) + root = Path(__file__).resolve().parents[1] + checkpoint = root / "notebooks" / "checkpoints" / "pinn_100d_poisson.pt" + model = load_tanh_mlp_checkpoint(checkpoint).double().eval() + box = IntervalTensor.from_bounds([-0.1] * 100, [0.1] * 100) + cell = PZIntegrationCell.from_affine_box(box) + + # One warm run initializes the activation-certificate and linear-algebra paths. + warm = scalar_hybrid_onejet_reverse(model, cell.domain) + assert isinstance(warm, DeepHybridOneJetResult) + integrate_deep_hybrid_onejet_squared(warm, cell) + + construction_times: list[float] = [] + integration_times: list[float] = [] + total_times: list[float] = [] + result = warm + squared = None + for _ in range(REPEATS): + start = perf_counter() + result = scalar_hybrid_onejet_reverse(model, cell.domain) + constructed = perf_counter() + squared = integrate_deep_hybrid_onejet_squared(result, cell) + finished = perf_counter() + construction_times.append(constructed - start) + integration_times.append(finished - constructed) + total_times.append(finished - start) + assert squared is not None + + generator = torch.Generator().manual_seed(SEED + 222) + samples = -0.1 + 0.2 * torch.rand( + (VALIDATION_SAMPLES, 100), generator=generator, dtype=torch.float64 + ) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + hidden = samples.detach() + value_noise_blocks = [hidden / 0.1] + derivative_noise_blocks = [] + hidden_linears = list(model.children())[0:-1:2] + for layer, factor in zip(hidden_linears, result.factors): + xi = torch.cat(value_noise_blocks, dim=1) + preactivation = layer(hidden) + polynomial_preactivation = ( + factor.preactivation_center.unsqueeze(0) + + xi @ factor.preactivation_coefficients.T + ) + if not torch.allclose( + preactivation, polynomial_preactivation, rtol=2e-11, atol=2e-11 + ): + raise AssertionError("Cached preactivation PZ does not realize the sample.") + hidden = torch.tanh(preactivation) + value_core = ( + factor.value_intercepts.unsqueeze(0) + + factor.value_slopes.unsqueeze(0) * preactivation + ) + value_eta = torch.where( + factor.value_approximation_radii.unsqueeze(0) == 0.0, + torch.zeros_like(hidden), + (hidden - value_core) / factor.value_approximation_radii.unsqueeze(0), + ) + affine_argument = xi @ factor.preactivation_coefficients.T + derivative_core = ( + factor.center.unsqueeze(0) + + xi @ factor.linear_coefficients.T + + factor.quadratic_coefficients.unsqueeze(0) * affine_argument.square() + ) + exact_derivative = 1.0 - hidden.square() + derivative_eta = torch.where( + factor.approximation_radii.unsqueeze(0) == 0.0, + torch.zeros_like(hidden), + (exact_derivative - derivative_core) + / factor.approximation_radii.unsqueeze(0), + ) + value_noise_blocks.append(value_eta) + derivative_noise_blocks.append(derivative_eta) + realizing_noise = torch.cat((*value_noise_blocks, *derivative_noise_blocks), dim=1) + factored_gradients = result.jacobian.evaluate(realizing_noise) + factor_gradient_max_error = float( + torch.max(torch.abs(factored_gradients - gradients)).detach().item() + ) + max_realizing_noise = float(torch.max(torch.abs(realizing_noise)).detach().item()) + value_enclosure = result.value.interval_enclosure() + jacobian_enclosure = result.jacobian.interval_enclosure() + value_lower = torch.as_tensor(value_enclosure.lower).reshape(1, 1) + value_upper = torch.as_tensor(value_enclosure.upper).reshape(1, 1) + jacobian_lower = torch.as_tensor(jacobian_enclosure.lower).reshape(1, 100) + jacobian_upper = torch.as_tensor(jacobian_enclosure.upper).reshape(1, 100) + value_violations = int(torch.count_nonzero((values < value_lower) | (values > value_upper))) + jacobian_violations = int( + torch.count_nonzero((gradients < jacobian_lower) | (gradients > jacobian_upper)) + ) + sampled_w12 = float( + (values.square().squeeze(1) + gradients.square().sum(1)).mean().detach() + ) + volume = float(cell.volume) + normalized_squared_lower = float(squared.lower) / volume + normalized_squared_upper = float(squared.upper) / volume + + print( + { + "architecture": [100, 50, 50, 50, 1], + "checkpoint": str(checkpoint.relative_to(root)), + "representation": "uncompressed_factored_reverse_hybrid", + "hidden_layers": len(result.factors), + "quadratic_neurons_per_layer": [ + int(torch.count_nonzero(factor.approximation_degrees == 2)) + for factor in result.factors + ], + "value_noise_symbols": result.jacobian.num_value_noise - 100, + "derivative_noise_symbols": result.jacobian.num_derivative_noise, + "factored_jacobian_degree": result.jacobian.max_degree, + "gradient_spectral_bound": result.gradient_spectral_bound, + "normalized_W12_squared_lower": normalized_squared_lower, + "normalized_W12_squared_upper": normalized_squared_upper, + "normalized_W12_lower": sqrt(max(0.0, normalized_squared_lower)), + "normalized_W12_upper": sqrt(max(0.0, normalized_squared_upper)), + "sampled_normalized_W12": sqrt(sampled_w12), + "construction_seconds_median": median(construction_times), + "integration_seconds_median": median(integration_times), + "total_seconds_median": median(total_times), + "total_seconds_range": [min(total_times), max(total_times)], + "value_soundness_violations": value_violations, + "jacobian_soundness_violations": jacobian_violations, + "factor_gradient_max_error": factor_gradient_max_error, + "max_realizing_noise_magnitude": max_realizing_noise, + } + ) + + +if __name__ == "__main__": + main() diff --git a/notebooks/pinn_100d_poisson_graph_hilbert_benchmark.py b/notebooks/pinn_100d_poisson_graph_hilbert_benchmark.py new file mode 100644 index 0000000..a4ecf1c --- /dev/null +++ b/notebooks/pinn_100d_poisson_graph_hilbert_benchmark.py @@ -0,0 +1,184 @@ +"""Benchmark dependency-preserving graph/Hilbert norm certificates.""" + +from __future__ import annotations + +from dataclasses import asdict +import json +from math import sqrt +from pathlib import Path +from time import perf_counter + +import torch + +from intervalnets import ( + IntervalTensor, + PZIntegrationCell, + certify_hybrid_graph_norms, + load_tanh_mlp_checkpoint, + scalar_hybrid_onejet_reverse, +) + + +SEED = 20260806 +VALIDATION_SAMPLES = 16_384 +POLYNOMIAL_DEGREE = 5 +RESIDUAL_SUBDIVISIONS = 2048 +DERIVATIVE_CERTIFICATE_SUBDIVISIONS = 64 + + +def _architecture(model: torch.nn.Sequential) -> list[int]: + linears = [layer for layer in model if isinstance(layer, torch.nn.Linear)] + return [linears[0].in_features, *(layer.out_features for layer in linears)] + + +def _norm_record(squared, volume: float, sampled: float) -> dict[str, float]: + normalized_squared_lower = max(0.0, float(squared.lower) / volume) + normalized_squared_upper = max(0.0, float(squared.upper) / volume) + lower = sqrt(normalized_squared_lower) + upper = sqrt(normalized_squared_upper) + width = upper - lower + return { + "domain_volume_normalized_lower": lower, + "domain_volume_normalized_upper": upper, + "domain_volume_normalized_width": width, + "relative_norm_width": width / upper if upper > 0.0 else 0.0, + "sampled_domain_volume_normalized": sampled, + "upper_over_sampled": upper / sampled, + "lower_over_sampled": lower / sampled, + "raw_squared_lower": float(squared.lower), + "raw_squared_upper": float(squared.upper), + } + + +def _benchmark(checkpoint: Path) -> dict[str, object]: + model = load_tanh_mlp_checkpoint(checkpoint).double().eval() + cell = PZIntegrationCell.from_affine_box( + IntervalTensor.from_bounds([-0.1] * 100, [0.1] * 100) + ) + construction_start = perf_counter() + result = scalar_hybrid_onejet_reverse(model, cell.domain) + construction_seconds = perf_counter() - construction_start + + certificate_start = perf_counter() + certificate = certify_hybrid_graph_norms( + model, + result, + cell, + polynomial_degree=POLYNOMIAL_DEGREE, + residual_subdivisions=RESIDUAL_SUBDIVISIONS, + derivative_certificate_subdivisions=DERIVATIVE_CERTIFICATE_SUBDIVISIONS, + ) + certificate_seconds = perf_counter() - certificate_start + + generator = torch.Generator().manual_seed(SEED + 222) + samples = -0.1 + 0.2 * torch.rand( + (VALIDATION_SAMPLES, 100), generator=generator, dtype=torch.float64 + ) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + sampled_l2 = sqrt(float(values.square().mean().detach())) + sampled_w12 = sqrt( + float( + ( + values.square().squeeze(1) + gradients.square().sum(dim=1) + ).mean().detach() + ) + ) + volume = float(cell.volume) + l2 = _norm_record(certificate.l2_squared, volume, sampled_l2) + w12 = _norm_record(certificate.w12_squared, volume, sampled_w12) + old_l2 = _norm_record(certificate.previous_l2_squared, volume, sampled_l2) + old_w12 = _norm_record(certificate.previous_w12_squared, volume, sampled_w12) + l2_contained = ( + l2["domain_volume_normalized_lower"] + <= sampled_l2 + <= l2["domain_volume_normalized_upper"] + ) + w12_contained = ( + w12["domain_volume_normalized_lower"] + <= sampled_w12 + <= w12["domain_volume_normalized_upper"] + ) + assert l2_contained and w12_contained + + return { + "checkpoint": str(checkpoint.relative_to(checkpoint.parents[2])), + "architecture": _architecture(model), + "physical_domain_volume": volume, + "method": "dependency_preserving_graph_hilbert", + "settings": { + "polynomial_degree": POLYNOMIAL_DEGREE, + "residual_subdivisions": RESIDUAL_SUBDIVISIONS, + "derivative_certificate_subdivisions": DERIVATIVE_CERTIFICATE_SUBDIVISIONS, + "validation_samples": VALIDATION_SAMPLES, + }, + "l2": l2, + "w12": w12, + "sample_containment_diagnostic": { + "l2": l2_contained, + "w12": w12_contained, + }, + "previous_absolute_moment_parity": {"l2": old_l2, "w12": old_w12}, + "improvement": { + "l2_relative_width_reduction": ( + old_l2["relative_norm_width"] - l2["relative_norm_width"] + ), + "w12_relative_width_reduction": ( + old_w12["relative_norm_width"] - w12["relative_norm_width"] + ), + "l2_upper_reduction_fraction": ( + old_l2["domain_volume_normalized_upper"] + - l2["domain_volume_normalized_upper"] + ) + / old_l2["domain_volume_normalized_upper"], + "w12_upper_reduction_fraction": ( + old_w12["domain_volume_normalized_upper"] + - w12["domain_volume_normalized_upper"] + ) + / old_w12["domain_volume_normalized_upper"], + }, + "decomposition": certificate.normalized_diagnostics, + "value_layers": [asdict(layer) for layer in certificate.value.layers], + "gradient": { + "factor_remainders": list(certificate.gradient.factor_remainders), + "product_projection_remainders": list( + certificate.gradient.product_projection_remainders + ), + "moment_states": certificate.gradient.moment_states, + }, + "w12_witness": asdict(certificate.w12_witness), + "complexity": { + "value_moment_states": certificate.value.moment_states, + "gradient_moment_states": certificate.gradient.moment_states, + }, + "timings_seconds": { + "hybrid_onejet_construction": construction_seconds, + "graph_hilbert_certification": certificate_seconds, + "total_certification": construction_seconds + certificate_seconds, + }, + } + + +def main() -> None: + torch.set_num_threads(1) + torch.manual_seed(SEED) + root = Path(__file__).resolve().parents[1] + checkpoints = [ + root / "notebooks" / "checkpoints" / "pinn_100d_poisson_shallow_300.pt", + root / "notebooks" / "checkpoints" / "pinn_100d_poisson.pt", + ] + records = [_benchmark(path) for path in checkpoints] + output = ( + root + / "notebooks" + / "benchmark_outputs" + / "hybrid_graph_hilbert_norms.json" + ) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(records, indent=2) + "\n", encoding="utf-8") + print(json.dumps(records, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/notebooks/pinn_100d_poisson_hybrid_norm_benchmark.py b/notebooks/pinn_100d_poisson_hybrid_norm_benchmark.py new file mode 100644 index 0000000..aa9f843 --- /dev/null +++ b/notebooks/pinn_100d_poisson_hybrid_norm_benchmark.py @@ -0,0 +1,173 @@ +"""Benchmark refined hybrid L2/W12 certificates on both saved 100D PINNs.""" + +from __future__ import annotations + +import json +from math import sqrt +from pathlib import Path +from statistics import median +from time import perf_counter + +import torch + +from intervalnets import ( + DeepHybridOneJetResult, + IntervalTensor, + PZIntegrationCell, + integrate_hybrid_onejet_squared, + integrate_hybrid_value_squared, + load_tanh_mlp_checkpoint, + scalar_hybrid_onejet_reverse, +) + + +SEED = 20260806 +REPEATS = 7 +VALIDATION_SAMPLES = 16_384 + + +def _architecture(model: torch.nn.Sequential) -> list[int]: + linears = [layer for layer in model if isinstance(layer, torch.nn.Linear)] + return [linears[0].in_features, *(layer.out_features for layer in linears)] + + +def _norm_record(squared, volume: float) -> dict[str, float]: + squared_lower = max(0.0, float(squared.lower)) + squared_upper = max(0.0, float(squared.upper)) + lower = sqrt(squared_lower) + upper = sqrt(squared_upper) + normalized_squared_lower = squared_lower / volume + normalized_squared_upper = squared_upper / volume + normalized_lower = sqrt(normalized_squared_lower) + normalized_upper = sqrt(normalized_squared_upper) + return { + "squared_lower": squared_lower, + "squared_upper": squared_upper, + "squared_absolute_width": squared_upper - squared_lower, + "squared_relative_width": ( + (squared_upper - squared_lower) / squared_upper + if squared_upper > 0.0 + else 0.0 + ), + "lower": lower, + "upper": upper, + "absolute_width": upper - lower, + "relative_width": (upper - lower) / upper if upper > 0.0 else 0.0, + "domain_volume_normalized_squared_lower": normalized_squared_lower, + "domain_volume_normalized_squared_upper": normalized_squared_upper, + "domain_volume_normalized_squared_width": ( + normalized_squared_upper - normalized_squared_lower + ), + "domain_volume_normalized_lower": normalized_lower, + "domain_volume_normalized_upper": normalized_upper, + "domain_volume_normalized_width": normalized_upper - normalized_lower, + } + + +def _benchmark(checkpoint: Path) -> dict[str, object]: + model = load_tanh_mlp_checkpoint(checkpoint).double().eval() + box = IntervalTensor.from_bounds([-0.1] * 100, [0.1] * 100) + cell = PZIntegrationCell.from_affine_box(box) + + warm = scalar_hybrid_onejet_reverse(model, cell.domain) + integrate_hybrid_value_squared(warm, cell) + integrate_hybrid_onejet_squared(warm, cell) + + construction_times: list[float] = [] + l2_times: list[float] = [] + w12_times: list[float] = [] + result = warm + l2_squared = None + w12_squared = None + for _ in range(REPEATS): + start = perf_counter() + result = scalar_hybrid_onejet_reverse(model, cell.domain) + constructed = perf_counter() + l2_squared = integrate_hybrid_value_squared(result, cell) + l2_done = perf_counter() + w12_squared = integrate_hybrid_onejet_squared(result, cell) + done = perf_counter() + construction_times.append(constructed - start) + l2_times.append(l2_done - constructed) + w12_times.append(done - l2_done) + assert l2_squared is not None and w12_squared is not None + + generator = torch.Generator().manual_seed(SEED + 222) + samples = -0.1 + 0.2 * torch.rand( + (VALIDATION_SAMPLES, 100), generator=generator, dtype=torch.float64 + ) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + sampled_l2_squared = float(values.square().mean().detach()) + sampled_w12_squared = float( + (values.square().squeeze(1) + gradients.square().sum(1)).mean().detach() + ) + + l2 = _norm_record(l2_squared, float(cell.volume)) + w12 = _norm_record(w12_squared, float(cell.volume)) + l2["sampled_domain_volume_normalized"] = sqrt(sampled_l2_squared) + w12["sampled_domain_volume_normalized"] = sqrt(sampled_w12_squared) + + value_enclosure = result.value.interval_enclosure() if isinstance(result, DeepHybridOneJetResult) else result.final.Y.interval_enclosure() + jacobian_enclosure = result.jacobian.interval_enclosure() if isinstance(result, DeepHybridOneJetResult) else result.final.J.interval_enclosure() + value_lower = torch.as_tensor(value_enclosure.lower).reshape(1, 1) + value_upper = torch.as_tensor(value_enclosure.upper).reshape(1, 1) + jacobian_lower = torch.as_tensor(jacobian_enclosure.lower).reshape(1, 100) + jacobian_upper = torch.as_tensor(jacobian_enclosure.upper).reshape(1, 100) + + record: dict[str, object] = { + "checkpoint": str(checkpoint.relative_to(checkpoint.parents[2])), + "architecture": _architecture(model), + "physical_domain_volume": float(cell.volume), + "integration_semantics": "pointwise_residual_absolute_moments_parity", + "l2": l2, + "w12": w12, + "timings_seconds_median": { + "onejet_construction": median(construction_times), + "l2_integration": median(l2_times), + "w12_integration": median(w12_times), + "construction_plus_l2_plus_w12": median( + [a + b + c for a, b, c in zip(construction_times, l2_times, w12_times)] + ), + }, + "sample_soundness_violations": { + "value": int(torch.count_nonzero((values < value_lower) | (values > value_upper))), + "jacobian_entries": int( + torch.count_nonzero((gradients < jacobian_lower) | (gradients > jacobian_upper)) + ), + }, + } + if isinstance(result, DeepHybridOneJetResult): + record["quadratic_neurons_per_layer"] = [ + int(torch.count_nonzero(factor.approximation_degrees == 2)) + for factor in result.factors + ] + record["deep_w12_integration_note"] = ( + "absolute moments and parity are applied after the exact factored " + "Jacobian is collapsed to its affine core plus certified pointwise remainder" + ) + else: + record["quadratic_neurons_per_layer"] = [ + int(torch.count_nonzero(result.derivative_degrees == 2)) + ] + return record + + +def main() -> None: + torch.set_num_threads(1) + torch.manual_seed(SEED) + root = Path(__file__).resolve().parents[1] + checkpoints = [ + root / "notebooks" / "checkpoints" / "pinn_100d_poisson_shallow_300.pt", + root / "notebooks" / "checkpoints" / "pinn_100d_poisson.pt", + ] + records = [_benchmark(path) for path in checkpoints] + output = root / "notebooks" / "benchmark_outputs" / "hybrid_absolute_moment_norms.json" + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(records, indent=2) + "\n", encoding="utf-8") + print(json.dumps(records, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/notebooks/pinn_100d_poisson_pz_certification.ipynb b/notebooks/pinn_100d_poisson_pz_certification.ipynb new file mode 100644 index 0000000..97ab163 --- /dev/null +++ b/notebooks/pinn_100d_poisson_pz_certification.ipynb @@ -0,0 +1,1575 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ee2e4d1b", + "metadata": {}, + "source": [ + "# A trained 100-dimensional PINN as a certification benchmark\n", + "\n", + "This notebook replaces the random target network by a standard physics-informed neural network for the manufactured Poisson problem\n", + "\n", + "\\[\n", + "-\\Delta u=f\\quad\\text{in }\\Omega=[-0.1,0.1]^{100},\n", + "\\qquad u=g\\quad\\text{on }\\partial\\Omega.\n", + "\\]\n", + "\n", + "The exact solution is a nonconstant two-direction ridge function,\n", + "\n", + "\\[\n", + "u_*(x)=\\sin(2.5\\,a^\\top x)+0.35\\cos(1.75\\,b^\\top x),\n", + "\\]\n", + "\n", + "where $a,b\\in\\mathbb R^{100}$ are orthonormal dense directions. Hence\n", + "\n", + "\\[\n", + "f(x)=2.5^2\\sin(2.5\\,a^\\top x)\n", + "+0.35\\,1.75^2\\cos(1.75\\,b^\\top x),\n", + "\\qquad g=u_*|_{\\partial\\Omega}.\n", + "\\]\n", + "\n", + "The target architecture is exactly $100$-$50$-$50$-$50$-$1$ with tanh activations. The checkpoint was trained from the interior PDE residual and sampled Dirichlet boundary loss only; the exact solution is used for validation, not as supervised training data.\n", + "\n", + "The half-width $0.1$ is an experimental scaling choice, not part of the Poisson equation itself. Because $a$ and $b$ are unit vectors, $a^\\top x$ and $b^\\top x$ can range on the order of one on this cube, so the two ridge modes remain nontrivial while the tanh preactivations stay in a range where a single-cell affine enclosure is still usable. On $[-1,1]^{100}$ the same frequencies would traverse much wider phase and preactivation intervals, making both ordinary PINN training and a global single-cell certificate substantially harder. The price of the small cube is $|\\Omega|=0.2^{100}$, which is why raw norms are tiny and volume-normalized norms are also reported.\n", + "\n", + "Certification compares interval arithmetic with the certified polynomial-Jacobian reductions Top-$k$, degree-capped Top-$k$, and coefficient-space PCA. The unreduced polynomial endpoint is structurally infeasible on the target architecture and is therefore not launched accidentally; reduced terms are always absorbed into a rigorously propagated pointwise remainder." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "228e15e2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "torch=2.13.0+cu130, dtype=torch.float64, threads=1\n", + "checkpoint=notebooks/checkpoints/pinn_100d_poisson.pt\n" + ] + } + ], + "source": [ + "from __future__ import annotations\n", + "\n", + "import math\n", + "import random\n", + "import sys\n", + "from pathlib import Path\n", + "from time import perf_counter\n", + "\n", + "import numpy as np\n", + "import torch\n", + "from torch import nn\n", + "\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / 'src' / 'intervalnets').exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "if str(repo_root / 'src') not in sys.path:\n", + " sys.path.insert(0, str(repo_root / 'src'))\n", + "\n", + "from intervalnets import (\n", + " IntervalTensor,\n", + " PZIntegrationCell,\n", + " enable_interval_eval,\n", + " integrate_pz_onejet_squared,\n", + " integrate_pz_value_squared,\n", + " load_tanh_mlp_checkpoint,\n", + " sequential_value_jacobian_laplacian,\n", + ")\n", + "\n", + "torch.set_num_threads(1)\n", + "torch.set_default_dtype(torch.float64)\n", + "enable_interval_eval()\n", + "\n", + "DIM = 100\n", + "HALF_WIDTH = 0.1\n", + "HIDDEN = (50, 50, 50)\n", + "SEED = 20260731\n", + "K1 = 2.5\n", + "K2 = 1.75\n", + "COS_AMPLITUDE = 0.35\n", + "CHECKPOINT = repo_root / 'notebooks' / 'checkpoints' / 'pinn_100d_poisson.pt'\n", + "\n", + "random.seed(SEED)\n", + "np.random.seed(SEED)\n", + "torch.manual_seed(SEED)\n", + "print(f'torch={torch.__version__}, dtype={torch.get_default_dtype()}, threads={torch.get_num_threads()}')\n", + "print(f'checkpoint={CHECKPOINT.relative_to(repo_root)}')" + ] + }, + { + "cell_type": "markdown", + "id": "7482e975", + "metadata": {}, + "source": [ + "## PDE, model, and efficient PINN residual" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2e78aeb3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(10201, Sequential(\n", + " (0): Linear(in_features=100, out_features=50, bias=True)\n", + " (1): Tanh()\n", + " (2): Linear(in_features=50, out_features=50, bias=True)\n", + " (3): Tanh()\n", + " (4): Linear(in_features=50, out_features=50, bias=True)\n", + " (5): Tanh()\n", + " (6): Linear(in_features=50, out_features=1, bias=True)\n", + "))" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def dense_directions(dim=DIM):\n", + " a = torch.ones(dim)\n", + " a /= torch.linalg.vector_norm(a)\n", + " b = torch.tensor([1.0 if i % 2 == 0 else -1.0 for i in range(dim)])\n", + " b -= torch.dot(a, b) * a\n", + " b /= torch.linalg.vector_norm(b)\n", + " return a, b\n", + "\n", + "\n", + "A, B = dense_directions()\n", + "\n", + "\n", + "def exact_solution(x):\n", + " return (torch.sin(K1 * (x @ A)) + COS_AMPLITUDE * torch.cos(K2 * (x @ B))).unsqueeze(-1)\n", + "\n", + "\n", + "def exact_gradient(x):\n", + " s = x @ A\n", + " t = x @ B\n", + " return K1 * torch.cos(K1 * s).unsqueeze(-1) * A - COS_AMPLITUDE * K2 * torch.sin(K2 * t).unsqueeze(-1) * B\n", + "\n", + "\n", + "def forcing(x):\n", + " return (K1**2 * torch.sin(K1 * (x @ A)) + COS_AMPLITUDE * K2**2 * torch.cos(K2 * (x @ B))).unsqueeze(-1)\n", + "\n", + "\n", + "def make_model():\n", + " layers, previous = [], DIM\n", + " for width in HIDDEN:\n", + " layers.extend([nn.Linear(previous, width), nn.Tanh()])\n", + " previous = width\n", + " layers.append(nn.Linear(previous, 1))\n", + " model = nn.Sequential(*layers)\n", + " for layer in model:\n", + " if isinstance(layer, nn.Linear):\n", + " nn.init.xavier_uniform_(layer.weight)\n", + " nn.init.zeros_(layer.bias)\n", + " return model\n", + "\n", + "\n", + "def sample_interior(n, generator):\n", + " return (2.0 * torch.rand((n, DIM), generator=generator) - 1.0) * HALF_WIDTH\n", + "\n", + "\n", + "def sample_boundary(n, generator):\n", + " x = sample_interior(n, generator)\n", + " coordinate = torch.randint(DIM, (n,), generator=generator)\n", + " sign = torch.where(torch.rand(n, generator=generator) < 0.5, -1.0, 1.0)\n", + " x[torch.arange(n), coordinate] = HALF_WIDTH * sign\n", + " return x\n", + "\n", + "\n", + "def pinn_residual(model, x):\n", + " value, jacobian, laplacian = sequential_value_jacobian_laplacian(model, x)\n", + " return value, jacobian, -laplacian - forcing(x)\n", + "\n", + "\n", + "model = make_model()\n", + "sum(parameter.numel() for parameter in model.parameters()), model" + ] + }, + { + "cell_type": "markdown", + "id": "ca0c7c2f", + "metadata": {}, + "source": [ + "## Reproducible PINN training\n", + "\n", + "Set `RETRAIN = True` to regenerate the checkpoint. The default loads the included deterministic checkpoint, so certification can be rerun in seconds. Training uses Adam with 512 fresh interior and 512 fresh boundary points per step and the ordinary loss\n", + "\n", + "\\[\n", + "\\mathcal L(\\theta)=\\mathbb E_\\Omega|{-\\Delta u_\\theta-f}|^2\n", + "+20\\,\\mathbb E_{\\partial\\Omega}|u_\\theta-g|^2.\n", + "\\]\n", + "\n", + "The Laplacian helper propagates the exact Hessian trace through the tanh MLP and remains differentiable with respect to its parameters; it changes computational organization, not the PINN objective." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "b60427a1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'loaded_checkpoint': True, 'training_records': []}" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def train_pinn(model, steps=1500, batch_size=512, lr=2e-3):\n", + " generator = torch.Generator().manual_seed(SEED + 1)\n", + " optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n", + " scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=steps, eta_min=2e-4)\n", + " history = []\n", + " model.train()\n", + " for step in range(1, steps + 1):\n", + " interior = sample_interior(batch_size, generator)\n", + " boundary = sample_boundary(batch_size, generator)\n", + " _, _, residual = pinn_residual(model, interior)\n", + " boundary_error = model(boundary) - exact_solution(boundary)\n", + " residual_loss = residual.square().mean()\n", + " boundary_loss = boundary_error.square().mean()\n", + " loss = residual_loss + 20.0 * boundary_loss\n", + " optimizer.zero_grad(set_to_none=True)\n", + " loss.backward()\n", + " torch.nn.utils.clip_grad_norm_(model.parameters(), 10.0)\n", + " optimizer.step()\n", + " scheduler.step()\n", + " if step == 1 or step % 100 == 0:\n", + " history.append({\n", + " 'step': step,\n", + " 'loss': float(loss.detach()),\n", + " 'residual_loss': float(residual_loss.detach()),\n", + " 'boundary_loss': float(boundary_loss.detach()),\n", + " })\n", + " model.eval()\n", + " return history\n", + "\n", + "\n", + "RETRAIN = False\n", + "if RETRAIN:\n", + " training_history = train_pinn(model)\n", + " CHECKPOINT.parent.mkdir(parents=True, exist_ok=True)\n", + " torch.save({'state_dict': model.state_dict(), 'seed': SEED}, CHECKPOINT)\n", + "else:\n", + " model = load_tanh_mlp_checkpoint(CHECKPOINT)\n", + " training_history = []\n", + "\n", + "{'loaded_checkpoint': not RETRAIN, 'training_records': training_history[-3:]}" + ] + }, + { + "cell_type": "markdown", + "id": "fb85ca69", + "metadata": {}, + "source": [ + "## Candidate-network validation" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "f643c401", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'solution_RMSE': 0.002859142422545456, 'solution_relative_L2_error': 0.007575039840436495, 'solution_max_sample_error': 0.030369810860190305, 'PDE_residual_RMSE': 0.01780581896203556, 'boundary_RMSE': 0.002867007950069476, 'network_normalized_L2_MC': 0.37731984665672036, 'network_normalized_W12_MC': 2.504706918197497, 'exact_normalized_L2_MC': 0.3774425590850363, 'exact_normalized_W12_MC': 2.5036437778283043}" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "validation_generator = torch.Generator().manual_seed(SEED + 222)\n", + "interior = sample_interior(8192, validation_generator)\n", + "boundary = sample_boundary(8192, validation_generator)\n", + "with torch.no_grad():\n", + " prediction, network_jacobian, residual = pinn_residual(model, interior)\n", + " target = exact_solution(interior)\n", + " target_gradient = exact_gradient(interior)\n", + " boundary_error = model(boundary) - exact_solution(boundary)\n", + " error = prediction - target\n", + " empirical_network_l2 = prediction.square().mean().sqrt()\n", + " empirical_network_w12 = (prediction.square() + network_jacobian.square().sum(dim=(-2, -1), keepdim=True)).mean().sqrt()\n", + " empirical_exact_l2 = target.square().mean().sqrt()\n", + " empirical_exact_w12 = (target.square() + target_gradient.square().sum(dim=-1, keepdim=True)).mean().sqrt()\n", + "\n", + "validation = {\n", + " 'solution_RMSE': float(error.square().mean().sqrt()),\n", + " 'solution_relative_L2_error': float(error.square().mean().sqrt() / target.square().mean().sqrt()),\n", + " 'solution_max_sample_error': float(error.abs().max()),\n", + " 'PDE_residual_RMSE': float(residual.square().mean().sqrt()),\n", + " 'boundary_RMSE': float(boundary_error.square().mean().sqrt()),\n", + " 'network_normalized_L2_MC': float(empirical_network_l2),\n", + " 'network_normalized_W12_MC': float(empirical_network_w12),\n", + " 'exact_normalized_L2_MC': float(empirical_exact_l2),\n", + " 'exact_normalized_W12_MC': float(empirical_exact_w12),\n", + "}\n", + "validation" + ] + }, + { + "cell_type": "markdown", + "id": "0aa2e549", + "metadata": {}, + "source": [ + "## Certification diagnostics\n", + "\n", + "The raw norm scales like $|\\Omega|^{1/2}=0.2^{50}$, so both raw and volume-normalized intervals are reported. Volume normalization is not relative error: it only removes the factor $|\\Omega|^{1/2}$. For an interval $[L,U]$, absolute width is $U-L$ and relative width is $(U-L)/\\max(|L|,|U|)$ when the denominator is nonzero; the lower bound is not used as the denominator.\n", + "\n", + "Before integration, the full certified Jacobian enclosure is summarized by mean component width, maximum component width, and the mean entrywise relative width\n", + "\n", + "\\[\n", + "\\frac1N\\sum_{ij}\\frac{\\overline J_{ij}-\\underline J_{ij}}\n", + "{\\max(|\\underline J_{ij}|,|\\overline J_{ij}|)},\n", + "\\]\n", + "\n", + "with exact-zero entries assigned zero. This relative width lies in $[0,2]$." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d9095157", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1.1258999068426271e-35" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SQRT_VOLUME = (2.0 * HALF_WIDTH) ** (DIM / 2.0)\n", + "VOLUME = SQRT_VOLUME ** 2\n", + "DOMAIN = IntervalTensor.from_bounds([-HALF_WIDTH] * DIM, [HALF_WIDTH] * DIM)\n", + "\n", + "\n", + "def norm_interval(squared):\n", + " return math.sqrt(max(0.0, float(squared.lower))), math.sqrt(max(0.0, float(squared.upper)))\n", + "\n", + "\n", + "def interval_metrics(bounds, prefix):\n", + " lower, upper = map(float, bounds)\n", + " width = upper - lower\n", + " return {\n", + " f'{prefix}_lower': lower,\n", + " f'{prefix}_upper': upper,\n", + " f'{prefix}_absolute_width': width,\n", + " f'{prefix}_relative_width': width / max(abs(lower), abs(upper)) if max(abs(lower), abs(upper)) > 0.0 else 0.0,\n", + " f'{prefix}_normalized_lower': lower / SQRT_VOLUME,\n", + " f'{prefix}_normalized_upper': upper / SQRT_VOLUME,\n", + " f'{prefix}_normalized_absolute_width': width / SQRT_VOLUME,\n", + " }\n", + "\n", + "\n", + "def jacobian_width_metrics(enclosure):\n", + " lower = torch.as_tensor(enclosure.lower)\n", + " upper = torch.as_tensor(enclosure.upper)\n", + " widths = upper - lower\n", + " scales = torch.maximum(lower.abs(), upper.abs())\n", + " relative_widths = torch.where(scales > 0.0, widths / scales, 0.0)\n", + " return {\n", + " 'J_mean_component_width_before_integration': float(widths.mean()),\n", + " 'J_max_component_width_before_integration': float(widths.max()),\n", + " 'J_relative_mean_component_width_before_integration': float(relative_widths.mean()),\n", + " }\n", + "\n", + "\n", + "def benchmark_polynomial(model, strategy='topk', **kwargs):\n", + " cell = PZIntegrationCell.from_affine_box(DOMAIN)\n", + " start = perf_counter()\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain, return_trace=True, reduction_strategy=strategy, **kwargs\n", + " )\n", + " forward_s = perf_counter() - start\n", + " jacobian_metrics = jacobian_width_metrics(traced.final.J.interval_enclosure())\n", + " start = perf_counter()\n", + " l2_integrated_pz = integrate_pz_value_squared(traced.final.Y, cell, output='pz')\n", + " l2_squared = l2_integrated_pz.interval_enclosure()\n", + " l2_integration_s = perf_counter() - start\n", + " start = perf_counter()\n", + " w12_integrated_pz = integrate_pz_onejet_squared(traced.final, cell, output='pz')\n", + " w12_squared = w12_integrated_pz.interval_enclosure()\n", + " w12_integration_s = perf_counter() - start\n", + " return {\n", + " 'strategy': strategy,\n", + " **kwargs,\n", + " 'forward_s': forward_s,\n", + " 'L2_integration_s': l2_integration_s,\n", + " 'W12_integration_s': w12_integration_s,\n", + " 'total_W12_s': forward_s + w12_integration_s,\n", + " 'J_terms': len(traced.final.J.terms),\n", + " 'J_degree': max(map(sum, traced.final.J.terms), default=0),\n", + " 'noise_count': traced.final.J.num_noise,\n", + " 'L2_integrated_PZ_terms': len(l2_integrated_pz.terms),\n", + " 'W12_integrated_PZ_terms': len(w12_integrated_pz.terms),\n", + " 'W12_integrated_PZ_noise': w12_integrated_pz.num_noise,\n", + " **jacobian_metrics,\n", + " **interval_metrics(norm_interval(l2_squared), 'L2'),\n", + " **interval_metrics(norm_interval(w12_squared), 'W12'),\n", + " 'trace': traced.records,\n", + " }\n", + "\n", + "\n", + "SQRT_VOLUME" + ] + }, + { + "cell_type": "markdown", + "id": "17ad3c19", + "metadata": {}, + "source": [ + "## Interval and certified polynomial-reduction benchmarks" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "feb9fdf7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'method': 'interval', 'total_W12_s': 0.18789246800042747, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 7.979522775780479, 'L2_normalized_absolute_width': 7.979522775780479, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 88.8468047131762, 'W12_normalized_absolute_width': 88.8468047131762, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 17.548814954264703, 'J_max_component_width_before_integration': 20.8648129804914, 'J_relative_mean_component_width_before_integration': 1.9897599352068311, 'J_terms': None, 'J_degree': None}, {'method': 'topk-32', 'total_W12_s': 1.3675986349990126, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 86.68683003796212, 'W12_normalized_absolute_width': 86.68683003796212, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 17.112409763051467, 'J_max_component_width_before_integration': 20.290219948868312, 'J_relative_mean_component_width_before_integration': 1.981436965912812, 'J_terms': 132, 'J_degree': 1}, {'method': 'topk-64', 'total_W12_s': 1.489973119001661, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 86.00679127398774, 'W12_normalized_absolute_width': 86.00679127398774, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.976826392261373, 'J_max_component_width_before_integration': 20.127466839615945, 'J_relative_mean_component_width_before_integration': 1.9812901838832613, 'J_terms': 164, 'J_degree': 1}, {'method': 'topk-96', 'total_W12_s': 2.1779780020006, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 85.37015269613306, 'W12_normalized_absolute_width': 85.37015269613306, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.849883223179123, 'J_max_component_width_before_integration': 19.977129769863755, 'J_relative_mean_component_width_before_integration': 1.9811506079606034, 'J_terms': 196, 'J_degree': 1}, {'method': 'topk-128', 'total_W12_s': 2.3470172980014468, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 85.22957171233983, 'W12_normalized_absolute_width': 85.22957171233983, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.821833951732813, 'J_max_component_width_before_integration': 19.945026651483428, 'J_relative_mean_component_width_before_integration': 1.9811194383171549, 'J_terms': 226, 'J_degree': 1}, {'method': 'topk-192', 'total_W12_s': 2.8487907859998813, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 85.06937919921694, 'W12_normalized_absolute_width': 85.06937919921694, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.789867184806653, 'J_max_component_width_before_integration': 19.907968632613045, 'J_relative_mean_component_width_before_integration': 1.981083775490849, 'J_terms': 278, 'J_degree': 1}, {'method': 'degree-64', 'total_W12_s': 1.627848776999599, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 86.00679127398774, 'W12_normalized_absolute_width': 86.00679127398774, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.976826392261373, 'J_max_component_width_before_integration': 20.127466839615945, 'J_relative_mean_component_width_before_integration': 1.9812901838832613, 'J_terms': 164, 'J_degree': 1}, {'method': 'pca-64', 'total_W12_s': 1.6332670800002234, 'L2_normalized_lower': 0.0, 'L2_normalized_upper': 2.6768306182215453, 'L2_normalized_absolute_width': 2.6768306182215453, 'L2_relative_width': 1.0, 'W12_normalized_lower': 0.0, 'W12_normalized_upper': 85.9003620724866, 'W12_normalized_absolute_width': 85.9003620724866, 'W12_relative_width': 1.0, 'J_mean_component_width_before_integration': 16.955616241634107, 'J_max_component_width_before_integration': 20.104078525652792, 'J_relative_mean_component_width_before_integration': 1.9812670435315718, 'J_terms': 168, 'J_degree': 1}]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "configurations = [\n", + " ('topk-32', 'topk', dict(max_terms=32)),\n", + " ('topk-64', 'topk', dict(max_terms=64)),\n", + " ('topk-96', 'topk', dict(max_terms=96)),\n", + " ('topk-128', 'topk', dict(max_terms=128)),\n", + " ('topk-192', 'topk', dict(max_terms=192)),\n", + " ('degree-64', 'degree', dict(max_terms=64, max_degree=2)),\n", + " ('pca-64', 'pca', dict(max_terms=64, pca_rank=4, pca_candidates=32)),\n", + "]\n", + "\n", + "polynomial_rows = []\n", + "for label, strategy, kwargs in configurations:\n", + " row = benchmark_polynomial(model, strategy=strategy, **kwargs)\n", + " row['method'] = label\n", + " polynomial_rows.append(row)\n", + "\n", + "start = perf_counter()\n", + "interval_w12 = model.sobolev_norm(DOMAIN, p=2.0, order=1, method='interval')\n", + "interval_total_s = perf_counter() - start\n", + "interval_l2 = model.lpnorm(DOMAIN, p=2.0, method='interval')\n", + "interval_jacobian = model.eval_jacobian(DOMAIN)\n", + "interval_row = {\n", + " 'method': 'interval',\n", + " 'total_W12_s': interval_total_s,\n", + " **jacobian_width_metrics(interval_jacobian),\n", + " **interval_metrics((interval_l2.lower, interval_l2.upper), 'L2'),\n", + " **interval_metrics((interval_w12.lower, interval_w12.upper), 'W12'),\n", + "}\n", + "\n", + "assert all(row['total_W12_s'] < 3.0 for row in polynomial_rows)\n", + "benchmark_rows = [interval_row, *polynomial_rows]\n", + "\n", + "columns = [\n", + " 'method', 'total_W12_s',\n", + " 'L2_normalized_lower', 'L2_normalized_upper', 'L2_normalized_absolute_width', 'L2_relative_width',\n", + " 'W12_normalized_lower', 'W12_normalized_upper', 'W12_normalized_absolute_width', 'W12_relative_width',\n", + " 'J_mean_component_width_before_integration', 'J_max_component_width_before_integration',\n", + " 'J_relative_mean_component_width_before_integration', 'J_terms', 'J_degree',\n", + "]\n", + "[{key: row.get(key) for key in columns} for row in benchmark_rows]" + ] + }, + { + "cell_type": "markdown", + "id": "f28cf10d", + "metadata": {}, + "source": [ + "### Raw norm intervals and absolute widths" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "c9ecd83c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'method': 'interval', 'L2_lower': 0.0, 'L2_upper': 8.984143949899863e-35, 'L2_absolute_width': 8.984143949899863e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 1.0003260914983017e-33, 'W12_absolute_width': 1.0003260914983017e-33, 'W12_relative_width': 1.0}, {'method': 'topk-32', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.76006938642242e-34, 'W12_absolute_width': 9.76006938642242e-34, 'W12_relative_width': 1.0}, {'method': 'topk-64', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.683503828321607e-34, 'W12_absolute_width': 9.683503828321607e-34, 'W12_relative_width': 1.0}, {'method': 'topk-96', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.611824696771706e-34, 'W12_absolute_width': 9.611824696771706e-34, 'W12_relative_width': 1.0}, {'method': 'topk-128', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.595996685116042e-34, 'W12_absolute_width': 9.595996685116042e-34, 'W12_relative_width': 1.0}, {'method': 'topk-192', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.577960611555848e-34, 'W12_absolute_width': 9.577960611555848e-34, 'W12_relative_width': 1.0}, {'method': 'degree-64', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.683503828321607e-34, 'W12_absolute_width': 9.683503828321607e-34, 'W12_relative_width': 1.0}, {'method': 'pca-64', 'L2_lower': 0.0, 'L2_upper': 3.0138433436891297e-35, 'L2_absolute_width': 3.0138433436891297e-35, 'L2_relative_width': 1.0, 'W12_lower': 0.0, 'W12_upper': 9.67152096551606e-34, 'W12_absolute_width': 9.67152096551606e-34, 'W12_relative_width': 1.0}]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "raw_columns = [\n", + " 'method',\n", + " 'L2_lower', 'L2_upper', 'L2_absolute_width', 'L2_relative_width',\n", + " 'W12_lower', 'W12_upper', 'W12_absolute_width', 'W12_relative_width',\n", + "]\n", + "[{key: row.get(key) for key in raw_columns} for row in benchmark_rows]" + ] + }, + { + "cell_type": "markdown", + "id": "a104ff89", + "metadata": {}, + "source": [ + "### Public default PZ APIs" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "856c0331", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'public_L2_s': 0.06607493099909334, 'public_W12_s': 2.057815976000711, 'public_L2_lower': -5e-324, 'public_L2_upper': 3.01384334368913e-35, 'public_L2_absolute_width': 3.01384334368913e-35, 'public_L2_relative_width': 1.0, 'public_L2_normalized_lower': -4.388184445513989e-289, 'public_L2_normalized_upper': 2.6768306182215458, 'public_L2_normalized_absolute_width': 2.6768306182215458, 'public_W12_lower': -5e-324, 'public_W12_upper': 9.611824696771708e-34, 'public_W12_absolute_width': 9.611824696771708e-34, 'public_W12_relative_width': 1.0, 'public_W12_normalized_lower': -4.388184445513989e-289, 'public_W12_normalized_upper': 85.37015269613308, 'public_W12_normalized_absolute_width': 85.37015269613308}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "start = perf_counter()\n", + "public_l2 = model.pz_l2norm(DOMAIN)\n", + "public_l2_s = perf_counter() - start\n", + "start = perf_counter()\n", + "public_w12 = model.pz_sobolev_norm(DOMAIN, order=1)\n", + "public_w12_s = perf_counter() - start\n", + "\n", + "default_row = next(row for row in polynomial_rows if row['method'] == 'topk-96')\n", + "assert math.isclose(float(public_l2.upper), default_row['L2_upper'], rel_tol=1e-12)\n", + "assert math.isclose(float(public_w12.upper), default_row['W12_upper'], rel_tol=1e-12)\n", + "{\n", + " 'public_L2_s': public_l2_s,\n", + " 'public_W12_s': public_w12_s,\n", + " **interval_metrics((public_l2.lower, public_l2.upper), 'public_L2'),\n", + " **interval_metrics((public_w12.lower, public_w12.upper), 'public_W12'),\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "8c689576", + "metadata": {}, + "source": [ + "## Why the target unreduced polynomial is not executed\n", + "\n", + "`reduction_strategy='none'` is a valid exact polynomial one-jet endpoint, but it is not a viable target-network benchmark. After the first tanh layer there are already roughly 100 domain-dependent derivative terms. The next chain-rule product couples these with about 150 derivative/value generators, producing on the order of $1.5\\times10^4$ candidates; the third activation can then produce millions of candidates before canonicalization. Launching this path would violate the benchmark's memory and runtime purpose.\n", + "\n", + "The unreduced endpoint remains covered by unit tests and by the small-network reference in `pz_w12_polynomial_reduction_benchmarks.ipynb`. Here, every target-network polynomial method is sound because the omitted tail is explicitly accumulated into a propagated pointwise remainder; no candidate term is simply dropped." + ] + }, + { + "cell_type": "markdown", + "id": "6930c60d", + "metadata": {}, + "source": [ + "## What Top-$k$ reduction does\n", + "\n", + "At each derivative-chain-rule product, every candidate monomial has a tensor coefficient $C_\\alpha$. Top-$k$ scores it by its largest absolute component, keeps the $k$ highest-scoring exponent/coefficient pairs as dependent polynomial terms, and adds every discarded coefficient componentwise to a certified pointwise remainder radius. Equal exponent vectors are canonicalized before the final enclosure. Thus Top-$k$ is sound: it trades dependency information for a box remainder, but never deletes uncertainty. Larger $k$ preserves more correlation and cancellation, at the cost of more polynomial products and integration work.\n", + "\n", + "PCA uses some of the discarded coefficient tensors differently: it retains a few shared coefficient-space directions and boxes only the orthogonal residual. This can preserve cancellation through later linear maps, although the benchmark below shows that the activation-approximation remainder, rather than the retained-support budget, dominates this particular PINN." + ] + }, + { + "cell_type": "markdown", + "id": "c211f09b", + "metadata": {}, + "source": [ + "## Layer diagnostics for the default Top-96 method\n", + "\n", + "For each hidden neuron, `tanh_prime_approximation_radius` is the certified coefficient $\\delta_{\\ell i}$ in the initial local enclosure\n", + "\n", + "\\[\\tanh'(z_i)\\in p_{\\ell i}z_i+q_{\\ell i}+\\delta_{\\ell i}[-1,1].\\]\n", + "\n", + "It is measured before multiplication by the incoming Jacobian and before any Top-$k$/PCA reduction, so it cleanly separates activation approximation error from compression error." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "c4f4c21a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'layer': 'Input', 'seconds': 0.0, 'Y_terms': 100, 'J_terms': 0, 'J_degree': 0, 'J_remainder_mean_radius': 0.0, 'J_remainder_max_radius': 0.0}, {'layer': 'Linear', 'seconds': 0.002622895999593311, 'Y_terms': 100, 'J_terms': 0, 'J_degree': 0, 'J_remainder_mean_radius': 0.0, 'J_remainder_max_radius': 0.0}, {'layer': 'Tanh', 'seconds': 0.02616049700009171, 'Y_terms': 150, 'J_terms': 96, 'J_degree': 1, 'J_remainder_mean_radius': 0.026446936553910286, 'J_remainder_max_radius': 0.07513935764656894}, {'layer': 'Linear', 'seconds': 0.009305661998951109, 'Y_terms': 150, 'J_terms': 96, 'J_degree': 1, 'J_remainder_mean_radius': 0.15294793951109945, 'J_remainder_max_radius': 0.23283848020502845}, {'layer': 'Tanh', 'seconds': 0.5622837949995301, 'Y_terms': 200, 'J_terms': 86, 'J_degree': 1, 'J_remainder_mean_radius': 0.17384077899371955, 'J_remainder_max_radius': 0.2799701365957077}, {'layer': 'Linear', 'seconds': 0.01211463400068169, 'Y_terms': 200, 'J_terms': 86, 'J_degree': 1, 'J_remainder_mean_radius': 1.0390345844288842, 'J_remainder_max_radius': 1.5042406548473115}, {'layer': 'Tanh', 'seconds': 0.670650341999135, 'Y_terms': 250, 'J_terms': 96, 'J_degree': 1, 'J_remainder_mean_radius': 1.0679993370671088, 'J_remainder_max_radius': 1.518823546002238}, {'layer': 'Linear', 'seconds': 0.011973935001151403, 'Y_terms': 250, 'J_terms': 96, 'J_degree': 1, 'J_remainder_mean_radius': 8.423915631599996, 'J_remainder_max_radius': 9.9871556306211}]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chosen = next(row for row in polynomial_rows if row['method'] == 'topk-96')\n", + "layer_diagnostics = [{\n", + " 'layer': record.layer_type,\n", + " 'seconds': record.elapsed_s,\n", + " 'Y_terms': record.summary['Y']['term_count'],\n", + " 'J_terms': record.summary['J']['term_count'],\n", + " 'J_degree': record.summary['J']['max_degree'],\n", + " 'J_remainder_mean_radius': record.summary['J']['remainder_mean_radius'],\n", + " 'J_remainder_max_radius': record.summary['J']['remainder_max_radius'],\n", + "} for record in chosen['trace']]\n", + "layer_diagnostics" + ] + }, + { + "cell_type": "markdown", + "id": "f7b7c5e1", + "metadata": {}, + "source": [ + "### Initial per-neuron $\\tanh'$ approximation-noise coefficients" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a43e7d0", + "metadata": {}, + "outputs": [], + "source": [ + "activation_records = [record for record in chosen['trace'] if record.layer_type == 'Tanh']\n", + "activation_error_columns = [\n", + " record.summary['tanh_prime_approximation_radii'].detach().cpu().tolist()\n", + " for record in activation_records\n", + "]\n", + "activation_error_summary = [{\n", + " 'hidden_layer': layer_index + 1,\n", + " 'min_delta': record.summary['tanh_prime_approximation_radius_min'],\n", + " 'mean_delta': record.summary['tanh_prime_approximation_radius_mean'],\n", + " 'max_delta': record.summary['tanh_prime_approximation_radius_max'],\n", + "} for layer_index, record in enumerate(activation_records)]\n", + "activation_error_rows = [{\n", + " 'neuron': neuron,\n", + " **{f'hidden_layer_{layer + 1}_delta': values[neuron] for layer, values in enumerate(activation_error_columns)},\n", + "} for neuron in range(len(activation_error_columns[0]))]\n", + "activation_error_summary, activation_error_rows" + ] + }, + { + "cell_type": "markdown", + "id": "medium-intro", + "metadata": {}, + "source": [ + "## Glossary-conformant medium benchmark\n", + "\n", + "This section is the canonical medium benchmark specified by\n", + "`docs/diagnostics_and_metrics_glossary.tex`. It executes two methods on the\n", + "same single-cell partition of $[-0.1,0.1]^{100}$:\n", + "\n", + "1. `interval`: interval propagation and interval norm integration;\n", + "2. `affine_pz_topk96_symbolic`: affine polynomial-zonotope value and one-jet\n", + " propagation with Top-96 reduction, followed by symbolic integration of the\n", + " squared $L^2$ and $W^{1,2}$ quantities. The integrated scalar PZ is\n", + " intervalized only after integration.\n", + "\n", + "The implementation writes one self-contained canonical output directory per\n", + "method. Missing values use the literal `NA`; unimplemented Hessian and\n", + "certified PDE-residual quantities remain explicit with a non-`ok` status.\n", + "Squared norm intervals are retained, and the canonical relative norm width is\n", + "computed before the square root." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-run", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import subprocess\n", + "from datetime import datetime, timezone\n", + "\n", + "import pandas as pd\n", + "\n", + "from intervalnets import interval_forward\n", + "\n", + "SCHEMA_VERSION = \"1.2\"\n", + "BENCHMARK_LEVEL = \"medium\"\n", + "PROBLEM_ID = \"poisson_100d_ridge\"\n", + "MODEL_ID = \"pinn_100d_poisson_seed_20260731\"\n", + "SPLIT_ID = \"single_cell\"\n", + "OUTPUT_ROOT = repo_root / \"notebooks\" / \"benchmark_outputs\" / \"pinn_100d_poisson_medium\"\n", + "\n", + "METRICS_COLUMNS = [\n", + " \"schema_version\", \"benchmark_level\", \"problem_id\", \"model_id\", \"method_id\",\n", + " \"run_id\", \"split_id\", \"quantity\", \"metric\", \"aggregation\",\n", + " \"derivative_order\", \"layer\", \"neuron\", \"output_index\", \"input_index_a\",\n", + " \"input_index_b\", \"cell_id\", \"value\", \"unit\", \"status\",\n", + "]\n", + "CELL_INTERVAL_COLUMNS = [\n", + " \"run_id\", \"split_id\", \"cell_id\", \"cell_weight\", \"quantity\", \"output_index\",\n", + " \"input_index_a\", \"input_index_b\", \"lower\", \"upper\", \"midpoint\", \"radius\",\n", + " \"width\", \"magnitude\", \"mignitude\", \"local_relative_radius\",\n", + " \"global_normalized_radius\", \"sign_certified\", \"status\",\n", + " \"local_relative_width\",\n", + "]\n", + "NORM_COLUMNS = [\n", + " \"run_id\", \"norm\", \"squared\", \"lower\", \"upper\", \"width\", \"relative_width\",\n", + " \"value_contribution_upper\", \"gradient_contribution_upper\",\n", + " \"hessian_contribution_upper\", \"value_contribution_width\",\n", + " \"gradient_contribution_width\", \"hessian_contribution_width\", \"status\",\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + "]\n", + "ACTIVATION_COLUMNS = [\n", + " \"run_id\", \"split_id\", \"cell_id\", \"cell_weight\", \"layer\", \"neuron\",\n", + " \"derivative_order\", \"preactivation_lower\", \"preactivation_upper\",\n", + " \"preactivation_midpoint\", \"preactivation_radius\", \"preactivation_width\",\n", + " \"approximation_kind\", \"approximation_error_radius\",\n", + " \"approximation_error_diameter\", \"activation_scale\",\n", + " \"normalized_approximation_radius\", \"noise_symbol_id\", \"shared_noise_group\",\n", + " \"status\",\n", + "]\n", + "COMPLEXITY_COLUMNS = [\n", + " \"run_id\", \"quantity\", \"n_alpha\", \"n_eta\", \"n_monomials\",\n", + " \"n_mixed_monomials\", \"max_degree\", \"n_coefficients\", \"status\",\n", + "]\n", + "TIMING_COLUMNS = [\"run_id\", \"stage\", \"seconds\", \"status\"]\n", + "SOUNDNESS_COLUMNS = [\n", + " \"run_id\", \"quantity\", \"sample_count\", \"failure_count\", \"max_violation\",\n", + " \"invalid_interval_count\", \"nan_endpoint_count\", \"infinite_endpoint_count\",\n", + " \"status\",\n", + "]\n", + "\n", + "\n", + "def _tensor(value):\n", + " return torch.as_tensor(value, dtype=torch.get_default_dtype()).detach().cpu()\n", + "\n", + "\n", + "def _nonnegative_squared_interval(enclosure):\n", + " lower = max(0.0, float(enclosure.lower))\n", + " upper = max(0.0, float(enclosure.upper))\n", + " return lower, upper\n", + "\n", + "\n", + "def _outward_square(bounds):\n", + " lower, upper = (max(0.0, float(v)) for v in bounds)\n", + " return (\n", + " float(np.nextafter(lower * lower, -np.inf)) if lower else 0.0,\n", + " float(np.nextafter(upper * upper, np.inf)) if upper else 0.0,\n", + " )\n", + "\n", + "\n", + "def _interval_trace(model, domain):\n", + " current = domain\n", + " records = []\n", + " hidden_layer = 0\n", + " for child in model:\n", + " if isinstance(child, nn.Linear):\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " elif isinstance(child, nn.Tanh):\n", + " preactivation = current\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " records.append({\n", + " \"layer\": hidden_layer,\n", + " \"preactivation_lower\": _tensor(preactivation.lower).reshape(-1),\n", + " \"preactivation_upper\": _tensor(preactivation.upper).reshape(-1),\n", + " \"postactivation_lower\": _tensor(current.lower).reshape(-1),\n", + " \"postactivation_upper\": _tensor(current.upper).reshape(-1),\n", + " })\n", + " hidden_layer += 1\n", + " else:\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " return current, records\n", + "\n", + "\n", + "def _run_medium_methods(model):\n", + " results = {}\n", + "\n", + " interval_start = perf_counter()\n", + " interval_l2 = model.lpnorm(DOMAIN, p=2.0, method=\"interval\")\n", + " interval_l2_s = perf_counter() - interval_start\n", + " interval_start = perf_counter()\n", + " interval_w12 = model.sobolev_norm(DOMAIN, p=2.0, order=1, method=\"interval\")\n", + " interval_w12_s = perf_counter() - interval_start\n", + " interval_start = perf_counter()\n", + " interval_y, interval_activation = _interval_trace(model, DOMAIN)\n", + " interval_j = model.eval_jacobian(DOMAIN)\n", + " interval_enclosure_s = perf_counter() - interval_start\n", + " results[\"interval\"] = {\n", + " \"method_id\": \"interval\",\n", + " \"run_id\": \"pinn100d_medium_interval\",\n", + " \"Y\": interval_y,\n", + " \"J\": interval_j,\n", + " \"activation\": interval_activation,\n", + " \"norms\": {\n", + " \"L2\": (float(interval_l2.lower), float(interval_l2.upper)),\n", + " \"L2_sq\": _outward_square((interval_l2.lower, interval_l2.upper)),\n", + " \"W12\": (float(interval_w12.lower), float(interval_w12.upper)),\n", + " \"W12_sq\": _outward_square((interval_w12.lower, interval_w12.upper)),\n", + " },\n", + " \"timings\": {\n", + " \"L2_total\": interval_l2_s,\n", + " \"W12_total\": interval_w12_s,\n", + " \"final_enclosures\": interval_enclosure_s,\n", + " },\n", + " }\n", + "\n", + " cell = PZIntegrationCell.from_affine_box(DOMAIN)\n", + " pz_start = perf_counter()\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain, return_trace=True, reduction_strategy=\"topk\", max_terms=96,\n", + " derivative_enclosure=\"affine\", reduction_variant=\"A\",\n", + " )\n", + " pz_forward_s = perf_counter() - pz_start\n", + " pz_l2_start = perf_counter()\n", + " pz_l2_integrated = integrate_pz_value_squared(traced.final.Y, cell, output=\"pz\")\n", + " pz_l2_sq = pz_l2_integrated.interval_enclosure()\n", + " pz_l2_s = perf_counter() - pz_l2_start\n", + " pz_w12_start = perf_counter()\n", + " pz_w12_integrated = integrate_pz_onejet_squared(traced.final, cell, output=\"pz\")\n", + " pz_w12_sq = pz_w12_integrated.interval_enclosure()\n", + " pz_w12_s = perf_counter() - pz_w12_start\n", + " pz_activation = []\n", + " hidden_layer = 0\n", + " for record in traced.records:\n", + " if record.layer_type != \"Tanh\":\n", + " continue\n", + " postactivation = record.value.interval_enclosure()\n", + " pz_activation.append({\n", + " \"layer\": hidden_layer,\n", + " \"preactivation_lower\": record.summary[\"preactivation_lower\"].detach().cpu(),\n", + " \"preactivation_upper\": record.summary[\"preactivation_upper\"].detach().cpu(),\n", + " \"postactivation_lower\": _tensor(postactivation.lower).reshape(-1),\n", + " \"postactivation_upper\": _tensor(postactivation.upper).reshape(-1),\n", + " \"rho0\": record.summary[\"tanh_approximation_radii\"].detach().cpu(),\n", + " \"rho1\": record.summary[\"tanh_prime_approximation_radii\"].detach().cpu(),\n", + " })\n", + " hidden_layer += 1\n", + " pz_l2_sq_bounds = _nonnegative_squared_interval(pz_l2_sq)\n", + " pz_w12_sq_bounds = _nonnegative_squared_interval(pz_w12_sq)\n", + " results[\"affine_pz_topk96_symbolic\"] = {\n", + " \"method_id\": \"affine_pz_topk96_symbolic\",\n", + " \"run_id\": \"pinn100d_medium_affine_pz_topk96_symbolic\",\n", + " \"Y\": traced.final.Y.interval_enclosure(),\n", + " \"J\": traced.final.J.interval_enclosure(),\n", + " \"Y_pz\": traced.final.Y,\n", + " \"J_pz\": traced.final.J,\n", + " \"activation\": pz_activation,\n", + " \"trace\": traced.records,\n", + " \"integrated_L2_pz\": pz_l2_integrated,\n", + " \"integrated_W12_pz\": pz_w12_integrated,\n", + " \"norms\": {\n", + " \"L2_sq\": pz_l2_sq_bounds,\n", + " \"L2\": norm_interval(pz_l2_sq),\n", + " \"W12_sq\": pz_w12_sq_bounds,\n", + " \"W12\": norm_interval(pz_w12_sq),\n", + " },\n", + " \"timings\": {\n", + " \"onejet_construction\": pz_forward_s,\n", + " \"L2_symbolic_integration\": pz_l2_s,\n", + " \"W12_symbolic_integration\": pz_w12_s,\n", + " \"W12_total\": pz_forward_s + pz_w12_s,\n", + " },\n", + " }\n", + " return results\n", + "\n", + "\n", + "medium_results = _run_medium_methods(model)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-schema", + "metadata": {}, + "outputs": [], + "source": [ + "def _interval_stats(lower, upper, scale):\n", + " lower = np.asarray(lower, dtype=float)\n", + " upper = np.asarray(upper, dtype=float)\n", + " midpoint = 0.5 * (lower + upper)\n", + " radius = 0.5 * (upper - lower)\n", + " width = upper - lower\n", + " magnitude = np.maximum(np.abs(lower), np.abs(upper))\n", + " mignitude = np.where((lower <= 0.0) & (upper >= 0.0), 0.0, np.minimum(np.abs(lower), np.abs(upper)))\n", + " local = np.divide(radius, magnitude, out=np.zeros_like(radius), where=magnitude > 0.0)\n", + " global_radius = np.divide(radius, scale, out=np.zeros_like(radius), where=scale > 0.0)\n", + " return midpoint, radius, width, magnitude, mignitude, local, global_radius\n", + "\n", + "\n", + "def _family_metrics(base, method, quantity, lower, upper, derivative_order):\n", + " lower = np.asarray(lower, dtype=float).reshape(-1)\n", + " upper = np.asarray(upper, dtype=float).reshape(-1)\n", + " scale = float(np.max(np.maximum(np.abs(lower), np.abs(upper)))) if lower.size else 0.0\n", + " _, _, width, _, _, _, global_radius = _interval_stats(lower, upper, scale)\n", + " specifications = [\n", + " (\"mean_width\", \"mean_weighted\", float(np.mean(width))),\n", + " (\"max_width\", \"max\", float(np.max(width))),\n", + " (\"q50_width\", \"q50\", float(np.quantile(width, 0.50, method=\"linear\"))),\n", + " (\"q90_width\", \"q90\", float(np.quantile(width, 0.90, method=\"linear\"))),\n", + " (\"q99_width\", \"q99\", float(np.quantile(width, 0.99, method=\"linear\"))),\n", + " (\"mean_global_normalized_radius\", \"mean_weighted\", float(np.mean(global_radius))),\n", + " (\"max_global_normalized_radius\", \"max\", float(np.max(global_radius))),\n", + " ]\n", + " rows = []\n", + " for metric, aggregation, value in specifications:\n", + " rows.append({**base, \"quantity\": quantity, \"metric\": metric,\n", + " \"aggregation\": aggregation, \"derivative_order\": derivative_order,\n", + " \"value\": value, \"unit\": \"dimensionless\", \"status\": \"ok\"})\n", + " if quantity == \"J\":\n", + " matrix_width = np.asarray(upper - lower, dtype=float)\n", + " frobenius = float(np.linalg.norm(matrix_width.reshape(-1)))\n", + " rows.extend([\n", + " {**base, \"quantity\": \"J\", \"metric\": \"mean_frobenius_width\",\n", + " \"aggregation\": \"mean_weighted\", \"derivative_order\": 1,\n", + " \"value\": frobenius, \"unit\": \"dimensionless\", \"status\": \"ok\"},\n", + " {**base, \"quantity\": \"J\", \"metric\": \"max_frobenius_width\",\n", + " \"aggregation\": \"max\", \"derivative_order\": 1,\n", + " \"value\": frobenius, \"unit\": \"dimensionless\", \"status\": \"ok\"},\n", + " ])\n", + " return rows\n", + "\n", + "\n", + "def _base_metric(result):\n", + " return {\n", + " \"schema_version\": SCHEMA_VERSION, \"benchmark_level\": BENCHMARK_LEVEL,\n", + " \"problem_id\": PROBLEM_ID, \"model_id\": MODEL_ID,\n", + " \"method_id\": result[\"method_id\"], \"run_id\": result[\"run_id\"],\n", + " \"split_id\": SPLIT_ID, \"layer\": pd.NA, \"neuron\": pd.NA,\n", + " \"output_index\": pd.NA, \"input_index_a\": pd.NA,\n", + " \"input_index_b\": pd.NA, \"cell_id\": pd.NA,\n", + " }\n", + "\n", + "\n", + "def _cell_interval_table(result):\n", + " rows = []\n", + " for quantity, enclosure in ((\"Y\", result[\"Y\"]), (\"J\", result[\"J\"])):\n", + " lower = np.asarray(_tensor(enclosure.lower), dtype=float)\n", + " upper = np.asarray(_tensor(enclosure.upper), dtype=float)\n", + " scale = float(np.max(np.maximum(np.abs(lower), np.abs(upper)))) if lower.size else 0.0\n", + " midpoint, radius, width, magnitude, mignitude, local, global_radius = _interval_stats(lower, upper, scale)\n", + " for index in np.ndindex(lower.shape):\n", + " if quantity == \"Y\":\n", + " output_index, input_a = (index[0] if index else 0), pd.NA\n", + " else:\n", + " output_index, input_a = index\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"split_id\": SPLIT_ID, \"cell_id\": 0,\n", + " \"cell_weight\": 1.0, \"quantity\": quantity,\n", + " \"output_index\": output_index, \"input_index_a\": input_a,\n", + " \"input_index_b\": pd.NA, \"lower\": float(lower[index]),\n", + " \"upper\": float(upper[index]), \"midpoint\": float(midpoint[index]),\n", + " \"radius\": float(radius[index]), \"width\": float(width[index]),\n", + " \"magnitude\": float(magnitude[index]), \"mignitude\": float(mignitude[index]),\n", + " \"local_relative_radius\": float(local[index]),\n", + " \"global_normalized_radius\": float(global_radius[index]),\n", + " \"sign_certified\": int(not (lower[index] <= 0.0 <= upper[index])),\n", + " \"status\": \"ok\",\n", + " \"local_relative_width\": float(2.0 * local[index]),\n", + " })\n", + " return pd.DataFrame(rows, columns=CELL_INTERVAL_COLUMNS).sort_values(\n", + " [\"cell_id\", \"quantity\", \"output_index\", \"input_index_a\", \"input_index_b\"],\n", + " na_position=\"last\", kind=\"stable\", ignore_index=True,\n", + " )\n", + "\n", + "\n", + "def _norm_table(result):\n", + " rows = []\n", + " for norm in (\"L2\", \"W12\"):\n", + " for squared in (1, 0):\n", + " key = norm + (\"_sq\" if squared else \"\")\n", + " lower, upper = map(float, result[\"norms\"][key])\n", + " width = upper - lower\n", + " volume_scale = VOLUME if squared else SQRT_VOLUME\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"norm\": norm, \"squared\": squared,\n", + " \"lower\": lower, \"upper\": upper, \"width\": width,\n", + " \"relative_width\": width / upper if upper > 0.0 else 0.0,\n", + " \"value_contribution_upper\": pd.NA,\n", + " \"gradient_contribution_upper\": pd.NA,\n", + " \"hessian_contribution_upper\": pd.NA,\n", + " \"value_contribution_width\": pd.NA,\n", + " \"gradient_contribution_width\": pd.NA,\n", + " \"hessian_contribution_width\": pd.NA,\n", + " \"status\": \"ok\",\n", + " \"domain_volume\": VOLUME,\n", + " \"domain_volume_normalized_lower\": lower / volume_scale,\n", + " \"domain_volume_normalized_upper\": upper / volume_scale,\n", + " \"domain_volume_normalized_width\": width / volume_scale,\n", + " })\n", + " return pd.DataFrame(rows, columns=NORM_COLUMNS)\n", + "\n", + "\n", + "def _tanh_prime_hull(lower, upper):\n", + " t_lo = np.tanh(lower)\n", + " t_hi = np.tanh(upper)\n", + " endpoint_lo = 1.0 - t_lo * t_lo\n", + " endpoint_hi = 1.0 - t_hi * t_hi\n", + " hull_lower = np.minimum(endpoint_lo, endpoint_hi)\n", + " hull_upper = np.where((lower <= 0.0) & (upper >= 0.0), 1.0, np.maximum(endpoint_lo, endpoint_hi))\n", + " return hull_lower, hull_upper\n", + "\n", + "\n", + "def _activation_tables(result):\n", + " rows = []\n", + " wide = {\"neuron\": np.arange(max(len(record[\"preactivation_lower\"]) for record in result[\"activation\"]))}\n", + " for record in result[\"activation\"]:\n", + " layer = int(record[\"layer\"])\n", + " lower = np.asarray(record[\"preactivation_lower\"], dtype=float)\n", + " upper = np.asarray(record[\"preactivation_upper\"], dtype=float)\n", + " midpoint = 0.5 * (lower + upper)\n", + " radius = 0.5 * (upper - lower)\n", + " width = upper - lower\n", + " y_lower = np.tanh(lower)\n", + " y_upper = np.tanh(upper)\n", + " d_lower, d_upper = _tanh_prime_hull(lower, upper)\n", + " hulls = {0: (y_lower, y_upper), 1: (d_lower, d_upper)}\n", + " post_lower = np.asarray(record[\"postactivation_lower\"], dtype=float)\n", + " post_upper = np.asarray(record[\"postactivation_upper\"], dtype=float)\n", + " y_scale = float(np.max(np.maximum(np.abs(post_lower), np.abs(post_upper))))\n", + " y_normalized = (0.5 * (post_upper - post_lower) / y_scale) if y_scale > 0.0 else np.zeros_like(post_lower)\n", + " padded = np.full(len(wide[\"neuron\"]), np.nan)\n", + " padded[:len(y_normalized)] = y_normalized\n", + " wide[f\"layer_{layer}\"] = padded\n", + " for derivative_order in (0, 1):\n", + " hull_lower, hull_upper = hulls[derivative_order]\n", + " scale = float(np.max(np.maximum(np.abs(hull_lower), np.abs(hull_upper))))\n", + " if result[\"method_id\"] == \"interval\":\n", + " rho = 0.5 * (hull_upper - hull_lower)\n", + " kind = \"interval\"\n", + " noise_ids = [pd.NA] * len(lower)\n", + " else:\n", + " rho = np.asarray(record[f\"rho{derivative_order}\"], dtype=float)\n", + " kind = \"affine\"\n", + " noise_ids = [f\"eta_l{layer}_n{neuron}_r{derivative_order}\" for neuron in range(len(lower))]\n", + " normalized = rho / scale if scale > 0.0 else np.zeros_like(rho)\n", + " for neuron in range(len(lower)):\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"split_id\": SPLIT_ID,\n", + " \"cell_id\": 0, \"cell_weight\": 1.0, \"layer\": layer,\n", + " \"neuron\": neuron, \"derivative_order\": derivative_order,\n", + " \"preactivation_lower\": lower[neuron],\n", + " \"preactivation_upper\": upper[neuron],\n", + " \"preactivation_midpoint\": midpoint[neuron],\n", + " \"preactivation_radius\": radius[neuron],\n", + " \"preactivation_width\": width[neuron],\n", + " \"approximation_kind\": kind,\n", + " \"approximation_error_radius\": rho[neuron],\n", + " \"approximation_error_diameter\": 2.0 * rho[neuron],\n", + " \"activation_scale\": scale,\n", + " \"normalized_approximation_radius\": normalized[neuron],\n", + " \"noise_symbol_id\": noise_ids[neuron],\n", + " \"shared_noise_group\": pd.NA, \"status\": \"ok\",\n", + " })\n", + " activation = pd.DataFrame(rows, columns=ACTIVATION_COLUMNS).sort_values(\n", + " [\"cell_id\", \"layer\", \"neuron\", \"derivative_order\"], kind=\"stable\", ignore_index=True,\n", + " )\n", + " wide_table = pd.DataFrame(wide)[[\"neuron\"] + sorted([key for key in wide if key.startswith(\"layer_\")])]\n", + " return activation, wide_table\n", + "\n", + "\n", + "def _pz_complexity_row(result, quantity, pz):\n", + " kinds = tuple(pz.noise_kinds)\n", + " domain = {index for index, kind in enumerate(kinds) if kind == \"domain\"}\n", + " approximation = set(range(len(kinds))) - domain\n", + " support = list(pz.terms)\n", + " mixed = sum(\n", + " int(any(exp[index] for index in domain) and any(exp[index] for index in approximation))\n", + " for exp in support\n", + " )\n", + " coefficient_dimension = int(np.prod(pz.shape)) if pz.shape else 1\n", + " return {\n", + " \"run_id\": result[\"run_id\"], \"quantity\": quantity,\n", + " \"n_alpha\": len(domain), \"n_eta\": len(approximation),\n", + " \"n_monomials\": len(support), \"n_mixed_monomials\": mixed,\n", + " \"max_degree\": max((sum(exp) for exp in support), default=0),\n", + " \"n_coefficients\": coefficient_dimension * len(support), \"status\": \"ok\",\n", + " }\n", + "\n", + "\n", + "def _complexity_table(result):\n", + " if \"Y_pz\" in result:\n", + " rows = [_pz_complexity_row(result, \"Y\", result[\"Y_pz\"]),\n", + " _pz_complexity_row(result, \"J\", result[\"J_pz\"])]\n", + " else:\n", + " rows = [{\"run_id\": result[\"run_id\"], \"quantity\": quantity, \"status\": \"not_implemented\"}\n", + " for quantity in (\"Y\", \"J\")]\n", + " rows.append({\"run_id\": result[\"run_id\"], \"quantity\": \"H\", \"status\": \"not_implemented\"})\n", + " return pd.DataFrame(rows).reindex(columns=COMPLEXITY_COLUMNS)\n", + "\n", + "\n", + "def _soundness_table(result):\n", + " rows = []\n", + " exact_values = {\n", + " \"Y\": prediction.detach().cpu().reshape(-1, 1).numpy(),\n", + " \"J\": network_jacobian.detach().cpu().reshape(-1, 1, DIM).numpy(),\n", + " }\n", + " for quantity, enclosure in ((\"Y\", result[\"Y\"]), (\"J\", result[\"J\"])):\n", + " lower = np.asarray(_tensor(enclosure.lower), dtype=float)\n", + " upper = np.asarray(_tensor(enclosure.upper), dtype=float)\n", + " values = exact_values[quantity]\n", + " violation = np.maximum(np.maximum(lower - values, values - upper), 0.0)\n", + " endpoints = np.concatenate([lower.reshape(-1), upper.reshape(-1)])\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"quantity\": quantity,\n", + " \"sample_count\": values.shape[0],\n", + " \"failure_count\": int(np.count_nonzero(np.any(violation > 0.0, axis=tuple(range(1, violation.ndim))))),\n", + " \"max_violation\": float(np.max(violation)),\n", + " \"invalid_interval_count\": int(np.count_nonzero(lower > upper)),\n", + " \"nan_endpoint_count\": int(np.count_nonzero(np.isnan(endpoints))),\n", + " \"infinite_endpoint_count\": int(np.count_nonzero(np.isinf(endpoints))),\n", + " \"status\": \"ok\",\n", + " })\n", + " return pd.DataFrame(rows, columns=SOUNDNESS_COLUMNS)\n", + "\n", + "\n", + "def _metrics_table(result, cell_intervals, norms, complexity, timings, soundness):\n", + " base = _base_metric(result)\n", + " rows = []\n", + " for quantity, derivative_order in ((\"Y\", 0), (\"J\", 1)):\n", + " family = cell_intervals[cell_intervals.quantity == quantity]\n", + " rows.extend(_family_metrics(base, result[\"method_id\"], quantity,\n", + " family.lower, family.upper, derivative_order))\n", + " for metric in (\"mean_width\", \"max_width\", \"q50_width\", \"q90_width\", \"q99_width\",\n", + " \"mean_global_normalized_radius\", \"max_global_normalized_radius\"):\n", + " rows.append({**base, \"quantity\": \"H\", \"metric\": metric, \"aggregation\": \"none\",\n", + " \"derivative_order\": 2, \"value\": pd.NA, \"unit\": pd.NA,\n", + " \"status\": \"not_implemented\"})\n", + " for _, row in norms.iterrows():\n", + " quantity = row[\"norm\"] + (\"_sq\" if row[\"squared\"] else \"\")\n", + " for metric in (\"lower\", \"upper\", \"width\", \"relative_norm_width\",\n", + " \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\",\n", + " \"domain_volume_normalized_width\"):\n", + " value = row[\"relative_width\"] if metric == \"relative_norm_width\" else row[metric]\n", + " rows.append({**base, \"quantity\": quantity, \"metric\": metric,\n", + " \"aggregation\": \"none\", \"derivative_order\": pd.NA,\n", + " \"value\": value, \"unit\": \"dimensionless\", \"status\": row[\"status\"]})\n", + " for residual_quantity, derivative_order in ((\"PDE_residual\", 2), (\"boundary_residual\", 0), (\"initial_residual\", 0)):\n", + " rows.append({**base, \"quantity\": residual_quantity, \"metric\": \"linf_upper\",\n", + " \"aggregation\": \"max\", \"derivative_order\": derivative_order, \"value\": pd.NA,\n", + " \"unit\": \"dimensionless\", \"status\": \"not_implemented\"})\n", + " for _, row in complexity.iterrows():\n", + " for column, metric in ((\"n_alpha\", \"n_alpha\"), (\"n_eta\", \"n_eta\"),\n", + " (\"n_monomials\", \"n_monomials\"),\n", + " (\"n_mixed_monomials\", \"n_mixed_monomials\"),\n", + " (\"max_degree\", \"max_degree\")):\n", + " rows.append({**base, \"quantity\": \"complexity\", \"metric\": metric,\n", + " \"aggregation\": \"none\", \"derivative_order\": pd.NA,\n", + " \"value\": row[column], \"unit\": \"dimensionless\", \"status\": row[\"status\"],\n", + " \"layer\": pd.NA, \"neuron\": pd.NA})\n", + " for _, row in timings.iterrows():\n", + " rows.append({**base, \"quantity\": \"runtime\", \"metric\": \"seconds\",\n", + " \"aggregation\": row[\"stage\"], \"derivative_order\": pd.NA,\n", + " \"value\": row[\"seconds\"], \"unit\": \"seconds\", \"status\": row[\"status\"]})\n", + " rows.append({**base, \"quantity\": \"memory\", \"metric\": \"bytes\",\n", + " \"aggregation\": \"peak\", \"derivative_order\": pd.NA,\n", + " \"value\": pd.NA, \"unit\": \"bytes\", \"status\": \"not_implemented\"})\n", + " for _, row in soundness.iterrows():\n", + " for column, metric in ((\"failure_count\", \"failure_count\"), (\"max_violation\", \"max_violation\")):\n", + " rows.append({**base, \"quantity\": \"soundness\", \"metric\": metric,\n", + " \"aggregation\": row[\"quantity\"], \"derivative_order\": pd.NA,\n", + " \"value\": row[column], \"unit\": \"dimensionless\", \"status\": row[\"status\"]})\n", + " return pd.DataFrame(rows).reindex(columns=METRICS_COLUMNS)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-write", + "metadata": {}, + "outputs": [], + "source": [ + "medium_outputs = {}\n", + "try:\n", + " git_commit = subprocess.check_output(\n", + " [\"git\", \"rev-parse\", \"HEAD\"], cwd=repo_root, text=True\n", + " ).strip()\n", + "except (OSError, subprocess.CalledProcessError):\n", + " git_commit = None\n", + "\n", + "for method_id, result in medium_results.items():\n", + " method_dir = OUTPUT_ROOT / method_id\n", + " method_dir.mkdir(parents=True, exist_ok=True)\n", + " cell_intervals = _cell_interval_table(result)\n", + " norms = _norm_table(result)\n", + " activation, layer_radius_y = _activation_tables(result)\n", + " complexity = _complexity_table(result)\n", + " timings = pd.DataFrame([\n", + " {\"run_id\": result[\"run_id\"], \"stage\": stage, \"seconds\": seconds, \"status\": \"ok\"}\n", + " for stage, seconds in result[\"timings\"].items()\n", + " ], columns=TIMING_COLUMNS)\n", + " soundness = _soundness_table(result)\n", + " metrics = _metrics_table(result, cell_intervals, norms, complexity, timings, soundness)\n", + "\n", + " assert list(metrics.columns) == METRICS_COLUMNS\n", + " assert list(cell_intervals.columns) == CELL_INTERVAL_COLUMNS\n", + " assert list(norms.columns) == NORM_COLUMNS\n", + " assert list(activation.columns) == ACTIVATION_COLUMNS\n", + " assert list(layer_radius_y.columns) == [\"neuron\", \"layer_0\", \"layer_1\", \"layer_2\"]\n", + " assert list(complexity.columns) == COMPLEXITY_COLUMNS\n", + " assert list(timings.columns) == TIMING_COLUMNS\n", + " assert list(soundness.columns) == SOUNDNESS_COLUMNS\n", + " assert activation.equals(activation.sort_values(\n", + " [\"cell_id\", \"layer\", \"neuron\", \"derivative_order\"], kind=\"stable\", ignore_index=True\n", + " ))\n", + " assert np.isclose(activation.cell_weight.groupby([activation.cell_id]).first().sum(), 1.0)\n", + " assert not bool((norms.relative_width < 0.0).any() or (norms.relative_width > 1.0).any())\n", + " assert int(soundness.failure_count.sum()) == 0\n", + " assert np.allclose(\n", + " cell_intervals.local_relative_width,\n", + " 2.0 * cell_intervals.local_relative_radius,\n", + " )\n", + " assert list(zip(norms[\"norm\"], norms[\"squared\"])) == [\n", + " (\"L2\", 1), (\"L2\", 0), (\"W12\", 1), (\"W12\", 0),\n", + " ]\n", + " assert not norms[[\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + " ]].isna().any().any()\n", + " norm_scales = np.where(norms.squared.astype(bool), VOLUME, SQRT_VOLUME)\n", + " assert np.all(norms.domain_volume.to_numpy() == VOLUME)\n", + " assert np.allclose(norms.domain_volume_normalized_lower, norms.lower / norm_scales)\n", + " assert np.allclose(norms.domain_volume_normalized_upper, norms.upper / norm_scales)\n", + " assert np.allclose(norms.domain_volume_normalized_width, norms.width / norm_scales)\n", + "\n", + " metadata = {\n", + " \"schema_version\": SCHEMA_VERSION,\n", + " \"benchmark_level\": BENCHMARK_LEVEL,\n", + " \"problem_id\": PROBLEM_ID,\n", + " \"model_id\": MODEL_ID,\n", + " \"method_id\": method_id,\n", + " \"git_commit\": git_commit,\n", + " \"dtype\": str(torch.get_default_dtype()).replace(\"torch.\", \"\"),\n", + " \"device\": \"cpu\",\n", + " \"quantile_interpolation\": \"linear\",\n", + " \"cell_average\": \"volume_weighted\",\n", + " \"timestamp_utc\": datetime.now(timezone.utc).isoformat(),\n", + " }\n", + " (method_dir / \"benchmark_metadata.json\").write_text(\n", + " json.dumps(metadata, indent=2) + \"\\n\", encoding=\"utf-8\"\n", + " )\n", + " for filename, frame in {\n", + " \"metrics.csv\": metrics,\n", + " \"cell_intervals.csv\": cell_intervals,\n", + " \"norms.csv\": norms,\n", + " \"complexity.csv\": complexity,\n", + " \"timings.csv\": timings,\n", + " \"soundness.csv\": soundness,\n", + " \"activation_approximation.csv\": activation,\n", + " \"layer_normalized_radius_Y.csv\": layer_radius_y,\n", + " }.items():\n", + " frame.to_csv(method_dir / filename, index=False, na_rep=\"NA\")\n", + " medium_outputs[method_id] = {\n", + " \"metadata\": metadata, \"metrics\": metrics, \"cell_intervals\": cell_intervals,\n", + " \"norms\": norms, \"complexity\": complexity, \"timings\": timings,\n", + " \"soundness\": soundness, \"activation\": activation,\n", + " \"layer_radius_Y\": layer_radius_y,\n", + " }\n", + "\n", + "medium_norms_summary = pd.concat([\n", + " output[\"norms\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\n", + " \"method_id\", \"norm\", \"squared\", \"lower\", \"upper\", \"width\", \"relative_width\",\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\", \"status\"\n", + "]]\n", + "medium_norms_summary" + ] + }, + { + "cell_type": "markdown", + "id": "medium-norms-heading", + "metadata": {}, + "source": [ + "### Canonical norm table\n", + "\n", + "Raw squared and unsquared intervals are shown together. Schema 1.2 stores\n", + "the physical domain volume and the mandatory domain-volume-normalized lower\n", + "endpoint, upper endpoint, and width directly in every canonical `norms.csv`\n", + "row. Squared rows are divided by $|\\Omega|$ and unsquared rows by\n", + "$|\\Omega|^{1/2}$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-normalized-norms", + "metadata": {}, + "outputs": [], + "source": [ + "normalized_norm_view = medium_norms_summary[medium_norms_summary.squared == 0].copy()\n", + "normalized_norm_view[[\n", + " \"method_id\", \"norm\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + " \"relative_width\",\n", + "]]" + ] + }, + { + "cell_type": "markdown", + "id": "medium-final-heading", + "metadata": {}, + "source": [ + "### Final enclosure and complexity summary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-enclosure-summary", + "metadata": {}, + "outputs": [], + "source": [ + "medium_enclosure_summary = pd.concat([\n", + " output[\"metrics\"].query(\n", + " \"quantity in ['Y', 'J'] and metric in ['mean_width', 'max_width', \"\n", + " \"'mean_global_normalized_radius', 'max_global_normalized_radius', \"\n", + " \"'mean_frobenius_width', 'max_frobenius_width']\"\n", + " )[[\"method_id\", \"quantity\", \"metric\", \"aggregation\", \"value\", \"status\"]]\n", + " for output in medium_outputs.values()\n", + "], ignore_index=True)\n", + "medium_enclosure_summary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-complexity-view", + "metadata": {}, + "outputs": [], + "source": [ + "pd.concat([\n", + " output[\"complexity\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\"method_id\"] + COMPLEXITY_COLUMNS]" + ] + }, + { + "cell_type": "markdown", + "id": "medium-activation-heading", + "metadata": {}, + "source": [ + "### Per-neuron activation diagnostics\n", + "\n", + "Each row below is canonical data from `activation_approximation.csv`. The\n", + "tables are ordered by hidden layer, neuron, and derivative order. The interval\n", + "method uses the constant interval-hull enclosure; the affine-PZ method reports\n", + "the certified approximation-error radius multiplying its fresh approximation\n", + "noise symbol." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-activation-summary", + "metadata": {}, + "outputs": [], + "source": [ + "medium_activation_summary = pd.concat([\n", + " output[\"activation\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)\n", + "medium_activation_layer_summary = (\n", + " medium_activation_summary\n", + " .groupby([\"method_id\", \"layer\", \"derivative_order\"], sort=True)\n", + " .agg(\n", + " preactivation_radius_mean=(\"preactivation_radius\", \"mean\"),\n", + " preactivation_radius_max=(\"preactivation_radius\", \"max\"),\n", + " approximation_error_radius_mean=(\"approximation_error_radius\", \"mean\"),\n", + " approximation_error_radius_max=(\"approximation_error_radius\", \"max\"),\n", + " normalized_approximation_radius_mean=(\"normalized_approximation_radius\", \"mean\"),\n", + " normalized_approximation_radius_max=(\"normalized_approximation_radius\", \"max\"),\n", + " )\n", + " .reset_index()\n", + ")\n", + "medium_activation_layer_summary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-activation-full", + "metadata": {}, + "outputs": [], + "source": [ + "activation_views = {}\n", + "for method_id, output in medium_outputs.items():\n", + " for derivative_order in (0, 1):\n", + " view = output[\"activation\"].query(\"derivative_order == @derivative_order\")[\n", + " [\"layer\", \"neuron\", \"preactivation_lower\", \"preactivation_upper\",\n", + " \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\", \"status\"]\n", + " ].reset_index(drop=True)\n", + " activation_views[(method_id, derivative_order)] = view\n", + " display(method_id, f\"derivative_order={derivative_order}\", view)" + ] + }, + { + "cell_type": "markdown", + "id": "medium-layer-radius-heading", + "metadata": {}, + "source": [ + "### Required layerwise normalized postactivation-radius tables\n", + "\n", + "For each hidden layer, the denominator is the maximum magnitude of the\n", + "postactivation interval hull over all neurons in that layer. The row and\n", + "column indices are zero-based." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-layer-radius-view", + "metadata": {}, + "outputs": [], + "source": [ + "for method_id, output in medium_outputs.items():\n", + " display(method_id, output[\"layer_radius_Y\"])" + ] + }, + { + "cell_type": "markdown", + "id": "medium-ranked-heading", + "metadata": {}, + "source": [ + "### Ranked worst activation enclosures" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-ranked-view", + "metadata": {}, + "outputs": [], + "source": [ + "largest_absolute_activation_radii = (\n", + " medium_activation_summary\n", + " .sort_values(\"approximation_error_radius\", ascending=False, kind=\"stable\")\n", + " [[\"method_id\", \"layer\", \"neuron\", \"derivative_order\", \"preactivation_lower\",\n", + " \"preactivation_upper\", \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\"]]\n", + " .head(20).reset_index(drop=True)\n", + ")\n", + "largest_normalized_activation_radii = (\n", + " medium_activation_summary\n", + " .sort_values(\"normalized_approximation_radius\", ascending=False, kind=\"stable\")\n", + " [[\"method_id\", \"layer\", \"neuron\", \"derivative_order\", \"preactivation_lower\",\n", + " \"preactivation_upper\", \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\"]]\n", + " .head(20).reset_index(drop=True)\n", + ")\n", + "largest_absolute_activation_radii, largest_normalized_activation_radii" + ] + }, + { + "cell_type": "markdown", + "id": "medium-soundness-heading", + "metadata": {}, + "source": [ + "### Validity and sampled-containment diagnostics\n", + "\n", + "Sampling is only a diagnostic and is not presented as a proof of soundness.\n", + "Any nonzero failure count would, however, invalidate the corresponding\n", + "enclosure." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "medium-soundness-view", + "metadata": {}, + "outputs": [], + "source": [ + "pd.concat([\n", + " output[\"soundness\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\"method_id\"] + SOUNDNESS_COLUMNS]" + ] + }, + { + "cell_type": "markdown", + "id": "medium-observed-results", + "metadata": {}, + "source": [ + "### Observed medium-benchmark result\n", + "\n", + "The current implementation gives the following single-cell, domain-volume-normalized norm intervals:\n", + "\n", + "| method | $L^2/|\\Omega|^{1/2}$ | $W^{1,2}/|\\Omega|^{1/2}$ |\n", + "|---|---:|---:|\n", + "| interval | $[0,7.979523]$ | $[0,88.846805]$ |\n", + "| affine PZ Top-96, symbolic integration | $[0,2.676831]$ | $[0,85.167723]$ |\n", + "\n", + "The affine-PZ final function-value hull has width $5.415735$ versus $15.909816$ for interval propagation. The corresponding mean Jacobian-entry widths are $16.809482$ and $17.548815$.\n", + "\n", + "Layerwise mean approximation-error radii $(\\rho^{(0)},\\rho^{(1)})$ for affine PZ are $(0.074179,0.275129)$, $(0.092422,0.305928)$, and $(0.133023,0.359351)$. For the constant interval-hull approximation they are $(0.741374,0.279480)$, $(0.999526,0.499543)$, and $(0.999971,0.499973)$. Thus the affine value enclosure yields a large improvement at every layer, while the affine $\\tanh'$ enclosure becomes only marginally better than its interval hull as the zero-crossing bump widens.\n", + "\n", + "Under schema 1.2 these values are no longer only a derived display: every squared and unsquared row in `norms.csv` stores the physical domain volume and its domain-volume-normalized lower endpoint, upper endpoint, and width. The squared normalized upper endpoints are $63.672784$ and $7893.754708$ for interval propagation, and $7.165422$ and $7253.541008$ for affine PZ. The complete per-neuron values, local relative widths, and familywise normalized radii are generated by the canonical cells above." + ] + }, + { + "cell_type": "markdown", + "id": "8b95b0f5", + "metadata": {}, + "source": [ + "## Interpretation\n", + "\n", + "- The checkpoint is a meaningful PDE candidate: sampled relative solution error is below one percent, while residual and boundary errors are independently reported.\n", + "- All reduced polynomial methods meet the three-second target on one CPU thread.\n", + "- The final raw norms are extremely small only because $|\\Omega|^{1/2}=0.2^{50}$. Volume-normalized bounds should be compared with the Monte Carlo RMS norms, but they are not relative errors.\n", + "- Standard PINN training does **not** automatically yield a certification-friendly parameterization. The sampled normalized $W^{1,2}$ norm is modest, but all single-cell certified lower bounds are zero and the upper bounds are much larger. The per-neuron $\\tanh'$ residual table shows sizeable initial derivative-approximation radii in every hidden layer, and the accumulated Jacobian remainder is then amplified by later linear maps. This explains why preserving more Top-$k$ terms or PCA directions yields only a small improvement.\n", + "- Squared $L^2$ and $W^{1,2}$ integration returns a scalar PZ. Pointwise residual uncertainty is converted to a fresh integrated global generator, and only this final PZ is intervalized before the square root.\n", + "- Top-192 is the tightest configuration that robustly remains below three seconds here: its normalized $W^{1,2}$ interval is $[0,85.0694]$, versus $[0,85.3702]$ for Top-96 and $[0,88.8468]$ for intervals. The gain from 96 to 192 terms is only about $0.35\\%$, so support growth is already saturating.\n", + "- This is an application-level finding: improving certificate-aware training, activation enclosures, or domain decomposition is more important here than simply increasing the retained support.\n", + "- Degree-64 coincides with Top-64 because the retained terms are degree one. PCA-64 gives only a small improvement relative to its added runtime. These outcomes are reported rather than selected away." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/pinn_100d_poisson_shallow_300_hybrid_certification.ipynb b/notebooks/pinn_100d_poisson_shallow_300_hybrid_certification.ipynb new file mode 100644 index 0000000..071fa45 --- /dev/null +++ b/notebooks/pinn_100d_poisson_shallow_300_hybrid_certification.ipynb @@ -0,0 +1,5050 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "shallow-00", + "metadata": {}, + "source": [ + "# Shallow 100-dimensional Poisson PINN: affine–quadratic hybrid certification\n", + "\n", + "This notebook trains (when explicitly requested) and certifies a shallow\n", + "$100$–$300$–$1$ tanh PINN for the same manufactured Poisson problem as the\n", + "deep benchmark. The committed checkpoint is loaded by default. Training uses\n", + "only the PDE residual and sampled Dirichlet boundary loss; the exact solution\n", + "is reserved for validation.\n", + "\n", + "The primary certified comparison uses one global cell on\n", + "$[-0.1,0.1]^{100}$:\n", + "\n", + "- interval propagation;\n", + "- affine-PZ value/one-jet propagation with symbolic integration;\n", + "- the affine–quadratic hybrid PZ one-jet, using a certified quadratic\n", + " approximation of $\\tanh'$ only on zero-crossing preactivation intervals\n", + " whose relative affine slope is at most $0.01$; the primary hybrid run is\n", + " uncompressed and uses scalar-output reverse-mode Jacobian propagation plus\n", + " direct structured symbolic integration.\n", + "\n", + "All canonical outputs implement schema 1.2 of\n", + "`docs/diagnostics_and_metrics_glossary.tex`." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "shallow-01", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "torch=2.8.0+cpu, dtype=torch.float64, threads=1\n", + "checkpoint=notebooks/checkpoints/pinn_100d_poisson_shallow_300.pt\n" + ] + } + ], + "source": [ + "from __future__ import annotations\n", + "\n", + "import math\n", + "import random\n", + "import sys\n", + "from pathlib import Path\n", + "from time import perf_counter\n", + "\n", + "import numpy as np\n", + "import torch\n", + "from torch import nn\n", + "\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / 'src' / 'intervalnets').exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "if str(repo_root / 'src') not in sys.path:\n", + " sys.path.insert(0, str(repo_root / 'src'))\n", + "\n", + "from intervalnets import (\n", + " IntervalTensor,\n", + " PZIntegrationCell,\n", + " enable_interval_eval,\n", + " integrate_pz_onejet_squared,\n", + " integrate_pz_value_squared,\n", + " integrate_shallow_hybrid_onejet_squared,\n", + " load_tanh_mlp_checkpoint,\n", + " sequential_value_jacobian_laplacian,\n", + " shallow_scalar_hybrid_onejet_reverse,\n", + ")\n", + "\n", + "torch.set_num_threads(1)\n", + "torch.set_default_dtype(torch.float64)\n", + "enable_interval_eval()\n", + "\n", + "DIM = 100\n", + "HALF_WIDTH = 0.1\n", + "HIDDEN = (300,)\n", + "SEED = 20260804\n", + "K1 = 2.5\n", + "K2 = 1.75\n", + "COS_AMPLITUDE = 0.35\n", + "CHECKPOINT = repo_root / 'notebooks' / 'checkpoints' / 'pinn_100d_poisson_shallow_300.pt'\n", + "\n", + "random.seed(SEED)\n", + "np.random.seed(SEED)\n", + "torch.manual_seed(SEED)\n", + "print(f'torch={torch.__version__}, dtype={torch.get_default_dtype()}, threads={torch.get_num_threads()}')\n", + "print(f'checkpoint={CHECKPOINT.relative_to(repo_root)}')" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-02", + "metadata": {}, + "source": [ + "## PDE, architecture, and reproducible checkpoint" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "shallow-03", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(30601,\n", + " Sequential(\n", + " (0): Linear(in_features=100, out_features=300, bias=True)\n", + " (1): Tanh()\n", + " (2): Linear(in_features=300, out_features=1, bias=True)\n", + " ))" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def dense_directions(dim=DIM):\n", + " a = torch.ones(dim)\n", + " a /= torch.linalg.vector_norm(a)\n", + " b = torch.tensor([1.0 if i % 2 == 0 else -1.0 for i in range(dim)])\n", + " b -= torch.dot(a, b) * a\n", + " b /= torch.linalg.vector_norm(b)\n", + " return a, b\n", + "\n", + "\n", + "A, B = dense_directions()\n", + "\n", + "\n", + "def exact_solution(x):\n", + " return (torch.sin(K1 * (x @ A)) + COS_AMPLITUDE * torch.cos(K2 * (x @ B))).unsqueeze(-1)\n", + "\n", + "\n", + "def exact_gradient(x):\n", + " s = x @ A\n", + " t = x @ B\n", + " return K1 * torch.cos(K1 * s).unsqueeze(-1) * A - COS_AMPLITUDE * K2 * torch.sin(K2 * t).unsqueeze(-1) * B\n", + "\n", + "\n", + "def forcing(x):\n", + " return (K1**2 * torch.sin(K1 * (x @ A)) + COS_AMPLITUDE * K2**2 * torch.cos(K2 * (x @ B))).unsqueeze(-1)\n", + "\n", + "\n", + "def make_model():\n", + " layers, previous = [], DIM\n", + " for width in HIDDEN:\n", + " layers.extend([nn.Linear(previous, width), nn.Tanh()])\n", + " previous = width\n", + " layers.append(nn.Linear(previous, 1))\n", + " model = nn.Sequential(*layers)\n", + " for layer in model:\n", + " if isinstance(layer, nn.Linear):\n", + " nn.init.xavier_uniform_(layer.weight)\n", + " nn.init.zeros_(layer.bias)\n", + " return model\n", + "\n", + "\n", + "def sample_interior(n, generator):\n", + " return (2.0 * torch.rand((n, DIM), generator=generator) - 1.0) * HALF_WIDTH\n", + "\n", + "\n", + "def sample_boundary(n, generator):\n", + " x = sample_interior(n, generator)\n", + " coordinate = torch.randint(DIM, (n,), generator=generator)\n", + " sign = torch.where(torch.rand(n, generator=generator) < 0.5, -1.0, 1.0)\n", + " x[torch.arange(n), coordinate] = HALF_WIDTH * sign\n", + " return x\n", + "\n", + "\n", + "def pinn_residual(model, x):\n", + " value, jacobian, laplacian = sequential_value_jacobian_laplacian(model, x)\n", + " return value, jacobian, -laplacian - forcing(x)\n", + "\n", + "\n", + "model = make_model()\n", + "sum(parameter.numel() for parameter in model.parameters()), model" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "shallow-04", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'loaded_checkpoint': True,\n", + " 'architecture': [100, 300, 1],\n", + " 'training_steps': 2200,\n", + " 'training_seconds': 262.240752271,\n", + " 'training_records': []}" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def train_pinn(model, steps=2200, batch_size=512, lr=2e-3):\n", + " generator = torch.Generator().manual_seed(SEED + 1)\n", + " optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n", + " scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=steps, eta_min=1e-4)\n", + " history = []\n", + " model.train()\n", + " for step in range(1, steps + 1):\n", + " interior = sample_interior(batch_size, generator)\n", + " boundary = sample_boundary(batch_size, generator)\n", + " _, _, residual = pinn_residual(model, interior)\n", + " boundary_error = model(boundary) - exact_solution(boundary)\n", + " residual_loss = residual.square().mean()\n", + " boundary_loss = boundary_error.square().mean()\n", + " loss = residual_loss + 20.0 * boundary_loss\n", + " optimizer.zero_grad(set_to_none=True)\n", + " loss.backward()\n", + " torch.nn.utils.clip_grad_norm_(model.parameters(), 10.0)\n", + " optimizer.step()\n", + " scheduler.step()\n", + " if step == 1 or step % 100 == 0:\n", + " history.append({\n", + " 'step': step,\n", + " 'loss': float(loss.detach()),\n", + " 'residual_loss': float(residual_loss.detach()),\n", + " 'boundary_loss': float(boundary_loss.detach()),\n", + " })\n", + " model.eval()\n", + " return history\n", + "\n", + "\n", + "RETRAIN = False\n", + "if RETRAIN:\n", + " training_start = perf_counter()\n", + " training_history = train_pinn(model)\n", + " training_seconds = perf_counter() - training_start\n", + " checkpoint_payload = {\n", + " 'state_dict': model.state_dict(),\n", + " 'seed': SEED,\n", + " 'architecture': [DIM, *HIDDEN, 1],\n", + " 'problem': 'poisson_100d_ridge',\n", + " 'domain_half_width': HALF_WIDTH,\n", + " 'training': {\n", + " 'steps': 2200,\n", + " 'batch_size': 512,\n", + " 'optimizer': 'Adam',\n", + " 'initial_lr': 2e-3,\n", + " 'final_lr': 1e-4,\n", + " 'boundary_weight': 20.0,\n", + " 'seconds': training_seconds,\n", + " 'history': training_history,\n", + " },\n", + " }\n", + " CHECKPOINT.parent.mkdir(parents=True, exist_ok=True)\n", + " torch.save(checkpoint_payload, CHECKPOINT)\n", + "else:\n", + " model = load_tanh_mlp_checkpoint(CHECKPOINT)\n", + " checkpoint_payload = torch.load(CHECKPOINT, map_location='cpu', weights_only=True)\n", + " training_history = []\n", + "\n", + "{'loaded_checkpoint': not RETRAIN,\n", + " 'architecture': checkpoint_payload.get('architecture'),\n", + " 'training_steps': checkpoint_payload.get('training', {}).get('steps'),\n", + " 'training_seconds': checkpoint_payload.get('training', {}).get('seconds'),\n", + " 'training_records': training_history[-3:]}\n" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-05", + "metadata": {}, + "source": [ + "## Independent sampled validation" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "shallow-06", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'solution_RMSE': 0.002764527838374597,\n", + " 'solution_relative_L2_error': 0.007386462596925562,\n", + " 'solution_max_sample_error': 0.027334537779656998,\n", + " 'gradient_RMSE': 0.007121730317469054,\n", + " 'PDE_residual_RMSE': 0.013159160676084783,\n", + " 'boundary_RMSE': 0.002782393594360182,\n", + " 'network_normalized_L2_MC': 0.374310346352375,\n", + " 'network_normalized_W12_MC': 2.502324834012471,\n", + " 'exact_normalized_L2_MC': 0.3742695237535306,\n", + " 'exact_normalized_W12_MC': 2.503330852539675}" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "validation_generator = torch.Generator().manual_seed(SEED + 222)\n", + "interior = sample_interior(16384, validation_generator)\n", + "boundary = sample_boundary(16384, validation_generator)\n", + "with torch.no_grad():\n", + " prediction, network_jacobian, residual = pinn_residual(model, interior)\n", + " target = exact_solution(interior)\n", + " target_gradient = exact_gradient(interior)\n", + " boundary_error = model(boundary) - exact_solution(boundary)\n", + " error = prediction - target\n", + " empirical_network_l2 = prediction.square().mean().sqrt()\n", + " empirical_network_w12 = (prediction.square() + network_jacobian.square().sum(dim=(-2, -1), keepdim=True)).mean().sqrt()\n", + " empirical_exact_l2 = target.square().mean().sqrt()\n", + " empirical_exact_w12 = (target.square() + target_gradient.square().sum(dim=-1, keepdim=True)).mean().sqrt()\n", + "\n", + "validation = {\n", + " 'solution_RMSE': float(error.square().mean().sqrt()),\n", + " 'solution_relative_L2_error': float(error.square().mean().sqrt() / target.square().mean().sqrt()),\n", + " 'solution_max_sample_error': float(error.abs().max()),\n", + " 'gradient_RMSE': float((network_jacobian.squeeze(1) - target_gradient).square().mean().sqrt()),\n", + " 'PDE_residual_RMSE': float(residual.square().mean().sqrt()),\n", + " 'boundary_RMSE': float(boundary_error.square().mean().sqrt()),\n", + " 'network_normalized_L2_MC': float(empirical_network_l2),\n", + " 'network_normalized_W12_MC': float(empirical_network_w12),\n", + " 'exact_normalized_L2_MC': float(empirical_exact_l2),\n", + " 'exact_normalized_W12_MC': float(empirical_exact_w12),\n", + "}\n", + "validation\n", + "if RETRAIN:\n", + " checkpoint_payload['validation'] = validation\n", + " torch.save(checkpoint_payload, CHECKPOINT)\n", + "validation\n" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-07", + "metadata": {}, + "source": [ + "## Domain and symbolic-integration helpers" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "shallow-08", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1.1258999068426271e-35" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "SQRT_VOLUME = (2.0 * HALF_WIDTH) ** (DIM / 2.0)\n", + "VOLUME = SQRT_VOLUME ** 2\n", + "DOMAIN = IntervalTensor.from_bounds([-HALF_WIDTH] * DIM, [HALF_WIDTH] * DIM)\n", + "\n", + "\n", + "def norm_interval(squared):\n", + " return math.sqrt(max(0.0, float(squared.lower))), math.sqrt(max(0.0, float(squared.upper)))\n", + "\n", + "\n", + "def interval_metrics(bounds, prefix):\n", + " lower, upper = map(float, bounds)\n", + " width = upper - lower\n", + " return {\n", + " f'{prefix}_lower': lower,\n", + " f'{prefix}_upper': upper,\n", + " f'{prefix}_absolute_width': width,\n", + " f'{prefix}_relative_width': width / max(abs(lower), abs(upper)) if max(abs(lower), abs(upper)) > 0.0 else 0.0,\n", + " f'{prefix}_normalized_lower': lower / SQRT_VOLUME,\n", + " f'{prefix}_normalized_upper': upper / SQRT_VOLUME,\n", + " f'{prefix}_normalized_absolute_width': width / SQRT_VOLUME,\n", + " }\n", + "\n", + "\n", + "def jacobian_width_metrics(enclosure):\n", + " lower = torch.as_tensor(enclosure.lower)\n", + " upper = torch.as_tensor(enclosure.upper)\n", + " widths = upper - lower\n", + " scales = torch.maximum(lower.abs(), upper.abs())\n", + " relative_widths = torch.where(scales > 0.0, widths / scales, 0.0)\n", + " return {\n", + " 'J_mean_component_width_before_integration': float(widths.mean()),\n", + " 'J_max_component_width_before_integration': float(widths.max()),\n", + " 'J_relative_mean_component_width_before_integration': float(relative_widths.mean()),\n", + " }\n", + "\n", + "\n", + "def benchmark_polynomial(model, strategy='topk', **kwargs):\n", + " cell = PZIntegrationCell.from_affine_box(DOMAIN)\n", + " start = perf_counter()\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain, return_trace=True, reduction_strategy=strategy, **kwargs\n", + " )\n", + " forward_s = perf_counter() - start\n", + " jacobian_metrics = jacobian_width_metrics(traced.final.J.interval_enclosure())\n", + " start = perf_counter()\n", + " l2_integrated_pz = integrate_pz_value_squared(traced.final.Y, cell, output='pz')\n", + " l2_squared = l2_integrated_pz.interval_enclosure()\n", + " l2_integration_s = perf_counter() - start\n", + " start = perf_counter()\n", + " w12_integrated_pz = integrate_pz_onejet_squared(traced.final, cell, output='pz')\n", + " w12_squared = w12_integrated_pz.interval_enclosure()\n", + " w12_integration_s = perf_counter() - start\n", + " return {\n", + " 'strategy': strategy,\n", + " **kwargs,\n", + " 'forward_s': forward_s,\n", + " 'L2_integration_s': l2_integration_s,\n", + " 'W12_integration_s': w12_integration_s,\n", + " 'total_W12_s': forward_s + w12_integration_s,\n", + " 'J_terms': len(traced.final.J.terms),\n", + " 'J_degree': max(map(sum, traced.final.J.terms), default=0),\n", + " 'noise_count': traced.final.J.num_noise,\n", + " 'L2_integrated_PZ_terms': len(l2_integrated_pz.terms),\n", + " 'W12_integrated_PZ_terms': len(w12_integrated_pz.terms),\n", + " 'W12_integrated_PZ_noise': w12_integrated_pz.num_noise,\n", + " **jacobian_metrics,\n", + " **interval_metrics(norm_interval(l2_squared), 'L2'),\n", + " **interval_metrics(norm_interval(w12_squared), 'W12'),\n", + " 'trace': traced.records,\n", + " }\n", + "\n", + "\n", + "SQRT_VOLUME" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-09", + "metadata": {}, + "source": [ + "## Glossary-conformant medium benchmark\n", + "\n", + "The medium benchmark writes the complete mini-benchmark outputs plus the\n", + "per-neuron activation table and the layerwise normalized postactivation-radius\n", + "table. The value activation always uses the certified affine enclosure. For\n", + "the hybrid derivative, the quadratic polynomial coefficient remains in the PZ\n", + "core and its certified approximation-error radius $\\rho_{0i}^{(1)}$ is kept\n", + "as one shared generator per neuron. The primary hybrid run is fully uncompressed:\n", + "it uses reverse-mode Jacobian propagation, retains every degree-one and degree-two\n", + "domain monomial, and performs direct symbolic integration of the structured square." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "shallow-10", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import subprocess\n", + "from datetime import datetime, timezone\n", + "\n", + "import pandas as pd\n", + "\n", + "from intervalnets import interval_forward\n", + "\n", + "SCHEMA_VERSION = \"1.2\"\n", + "BENCHMARK_LEVEL = \"medium\"\n", + "PROBLEM_ID = \"poisson_100d_ridge\"\n", + "MODEL_ID = \"pinn_100d_poisson_shallow_300_seed_20260804\"\n", + "SPLIT_ID = \"single_cell\"\n", + "OUTPUT_ROOT = repo_root / \"notebooks\" / \"benchmark_outputs\" / \"pinn_100d_poisson_shallow_300_medium\"\n", + "\n", + "METRICS_COLUMNS = [\n", + " \"schema_version\", \"benchmark_level\", \"problem_id\", \"model_id\", \"method_id\",\n", + " \"run_id\", \"split_id\", \"quantity\", \"metric\", \"aggregation\",\n", + " \"derivative_order\", \"layer\", \"neuron\", \"output_index\", \"input_index_a\",\n", + " \"input_index_b\", \"cell_id\", \"value\", \"unit\", \"status\",\n", + "]\n", + "CELL_INTERVAL_COLUMNS = [\n", + " \"run_id\", \"split_id\", \"cell_id\", \"cell_weight\", \"quantity\", \"output_index\",\n", + " \"input_index_a\", \"input_index_b\", \"lower\", \"upper\", \"midpoint\", \"radius\",\n", + " \"width\", \"magnitude\", \"mignitude\", \"local_relative_radius\",\n", + " \"global_normalized_radius\", \"sign_certified\", \"status\",\n", + " \"local_relative_width\",\n", + "]\n", + "NORM_COLUMNS = [\n", + " \"run_id\", \"norm\", \"squared\", \"lower\", \"upper\", \"width\", \"relative_width\",\n", + " \"value_contribution_upper\", \"gradient_contribution_upper\",\n", + " \"hessian_contribution_upper\", \"value_contribution_width\",\n", + " \"gradient_contribution_width\", \"hessian_contribution_width\", \"status\",\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + "]\n", + "ACTIVATION_COLUMNS = [\n", + " \"run_id\", \"split_id\", \"cell_id\", \"cell_weight\", \"layer\", \"neuron\",\n", + " \"derivative_order\", \"preactivation_lower\", \"preactivation_upper\",\n", + " \"preactivation_midpoint\", \"preactivation_radius\", \"preactivation_width\",\n", + " \"approximation_kind\", \"approximation_error_radius\",\n", + " \"approximation_error_diameter\", \"activation_scale\",\n", + " \"normalized_approximation_radius\", \"noise_symbol_id\", \"shared_noise_group\",\n", + " \"status\",\n", + "]\n", + "COMPLEXITY_COLUMNS = [\n", + " \"run_id\", \"quantity\", \"n_alpha\", \"n_eta\", \"n_monomials\",\n", + " \"n_mixed_monomials\", \"max_degree\", \"n_coefficients\", \"status\",\n", + "]\n", + "TIMING_COLUMNS = [\"run_id\", \"stage\", \"seconds\", \"status\"]\n", + "SOUNDNESS_COLUMNS = [\n", + " \"run_id\", \"quantity\", \"sample_count\", \"failure_count\", \"max_violation\",\n", + " \"invalid_interval_count\", \"nan_endpoint_count\", \"infinite_endpoint_count\",\n", + " \"status\",\n", + "]\n", + "\n", + "\n", + "def _tensor(value):\n", + " return torch.as_tensor(value, dtype=torch.get_default_dtype()).detach().cpu()\n", + "\n", + "\n", + "def _nonnegative_squared_interval(enclosure):\n", + " lower = max(0.0, float(enclosure.lower))\n", + " upper = max(0.0, float(enclosure.upper))\n", + " return lower, upper\n", + "\n", + "\n", + "def _outward_square(bounds):\n", + " lower, upper = (max(0.0, float(v)) for v in bounds)\n", + " return (\n", + " float(np.nextafter(lower * lower, -np.inf)) if lower else 0.0,\n", + " float(np.nextafter(upper * upper, np.inf)) if upper else 0.0,\n", + " )\n", + "\n", + "\n", + "def _interval_trace(model, domain):\n", + " current = domain\n", + " records = []\n", + " hidden_layer = 0\n", + " for child in model:\n", + " if isinstance(child, nn.Linear):\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " elif isinstance(child, nn.Tanh):\n", + " preactivation = current\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " records.append({\n", + " \"layer\": hidden_layer,\n", + " \"preactivation_lower\": _tensor(preactivation.lower).reshape(-1),\n", + " \"preactivation_upper\": _tensor(preactivation.upper).reshape(-1),\n", + " \"postactivation_lower\": _tensor(current.lower).reshape(-1),\n", + " \"postactivation_upper\": _tensor(current.upper).reshape(-1),\n", + " })\n", + " hidden_layer += 1\n", + " else:\n", + " current = interval_forward(child, current, enclosure_mode=\"box\")\n", + " return current, records\n", + "\n", + "\n", + "\n", + "def _run_medium_methods(model):\n", + " results = {}\n", + "\n", + " interval_start = perf_counter()\n", + " interval_l2 = model.lpnorm(DOMAIN, p=2.0, method=\"interval\")\n", + " interval_l2_s = perf_counter() - interval_start\n", + " interval_start = perf_counter()\n", + " interval_w12 = model.sobolev_norm(DOMAIN, p=2.0, order=1, method=\"interval\")\n", + " interval_w12_s = perf_counter() - interval_start\n", + " interval_start = perf_counter()\n", + " interval_y, interval_activation = _interval_trace(model, DOMAIN)\n", + " interval_j = model.eval_jacobian(DOMAIN)\n", + " interval_enclosure_s = perf_counter() - interval_start\n", + " results[\"interval\"] = {\n", + " \"method_id\": \"interval\",\n", + " \"run_id\": \"pinn100d_shallow300_medium_interval\",\n", + " \"Y\": interval_y,\n", + " \"J\": interval_j,\n", + " \"activation\": interval_activation,\n", + " \"norms\": {\n", + " \"L2\": (float(interval_l2.lower), float(interval_l2.upper)),\n", + " \"L2_sq\": _outward_square((interval_l2.lower, interval_l2.upper)),\n", + " \"W12\": (float(interval_w12.lower), float(interval_w12.upper)),\n", + " \"W12_sq\": _outward_square((interval_w12.lower, interval_w12.upper)),\n", + " },\n", + " \"timings\": {\n", + " \"L2_total\": interval_l2_s,\n", + " \"W12_total\": interval_w12_s,\n", + " \"final_enclosures\": interval_enclosure_s,\n", + " },\n", + " }\n", + "\n", + " cell = PZIntegrationCell.from_affine_box(DOMAIN)\n", + " configurations = {\n", + " \"affine_pz_topk96_symbolic\": {\n", + " \"run_id\": \"pinn100d_shallow300_medium_affine_pz_topk96_symbolic\",\n", + " \"derivative_enclosure\": \"affine\",\n", + " \"derivative_flatness_threshold\": 0.01,\n", + " },\n", + " \"hybrid_pz_topk96_B_symbolic\": {\n", + " \"run_id\": \"pinn100d_shallow300_medium_hybrid_pz_topk96_B_symbolic\",\n", + " \"derivative_enclosure\": \"quadratic_flat\",\n", + " \"derivative_flatness_threshold\": 0.01,\n", + " },\n", + " }\n", + " for method_id, config in configurations.items():\n", + " pz_start = perf_counter()\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain,\n", + " return_trace=True,\n", + " reduction_strategy=\"topk\",\n", + " max_terms=96,\n", + " reduction_variant=\"B\",\n", + " derivative_enclosure=config[\"derivative_enclosure\"],\n", + " derivative_flatness_threshold=config[\"derivative_flatness_threshold\"],\n", + " quadratic_certificate_subdivisions=64,\n", + " quadratic_compression_guard=False,\n", + " )\n", + " pz_forward_s = perf_counter() - pz_start\n", + " pz_l2_start = perf_counter()\n", + " pz_l2_integrated = integrate_pz_value_squared(traced.final.Y, cell, output=\"pz\")\n", + " pz_l2_sq = pz_l2_integrated.interval_enclosure()\n", + " pz_l2_s = perf_counter() - pz_l2_start\n", + " pz_w12_start = perf_counter()\n", + " pz_w12_integrated = integrate_pz_onejet_squared(traced.final, cell, output=\"pz\")\n", + " pz_w12_sq = pz_w12_integrated.interval_enclosure()\n", + " pz_w12_s = perf_counter() - pz_w12_start\n", + " pz_activation = []\n", + " hidden_layer = 0\n", + " for record in traced.records:\n", + " if record.layer_type != \"Tanh\":\n", + " continue\n", + " postactivation = record.value.interval_enclosure()\n", + " degrees = record.summary[\"tanh_prime_approximation_degrees\"].detach().cpu().numpy()\n", + " pz_activation.append({\n", + " \"layer\": hidden_layer,\n", + " \"preactivation_lower\": record.summary[\"preactivation_lower\"].detach().cpu(),\n", + " \"preactivation_upper\": record.summary[\"preactivation_upper\"].detach().cpu(),\n", + " \"postactivation_lower\": _tensor(postactivation.lower).reshape(-1),\n", + " \"postactivation_upper\": _tensor(postactivation.upper).reshape(-1),\n", + " \"rho0\": record.summary[\"tanh_approximation_radii\"].detach().cpu(),\n", + " \"rho1\": record.summary[\"tanh_prime_approximation_radii\"].detach().cpu(),\n", + " \"kind0\": np.full(len(degrees), \"affine\", dtype=object),\n", + " \"kind1\": np.where(degrees == 2, \"quadratic\", \"affine\"),\n", + " \"affine_rho1\": record.summary[\"tanh_prime_affine_radii\"].detach().cpu(),\n", + " \"quadratic_core_box_radius\": record.summary[\"tanh_prime_polynomial_reduction_radii\"].detach().cpu(),\n", + " \"relative_slope\": record.summary[\"tanh_prime_relative_slopes\"].detach().cpu(),\n", + " })\n", + " hidden_layer += 1\n", + " pz_l2_sq_bounds = _nonnegative_squared_interval(pz_l2_sq)\n", + " pz_w12_sq_bounds = _nonnegative_squared_interval(pz_w12_sq)\n", + " results[method_id] = {\n", + " \"method_id\": method_id,\n", + " \"run_id\": config[\"run_id\"],\n", + " \"Y\": traced.final.Y.interval_enclosure(),\n", + " \"J\": traced.final.J.interval_enclosure(),\n", + " \"Y_pz\": traced.final.Y,\n", + " \"J_pz\": traced.final.J,\n", + " \"activation\": pz_activation,\n", + " \"trace\": traced.records,\n", + " \"integrated_L2_pz\": pz_l2_integrated,\n", + " \"integrated_W12_pz\": pz_w12_integrated,\n", + " \"norms\": {\n", + " \"L2_sq\": pz_l2_sq_bounds,\n", + " \"L2\": norm_interval(pz_l2_sq),\n", + " \"W12_sq\": pz_w12_sq_bounds,\n", + " \"W12\": norm_interval(pz_w12_sq),\n", + " },\n", + " \"timings\": {\n", + " \"onejet_construction\": pz_forward_s,\n", + " \"L2_symbolic_integration\": pz_l2_s,\n", + " \"W12_symbolic_integration\": pz_w12_s,\n", + " \"W12_total\": pz_forward_s + pz_w12_s,\n", + " },\n", + " }\n", + "\n", + " method_id = \"hybrid_pz_uncompressed_reverse_symbolic\"\n", + " reverse_start = perf_counter()\n", + " reverse = shallow_scalar_hybrid_onejet_reverse(\n", + " model,\n", + " cell.domain,\n", + " derivative_flatness_threshold=0.01,\n", + " quadratic_certificate_subdivisions=64,\n", + " )\n", + " reverse_forward_s = perf_counter() - reverse_start\n", + " pz_l2_start = perf_counter()\n", + " pz_l2_integrated = integrate_pz_value_squared(reverse.final.Y, cell, output=\"pz\")\n", + " pz_l2_sq = pz_l2_integrated.interval_enclosure()\n", + " pz_l2_s = perf_counter() - pz_l2_start\n", + " pz_w12_start = perf_counter()\n", + " pz_w12_integrated = integrate_shallow_hybrid_onejet_squared(reverse, cell, output=\"pz\")\n", + " pz_w12_sq = pz_w12_integrated.interval_enclosure()\n", + " pz_w12_s = perf_counter() - pz_w12_start\n", + " compressed_activation = results[\"hybrid_pz_topk96_B_symbolic\"][\"activation\"][0]\n", + " degrees = reverse.derivative_degrees.detach().cpu().numpy()\n", + " reverse_activation = [{\n", + " **compressed_activation,\n", + " \"preactivation_lower\": reverse.preactivation_lower.detach().cpu(),\n", + " \"preactivation_upper\": reverse.preactivation_upper.detach().cpu(),\n", + " \"rho1\": reverse.derivative_approximation_radii.detach().cpu(),\n", + " \"kind1\": np.where(degrees == 2, \"quadratic\", \"affine\"),\n", + " \"affine_rho1\": reverse.affine_derivative_approximation_radii.detach().cpu(),\n", + " \"quadratic_core_box_radius\": torch.zeros_like(reverse.derivative_approximation_radii).cpu(),\n", + " \"relative_slope\": reverse.derivative_relative_slopes.detach().cpu(),\n", + " }]\n", + " results[method_id] = {\n", + " \"method_id\": method_id,\n", + " \"run_id\": \"pinn100d_shallow300_medium_hybrid_pz_uncompressed_reverse_symbolic\",\n", + " \"Y\": reverse.final.Y.interval_enclosure(),\n", + " \"J\": reverse.final.J.interval_enclosure(),\n", + " \"Y_pz\": reverse.final.Y,\n", + " \"J_pz\": reverse.final.J,\n", + " \"activation\": reverse_activation,\n", + " \"integrated_L2_pz\": pz_l2_integrated,\n", + " \"integrated_W12_pz\": pz_w12_integrated,\n", + " \"norms\": {\n", + " \"L2_sq\": _nonnegative_squared_interval(pz_l2_sq),\n", + " \"L2\": norm_interval(pz_l2_sq),\n", + " \"W12_sq\": _nonnegative_squared_interval(pz_w12_sq),\n", + " \"W12\": norm_interval(pz_w12_sq),\n", + " },\n", + " \"timings\": {\n", + " **reverse.timings,\n", + " \"onejet_construction_measured\": reverse_forward_s,\n", + " \"L2_symbolic_integration\": pz_l2_s,\n", + " \"W12_symbolic_integration\": pz_w12_s,\n", + " \"W12_total\": reverse_forward_s + pz_w12_s,\n", + " },\n", + " }\n", + " return results\n", + "\n", + "\n", + "medium_results = _run_medium_methods(model)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "shallow-11", + "metadata": {}, + "outputs": [], + "source": [ + "def _interval_stats(lower, upper, scale):\n", + " lower = np.asarray(lower, dtype=float)\n", + " upper = np.asarray(upper, dtype=float)\n", + " midpoint = 0.5 * (lower + upper)\n", + " radius = 0.5 * (upper - lower)\n", + " width = upper - lower\n", + " magnitude = np.maximum(np.abs(lower), np.abs(upper))\n", + " mignitude = np.where((lower <= 0.0) & (upper >= 0.0), 0.0, np.minimum(np.abs(lower), np.abs(upper)))\n", + " local = np.divide(radius, magnitude, out=np.zeros_like(radius), where=magnitude > 0.0)\n", + " global_radius = np.divide(radius, scale, out=np.zeros_like(radius), where=scale > 0.0)\n", + " return midpoint, radius, width, magnitude, mignitude, local, global_radius\n", + "\n", + "\n", + "def _family_metrics(base, method, quantity, lower, upper, derivative_order):\n", + " lower = np.asarray(lower, dtype=float).reshape(-1)\n", + " upper = np.asarray(upper, dtype=float).reshape(-1)\n", + " scale = float(np.max(np.maximum(np.abs(lower), np.abs(upper)))) if lower.size else 0.0\n", + " _, _, width, _, _, _, global_radius = _interval_stats(lower, upper, scale)\n", + " specifications = [\n", + " (\"mean_width\", \"mean_weighted\", float(np.mean(width))),\n", + " (\"max_width\", \"max\", float(np.max(width))),\n", + " (\"q50_width\", \"q50\", float(np.quantile(width, 0.50, method=\"linear\"))),\n", + " (\"q90_width\", \"q90\", float(np.quantile(width, 0.90, method=\"linear\"))),\n", + " (\"q99_width\", \"q99\", float(np.quantile(width, 0.99, method=\"linear\"))),\n", + " (\"mean_global_normalized_radius\", \"mean_weighted\", float(np.mean(global_radius))),\n", + " (\"max_global_normalized_radius\", \"max\", float(np.max(global_radius))),\n", + " ]\n", + " rows = []\n", + " for metric, aggregation, value in specifications:\n", + " rows.append({**base, \"quantity\": quantity, \"metric\": metric,\n", + " \"aggregation\": aggregation, \"derivative_order\": derivative_order,\n", + " \"value\": value, \"unit\": \"dimensionless\", \"status\": \"ok\"})\n", + " if quantity == \"J\":\n", + " matrix_width = np.asarray(upper - lower, dtype=float)\n", + " frobenius = float(np.linalg.norm(matrix_width.reshape(-1)))\n", + " rows.extend([\n", + " {**base, \"quantity\": \"J\", \"metric\": \"mean_frobenius_width\",\n", + " \"aggregation\": \"mean_weighted\", \"derivative_order\": 1,\n", + " \"value\": frobenius, \"unit\": \"dimensionless\", \"status\": \"ok\"},\n", + " {**base, \"quantity\": \"J\", \"metric\": \"max_frobenius_width\",\n", + " \"aggregation\": \"max\", \"derivative_order\": 1,\n", + " \"value\": frobenius, \"unit\": \"dimensionless\", \"status\": \"ok\"},\n", + " ])\n", + " return rows\n", + "\n", + "\n", + "def _base_metric(result):\n", + " return {\n", + " \"schema_version\": SCHEMA_VERSION, \"benchmark_level\": BENCHMARK_LEVEL,\n", + " \"problem_id\": PROBLEM_ID, \"model_id\": MODEL_ID,\n", + " \"method_id\": result[\"method_id\"], \"run_id\": result[\"run_id\"],\n", + " \"split_id\": SPLIT_ID, \"layer\": pd.NA, \"neuron\": pd.NA,\n", + " \"output_index\": pd.NA, \"input_index_a\": pd.NA,\n", + " \"input_index_b\": pd.NA, \"cell_id\": pd.NA,\n", + " }\n", + "\n", + "\n", + "def _cell_interval_table(result):\n", + " rows = []\n", + " for quantity, enclosure in ((\"Y\", result[\"Y\"]), (\"J\", result[\"J\"])):\n", + " lower = np.asarray(_tensor(enclosure.lower), dtype=float)\n", + " upper = np.asarray(_tensor(enclosure.upper), dtype=float)\n", + " scale = float(np.max(np.maximum(np.abs(lower), np.abs(upper)))) if lower.size else 0.0\n", + " midpoint, radius, width, magnitude, mignitude, local, global_radius = _interval_stats(lower, upper, scale)\n", + " for index in np.ndindex(lower.shape):\n", + " if quantity == \"Y\":\n", + " output_index, input_a = (index[0] if index else 0), pd.NA\n", + " else:\n", + " output_index, input_a = index\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"split_id\": SPLIT_ID, \"cell_id\": 0,\n", + " \"cell_weight\": 1.0, \"quantity\": quantity,\n", + " \"output_index\": output_index, \"input_index_a\": input_a,\n", + " \"input_index_b\": pd.NA, \"lower\": float(lower[index]),\n", + " \"upper\": float(upper[index]), \"midpoint\": float(midpoint[index]),\n", + " \"radius\": float(radius[index]), \"width\": float(width[index]),\n", + " \"magnitude\": float(magnitude[index]), \"mignitude\": float(mignitude[index]),\n", + " \"local_relative_radius\": float(local[index]),\n", + " \"global_normalized_radius\": float(global_radius[index]),\n", + " \"sign_certified\": int(not (lower[index] <= 0.0 <= upper[index])),\n", + " \"status\": \"ok\",\n", + " \"local_relative_width\": float(2.0 * local[index]),\n", + " })\n", + " return pd.DataFrame(rows, columns=CELL_INTERVAL_COLUMNS).sort_values(\n", + " [\"cell_id\", \"quantity\", \"output_index\", \"input_index_a\", \"input_index_b\"],\n", + " na_position=\"last\", kind=\"stable\", ignore_index=True,\n", + " )\n", + "\n", + "\n", + "def _norm_table(result):\n", + " rows = []\n", + " for norm in (\"L2\", \"W12\"):\n", + " for squared in (1, 0):\n", + " key = norm + (\"_sq\" if squared else \"\")\n", + " lower, upper = map(float, result[\"norms\"][key])\n", + " width = upper - lower\n", + " volume_scale = VOLUME if squared else SQRT_VOLUME\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"norm\": norm, \"squared\": squared,\n", + " \"lower\": lower, \"upper\": upper, \"width\": width,\n", + " \"relative_width\": width / upper if upper > 0.0 else 0.0,\n", + " \"value_contribution_upper\": pd.NA,\n", + " \"gradient_contribution_upper\": pd.NA,\n", + " \"hessian_contribution_upper\": pd.NA,\n", + " \"value_contribution_width\": pd.NA,\n", + " \"gradient_contribution_width\": pd.NA,\n", + " \"hessian_contribution_width\": pd.NA,\n", + " \"status\": \"ok\",\n", + " \"domain_volume\": VOLUME,\n", + " \"domain_volume_normalized_lower\": lower / volume_scale,\n", + " \"domain_volume_normalized_upper\": upper / volume_scale,\n", + " \"domain_volume_normalized_width\": width / volume_scale,\n", + " })\n", + " return pd.DataFrame(rows, columns=NORM_COLUMNS)\n", + "\n", + "\n", + "def _tanh_prime_hull(lower, upper):\n", + " t_lo = np.tanh(lower)\n", + " t_hi = np.tanh(upper)\n", + " endpoint_lo = 1.0 - t_lo * t_lo\n", + " endpoint_hi = 1.0 - t_hi * t_hi\n", + " hull_lower = np.minimum(endpoint_lo, endpoint_hi)\n", + " hull_upper = np.where((lower <= 0.0) & (upper >= 0.0), 1.0, np.maximum(endpoint_lo, endpoint_hi))\n", + " return hull_lower, hull_upper\n", + "\n", + "\n", + "def _activation_tables(result):\n", + " rows = []\n", + " wide = {\"neuron\": np.arange(max(len(record[\"preactivation_lower\"]) for record in result[\"activation\"]))}\n", + " for record in result[\"activation\"]:\n", + " layer = int(record[\"layer\"])\n", + " lower = np.asarray(record[\"preactivation_lower\"], dtype=float)\n", + " upper = np.asarray(record[\"preactivation_upper\"], dtype=float)\n", + " midpoint = 0.5 * (lower + upper)\n", + " radius = 0.5 * (upper - lower)\n", + " width = upper - lower\n", + " y_lower = np.tanh(lower)\n", + " y_upper = np.tanh(upper)\n", + " d_lower, d_upper = _tanh_prime_hull(lower, upper)\n", + " hulls = {0: (y_lower, y_upper), 1: (d_lower, d_upper)}\n", + " post_lower = np.asarray(record[\"postactivation_lower\"], dtype=float)\n", + " post_upper = np.asarray(record[\"postactivation_upper\"], dtype=float)\n", + " y_scale = float(np.max(np.maximum(np.abs(post_lower), np.abs(post_upper))))\n", + " y_normalized = (0.5 * (post_upper - post_lower) / y_scale) if y_scale > 0.0 else np.zeros_like(post_lower)\n", + " padded = np.full(len(wide[\"neuron\"]), np.nan)\n", + " padded[:len(y_normalized)] = y_normalized\n", + " wide[f\"layer_{layer}\"] = padded\n", + " for derivative_order in (0, 1):\n", + " hull_lower, hull_upper = hulls[derivative_order]\n", + " scale = float(np.max(np.maximum(np.abs(hull_lower), np.abs(hull_upper))))\n", + " if result[\"method_id\"] == \"interval\":\n", + " rho = 0.5 * (hull_upper - hull_lower)\n", + " kinds = np.full(len(lower), \"interval\", dtype=object)\n", + " noise_ids = [pd.NA] * len(lower)\n", + " else:\n", + " rho = np.asarray(record[f\"rho{derivative_order}\"], dtype=float)\n", + " kinds = np.asarray(record.get(f\"kind{derivative_order}\", np.full(len(lower), \"affine\")), dtype=object)\n", + " noise_ids = [f\"eta_l{layer}_n{neuron}_r{derivative_order}\" for neuron in range(len(lower))]\n", + " normalized = rho / scale if scale > 0.0 else np.zeros_like(rho)\n", + " for neuron in range(len(lower)):\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"split_id\": SPLIT_ID,\n", + " \"cell_id\": 0, \"cell_weight\": 1.0, \"layer\": layer,\n", + " \"neuron\": neuron, \"derivative_order\": derivative_order,\n", + " \"preactivation_lower\": lower[neuron],\n", + " \"preactivation_upper\": upper[neuron],\n", + " \"preactivation_midpoint\": midpoint[neuron],\n", + " \"preactivation_radius\": radius[neuron],\n", + " \"preactivation_width\": width[neuron],\n", + " \"approximation_kind\": kinds[neuron],\n", + " \"approximation_error_radius\": rho[neuron],\n", + " \"approximation_error_diameter\": 2.0 * rho[neuron],\n", + " \"activation_scale\": scale,\n", + " \"normalized_approximation_radius\": normalized[neuron],\n", + " \"noise_symbol_id\": noise_ids[neuron],\n", + " \"shared_noise_group\": pd.NA, \"status\": \"ok\",\n", + " })\n", + " activation = pd.DataFrame(rows, columns=ACTIVATION_COLUMNS).sort_values(\n", + " [\"cell_id\", \"layer\", \"neuron\", \"derivative_order\"], kind=\"stable\", ignore_index=True,\n", + " )\n", + " wide_table = pd.DataFrame(wide)[[\"neuron\"] + sorted([key for key in wide if key.startswith(\"layer_\")])]\n", + " return activation, wide_table\n", + "\n", + "\n", + "def _pz_complexity_row(result, quantity, pz):\n", + " kinds = tuple(pz.noise_kinds)\n", + " domain = {index for index, kind in enumerate(kinds) if kind == \"domain\"}\n", + " approximation = set(range(len(kinds))) - domain\n", + " support = list(pz.terms)\n", + " mixed = sum(\n", + " int(any(exp[index] for index in domain) and any(exp[index] for index in approximation))\n", + " for exp in support\n", + " )\n", + " coefficient_dimension = int(np.prod(pz.shape)) if pz.shape else 1\n", + " return {\n", + " \"run_id\": result[\"run_id\"], \"quantity\": quantity,\n", + " \"n_alpha\": len(domain), \"n_eta\": len(approximation),\n", + " \"n_monomials\": len(support), \"n_mixed_monomials\": mixed,\n", + " \"max_degree\": max((sum(exp) for exp in support), default=0),\n", + " \"n_coefficients\": coefficient_dimension * len(support), \"status\": \"ok\",\n", + " }\n", + "\n", + "\n", + "def _complexity_table(result):\n", + " if \"Y_pz\" in result:\n", + " rows = [_pz_complexity_row(result, \"Y\", result[\"Y_pz\"]),\n", + " _pz_complexity_row(result, \"J\", result[\"J_pz\"])]\n", + " else:\n", + " rows = [{\"run_id\": result[\"run_id\"], \"quantity\": quantity, \"status\": \"not_implemented\"}\n", + " for quantity in (\"Y\", \"J\")]\n", + " rows.append({\"run_id\": result[\"run_id\"], \"quantity\": \"H\", \"status\": \"not_implemented\"})\n", + " return pd.DataFrame(rows).reindex(columns=COMPLEXITY_COLUMNS)\n", + "\n", + "\n", + "def _soundness_table(result):\n", + " rows = []\n", + " exact_values = {\n", + " \"Y\": prediction.detach().cpu().reshape(-1, 1).numpy(),\n", + " \"J\": network_jacobian.detach().cpu().reshape(-1, 1, DIM).numpy(),\n", + " }\n", + " for quantity, enclosure in ((\"Y\", result[\"Y\"]), (\"J\", result[\"J\"])):\n", + " lower = np.asarray(_tensor(enclosure.lower), dtype=float)\n", + " upper = np.asarray(_tensor(enclosure.upper), dtype=float)\n", + " values = exact_values[quantity]\n", + " violation = np.maximum(np.maximum(lower - values, values - upper), 0.0)\n", + " endpoints = np.concatenate([lower.reshape(-1), upper.reshape(-1)])\n", + " rows.append({\n", + " \"run_id\": result[\"run_id\"], \"quantity\": quantity,\n", + " \"sample_count\": values.shape[0],\n", + " \"failure_count\": int(np.count_nonzero(np.any(violation > 0.0, axis=tuple(range(1, violation.ndim))))),\n", + " \"max_violation\": float(np.max(violation)),\n", + " \"invalid_interval_count\": int(np.count_nonzero(lower > upper)),\n", + " \"nan_endpoint_count\": int(np.count_nonzero(np.isnan(endpoints))),\n", + " \"infinite_endpoint_count\": int(np.count_nonzero(np.isinf(endpoints))),\n", + " \"status\": \"ok\",\n", + " })\n", + " return pd.DataFrame(rows, columns=SOUNDNESS_COLUMNS)\n", + "\n", + "\n", + "def _metrics_table(result, cell_intervals, norms, complexity, timings, soundness):\n", + " base = _base_metric(result)\n", + " rows = []\n", + " for quantity, derivative_order in ((\"Y\", 0), (\"J\", 1)):\n", + " family = cell_intervals[cell_intervals.quantity == quantity]\n", + " rows.extend(_family_metrics(base, result[\"method_id\"], quantity,\n", + " family.lower, family.upper, derivative_order))\n", + " for metric in (\"mean_width\", \"max_width\", \"q50_width\", \"q90_width\", \"q99_width\",\n", + " \"mean_global_normalized_radius\", \"max_global_normalized_radius\"):\n", + " rows.append({**base, \"quantity\": \"H\", \"metric\": metric, \"aggregation\": \"none\",\n", + " \"derivative_order\": 2, \"value\": pd.NA, \"unit\": pd.NA,\n", + " \"status\": \"not_implemented\"})\n", + " for _, row in norms.iterrows():\n", + " quantity = row[\"norm\"] + (\"_sq\" if row[\"squared\"] else \"\")\n", + " for metric in (\"lower\", \"upper\", \"width\", \"relative_norm_width\",\n", + " \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\",\n", + " \"domain_volume_normalized_width\"):\n", + " value = row[\"relative_width\"] if metric == \"relative_norm_width\" else row[metric]\n", + " rows.append({**base, \"quantity\": quantity, \"metric\": metric,\n", + " \"aggregation\": \"none\", \"derivative_order\": pd.NA,\n", + " \"value\": value, \"unit\": \"dimensionless\", \"status\": row[\"status\"]})\n", + " for residual_quantity, derivative_order in ((\"PDE_residual\", 2), (\"boundary_residual\", 0), (\"initial_residual\", 0)):\n", + " rows.append({**base, \"quantity\": residual_quantity, \"metric\": \"linf_upper\",\n", + " \"aggregation\": \"max\", \"derivative_order\": derivative_order, \"value\": pd.NA,\n", + " \"unit\": \"dimensionless\", \"status\": \"not_implemented\"})\n", + " for _, row in complexity.iterrows():\n", + " for column, metric in ((\"n_alpha\", \"n_alpha\"), (\"n_eta\", \"n_eta\"),\n", + " (\"n_monomials\", \"n_monomials\"),\n", + " (\"n_mixed_monomials\", \"n_mixed_monomials\"),\n", + " (\"max_degree\", \"max_degree\")):\n", + " rows.append({**base, \"quantity\": \"complexity\", \"metric\": metric,\n", + " \"aggregation\": \"none\", \"derivative_order\": pd.NA,\n", + " \"value\": row[column], \"unit\": \"dimensionless\", \"status\": row[\"status\"],\n", + " \"layer\": pd.NA, \"neuron\": pd.NA})\n", + " for _, row in timings.iterrows():\n", + " rows.append({**base, \"quantity\": \"runtime\", \"metric\": \"seconds\",\n", + " \"aggregation\": row[\"stage\"], \"derivative_order\": pd.NA,\n", + " \"value\": row[\"seconds\"], \"unit\": \"seconds\", \"status\": row[\"status\"]})\n", + " rows.append({**base, \"quantity\": \"memory\", \"metric\": \"bytes\",\n", + " \"aggregation\": \"peak\", \"derivative_order\": pd.NA,\n", + " \"value\": pd.NA, \"unit\": \"bytes\", \"status\": \"not_implemented\"})\n", + " for _, row in soundness.iterrows():\n", + " for column, metric in ((\"failure_count\", \"failure_count\"), (\"max_violation\", \"max_violation\")):\n", + " rows.append({**base, \"quantity\": \"soundness\", \"metric\": metric,\n", + " \"aggregation\": row[\"quantity\"], \"derivative_order\": pd.NA,\n", + " \"value\": row[column], \"unit\": \"dimensionless\", \"status\": row[\"status\"]})\n", + " return pd.DataFrame(rows).reindex(columns=METRICS_COLUMNS)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "shallow-12", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idnormsquaredlowerupperwidthrelative_widthdomain_volumedomain_volume_normalized_lowerdomain_volume_normalized_upperdomain_volume_normalized_widthstatus
0intervalL210.03.188219e-683.188219e-681.01.267651e-700.0251.506108251.506108ok
1intervalL200.01.785558e-341.785558e-341.01.267651e-700.015.85894415.858944ok
2intervalW1210.04.013416e-684.013416e-681.01.267651e-700.0316.602733316.602733ok
3intervalW1200.02.003351e-342.003351e-341.01.267651e-700.017.79333417.793334ok
4affine_pz_topk96_symbolicL210.01.244074e-691.244074e-691.01.267651e-700.09.8140119.814011ok
5affine_pz_topk96_symbolicL200.03.527143e-353.527143e-351.01.267651e-700.03.1327323.132732ok
6affine_pz_topk96_symbolicW1210.09.340887e-699.340887e-691.01.267651e-700.073.68660873.686608ok
7affine_pz_topk96_symbolicW1200.09.664827e-359.664827e-351.01.267651e-700.08.5840908.584090ok
8hybrid_pz_topk96_B_symbolicL210.01.244074e-691.244074e-691.01.267651e-700.09.8140119.814011ok
9hybrid_pz_topk96_B_symbolicL200.03.527143e-353.527143e-351.01.267651e-700.03.1327323.132732ok
10hybrid_pz_topk96_B_symbolicW1210.09.771853e-699.771853e-691.01.267651e-700.077.08632877.086328ok
11hybrid_pz_topk96_B_symbolicW1200.09.885268e-359.885268e-351.01.267651e-700.08.7798828.779882ok
12hybrid_pz_uncompressed_reverse_symbolicL210.01.244074e-691.244074e-691.01.267651e-700.09.8140119.814011ok
13hybrid_pz_uncompressed_reverse_symbolicL200.03.527143e-353.527143e-351.01.267651e-700.03.1327323.132732ok
14hybrid_pz_uncompressed_reverse_symbolicW1210.02.883488e-692.883488e-691.01.267651e-700.022.74670922.746709ok
15hybrid_pz_uncompressed_reverse_symbolicW1200.05.369812e-355.369812e-351.01.267651e-700.04.7693514.769351ok
\n", + "
" + ], + "text/plain": [ + " method_id ... status\n", + "0 interval ... ok\n", + "1 interval ... ok\n", + "2 interval ... ok\n", + "3 interval ... ok\n", + "4 affine_pz_topk96_symbolic ... ok\n", + "5 affine_pz_topk96_symbolic ... ok\n", + "6 affine_pz_topk96_symbolic ... ok\n", + "7 affine_pz_topk96_symbolic ... ok\n", + "8 hybrid_pz_topk96_B_symbolic ... ok\n", + "9 hybrid_pz_topk96_B_symbolic ... ok\n", + "10 hybrid_pz_topk96_B_symbolic ... ok\n", + "11 hybrid_pz_topk96_B_symbolic ... ok\n", + "12 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "13 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "14 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "15 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "\n", + "[16 rows x 12 columns]" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "medium_outputs = {}\n", + "try:\n", + " git_commit = subprocess.check_output(\n", + " [\"git\", \"rev-parse\", \"HEAD\"], cwd=repo_root, text=True\n", + " ).strip()\n", + "except (OSError, subprocess.CalledProcessError):\n", + " git_commit = None\n", + "\n", + "for method_id, result in medium_results.items():\n", + " method_dir = OUTPUT_ROOT / method_id\n", + " method_dir.mkdir(parents=True, exist_ok=True)\n", + " cell_intervals = _cell_interval_table(result)\n", + " norms = _norm_table(result)\n", + " activation, layer_radius_y = _activation_tables(result)\n", + " complexity = _complexity_table(result)\n", + " timings = pd.DataFrame([\n", + " {\"run_id\": result[\"run_id\"], \"stage\": stage, \"seconds\": seconds, \"status\": \"ok\"}\n", + " for stage, seconds in result[\"timings\"].items()\n", + " ], columns=TIMING_COLUMNS)\n", + " soundness = _soundness_table(result)\n", + " metrics = _metrics_table(result, cell_intervals, norms, complexity, timings, soundness)\n", + "\n", + " assert list(metrics.columns) == METRICS_COLUMNS\n", + " assert list(cell_intervals.columns) == CELL_INTERVAL_COLUMNS\n", + " assert list(norms.columns) == NORM_COLUMNS\n", + " assert list(activation.columns) == ACTIVATION_COLUMNS\n", + " assert list(layer_radius_y.columns) == [\"neuron\", \"layer_0\"]\n", + " assert list(complexity.columns) == COMPLEXITY_COLUMNS\n", + " assert list(timings.columns) == TIMING_COLUMNS\n", + " assert list(soundness.columns) == SOUNDNESS_COLUMNS\n", + " assert activation.equals(activation.sort_values(\n", + " [\"cell_id\", \"layer\", \"neuron\", \"derivative_order\"], kind=\"stable\", ignore_index=True\n", + " ))\n", + " assert np.isclose(activation.cell_weight.groupby([activation.cell_id]).first().sum(), 1.0)\n", + " assert not bool((norms.relative_width < 0.0).any() or (norms.relative_width > 1.0).any())\n", + " assert int(soundness.failure_count.sum()) == 0\n", + " assert np.allclose(\n", + " cell_intervals.local_relative_width,\n", + " 2.0 * cell_intervals.local_relative_radius,\n", + " )\n", + " assert list(zip(norms[\"norm\"], norms[\"squared\"])) == [\n", + " (\"L2\", 1), (\"L2\", 0), (\"W12\", 1), (\"W12\", 0),\n", + " ]\n", + " assert not norms[[\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + " ]].isna().any().any()\n", + " norm_scales = np.where(norms.squared.astype(bool), VOLUME, SQRT_VOLUME)\n", + " assert np.all(norms.domain_volume.to_numpy() == VOLUME)\n", + " assert np.allclose(norms.domain_volume_normalized_lower, norms.lower / norm_scales)\n", + " assert np.allclose(norms.domain_volume_normalized_upper, norms.upper / norm_scales)\n", + " assert np.allclose(norms.domain_volume_normalized_width, norms.width / norm_scales)\n", + "\n", + " metadata = {\n", + " \"schema_version\": SCHEMA_VERSION,\n", + " \"benchmark_level\": BENCHMARK_LEVEL,\n", + " \"problem_id\": PROBLEM_ID,\n", + " \"model_id\": MODEL_ID,\n", + " \"method_id\": method_id,\n", + " \"git_commit\": git_commit,\n", + " \"dtype\": str(torch.get_default_dtype()).replace(\"torch.\", \"\"),\n", + " \"device\": \"cpu\",\n", + " \"quantile_interpolation\": \"linear\",\n", + " \"cell_average\": \"volume_weighted\",\n", + " \"timestamp_utc\": datetime.now(timezone.utc).isoformat(),\n", + " }\n", + " (method_dir / \"benchmark_metadata.json\").write_text(\n", + " json.dumps(metadata, indent=2) + \"\\n\", encoding=\"utf-8\"\n", + " )\n", + " for filename, frame in {\n", + " \"metrics.csv\": metrics,\n", + " \"cell_intervals.csv\": cell_intervals,\n", + " \"norms.csv\": norms,\n", + " \"complexity.csv\": complexity,\n", + " \"timings.csv\": timings,\n", + " \"soundness.csv\": soundness,\n", + " \"activation_approximation.csv\": activation,\n", + " \"layer_normalized_radius_Y.csv\": layer_radius_y,\n", + " }.items():\n", + " frame.to_csv(method_dir / filename, index=False, na_rep=\"NA\")\n", + " medium_outputs[method_id] = {\n", + " \"metadata\": metadata, \"metrics\": metrics, \"cell_intervals\": cell_intervals,\n", + " \"norms\": norms, \"complexity\": complexity, \"timings\": timings,\n", + " \"soundness\": soundness, \"activation\": activation,\n", + " \"layer_radius_Y\": layer_radius_y,\n", + " }\n", + "\n", + "medium_norms_summary = pd.concat([\n", + " output[\"norms\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\n", + " \"method_id\", \"norm\", \"squared\", \"lower\", \"upper\", \"width\", \"relative_width\",\n", + " \"domain_volume\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\", \"status\"\n", + "]]\n", + "medium_norms_summary" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-13", + "metadata": {}, + "source": [ + "### Canonical squared and unsquared norm intervals" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "shallow-14", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idnormdomain_volume_normalized_lowerdomain_volume_normalized_upperdomain_volume_normalized_widthrelative_width
1intervalL20.015.85894415.8589441.0
3intervalW120.017.79333417.7933341.0
5affine_pz_topk96_symbolicL20.03.1327323.1327321.0
7affine_pz_topk96_symbolicW120.08.5840908.5840901.0
9hybrid_pz_topk96_B_symbolicL20.03.1327323.1327321.0
11hybrid_pz_topk96_B_symbolicW120.08.7798828.7798821.0
13hybrid_pz_uncompressed_reverse_symbolicL20.03.1327323.1327321.0
15hybrid_pz_uncompressed_reverse_symbolicW120.04.7693514.7693511.0
\n", + "
" + ], + "text/plain": [ + " method_id ... relative_width\n", + "1 interval ... 1.0\n", + "3 interval ... 1.0\n", + "5 affine_pz_topk96_symbolic ... 1.0\n", + "7 affine_pz_topk96_symbolic ... 1.0\n", + "9 hybrid_pz_topk96_B_symbolic ... 1.0\n", + "11 hybrid_pz_topk96_B_symbolic ... 1.0\n", + "13 hybrid_pz_uncompressed_reverse_symbolic ... 1.0\n", + "15 hybrid_pz_uncompressed_reverse_symbolic ... 1.0\n", + "\n", + "[8 rows x 6 columns]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "normalized_norm_view = medium_norms_summary[medium_norms_summary.squared == 0].copy()\n", + "normalized_norm_view[[\n", + " \"method_id\", \"norm\", \"domain_volume_normalized_lower\",\n", + " \"domain_volume_normalized_upper\", \"domain_volume_normalized_width\",\n", + " \"relative_width\",\n", + "]]" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-15", + "metadata": {}, + "source": [ + "### Final enclosure, complexity, and runtime diagnostics" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "shallow-16", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idquantitymetricaggregationvaluestatus
0intervalYmean_widthmean_weighted31.450925ok
1intervalYmax_widthmax31.450925ok
2intervalYmean_global_normalized_radiusmean_weighted0.991583ok
3intervalYmax_global_normalized_radiusmax0.991583ok
4intervalJmean_widthmean_weighted1.353524ok
5intervalJmax_widthmax1.682731ok
6intervalJmean_global_normalized_radiusmean_weighted0.694703ok
7intervalJmax_global_normalized_radiusmax0.863669ok
8intervalJmean_frobenius_widthmean_weighted13.575215ok
9intervalJmax_frobenius_widthmax13.575215ok
10affine_pz_topk96_symbolicYmean_widthmean_weighted6.497835ok
11affine_pz_topk96_symbolicYmax_widthmax6.497835ok
12affine_pz_topk96_symbolicYmean_global_normalized_radiusmean_weighted0.926557ok
13affine_pz_topk96_symbolicYmax_global_normalized_radiusmax0.926557ok
14affine_pz_topk96_symbolicJmean_widthmean_weighted1.337706ok
15affine_pz_topk96_symbolicJmax_widthmax1.662853ok
16affine_pz_topk96_symbolicJmean_global_normalized_radiusmean_weighted0.693134ok
17affine_pz_topk96_symbolicJmax_global_normalized_radiusmax0.861609ok
18affine_pz_topk96_symbolicJmean_frobenius_widthmean_weighted13.416511ok
19affine_pz_topk96_symbolicJmax_frobenius_widthmax13.416511ok
20hybrid_pz_topk96_B_symbolicYmean_widthmean_weighted6.497835ok
21hybrid_pz_topk96_B_symbolicYmax_widthmax6.497835ok
22hybrid_pz_topk96_B_symbolicYmean_global_normalized_radiusmean_weighted0.926557ok
23hybrid_pz_topk96_B_symbolicYmax_global_normalized_radiusmax0.926557ok
24hybrid_pz_topk96_B_symbolicJmean_widthmean_weighted1.369178ok
25hybrid_pz_topk96_B_symbolicJmax_widthmax1.705081ok
26hybrid_pz_topk96_B_symbolicJmean_global_normalized_radiusmean_weighted0.690085ok
27hybrid_pz_topk96_B_symbolicJmax_global_normalized_radiusmax0.859385ok
28hybrid_pz_topk96_B_symbolicJmean_frobenius_widthmean_weighted13.732614ok
29hybrid_pz_topk96_B_symbolicJmax_frobenius_widthmax13.732614ok
30hybrid_pz_uncompressed_reverse_symbolicYmean_widthmean_weighted6.497835ok
31hybrid_pz_uncompressed_reverse_symbolicYmax_widthmax6.497835ok
32hybrid_pz_uncompressed_reverse_symbolicYmean_global_normalized_radiusmean_weighted0.926557ok
33hybrid_pz_uncompressed_reverse_symbolicYmax_global_normalized_radiusmax0.926557ok
34hybrid_pz_uncompressed_reverse_symbolicJmean_widthmean_weighted1.351796ok
35hybrid_pz_uncompressed_reverse_symbolicJmax_widthmax1.677475ok
36hybrid_pz_uncompressed_reverse_symbolicJmean_global_normalized_radiusmean_weighted0.690858ok
37hybrid_pz_uncompressed_reverse_symbolicJmax_global_normalized_radiusmax0.857302ok
38hybrid_pz_uncompressed_reverse_symbolicJmean_frobenius_widthmean_weighted13.557322ok
39hybrid_pz_uncompressed_reverse_symbolicJmax_frobenius_widthmax13.557322ok
\n", + "
" + ], + "text/plain": [ + " method_id quantity ... value status\n", + "0 interval Y ... 31.450925 ok\n", + "1 interval Y ... 31.450925 ok\n", + "2 interval Y ... 0.991583 ok\n", + "3 interval Y ... 0.991583 ok\n", + "4 interval J ... 1.353524 ok\n", + "5 interval J ... 1.682731 ok\n", + "6 interval J ... 0.694703 ok\n", + "7 interval J ... 0.863669 ok\n", + "8 interval J ... 13.575215 ok\n", + "9 interval J ... 13.575215 ok\n", + "10 affine_pz_topk96_symbolic Y ... 6.497835 ok\n", + "11 affine_pz_topk96_symbolic Y ... 6.497835 ok\n", + "12 affine_pz_topk96_symbolic Y ... 0.926557 ok\n", + "13 affine_pz_topk96_symbolic Y ... 0.926557 ok\n", + "14 affine_pz_topk96_symbolic J ... 1.337706 ok\n", + "15 affine_pz_topk96_symbolic J ... 1.662853 ok\n", + "16 affine_pz_topk96_symbolic J ... 0.693134 ok\n", + "17 affine_pz_topk96_symbolic J ... 0.861609 ok\n", + "18 affine_pz_topk96_symbolic J ... 13.416511 ok\n", + "19 affine_pz_topk96_symbolic J ... 13.416511 ok\n", + "20 hybrid_pz_topk96_B_symbolic Y ... 6.497835 ok\n", + "21 hybrid_pz_topk96_B_symbolic Y ... 6.497835 ok\n", + "22 hybrid_pz_topk96_B_symbolic Y ... 0.926557 ok\n", + "23 hybrid_pz_topk96_B_symbolic Y ... 0.926557 ok\n", + "24 hybrid_pz_topk96_B_symbolic J ... 1.369178 ok\n", + "25 hybrid_pz_topk96_B_symbolic J ... 1.705081 ok\n", + "26 hybrid_pz_topk96_B_symbolic J ... 0.690085 ok\n", + "27 hybrid_pz_topk96_B_symbolic J ... 0.859385 ok\n", + "28 hybrid_pz_topk96_B_symbolic J ... 13.732614 ok\n", + "29 hybrid_pz_topk96_B_symbolic J ... 13.732614 ok\n", + "30 hybrid_pz_uncompressed_reverse_symbolic Y ... 6.497835 ok\n", + "31 hybrid_pz_uncompressed_reverse_symbolic Y ... 6.497835 ok\n", + "32 hybrid_pz_uncompressed_reverse_symbolic Y ... 0.926557 ok\n", + "33 hybrid_pz_uncompressed_reverse_symbolic Y ... 0.926557 ok\n", + "34 hybrid_pz_uncompressed_reverse_symbolic J ... 1.351796 ok\n", + "35 hybrid_pz_uncompressed_reverse_symbolic J ... 1.677475 ok\n", + "36 hybrid_pz_uncompressed_reverse_symbolic J ... 0.690858 ok\n", + "37 hybrid_pz_uncompressed_reverse_symbolic J ... 0.857302 ok\n", + "38 hybrid_pz_uncompressed_reverse_symbolic J ... 13.557322 ok\n", + "39 hybrid_pz_uncompressed_reverse_symbolic J ... 13.557322 ok\n", + "\n", + "[40 rows x 6 columns]" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "medium_enclosure_summary = pd.concat([\n", + " output[\"metrics\"].query(\n", + " \"quantity in ['Y', 'J'] and metric in ['mean_width', 'max_width', \"\n", + " \"'mean_global_normalized_radius', 'max_global_normalized_radius', \"\n", + " \"'mean_frobenius_width', 'max_frobenius_width']\"\n", + " )[[\"method_id\", \"quantity\", \"metric\", \"aggregation\", \"value\", \"status\"]]\n", + " for output in medium_outputs.values()\n", + "], ignore_index=True)\n", + "medium_enclosure_summary" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "shallow-17", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idrun_idquantityn_alphan_etan_monomialsn_mixed_monomialsmax_degreen_coefficientsstatus
0intervalpinn100d_shallow300_medium_intervalYNaNNaNNaNNaNNaNNaNnot_implemented
1intervalpinn100d_shallow300_medium_intervalJNaNNaNNaNNaNNaNNaNnot_implemented
2intervalpinn100d_shallow300_medium_intervalHNaNNaNNaNNaNNaNNaNnot_implemented
3affine_pz_topk96_symbolicpinn100d_shallow300_medium_affine_pz_topk96_sy...Y100.0400.0400.00.01.0400.0ok
4affine_pz_topk96_symbolicpinn100d_shallow300_medium_affine_pz_topk96_sy...J100.0400.0196.00.01.019600.0ok
5affine_pz_topk96_symbolicpinn100d_shallow300_medium_affine_pz_topk96_sy...HNaNNaNNaNNaNNaNNaNnot_implemented
6hybrid_pz_topk96_B_symbolicpinn100d_shallow300_medium_hybrid_pz_topk96_B_...Y100.0400.0400.00.01.0400.0ok
7hybrid_pz_topk96_B_symbolicpinn100d_shallow300_medium_hybrid_pz_topk96_B_...J100.0400.0196.00.01.019600.0ok
8hybrid_pz_topk96_B_symbolicpinn100d_shallow300_medium_hybrid_pz_topk96_B_...HNaNNaNNaNNaNNaNNaNnot_implemented
9hybrid_pz_uncompressed_reverse_symbolicpinn100d_shallow300_medium_hybrid_pz_uncompres...Y100.0600.0400.00.01.0400.0ok
10hybrid_pz_uncompressed_reverse_symbolicpinn100d_shallow300_medium_hybrid_pz_uncompres...J100.0600.05450.00.02.0545000.0ok
11hybrid_pz_uncompressed_reverse_symbolicpinn100d_shallow300_medium_hybrid_pz_uncompres...HNaNNaNNaNNaNNaNNaNnot_implemented
\n", + "
" + ], + "text/plain": [ + " method_id ... status\n", + "0 interval ... not_implemented\n", + "1 interval ... not_implemented\n", + "2 interval ... not_implemented\n", + "3 affine_pz_topk96_symbolic ... ok\n", + "4 affine_pz_topk96_symbolic ... ok\n", + "5 affine_pz_topk96_symbolic ... not_implemented\n", + "6 hybrid_pz_topk96_B_symbolic ... ok\n", + "7 hybrid_pz_topk96_B_symbolic ... ok\n", + "8 hybrid_pz_topk96_B_symbolic ... not_implemented\n", + "9 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "10 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "11 hybrid_pz_uncompressed_reverse_symbolic ... not_implemented\n", + "\n", + "[12 rows x 10 columns]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pd.concat([\n", + " output[\"complexity\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\"method_id\"] + COMPLEXITY_COLUMNS]" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-18", + "metadata": {}, + "source": [ + "### Per-neuron activation diagnostics" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "shallow-19", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idlayerderivative_orderpreactivation_radius_meanpreactivation_radius_maxapproximation_error_radius_meanapproximation_error_radius_maxnormalized_approximation_radius_meannormalized_approximation_radius_max
0affine_pz_topk96_symbolic001.0007721.6712610.0848340.2180430.0910230.233953
1affine_pz_topk96_symbolic011.0007721.6712610.2830510.4340490.2830510.434049
2hybrid_pz_topk96_B_symbolic001.0007721.6712610.0848340.2180430.0910230.233953
3hybrid_pz_topk96_B_symbolic011.0007721.6712610.2794910.4141730.2794910.414173
4hybrid_pz_uncompressed_reverse_symbolic001.0007721.6712610.0848340.2180430.0910230.233953
5hybrid_pz_uncompressed_reverse_symbolic011.0007721.6712610.2794910.4141730.2794910.414173
6interval001.0007721.6712610.7471990.9317180.8017190.999701
7interval011.0007721.6712610.2880930.4343090.2880930.434309
\n", + "
" + ], + "text/plain": [ + " method_id ... normalized_approximation_radius_max\n", + "0 affine_pz_topk96_symbolic ... 0.233953\n", + "1 affine_pz_topk96_symbolic ... 0.434049\n", + "2 hybrid_pz_topk96_B_symbolic ... 0.233953\n", + "3 hybrid_pz_topk96_B_symbolic ... 0.414173\n", + "4 hybrid_pz_uncompressed_reverse_symbolic ... 0.233953\n", + "5 hybrid_pz_uncompressed_reverse_symbolic ... 0.414173\n", + "6 interval ... 0.999701\n", + "7 interval ... 0.434309\n", + "\n", + "[8 rows x 9 columns]" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "medium_activation_summary = pd.concat([\n", + " output[\"activation\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)\n", + "medium_activation_layer_summary = (\n", + " medium_activation_summary\n", + " .groupby([\"method_id\", \"layer\", \"derivative_order\"], sort=True)\n", + " .agg(\n", + " preactivation_radius_mean=(\"preactivation_radius\", \"mean\"),\n", + " preactivation_radius_max=(\"preactivation_radius\", \"max\"),\n", + " approximation_error_radius_mean=(\"approximation_error_radius\", \"mean\"),\n", + " approximation_error_radius_max=(\"approximation_error_radius\", \"max\"),\n", + " normalized_approximation_radius_mean=(\"normalized_approximation_radius\", \"mean\"),\n", + " normalized_approximation_radius_max=(\"normalized_approximation_radius\", \"max\"),\n", + " )\n", + " .reset_index()\n", + ")\n", + "medium_activation_layer_summary" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "shallow-20", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'interval'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=0'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877interval0.6614960.709762ok
101-0.6668100.693385interval0.5915150.634675ok
202-0.8409690.870502interval0.6939750.744612ok
303-0.8131650.846186interval0.6802020.729833ok
404-1.0376261.001014interval0.7694840.825630ok
...........................
2950295-1.4165491.448714interval0.8921580.957254ok
2960296-0.7369820.710152interval0.6190450.664214ok
2970297-0.9825141.013824interval0.7607460.816254ok
2980298-0.8332060.854856interval0.6878960.738088ok
2990299-0.7428390.726907interval0.6260130.671691ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.709762 ok\n", + "1 0 1 ... 0.634675 ok\n", + "2 0 2 ... 0.744612 ok\n", + "3 0 3 ... 0.729833 ok\n", + "4 0 4 ... 0.825630 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.957254 ok\n", + "296 0 296 ... 0.664214 ok\n", + "297 0 297 ... 0.816254 ok\n", + "298 0 298 ... 0.738088 ok\n", + "299 0 299 ... 0.671691 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'interval'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=1'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877interval0.2205050.220505ok
101-0.6668100.693385interval0.1800910.180091ok
202-0.8409690.870502interval0.2461420.246142ok
303-0.8131650.846186interval0.2374100.237410ok
404-1.0376261.001014interval0.3018250.301825ok
...........................
2950295-1.4165491.448714interval0.4009050.400905ok
2960296-0.7369820.710152interval0.1967640.196764ok
2970297-0.9825141.013824interval0.2944050.294405ok
2980298-0.8332060.854856interval0.2405390.240539ok
2990299-0.7428390.726907interval0.1989910.198991ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.220505 ok\n", + "1 0 1 ... 0.180091 ok\n", + "2 0 2 ... 0.246142 ok\n", + "3 0 3 ... 0.237410 ok\n", + "4 0 4 ... 0.301825 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.400905 ok\n", + "296 0 296 ... 0.196764 ok\n", + "297 0 297 ... 0.294405 ok\n", + "298 0 298 ... 0.240539 ok\n", + "299 0 299 ... 0.198991 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'affine_pz_topk96_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=0'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.0477610.051246ok
101-0.6668100.693385affine0.0322040.034554ok
202-0.8409690.870502affine0.0570500.061212ok
303-0.8131650.846186affine0.0529740.056839ok
404-1.0376261.001014affine0.0853450.091572ok
...........................
2950295-1.4165491.448714affine0.1685430.180841ok
2960296-0.7369820.710152affine0.0377300.040483ok
2970297-0.9825141.013824affine0.0814500.087393ok
2980298-0.8332060.854856affine0.0551780.059205ok
2990299-0.7428390.726907affine0.0392130.042075ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.051246 ok\n", + "1 0 1 ... 0.034554 ok\n", + "2 0 2 ... 0.061212 ok\n", + "3 0 3 ... 0.056839 ok\n", + "4 0 4 ... 0.091572 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.180841 ok\n", + "296 0 296 ... 0.040483 ok\n", + "297 0 297 ... 0.087393 ok\n", + "298 0 298 ... 0.059205 ok\n", + "299 0 299 ... 0.042075 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'affine_pz_topk96_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=1'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.2187840.218784ok
101-0.6668100.693385affine0.1749110.174911ok
202-0.8409690.870502affine0.2407580.240758ok
303-0.8131650.846186affine0.2312830.231283ok
404-1.0376261.001014affine0.2959940.295994ok
...........................
2950295-1.4165491.448714affine0.3979470.397947ok
2960296-0.7369820.710152affine0.1915730.191573ok
2970297-0.9825141.013824affine0.2893230.289323ok
2980298-0.8332060.854856affine0.2365770.236577ok
2990299-0.7428390.726907affine0.1959340.195934ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.218784 ok\n", + "1 0 1 ... 0.174911 ok\n", + "2 0 2 ... 0.240758 ok\n", + "3 0 3 ... 0.231283 ok\n", + "4 0 4 ... 0.295994 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.397947 ok\n", + "296 0 296 ... 0.191573 ok\n", + "297 0 297 ... 0.289323 ok\n", + "298 0 298 ... 0.236577 ok\n", + "299 0 299 ... 0.195934 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_topk96_B_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=0'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.0477610.051246ok
101-0.6668100.693385affine0.0322040.034554ok
202-0.8409690.870502affine0.0570500.061212ok
303-0.8131650.846186affine0.0529740.056839ok
404-1.0376261.001014affine0.0853450.091572ok
...........................
2950295-1.4165491.448714affine0.1685430.180841ok
2960296-0.7369820.710152affine0.0377300.040483ok
2970297-0.9825141.013824affine0.0814500.087393ok
2980298-0.8332060.854856affine0.0551780.059205ok
2990299-0.7428390.726907affine0.0392130.042075ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.051246 ok\n", + "1 0 1 ... 0.034554 ok\n", + "2 0 2 ... 0.061212 ok\n", + "3 0 3 ... 0.056839 ok\n", + "4 0 4 ... 0.091572 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.180841 ok\n", + "296 0 296 ... 0.040483 ok\n", + "297 0 297 ... 0.087393 ok\n", + "298 0 298 ... 0.059205 ok\n", + "299 0 299 ... 0.042075 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_topk96_B_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=1'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.2187840.218784ok
101-0.6668100.693385affine0.1749110.174911ok
202-0.8409690.870502affine0.2407580.240758ok
303-0.8131650.846186affine0.2312830.231283ok
404-1.0376261.001014affine0.2959940.295994ok
...........................
2950295-1.4165491.448714affine0.3979470.397947ok
2960296-0.7369820.710152affine0.1915730.191573ok
2970297-0.9825141.013824affine0.2893230.289323ok
2980298-0.8332060.854856affine0.2365770.236577ok
2990299-0.7428390.726907affine0.1959340.195934ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.218784 ok\n", + "1 0 1 ... 0.174911 ok\n", + "2 0 2 ... 0.240758 ok\n", + "3 0 3 ... 0.231283 ok\n", + "4 0 4 ... 0.295994 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.397947 ok\n", + "296 0 296 ... 0.191573 ok\n", + "297 0 297 ... 0.289323 ok\n", + "298 0 298 ... 0.236577 ok\n", + "299 0 299 ... 0.195934 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_uncompressed_reverse_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=0'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.0477610.051246ok
101-0.6668100.693385affine0.0322040.034554ok
202-0.8409690.870502affine0.0570500.061212ok
303-0.8131650.846186affine0.0529740.056839ok
404-1.0376261.001014affine0.0853450.091572ok
...........................
2950295-1.4165491.448714affine0.1685430.180841ok
2960296-0.7369820.710152affine0.0377300.040483ok
2970297-0.9825141.013824affine0.0814500.087393ok
2980298-0.8332060.854856affine0.0551780.059205ok
2990299-0.7428390.726907affine0.0392130.042075ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.051246 ok\n", + "1 0 1 ... 0.034554 ok\n", + "2 0 2 ... 0.061212 ok\n", + "3 0 3 ... 0.056839 ok\n", + "4 0 4 ... 0.091572 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.180841 ok\n", + "296 0 296 ... 0.040483 ok\n", + "297 0 297 ... 0.087393 ok\n", + "298 0 298 ... 0.059205 ok\n", + "299 0 299 ... 0.042075 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_uncompressed_reverse_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'derivative_order=1'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
layerneuronpreactivation_lowerpreactivation_upperapproximation_kindapproximation_error_radiusnormalized_approximation_radiusstatus
000-0.8000890.790877affine0.2187840.218784ok
101-0.6668100.693385affine0.1749110.174911ok
202-0.8409690.870502affine0.2407580.240758ok
303-0.8131650.846186affine0.2312830.231283ok
404-1.0376261.001014affine0.2959940.295994ok
...........................
2950295-1.4165491.448714affine0.3979470.397947ok
2960296-0.7369820.710152affine0.1915730.191573ok
2970297-0.9825141.013824affine0.2893230.289323ok
2980298-0.8332060.854856affine0.2365770.236577ok
2990299-0.7428390.726907affine0.1959340.195934ok
\n", + "

300 rows × 8 columns

\n", + "
" + ], + "text/plain": [ + " layer neuron ... normalized_approximation_radius status\n", + "0 0 0 ... 0.218784 ok\n", + "1 0 1 ... 0.174911 ok\n", + "2 0 2 ... 0.240758 ok\n", + "3 0 3 ... 0.231283 ok\n", + "4 0 4 ... 0.295994 ok\n", + ".. ... ... ... ... ...\n", + "295 0 295 ... 0.397947 ok\n", + "296 0 296 ... 0.191573 ok\n", + "297 0 297 ... 0.289323 ok\n", + "298 0 298 ... 0.236577 ok\n", + "299 0 299 ... 0.195934 ok\n", + "\n", + "[300 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "activation_views = {}\n", + "for method_id, output in medium_outputs.items():\n", + " for derivative_order in (0, 1):\n", + " view = output[\"activation\"].query(\"derivative_order == @derivative_order\")[\n", + " [\"layer\", \"neuron\", \"preactivation_lower\", \"preactivation_upper\",\n", + " \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\", \"status\"]\n", + " ].reset_index(drop=True)\n", + " activation_views[(method_id, derivative_order)] = view\n", + " display(method_id, f\"derivative_order={derivative_order}\", view)" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-21", + "metadata": {}, + "source": [ + "### Hybrid switch and quadratic-core diagnostics" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "shallow-22", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
flatness_thresholdzero_crossing_neuronsquadratic_neuronsmean_affine_rho1mean_selected_rho1mean_quadratic_core_box_radius
00.0130040.2830510.2794910.0
\n", + "
" + ], + "text/plain": [ + " flatness_threshold ... mean_quadratic_core_box_radius\n", + "0 0.01 ... 0.0\n", + "\n", + "[1 rows x 6 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
neuronpreactivation_lowerpreactivation_upperrelative_slopeapproximation_kindaffine_approximation_error_radiusselected_approximation_error_radiusquadratic_core_box_radius
018-1.6733731.6691490.001195quadratic0.4340490.1354970.0
156-0.7083440.7036150.009753quadratic0.1849200.0143510.0
260-1.5860531.5733840.004301quadratic0.4218660.1230050.0
3101-1.5837881.5801230.001241quadratic0.4221940.1222940.0
\n", + "
" + ], + "text/plain": [ + " neuron ... quadratic_core_box_radius\n", + "0 18 ... 0.0\n", + "1 56 ... 0.0\n", + "2 60 ... 0.0\n", + "3 101 ... 0.0\n", + "\n", + "[4 rows x 8 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "hybrid_record = medium_results[\"hybrid_pz_uncompressed_reverse_symbolic\"][\"activation\"][0]\n", + "hybrid_neuron_diagnostics = pd.DataFrame({\n", + " \"neuron\": np.arange(len(hybrid_record[\"rho1\"])),\n", + " \"preactivation_lower\": np.asarray(hybrid_record[\"preactivation_lower\"], dtype=float),\n", + " \"preactivation_upper\": np.asarray(hybrid_record[\"preactivation_upper\"], dtype=float),\n", + " \"relative_slope\": np.asarray(hybrid_record[\"relative_slope\"], dtype=float),\n", + " \"approximation_kind\": np.asarray(hybrid_record[\"kind1\"], dtype=object),\n", + " \"affine_approximation_error_radius\": np.asarray(hybrid_record[\"affine_rho1\"], dtype=float),\n", + " \"selected_approximation_error_radius\": np.asarray(hybrid_record[\"rho1\"], dtype=float),\n", + " \"quadratic_core_box_radius\": np.asarray(hybrid_record[\"quadratic_core_box_radius\"], dtype=float),\n", + "})\n", + "hybrid_summary = pd.DataFrame([{\n", + " \"flatness_threshold\": 0.01,\n", + " \"zero_crossing_neurons\": int(np.count_nonzero(\n", + " (hybrid_neuron_diagnostics.preactivation_lower <= 0.0)\n", + " & (hybrid_neuron_diagnostics.preactivation_upper >= 0.0)\n", + " )),\n", + " \"quadratic_neurons\": int(np.count_nonzero(hybrid_neuron_diagnostics.approximation_kind == \"quadratic\")),\n", + " \"mean_affine_rho1\": hybrid_neuron_diagnostics.affine_approximation_error_radius.mean(),\n", + " \"mean_selected_rho1\": hybrid_neuron_diagnostics.selected_approximation_error_radius.mean(),\n", + " \"mean_quadratic_core_box_radius\": hybrid_neuron_diagnostics.quadratic_core_box_radius.mean(),\n", + "}])\n", + "display(hybrid_summary)\n", + "display(hybrid_neuron_diagnostics.query(\"approximation_kind == 'quadratic'\").reset_index(drop=True))\n" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-23", + "metadata": {}, + "source": [ + "### Required layerwise normalized postactivation-radius tables" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "shallow-24", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'interval'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
neuronlayer_0
000.709762
110.634675
220.744612
330.729833
440.825630
.........
2952950.957255
2962960.664214
2972970.816254
2982980.738089
2992990.671691
\n", + "

300 rows × 2 columns

\n", + "
" + ], + "text/plain": [ + " neuron layer_0\n", + "0 0 0.709762\n", + "1 1 0.634675\n", + "2 2 0.744612\n", + "3 3 0.729833\n", + "4 4 0.825630\n", + ".. ... ...\n", + "295 295 0.957255\n", + "296 296 0.664214\n", + "297 297 0.816254\n", + "298 298 0.738089\n", + "299 299 0.671691\n", + "\n", + "[300 rows x 2 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'affine_pz_topk96_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
neuronlayer_0
000.616242
110.541922
220.652533
330.637024
440.742724
.........
2952950.921597
2962960.570643
2972970.731747
2982980.645625
2992990.577986
\n", + "

300 rows × 2 columns

\n", + "
" + ], + "text/plain": [ + " neuron layer_0\n", + "0 0 0.616242\n", + "1 1 0.541922\n", + "2 2 0.652533\n", + "3 3 0.637024\n", + "4 4 0.742724\n", + ".. ... ...\n", + "295 295 0.921597\n", + "296 296 0.570643\n", + "297 297 0.731747\n", + "298 298 0.645625\n", + "299 299 0.577986\n", + "\n", + "[300 rows x 2 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_topk96_B_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
neuronlayer_0
000.616242
110.541922
220.652533
330.637024
440.742724
.........
2952950.921597
2962960.570643
2972970.731747
2982980.645625
2992990.577986
\n", + "

300 rows × 2 columns

\n", + "
" + ], + "text/plain": [ + " neuron layer_0\n", + "0 0 0.616242\n", + "1 1 0.541922\n", + "2 2 0.652533\n", + "3 3 0.637024\n", + "4 4 0.742724\n", + ".. ... ...\n", + "295 295 0.921597\n", + "296 296 0.570643\n", + "297 297 0.731747\n", + "298 298 0.645625\n", + "299 299 0.577986\n", + "\n", + "[300 rows x 2 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "'hybrid_pz_uncompressed_reverse_symbolic'" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
neuronlayer_0
000.616242
110.541922
220.652533
330.637024
440.742724
.........
2952950.921597
2962960.570643
2972970.731747
2982980.645625
2992990.577986
\n", + "

300 rows × 2 columns

\n", + "
" + ], + "text/plain": [ + " neuron layer_0\n", + "0 0 0.616242\n", + "1 1 0.541922\n", + "2 2 0.652533\n", + "3 3 0.637024\n", + "4 4 0.742724\n", + ".. ... ...\n", + "295 295 0.921597\n", + "296 296 0.570643\n", + "297 297 0.731747\n", + "298 298 0.645625\n", + "299 299 0.577986\n", + "\n", + "[300 rows x 2 columns]" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "for method_id, output in medium_outputs.items():\n", + " display(method_id, output[\"layer_radius_Y\"])" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-25", + "metadata": {}, + "source": [ + "### Ranked worst activation enclosures" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "shallow-26", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "( method_id ... normalized_approximation_radius\n", + " 0 interval ... 0.999701\n", + " 1 interval ... 0.985955\n", + " 2 interval ... 0.985575\n", + " 3 interval ... 0.976566\n", + " 4 interval ... 0.975547\n", + " 5 interval ... 0.975511\n", + " 6 interval ... 0.966316\n", + " 7 interval ... 0.963651\n", + " 8 interval ... 0.963562\n", + " 9 interval ... 0.963072\n", + " 10 interval ... 0.961890\n", + " 11 interval ... 0.961794\n", + " 12 interval ... 0.957254\n", + " 13 interval ... 0.953726\n", + " 14 interval ... 0.953364\n", + " 15 interval ... 0.951152\n", + " 16 interval ... 0.950026\n", + " 17 interval ... 0.948308\n", + " 18 interval ... 0.946785\n", + " 19 interval ... 0.946229\n", + " \n", + " [20 rows x 9 columns],\n", + " method_id ... normalized_approximation_radius\n", + " 0 interval ... 0.999701\n", + " 1 interval ... 0.985955\n", + " 2 interval ... 0.985575\n", + " 3 interval ... 0.976566\n", + " 4 interval ... 0.975547\n", + " 5 interval ... 0.975511\n", + " 6 interval ... 0.966316\n", + " 7 interval ... 0.963651\n", + " 8 interval ... 0.963562\n", + " 9 interval ... 0.963072\n", + " 10 interval ... 0.961890\n", + " 11 interval ... 0.961794\n", + " 12 interval ... 0.957254\n", + " 13 interval ... 0.953726\n", + " 14 interval ... 0.953364\n", + " 15 interval ... 0.951152\n", + " 16 interval ... 0.950026\n", + " 17 interval ... 0.948308\n", + " 18 interval ... 0.946785\n", + " 19 interval ... 0.946229\n", + " \n", + " [20 rows x 9 columns])" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "largest_absolute_activation_radii = (\n", + " medium_activation_summary\n", + " .sort_values(\"approximation_error_radius\", ascending=False, kind=\"stable\")\n", + " [[\"method_id\", \"layer\", \"neuron\", \"derivative_order\", \"preactivation_lower\",\n", + " \"preactivation_upper\", \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\"]]\n", + " .head(20).reset_index(drop=True)\n", + ")\n", + "largest_normalized_activation_radii = (\n", + " medium_activation_summary\n", + " .sort_values(\"normalized_approximation_radius\", ascending=False, kind=\"stable\")\n", + " [[\"method_id\", \"layer\", \"neuron\", \"derivative_order\", \"preactivation_lower\",\n", + " \"preactivation_upper\", \"approximation_kind\", \"approximation_error_radius\",\n", + " \"normalized_approximation_radius\"]]\n", + " .head(20).reset_index(drop=True)\n", + ")\n", + "largest_absolute_activation_radii, largest_normalized_activation_radii" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-27", + "metadata": {}, + "source": [ + "### Validity and sampled-containment diagnostics" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "shallow-28", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
method_idrun_idquantitysample_countfailure_countmax_violationinvalid_interval_countnan_endpoint_countinfinite_endpoint_countstatus
0intervalpinn100d_shallow300_medium_intervalY1638400.0000ok
1intervalpinn100d_shallow300_medium_intervalJ1638400.0000ok
2affine_pz_topk96_symbolicpinn100d_shallow300_medium_affine_pz_topk96_sy...Y1638400.0000ok
3affine_pz_topk96_symbolicpinn100d_shallow300_medium_affine_pz_topk96_sy...J1638400.0000ok
4hybrid_pz_topk96_B_symbolicpinn100d_shallow300_medium_hybrid_pz_topk96_B_...Y1638400.0000ok
5hybrid_pz_topk96_B_symbolicpinn100d_shallow300_medium_hybrid_pz_topk96_B_...J1638400.0000ok
6hybrid_pz_uncompressed_reverse_symbolicpinn100d_shallow300_medium_hybrid_pz_uncompres...Y1638400.0000ok
7hybrid_pz_uncompressed_reverse_symbolicpinn100d_shallow300_medium_hybrid_pz_uncompres...J1638400.0000ok
\n", + "
" + ], + "text/plain": [ + " method_id ... status\n", + "0 interval ... ok\n", + "1 interval ... ok\n", + "2 affine_pz_topk96_symbolic ... ok\n", + "3 affine_pz_topk96_symbolic ... ok\n", + "4 hybrid_pz_topk96_B_symbolic ... ok\n", + "5 hybrid_pz_topk96_B_symbolic ... ok\n", + "6 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "7 hybrid_pz_uncompressed_reverse_symbolic ... ok\n", + "\n", + "[8 rows x 10 columns]" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pd.concat([\n", + " output[\"soundness\"].assign(method_id=method_id)\n", + " for method_id, output in medium_outputs.items()\n", + "], ignore_index=True)[[\"method_id\"] + SOUNDNESS_COLUMNS]" + ] + }, + { + "cell_type": "markdown", + "id": "shallow-29", + "metadata": {}, + "source": [ + "## Interpretation\n", + "\n", + "The shallow architecture is a useful positive result for certifiability: both\n", + "PZ variants are far tighter than interval arithmetic and their certified\n", + "$W^{1,2}$ upper bounds are much closer to the sampled network norm than in the\n", + "three-hidden-layer benchmark. The strict hybrid switch selects only the most\n", + "symmetric derivative bumps. Its local approximation-error radii decrease, but\n", + "with the current expanded Top-96 representation the resulting degree-two terms\n", + "still introduce enough reduction radius to make the final hybrid certificate\n", + "slightly wider than the purely affine derivative enclosure. The notebook\n", + "therefore reports both quantities separately rather than attributing the final\n", + "width to the quadratic approximation itself." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/pinn_aposteriori_square_poisson.ipynb b/notebooks/pinn_aposteriori_square_poisson.ipynb new file mode 100644 index 0000000..13016c2 --- /dev/null +++ b/notebooks/pinn_aposteriori_square_poisson.ipynb @@ -0,0 +1,820 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f05571f6", + "metadata": {}, + "source": [ + "# PINN a posteriori estimator on the unit square\n", + "\n", + "This notebook trains a PINN for the manufactured Poisson problem on $\\Omega=[0,1]^2$, and computes certified interval enclosures for:\n", + "\n", + "- the interior residual norm $\\|r_\\theta\\|_{L^2(\\Omega)}$ using **interval Hessian bounds**,\n", + "- the boundary mismatch norm $\\|u_\\theta-g\\|_{L^2(\\partial\\Omega)}$,\n", + "- and the combined estimator $\\eta_\\theta$." + ] + }, + { + "cell_type": "markdown", + "id": "d552ece5", + "metadata": {}, + "source": [ + "## 1) Setup and imports" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "3452a527", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "torch=2.0.0+cu117, dtype=torch.float64, seed=1234\n" + ] + } + ], + "source": [ + "from __future__ import annotations\n", + "\n", + "import math\n", + "import random\n", + "from pathlib import Path\n", + "\n", + "import matplotlib.pyplot as plt\n", + "from pathlib import Path as _Path\n", + "from IPython.display import Image, display\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "import torch.nn as nn\n", + "\n", + "import os\n", + "os.environ.setdefault(\"KMP_DUPLICATE_LIB_OK\", \"TRUE\")\n", + "\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / \"src\" / \"intervalnets\").exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "if str(repo_root / \"src\") not in __import__('sys').path:\n", + " __import__('sys').path.insert(0, str(repo_root / \"src\"))\n", + "\n", + "from intervalnets import Interval, IntervalTensor, enable_interval_eval\n", + "\n", + "enable_interval_eval(enclosure_mode=\"slope\")\n", + "\n", + "dtype = torch.float64\n", + "device = torch.device(\"cpu\")\n", + "SEED = 1234\n", + "random.seed(SEED)\n", + "np.random.seed(SEED)\n", + "torch.manual_seed(SEED)\n", + "print(f\"torch={torch.__version__}, dtype={dtype}, seed={SEED}\")\n", + "\n", + "\n", + "PLOT_OUTPUT_DIR = _Path(\"notebooks\") / \"artifacts\"\n", + "PLOT_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)\n", + "def finalize_figure(fig, filename: str, show_inline: bool = True):\n", + " out_path = PLOT_OUTPUT_DIR / filename\n", + " fig.savefig(out_path, dpi=160, bbox_inches=\"tight\")\n", + " print(f\"saved figure: {out_path}\")\n", + " if show_inline:\n", + " display(Image(filename=str(out_path)))\n", + " plt.close(fig)" + ] + }, + { + "cell_type": "markdown", + "id": "25025f33", + "metadata": {}, + "source": [ + "## 2) Problem definition (PDE, localized forcing, BC)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "5a367b34", + "metadata": {}, + "outputs": [], + "source": [ + "PI = math.pi\n", + "\n", + "# Two strongly localized Gaussian bumps for the RHS forcing term.\n", + "LOCAL_BUMPS = [\n", + " # (x_center, y_center, sigma, amplitude)\n", + " (0.28, 0.72, 0.055, 95.0),\n", + " (0.74, 0.31, 0.045, -80.0),\n", + "]\n", + "\n", + "\n", + "def forcing_f(xy: torch.Tensor) -> torch.Tensor:\n", + " x = xy[:, 0:1]\n", + " y = xy[:, 1:2]\n", + " f = torch.zeros_like(x)\n", + " for cx, cy, sigma, amp in LOCAL_BUMPS:\n", + " r2 = (x - cx) ** 2 + (y - cy) ** 2\n", + " f = f + amp * torch.exp(-r2 / (2.0 * sigma ** 2))\n", + " return f\n", + "\n", + "\n", + "def forcing_interval_on_box(box: IntervalTensor) -> Interval:\n", + " x_lo, y_lo = float(box.lower[0]), float(box.lower[1])\n", + " x_hi, y_hi = float(box.upper[0]), float(box.upper[1])\n", + "\n", + " f_lo = 0.0\n", + " f_hi = 0.0\n", + "\n", + " for cx, cy, sigma, amp in LOCAL_BUMPS:\n", + " # Squared-distance range from a box to bump center.\n", + " if x_lo <= cx <= x_hi:\n", + " dx_min = 0.0\n", + " else:\n", + " dx_min = min(abs(x_lo - cx), abs(x_hi - cx))\n", + " if y_lo <= cy <= y_hi:\n", + " dy_min = 0.0\n", + " else:\n", + " dy_min = min(abs(y_lo - cy), abs(y_hi - cy))\n", + " r2_min = dx_min ** 2 + dy_min ** 2\n", + "\n", + " dx_max = max(abs(x_lo - cx), abs(x_hi - cx))\n", + " dy_max = max(abs(y_lo - cy), abs(y_hi - cy))\n", + " r2_max = dx_max ** 2 + dy_max ** 2\n", + "\n", + " gauss_min = math.exp(-r2_max / (2.0 * sigma ** 2))\n", + " gauss_max = math.exp(-r2_min / (2.0 * sigma ** 2))\n", + "\n", + " if amp >= 0.0:\n", + " term_lo = amp * gauss_min\n", + " term_hi = amp * gauss_max\n", + " else:\n", + " term_lo = amp * gauss_max\n", + " term_hi = amp * gauss_min\n", + "\n", + " f_lo += term_lo\n", + " f_hi += term_hi\n", + "\n", + " return Interval.from_bounds(f_lo, f_hi)\n", + "\n", + "\n", + "def g_boundary(xy: torch.Tensor) -> torch.Tensor:\n", + " return torch.zeros((xy.shape[0], 1), dtype=xy.dtype, device=xy.device)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "fd0031c8", + "metadata": {}, + "source": [ + "## 3) PINN model" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "1f4f4c83", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Sequential(\n", + " (0): Linear(in_features=2, out_features=64, bias=True)\n", + " (1): Tanh()\n", + " (2): Linear(in_features=64, out_features=64, bias=True)\n", + " (3): Tanh()\n", + " (4): Linear(in_features=64, out_features=1, bias=True)\n", + ")" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def make_pinn(width: int = 64, hidden_layers: int = 4) -> nn.Sequential:\n", + " layers: list[nn.Module] = [nn.Linear(2, width), nn.Tanh()]\n", + " for _ in range(hidden_layers - 1):\n", + " layers += [nn.Linear(width, width), nn.Tanh()]\n", + " layers += [nn.Linear(width, 1)]\n", + " return nn.Sequential(*layers)\n", + "\n", + "model = make_pinn(width=64, hidden_layers=2).to(device=device, dtype=dtype)\n", + "model" + ] + }, + { + "cell_type": "markdown", + "id": "3c34555c", + "metadata": {}, + "source": [ + "## 4) Training data sampling (interior + boundary)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "bcd94222", + "metadata": {}, + "outputs": [], + "source": [ + "def sample_interior(n: int) -> torch.Tensor:\n", + " return torch.rand((n, 2), dtype=dtype, device=device)\n", + "\n", + "\n", + "def sample_boundary(n_per_edge: int) -> torch.Tensor:\n", + " t = torch.rand((n_per_edge, 1), dtype=dtype, device=device)\n", + " z = torch.zeros_like(t)\n", + " o = torch.ones_like(t)\n", + " return torch.cat([\n", + " torch.cat([t, z], dim=1),\n", + " torch.cat([t, o], dim=1),\n", + " torch.cat([z, t], dim=1),\n", + " torch.cat([o, t], dim=1),\n", + " ], dim=0)\n", + "\n", + "\n", + "def laplacian_u_autograd(model: nn.Module, xy: torch.Tensor, create_graph: bool = True) -> torch.Tensor:\n", + " # We always need first-derivative graph construction to differentiate once more.\n", + " # The create_graph flag controls whether we keep graph info beyond second derivatives.\n", + " with torch.enable_grad():\n", + " xy_req = xy.detach().clone().requires_grad_(True)\n", + " u = model(xy_req)\n", + " grad_u = torch.autograd.grad(u, xy_req, grad_outputs=torch.ones_like(u), create_graph=True)[0]\n", + " u_xx = torch.autograd.grad(\n", + " grad_u[:, 0:1],\n", + " xy_req,\n", + " grad_outputs=torch.ones_like(grad_u[:, 0:1]),\n", + " retain_graph=True,\n", + " create_graph=create_graph,\n", + " )[0][:, 0:1]\n", + " u_yy = torch.autograd.grad(\n", + " grad_u[:, 1:2],\n", + " xy_req,\n", + " grad_outputs=torch.ones_like(grad_u[:, 1:2]),\n", + " create_graph=create_graph,\n", + " )[0][:, 1:2]\n", + " return u_xx + u_yy\n", + "\n", + "\n", + "def residual_r(model: nn.Module, xy: torch.Tensor, create_graph: bool = True) -> torch.Tensor:\n", + " return -laplacian_u_autograd(model, xy, create_graph=create_graph) - forcing_f(xy)\n" + ] + }, + { + "cell_type": "markdown", + "id": "0111725c", + "metadata": {}, + "source": [ + "## 5) Training loop" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "2cb193c8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch= 200 total=5.317e+01 interior=5.307e+01 boundary=4.861e-03\n", + "epoch= 400 total=2.171e+01 interior=2.161e+01 boundary=5.158e-03\n", + "epoch= 600 total=1.002e+01 interior=9.944e+00 boundary=3.642e-03\n", + "epoch= 800 total=6.141e+00 interior=6.042e+00 boundary=4.927e-03\n", + "epoch=1000 total=3.956e+00 interior=3.866e+00 boundary=4.492e-03\n", + "epoch=1200 total=4.350e+00 interior=4.279e+00 boundary=3.555e-03\n", + "epoch=1400 total=2.896e+00 interior=2.865e+00 boundary=1.561e-03\n", + "epoch=1600 total=2.322e+00 interior=2.288e+00 boundary=1.712e-03\n", + "epoch=1800 total=2.002e+00 interior=1.969e+00 boundary=1.656e-03\n", + "epoch=2000 total=1.669e+00 interior=1.635e+00 boundary=1.708e-03\n" + ] + } + ], + "source": [ + "EPOCHS = 2000\n", + "LR = 1e-3\n", + "W_INTERIOR = 1.0\n", + "W_BOUNDARY = 20.0\n", + "N_INTERIOR = 1024\n", + "N_BDRY_PER_EDGE = 256\n", + "WEIGHT_DECAY = 1e-5\n", + "\n", + "opt = torch.optim.Adam(model.parameters(), lr=LR, weight_decay=WEIGHT_DECAY)\n", + "history = {\"total\": [], \"interior\": [], \"boundary\": []}\n", + "\n", + "for epoch in range(1, EPOCHS + 1):\n", + " xi = sample_interior(N_INTERIOR)\n", + " xb = sample_boundary(N_BDRY_PER_EDGE)\n", + "\n", + " ri = residual_r(model, xi)\n", + " bm = model(xb) - g_boundary(xb)\n", + "\n", + " li = torch.mean(ri ** 2)\n", + " lb = torch.mean(bm ** 2)\n", + " loss = W_INTERIOR * li + W_BOUNDARY * lb\n", + "\n", + " opt.zero_grad()\n", + " loss.backward()\n", + " opt.step()\n", + "\n", + " history[\"total\"].append(float(loss.detach().cpu()))\n", + " history[\"interior\"].append(float(li.detach().cpu()))\n", + " history[\"boundary\"].append(float(lb.detach().cpu()))\n", + "\n", + " if epoch % 200 == 0:\n", + " print(f\"epoch={epoch:4d} total={history['total'][-1]:.3e} interior={history['interior'][-1]:.3e} boundary={history['boundary'][-1]:.3e}\")\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "450dafae", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnEAAAGHCAYAAADfrgShAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAChJElEQVR4nOzdd3RURRvA4d/d9B5SCRAIHULvvSMQuiigqBQBQVBBsGD5FBDFilgCVqoNUKR36dJ7Cb2GmhBI77v3+2PNJpvdTSOVvM85HPfOnTt3JgnyZqqiqqqKEEIIIYQoUTRFXQEhhBBCCJF7EsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAEsQJIQwWLFiAoiiGP9bW1lSoUIERI0Zw8+ZNQ77t27ejKAp//vmnybP29vZcu3bNpOyOHTtSt25do7SAgAAURWHs2LEm+c29w5xbt24xdepUjh07lsvW5kxau65evZrrZ69evYqiKCxYsCDf6yWEEBLECSFMzJ8/n71797J582ZGjx7N77//Trt27YiLi8v22aSkJN59991cve/nn3/m3LlzearrrVu3mDZtWoEFcb169WLv3r34+fnl+lk/Pz/27t1Lr169CqBmQojSToI4IYSJunXr0rJlSzp16sT777/PG2+8wZUrV1ixYkW2z/bo0YPffvuN48eP5+hdrVq1wsnJibfffvsha50z8fHxucrv7e1Ny5YtsbOzy/W77OzsaNmyJd7e3rl+tiTI7ddSCJG/JIgTQmSrZcuWAGaHSTN744038PT05M0338xR2R4eHkyZMoXly5ezb9++XNVr+/btNGvWDIARI0YYhoGnTp0KwPDhw3F2dubkyZN069YNFxcXunTpAsDmzZvp168fFSpUwN7enmrVqjFmzBju3btn9A5zw6lpQ8MHDx6kXbt2ODo6UqVKFT7++GN0Op0hn7nh1KlTp6IoCqdPn+bpp5/Gzc0NX19fnn/+eaKioozeHRkZyciRI/Hw8MDZ2ZlevXpx+fJlozZmJTIyksmTJ1OlShXs7Ozw8fGhZ8+enD171vD1UxSF7du3Gz1nrt6WvpYTJ07EycmJ6Ohok/cPHjwYX19fUlJSDGlLliwxBO7Ozs50796do0ePGj13+fJlnnrqKcqVK4ednR2+vr506dKlwHpbhSipJIgTQmTr4sWLADnqUXJxceHdd99l48aNbN26NUflT5gwgfLly/PGG2/kql6NGzdm/vz5ALz77rvs3buXvXv3MmrUKEOe5ORk+vbtS+fOnVm5ciXTpk0D4NKlS7Rq1Yq5c+eyadMm3nvvPfbv30/btm2Ngg5L7ty5wzPPPMOzzz7LqlWrCAoK4q233uKXX37JUd2feOIJatSowV9//cWUKVP47bffePXVVw33dTodffr04bfffuPNN9/k77//pkWLFvTo0SNH5cfExNC2bVu+//57RowYwerVq/nuu++oUaMGt2/fzlEZmZn7Wj7//PPEx8ezdOlSo7yRkZGsXLmSZ599FhsbGwA++ugjnn76aQIDA1m6dCmLFy8mJiaGdu3aERISYni2Z8+eHD58mE8//ZTNmzczd+5cGjVqRGRkZJ7qLcQjSxVCiP/Mnz9fBdR9+/apKSkpakxMjLpmzRrV29tbdXFxUe/cuaOqqqpu27ZNBdRly5aZPHvw4EE1KSlJrVKlitq0aVNVp9OpqqqqHTp0UOvUqWP0vkqVKqm9evVSVVVVf/zxRxVQV69ebfEd5hw8eFAF1Pnz55vcGzZsmAqo8+bNy7IMnU6npqSkqNeuXVMBdeXKlSbtunLliiGtQ4cOKqDu37/fqJzAwEC1e/fuhusrV66Y1O39999XAfXTTz81enbcuHGqvb294eu1du1aFVDnzp1rlG/mzJkqoL7//vtZtmn69OkqoG7evNlinrSv8bZt24zSzdU7q69l48aN1datWxulzZkzRwXUkydPqqqqqtevX1etra3Vl19+2ShfTEyMWrZsWXXQoEGqqqrqvXv3VECdPXt2lu0TQqiq9MQJIUy0bNkSGxsbXFxc6N27N2XLlmX9+vX4+vrm6HlbW1tmzJjBoUOHTHpoLBkxYgSBgYFMmTLFaEgyPzzxxBMmaWFhYYwdOxZ/f3+sra2xsbGhUqVKAJw5cybbMsuWLUvz5s2N0urXr5+jIWeAvn37mjybmJhIWFgYADt27ABg0KBBRvmefvrpHJW/fv16atSoQdeuXXOUP6fMfS1HjBjBnj17jBanzJ8/n2bNmhlWJG/cuJHU1FSGDh1Kamqq4Y+9vT0dOnQwDOl6eHhQtWpVPvvsM2bNmsXRo0fz/edBiEeFBHFCCBOLFi3i4MGDHD16lFu3bnHixAnatGmTqzKeeuopGjduzDvvvJOj4UkrKys++ugjTp8+zcKFC/NadROOjo64uroapel0Orp168by5ct54403+Oeffzhw4IBhTl5CQkK25Xp6epqk2dnZ5ehZc8+nLZxIez4iIgJra2s8PDyM8uU0kA4PD6dChQo5yptT5r6WAM888wx2dnaGOXQhISEcPHiQESNGGPLcvXsXgGbNmmFjY2P0Z8mSJYa5iIqi8M8//9C9e3c+/fRTGjdujLe3N6+88goxMTH52h4hSjrroq6AEKL4qV27Nk2bNn2oMhRF4ZNPPuGxxx7jhx9+yNEz/fr1o02bNrz//vs5fiYn9cjs1KlTHD9+nAULFjBs2DBDetrcv+LA09OT1NRU7t+/bxTI3blzJ0fPe3t7c+PGjSzz2NvbA/ptYTLKvLgjjbmvJUCZMmXo168fixYtYsaMGcyfPx97e3ujXkMvLy8A/vzzT0OPpyWVKlXi559/BuD8+fMsXbqUqVOnkpyczHfffZfls0KUJtITJ4QoMF27duWxxx5j+vTpxMbG5uiZTz75hNDQUL7++usc5c/cg5UTacFI5m1Dvv/++xyXUdA6dOgA6FdzZvTHH3/k6PmgoCDOnz+f5eKSgIAAAE6cOGGUvmrVqlzUVG/EiBHcunWLdevW8csvv/D444/j7u5uuN+9e3esra25dOkSTZs2NfvHnBo1avDuu+9Sr149jhw5kut6CfEok544IUSB+uSTT2jSpAlhYWHUqVMn2/xt2rShX79+rFy5MkflV61aFQcHB3799Vdq166Ns7Mz5cqVo1y5chafqVWrFlWrVmXKlCmoqoqHhwerV69m8+bNOW5XQevRowdt2rRh8uTJREdH06RJE/bu3cuiRYsA0Giy/h184sSJLFmyhH79+jFlyhSaN29OQkICO3bsoHfv3nTq1ImyZcvStWtXZs6cSZkyZahUqRL//PMPy5cvz3V9u3XrRoUKFRg3bhx37twxGkoFfcA4ffp03nnnHS5fvkyPHj0oU6YMd+/e5cCBAzg5OTFt2jROnDjBSy+9xMCBA6levTq2trZs3bqVEydOMGXKlFzXS4hHmfTECSEKVKNGjXI8GT/NzJkzsbKyylFeR0dH5s2bR0REBN26daNZs2bZDsXa2NiwevVqatSowZgxY3j66acJCwtjy5YtuapnQdJoNKxevZqnnnqKjz/+mH79+rFr1y7DFiYZe7nMcXFxYffu3YwcOZIffviBXr16MXr0aM6dO2cU4C5evJguXbrw5ptvMnDgQG7evMnvv/+ep/oOHTqUGzdu4O/vb9iPL6O33nqLP//8k/PnzzNs2DC6d+/OG2+8wbVr12jfvj2gXzBStWpV5syZw5NPPkm/fv1YvXo1X3zxBdOnT891vYR4lCmqqqpFXQkhhBA589tvv/HMM8/w77//0rp166KujhCiCEkQJ4QQxdTvv//OzZs3qVevHhqNhn379vHZZ5/RqFEjwxYkQojSS+bECSFEMeXi4sIff/zBjBkziIuLw8/Pj+HDhzNjxoyirpoQohiQnjghhBBCiBJIFjYIIYQQQpRAEsQJIYQQQpRAEsQJIYQQQpRAsrAhGzqdjlu3buHi4mLxyBkhhBBCiPygqioxMTGUK1cu2029JYjLxq1bt/D39y/qagghhBCiFAkNDaVChQpZ5pEgLhsuLi6A/ovp6uqa7+XrdDrCw8Px9vbONuJ+lEi7S0+7S2ObQdpdmtpdGtsM0u6Cand0dDT+/v6G+CMrEsRlI20I1dXVtcCCuMTERFxdXUvdXwJpd+lod2lsM0i7S1O7S2ObQdpd0O3OyRSu0vNVz6Xg4GACAwNp1qxZUVdFCCGEEMKEBHEWjB8/npCQEA4ePFjUVRFCCCGEMCFBnBBCCCFECSRz4oQQQogSQlVVUlNT0Wq1RV0VdDodKSkpJCYmlro5cQ/TbisrK6ytrfNl2zIJ4oQQQogSIDk5mdu3bxMfH1/UVQH0AaVOpyMmJqZU7aOaH+12dHTEz88PW1vbh6qLBHFCCCFEMafT6bhy5QpWVlaUK1cOW1vbIg+c0noF86tXqaR4mHarqkpycjLh4eFcuXKF6tWrP1QvpgRxQgghRDGXnJyMTqfD398fR0fHoq4OIEFcXtvt4OCAjY0N165dIzk5GXt7+zzXpfQMYgshhBAlXGmae/Yoy6/vo/w0CCGEEEKUQBLEWVDcNvu9dv444XdvFHU1hBBCCFFMSBBnQXHa7PfuzctU+q093nPrFHVVhBBCiGJr+PDh9O/fv6irUWgkiCsBbofsK+oqCCGEEHnSsWNHJk6cWODPlEYSxBUj925f5/wHTdi35FOj9JTEuCKqkRBCCCGKKwniipGrv02ghvYiLc98aEjb++sMmh1+zSRvcditWwghRNFQVZX45NQi+aOqao7rOXz4cHbs2MFXX32FoigoisLVq1fZsWMHzZs3x87ODj8/P6ZMmUJqamqWz2i1WkaOHEnlypVxcHCgZs2afPXVVwX1JS4RZJ+4YuLBvbv4xxw3SW914TOja1Wn4/S+jVTc+Dxn6k+hxRMTCquKQgghiomEFC2B720skneHTO+Oo23OwoevvvqK8+fPU7duXaZPnw7oOyF69uzJ8OHDWbRoEWfPnmX06NHY29szdepUs894e3uj0+moUKECS5cuxcvLiz179vDCCy/g5+fHoEGDCqy9xZkEccVAakoyZX+uZ5R24/IZbO0c8cmU9+j25TTeORIUaHHyPfgviFN1OvavW0DZms0IqF4PIYQQoqi5ublha2uLo6MjZcuWBeCdd97B39+fb7/9FkVRqFWrFrdu3eLNN9/kvffeM/sM6M8cnTZtmuG6cuXK7Nmzh6VLl0oQJ4rOlb/+R4VMaRUWtTSbt/HOkWbTD37en5bxO+AQMDUqfysohBCiWHGwsSJkevcie/fDOHPmDK1atTI67aBNmzbExsZy48YNKlasaPHZ7777jp9++olr166RkJBAcnIyDRs2fKj6lGQSxBWx8NvXaBe54qHKSEyIp3n8jizzXDx1gLCtwVR7Yjo+5Ss91PuEEEIULUVRcjykWdyoqmpyXFXaPLusjrFaunQpr776Kl988QWtWrXCxcWFzz77jP379xdofYuzkvkT8Ai5snw6vg9ZxoOwG/hlk6f8sl5UU5I5tvgmPlO2POQbhRBCiJyxtbU1WowXGBjIX3/9ZRTM7dmzBxcXF8qXL2/2GYBdu3bRunVrxo0bZ0i7dOlSIbSg+JLVqUXMMe76Qz3/73cvEXp8q1FaSkqyST4HRZ8WkHjmod4nhBBC5EZAQAD79+/n6tWr3Lt3j3HjxhEaGsrLL7/M2bNnWblyJe+//z6TJk0ynCma+RmdTke1atU4dOgQGzdu5Pz58/zvf/8rFhvyF6VHPogLDQ2lY8eOBAYGUr9+fZYtW1bUVTKSZOf1UM+3ubOY5kfeNEqLeXAPgL3LZnHws35ERd433HNSE9gwrTdb1ix5qPcKIYQQOfHaa69hZWVFYGAg3t7epKSksG7dOg4cOECDBg0YO3YsI0eO5N1337X4zPXr1xk7diwDBgxg8ODBtGjRgoiICKNeudJIUXOz4UsJdPv2be7evUvDhg0JCwujcePGnDt3Dicnpxw9Hx0djZubG1FRUbi6uuZ7/Y581pvGcbvytcw7ahkO+w6iV9j3AOzxeYrWYX+YZizCBRA6nY6wsDB8fHwMv3mVBqWx3aWxzSDtLk3tLow2JyYmcuXKFSpXroy9vX2BvCO3VFUlNTUVa2vrLOeyPWryo91ZfT9zE3c88n/D/Pz8DCtXfHx88PDw4P79+1k/VIistQkmaResa3CopukGvzlVVnlgCOAAnKPOm82nqipHti1n3yd9CbsdCkBMQjI63SMd1wshhBCPhGIfxO3cuZM+ffpQrlw5FEVhxYoVJnnmzJljiGabNGnCrl3me7YOHTqETqfD39+/gGudczZmgrgI31Y0ffp/HHbumC/vqJ90xGz6+l+/pPGOEbRM2MHNn59hzc8zSPy4Gv98OZxDO9aw9ZuxxMfHcf9eGIdmdmPvqp/ypT5CCCGEeHjFfnVqXFwcDRo0YMSIETzxxBMm95csWcLEiROZM2cObdq04fvvvycoKIiQkBCjvWYiIiIYOnQoP/1UvAKRWOcAeHDaKE39rydMo0sp0Hf3vJi+aWKj1OM0Cj0OCjwWswK2rQBg5x8eWMffpXXSfjiyH/qOKtA6CSGEECJnin0QFxQURFBQkMX7s2bNYuTIkYwapQ8uZs+ezcaNG5k7dy4zZ84EICkpiccff5y33nqL1q1bZ/m+pKQkkpKSDNfR0dGAfs6DTqd72OaYaDR+Efu3LKPFvrHpiaoWnU6HRk0P4sJHHcH7p8YP9a5QpRz+6q1cPeN4/wwBMYfgv2H//Poa6HQ6VFUtkK9pcVYa210a2wzS7tLU7sJoc9o70v4UF2l1KU51KgwP2+6076O52CI3P0fFPojLSnJyMocPH2bKlClG6d26dWPPnj2A/gs1fPhwOnfuzHPPPZdtmTNnzjQ61iNNeHg4iYmJ+VPxDHQ6Ha6VGnLMeQl+/4zHV72HfWBPwsLCUFLTtwrRWjtxzK4pDZMOWSzr/IDN1Fj+mMX7ETZ++CfnLohrGrvVEMABhIWFGd2PvB+OvYMT9g6OuSpXp9MRFRWFqqqlZvIzlM52l8Y2g7S7NLW7MNqckpKCTqcjNTXVcFB8UVNV1bCXW2lb2PCw7U5NTUWn0xEREYGNjY3RvZiYmByXU6KDuHv37qHVavH1Nd4u19fXlzt37gDw77//smTJEurXr2+YT7d48WLq1TN/vuhbb73FpEmTDNfR0dH4+/vj7e1dIKtTdTodiqLgXb062ibHCQ+7SQP/6gDEBr0Pqx7nUNnBNPbxQffUN7CwlcWyqgY2huWW35XsVA6SDz9UfX180k9zjQi7Sa2lbQnHHc/3ruSqHEO7vb1Lzf/ooXS2uzS2GaTdpandhdHmxMREYmJisLa2xtq6eP3TnTkIKS0ept3W1tZoNBo8PT1NVqfmZvVx8fpJyCNzx3ekpbVt2zZXXZN2dnbY2dkRHBxMcHCwIdrWaDQF9pdTURQ0Gg3WDs54V6ppSK/WuDNJta7T1NENgHKVAznQYAbNj79rthyrbP5i61zKwYOHq2va1yA5OYVLe1fhDXgTCXn42qS1u7T8jz5NaWx3aWwzSLtLU7sLus0ajQZFUQx/ioOM/9YWlzoVhvxod9r30dzPTG5+hkr03zAvLy+srKwMvW5pwsLCTHrncmv8+PGEhIQU+W7Qdv8FcGkCOw8hjrztEWTtmX9npp7/pC0tj79tuC5t8yGEEEKIolaigzhbW1uaNGnC5s2bjdI3b96c7QKGksrZzRP7t69xvEP2q2wvdQzmaOAb3FM8OGPfCI+A+mbzXbA3P7Rsjlar79Wsqz1rlH5qz3r2zh7C/fDbOS5LCCGEEHlX7IO42NhYjh07xrFjxwC4cuUKx44d4/p1/ZmjkyZN4qeffmLevHmcOXOGV1991XA8x8MIDg4mMDCQZs2aPWwT8p2VrT265Pgs85y3qUXVjs/SaNA7eP7vIrXf3EaVBu3M5i33wjJOVh6Zo3fvWfC22fR6m5+mVeRaLi6ekKNyhBBCPPo6duzIxIkTi+TdCxYswN3dvUjeXViKfRB36NAhGjVqRKNGjQB90NaoUSPee+89AAYPHszs2bOZPn06DRs2ZOfOnaxbt45KlR5u6LC4DKdakhKf9ZFZ8TYehs+Kxgr+G7c/Xe5Jo3wh3j1x8vCj3rBZOXpvu9C5pGSxMsot/mqOyhFCCPHoW758OR988EGO8l69ehVFUQydNg9r8ODBnD9v/sSiR0WxD+I6duxotDdO2p8FCxYY8owbN46rV6+SlJTE4cOHad++fdFVuJC4+ARkeV+rsTWbXvPZL4yuA8f/bvgcUj+9l+2wk+Wvoc0MT4v3PFNuZxnkCSGEKD08PDxwcXEp9PempKTg4OBgtKNCXsspzop9EFdUivNwKkCtVr05Wv89jgZOIdTKn5BOPxvd11mbX/xg7ejOpWb6ffDC7AOM7gUOeDM9n71znurlpUSz/5sRnD2ykwvTG3F8x995KkcIIUQWVBWS44rmTy4WsmUcTg0ICOCjjz7i+eefx8XFhYoVK/LDDz8Y8lauXBmARo0aoSgKHTt2NNybP38+tWvXxt7enlq1ajFnzhzDvbQevKVLl9KxY0fs7e355ZdfzA6nzp07l6pVq2Jra0vNmjVZvHix0X1FUfjuu+/o168fTk5OzJgxI8dtLQqPxBYjBWH8+PGMHz+e6Oho3Nzcsn+gkCkaDY0GTP7v6i2T+3YBLS0+W7XHeKLLVcKruukcuUttP8fq+G8EDJgOP67LU93aRq3iwaptlCEGtg2HDo/nqRwhhBAWpMTDR+WK5t1v3wJbpzw9+sUXX/DBBx/w9ttv8+eff/Liiy/Svn17atWqxYEDB2jevDlbtmyhTp062NrqR5R+/PFH3n//fb799lsaNWrE0aNHGT16NE5OTgwbNsxQ9ptvvskXX3zB/PnzsbOzY9OmTUbv/vvvv5kwYQKzZ8+ma9eurFmzhhEjRlChQgU6depkyPf+++8zc+ZMvvzyS6ysrPLUzsIiQdwjJmzIFiJOb6Ven5ctZ7KywbWR+cCqatfR0HX0Q9ejDDnfcVoIIUTp0LNnT8aNGwfog64vv/yS7du3U6tWLby9vQHw9PSkbNmyhmc++OADvvjiCwYMGADoe+xCQkL4/vvvjYK4iRMnGvKY8/nnnzN8+HDD+ydNmsS+ffv4/PPPjYK4IUOG8Pzzz+dfowuQBHGPGJ8azfCpkT9DwNo3rmL1aUC+lCWEECIf2Tjqe8SK6t15VL9++lZXiqJQtmxZk+McMwoPDyc0NJSRI0cyenR6B0NqaqrJKFnTpk2zfPeZM2d44YUXjNLatGnDV199latyihMJ4izIfGJDaWTlWIZTDs2om1AwK3Rj4hNxSUrBycGuQMoXQohHlqLkeUizKGU+qkpRlCxPVUq79+OPP9KiRQuje5mHOp2csv96ZHXCU27KKS5kYYMFxX2LkcLi2HiQ0fXlrj/m6vlQxfycjdi4OBwWdiHk4w55rpsQQohHR9ocuIydJ76+vpQvX57Lly9TrVo1oz9pCyFyqnbt2uzevdsobc+ePdSuXfvhK19EpCdOZKlK19Hcc3XD1rMS0XEJVGnQCbbkfM6cv3qL43s2Eti8KzbW6b81hZ7eQx3lHhW4Z0jbtPAjrMJO0v7VxdgUswOehRBCFCwfHx8cHBzYsGEDFSpUwN7eHjc3N6ZOncorr7yCq6srQUFBJCUlcejQIR48eMCkSZNyXP7rr7/OoEGDaNy4MV26dGH16tUsX76cLVu2FGCrCpb0xImsKQpeLQbjWq0lFRp0MpvlYK03siyiwaZBhHzU1ui3K11KUvrn/47y6nblE7rErePEzpX5UHEhhBAlibW1NV9//TXff/895cqVo1+/fgCMGjWKn376iQULFlCvXj06dOjAggULct0T179/f7766is+++wz6tSpw/fff8/8+fONtjIpaaS7wwKZE2fZ2RYfU2v/FMO1U8VGcDaLB4AGuhBOHt5BveadAbh/dJXhnlabgsYqfV6cJiUufysshBCiSGzfvt3w+erVqyb3M5/OMGrUKEaNGmWSb8iQIQwZMsTsOwICAlDN7F03fPhwhg8fbpT24osv8uKLL1qsr7lyijPpibNA5sRZVivoRaJfOsdRjx6c77mEms0fy9FzAWuHcC00lJ3bN9Hh/jJDujY1hdt37xquNVY25h4XQgghRAbSEyfyxNWrLI1eWWK4DtWUx193M8tnXJQEXH6uS+ZTbVNSknkwtzt+/10r1hLECSGEENmRnjiRL1L6zuUBbly2rZHrZ8/PCiKQK4Zr6YkTQgghsic9cSJfVGnYARpcwyoqAmZXzdWzTRTjCXXndy7BvkJDLu74FTX6Fj1e/sZoHx+dTkWjUTIXI4QQQpQqEsRZIAsb8kBRsM6HodABKWtJWrCRakoqAOdOPEvNBi1RVZUNP71HpZtr8BizmrJ+FR76XUIIIURJJcOpFsjChryxtrbNl3Ls/gvgAHSJ0aiqyrrZYwm6+TWBXObSn+9n+fzNO7cJCTmVq3dGPbjPvvW/kBAfn6c6CyGEEIVJeuJEvrK2yZ8gLqPzJ/dTe/1AemVIs02NBeDO7RuE/vwsD/zaU/bmRiLrPU/7x8fgNbcu5ZVUbo44SvlKVXL0nktzB9Ey+SD/XthKm1fm5Xs7hBBCiPwkPXEiX2kynWUHcLrWyw9VZr8bn5ukKaq+p+7SH2/SLPUo3UK/or7uLO2Pv4Gq0xl68sLP7yM6PsHo2Tvh99i9ZSVbP3+GIzvXGNIbJ+t7XZtGrEIIIYQo7iSIEwXOoW5vTjf/OF/LVHSpRDx4QJuoNSb3ju5eZ/icsv8nXD8ty5Y/vwdg1/K5lA2uStvdQ+kcu4bGW58xeV4rfy2EEEKUAPKvlch3t55czdVu8w3X9vaO1On5Iuq7YdyxyZ/FCE1it+P5VYDZezX+GWn43Cz1MABdT73Bjdt3aXdiitlnMtLJXwshhMgXHTt2ZOLEiUVdDbMCAgKYPXt2UVfjocicOJHvytVtD8Cx05vRJEVSv2o9ABRrO2441KJsyo0Cfb+zkmg23ee72mBmZ5Lt37yAioa0k2GdlQQ2//QOjfpPxMvLu+AqKoQQQjwECeIskC1GHl7D0XNN0hRVVwQ10bNVzH8vO0YsMUl77Ma38O23bC/7PB3HflnQVRNCCFHCaLVaNBoNGk3Rjd7IuJEFssVIwXCo3t7wOaTFp6QW898jOt6ZR1RMrNl7qqoSHRdXyDUSQgj9/3/iU+KL5E9uD4lPTU3lpZdewt3dHU9PT959911DGQ8ePGDo0KGUKVMGR0dHgoKCuHDhguHZqVOn0rBhQ6PyZs+eTUBAgOF6+PDh9O/fn88//xw/Pz88PT0ZP348KSkphjxhYWH06dMHBwcHKleuzK+//mpSz1mzZlGvXj2cnJzw9/dn3LhxxMam//9/wYIFuLu7s2bNGurXr4+9vT27du3CxsaGO3fuGJU1efJk2rdvn/kV+a54/wsqHjm1e73MNUcHytTqgId1GTTdR8P0MkVdrSylfF6biFdPcfXcCRo2bYOVlf53n+3fjKFFxAouP7WeKrWbFHEthRClSUJqAi1+a1Ek794/ZD+ONo45zr9w4UJGjhzJ/v37OXToEC+88AKVKlVi9OjRDB8+nAsXLrBq1SpcXV1588036dmzJyEhIdjY5Hzz+G3btuHn58e2bdu4ePEigwcPpmHDhowePRrQB3qhoaFs3boVW1tbXnnlFcLCwozK0Gg0fP311wQEBHDlyhXGjRvHG2+8wZw5cwx54uPj+fjjj/n+++/x8fHB39+fKlWqsHjxYl5//XVAH7T+8ssvfPxx/i7oM0eCOFGoFCtrKnUdi06nI/6/v0BJij12avo8tmj78rgm3iyqKprwUqJhdkU8gd23PqFWuye5HHKQTveXgAK3N39D5VrzDUeDXbpwlnIVAnBwsC/aigshRDHg7+/Pl19+iaIo1KxZk5MnT/Lll1/SsWNHVq1axb///kvr1q0B+PXXX/H392fFihUMHDgwx+8oU6YM3377LVZWVtSqVYtevXrxzz//MHr0aM6fP8/69evZt28fLVroA9+ff/6Z2rVrG5WRcQFG5cqV+eCDD3jxxReNgriUlBSCg4OpU6cO1tbWKIrCyJEjmT9/viGIW7t2LfHx8QwaNCivX7IckyBOFDm7t67AR34ARHSdjUO9PvCl/vzV41XHUOnqMty1942euT/8XzwWtCn0urY9/iYcfxOvDGlt7v9N7NT1nHJsjn98CFWVe5yjMndqD6Nhj+G4uRXvnkYhRMnjYO3A/iH7i+zdudGyZUuj869btWrFF198QUhICNbW1obACsDT05OaNWty5syZXL2jTp06WGXYp9TPz4+TJ08CcObMGaytrWnatKnhfq1atXB3dzcqY9u2bXz00UeEhIQQHR1NamoqiYmJxMXF4eTkBICtrS3169c3mi8/fPhw3n33Xfbt20fLli2ZN28egwYNMjxTkCSIE0XP1pGY5zaScPs8Pm2HkpqcZLil+LfA5akZ8KHxKlGdvWth1zJLzkoiLRN2Gla/1uQKNc9MZe+NvbSavLRoKyeEeOQoipKrIc2SRFVVQ9Cn0WhM5uBlnOuWJvPQq6Io6HQ6Q3lpaZZcu3aNnj17MnbsWD744AM8PDzYvXs3I0eONHqfg4ODSTk+Pj706dOH+fPnU6VKFdatW8f27dtz3uCHIAsbRLHgUrUlPm2HAsZHdykaBSsbW677duGOSz1DurOzK5ceK/5HY7WK2VjUVRBCiCK1b98+k+vq1asTGBhIamoq+/en9yhGRERw/vx5w1Cnt7c3d+7cMQrkjh07lqv3165dm9TUVA4dOmRIO3fuHJGRkYbrQ4cOkZqayhdffEHLli2pUaMGt27dyvE7Ro0axR9//MH3339P1apVadOmcEaKJIgTxU+G33I83NwBqPjicspO3k3EwL8Jf+JP7J3d0bgbbxx8ttkMTlZ81igt1K87TI0q8CpnRVVVEpNTOLRjDedPHy7SugghRGELDQ1l0qRJnDt3jt9//51vvvmGCRMmUL16dfr168fo0aPZvXs3x48f59lnn6V8+fL069cP0G8WHB4ezqeffsqlS5cIDg5m/fr1uXp/zZo16dGjB6NHj2b//v0cPnyYUaNG4eCQPixctWpVUlNT+eabb7h8+TKLFy/mu+++y/E7unfvjpubGzNmzGDEiBG5qt/DkCBOFEs36r3MjQo9KV+vo1G6Z53OeNd7TH+hpP/4xj63iVo9X8Kx8WCj/P6jfgMgWS26mQPKNHfsP/Ki6bZnqPVX1yKrhxBCFIWhQ4eSkJBA8+bNGT9+PC+//DIvvPACAPPnz6dJkyb07t2bVq1aoaoq69atMwyP1q5dmzlz5hAcHEyDBg04cOAAr732Wq7rMH/+fPz9/enQoQMDBgzghRdewMfHx3C/YcOGzJo1i08++YS6devy66+/MnPmzByXr9FoGD58OFqtlqFDh+a6fnmlqLnd8KWUiY6Oxs3NjaioKFxd838elk6nIywsDB8fnyLdMLCw5Ue7r4UcpNJSfVCU+vZdrG3tSUmIweaTDD10//XCXfp7BlWPf/bQ9c4P+/vuoH6dQBzsbLPMl5qayv41P+NfrwMVq9YqpNrlP/kZl3Y/6gqjzYmJiVy5coXKlStjb188Vr6rqkpqaqphlWZpYando0eP5u7du6xatSrbMrL6fuYm7igdf8PyIDg4mMDAQJo1a1bUVREW+FetA0CSYoe1jR0ANg4uXNeUN8lbtd/bJI/ehTollPtDctcVn99arOrArjnjss134O+vaXPsDcoual0ItRJCCJEXUVFRbNmyhV9//ZWXX365UN8tQZwFcmJD8aexc0Sdch27t64azaMze4C9RoNt+foo9q44Vqhneh846dPPbHq8akeomcDwYXSLWmb4HBF+hx0/T+H8yQNGeeyvbQMsHxcmhBCi6PXr14++ffsyZswYHnvssUJ9t2wxIko0xd7NJE3N5ncTewdn7tpUwE6XgKJqcdNFEtpjPpVrtIKvVwJwqtxA6t7SB1o6NNz3bY3/7fTA61yNsThdXk+F1Gt5rvuhLUtx3v0h/uptOihJEDqXkGWVSHr8Zxo1amZ0zqxWq+XQ5/1IdCpPh5e+z/M7hRBC5K/C2k7EHAnixCMnycUforIIrhQF3ynHQFXRJscRefsS/lX1m0A+GPoP1nZO1C1fG6bqgzatoqH2s59zcV4E1SK28kBxp+aQT9j3QzwVbi3Kcz2b7h79X33S0wI112BlV3Zv74NThiDu/NEdtEjYBQmg087l9p2bODm74u5mGsQKIYQoHSSIE4+cisO+5/IfE3BsM5ayljJZ6Vc+WVnb4l41fRfvMlWammTVYoWtkzvVXv6bGye24epT+b8y7PK55unaRq02unZZPcYQ7N28fhH/hc24pXriPu1ygdVBCFH8yFrER0N+fR8liBOPHEePClQZ91e+lZdxjl2F+p0Mn63dykJovr0mSxWU9IOa4xY9BUA5JYKE+DgcHC0f7XLj9m28vbywy8VB0kKI4idty434+Hij/c1EyRQfHw+YnjSRWxLECZENLVZm031rtYJT+s/qO3dQPtT3+5326oFNlbbUOPBugdSnlnrJ8DnsxkUq1WhgNl/I8X0E/t2df62a0/rdTaVqCwAhHjVWVla4u7sTFqb/hc7R0bHI/07LFiO5b7eqqsTHxxMWFoa7u7vRea95IUGcEBZEaDzx1EVw17cDvmbu+9dtw62kX3GtUAdnGwe0U25y5+Q2Ahs+hmJjz6mr+6kbttrMk/knOT7GJC0pVYudtRWxO74FoI32AFuDx9NmzNdYKQrW1g/3Pw0hRNEoW1b/i2JaIFfUVFVFp9Oh0WhKXRD3sO12d3c3fD8fhgRxQlhgO3Yr5/f9Se3HXrCYp1yT3obPVvbOlG/Wx3Bdd9wvRFw9TsytSwRsKphjWLRJ8ezc8CfawwupPiyYc5t/pubVX0l6ZhWKNv3Q5s73fuX0h/uIt3Kh1utbcLG3ISoqin1LPsG/9UAC6zYqkPoJIfKPoij4+fnh4+Nj9hD4wqbT6YiIiMDT07PUbOwMD99uGxubh+6BSyNBnBAWuPgE4NI398e7ZOQZ0ADPgAac2/0NNeOP5FPN0t09soYOdxcCcOTnQXRRz4ACB1e8iVv8daO8dbgEWrg7sxrHO8xBG7KK7vf+QLdsDtSNzPe6CSEKhpWVVb4FAQ9Dp9NhY2ODvb19qQviiku7S8VX/fHHH6dMmTI8+eSTRV0VUUpVfGFpgZSbFsABNFbPGD47pdynBlfNPuOrRBK4YwxlI48CoFFktZsQQpREpSKIe+WVV1i0KO/7eQnxsBzcvS3eO+OUfrRbSj51jgemnMryvi2p1Ew9ly/vEkIIUTRKRRDXqVMnXFxciroaQpi41fcPak5YY7g+6dyaGBwL/L2Z+960OtWwb9Htu3dZPfdtzp/PWZCXmqrln8+GsOmnd/K5lkIIIbJS7IO4nTt30qdPH8qVK4eiKKxYscIkz5w5c6hcuTL29vY0adKEXbt2FX5FhciFsIGriByxm3KNg9DY2hvdu9L0vQJ/v4uSYHR9bWotVn4ynNNnz3Fp3mj63A3G+lfzZ8lmdnr/RrrEraXbjW8LoqpCCCEsKPZBXFxcHA0aNODbb83/A7FkyRImTpzIO++8w9GjR2nXrh1BQUFcv37dbH4hisrJWhM449ae26NP4VW7He6V6pnNV6f7SJKwBeBsi48N6Vetq3C8+RcFUrcqmjv0T1xBnT+a0zZphz5NuU1CsjbL53bu3cvNMwcLpE5CCCGyVuxXpwYFBREUFGTx/qxZsxg5ciSjRo0CYPbs2WzcuJG5c+cyc+bMXL8vKSmJpKQkw3V0dDSgX42i0+ksPZZnOp3OsOdMaVIa211n0FR0Oh3h4eEm7U77bUpFQbGyxvqtq2iBGjYOXC8XSOw/n+HZ/2MSbqdv9JuAHQ4kUZCuzWjAtabv8FjvwYa067ducfifZVRr2pX2G3sY5Tf3/SyN32uQdpemdpfGNoO0u6DanZtyi30Ql5Xk5GQOHz7MlClTjNK7devGnj178lTmzJkzmTZtmkl6eHg4iYmJeSozKzqdjqioKFRVLfKlyoVJ2m3cbg9ssCUFtWLbTBt5xmDrWxOPIT+hAu4B7px2bkO8WzWSEuNoG/Fngda3liaUWkfGcrhSHY4f3IWDhx8BJ2fzhHqchMsfGc5zTXP79m2TrQ/y+r3WqSqpWhVb65L58yE/46Wn3aWxzSDtLqh2x8SYbuJuSYkO4u7du4dWq8XX13g/fV9fX+7cuWO47t69O0eOHCEuLo4KFSrw999/06xZs8zFAfDWW28xadIkw3V0dDT+/v54e3vj6uqa723Q6XQoioK3t3ep+0sg7U5vd8K4Q9w5v4dGLQeiaLLe/8l3kn4hxIGFb0GEPi2kyQeUOfEjSUGzCFiV/1vpVFzemyZKFNxIT3NQkk3yOTm7sPvQEWrVaYCrgx1eLvYmbd60aQ06nY4ePfpm+c5Nnz+HX9wZKr22A1cny+fDFlfyM1562l0a2wzS7oJqt729ffaZ/lOig7g0mY+9UFXVKG3jxo05LsvOzg47OzuCg4MJDg5Gq9XPCdJoNAX2Q6ooSoGWX1xJu9Pb7eQTgJNPQK7K0Ti4GT4H9nkF+rxCQmy0Ie2yQz2qJJx86PoCeCtROcq3fNY4nldWw39ri35M7UmvyT+jURRStSqrfpzOoLtfAhDX/gZOzuZXja9bv5qe8WtAgcP/rqBJj+fypR2FTX7GS0+7S2ObQdpdEO3OTZkl+qvu5eWFlZWVUa8b6M+Vy9w7l1vjx48nJCSEgwdl0rYonqq00q8eTcLOkGbvmN5j5Tvm70Kv0/OK8Vmxo63XUe4rPw5s+p09axcaAjiAlMR4o7wPYhOJS0zhzOnj9Nz/rCFdIevFFYkpWd8XQohHVYkO4mxtbWnSpAmbN282St+8eTOtW7cuoloJUTi8KtUh+vndKK+mb+yraKyIGrad+0+vxymLDYYtOVN9TH5W0aDv5al0PjHZKC0lJX1RRlx8AuGfNubwzK5cOrzFKF/s3vlExRlviZJm69YNHJnelm3bNuV/pYUQopgr9kFcbGwsx44d49ixYwBcuXKFY8eOGbYQmTRpEj/99BPz5s3jzJkzvPrqq1y/fp2xY8c+1HuDg4MJDAy0OHdOiOLAtWI9bN18jNLcKjfCo6b+l5hzdSdzy64K53x7o0VDvMYZgNBKT5gtr/Yzn3LTvnrBVvo/RxZMZvf29QAcPfQvNTQ3aa8co/fl6Ub52ivH2PvnlybPnzx3kc47B9PaKoSm24eafUdkfDJbjl4gObV0rZ4TQpQOipq2TXsxtX37djp16mSSPmzYMBYsWADoN/v99NNPuX37NnXr1uXLL7+kffv2+fL+6Oho3NzciIqKKrCFDWFhYfj4+JSqOQXS7iJot6pChrmiN756jAoPDvDAMYAy8Ve55hdEpTF/kBL3gAdftcUnOX0Vw92A/vheXVEg1VqU+hhDrTdnm099P9JoruuD98tTRok1XB8depZGVfyMnln6ySgGJSzj71pf8PhTo/Kv0jkgP+Olp92lsc0g7S6oducm7ij2QVxRkyCuYEi7i0G7k+NIvn4I2yptUe9fQfGoDBlXxk5NXzgRNuooPj81KoJKptv92BratmmXnpChfgAxqgNrfV4gSafg8OA8jk0G0/vgMACuq76c6LWabo2rF9qWJcXqe12ISmO7S2ObQdpdHIK40vNVzyUZThWPPFsnbKt1AI0Vilc14wAOUN+6QZhnM252+x6NtfmF7PfsKxVGTQFw3v4uAGduRTFnziyT+y5KAk+Ff8WwiNkM0q0zBHAAFZW79F7XnFV//270TFxiCj9/9T5rNm1Cq9P/PvsgNolVu44Qk5hSgK0RQoiHJ0GcBbI6VZR2ip0LPi9voXzrp/DwSJ93F9rmI8Nn99GrCq0+LslhzJy3lNo/VGRcmOmG3DnR6vQ0Mg4+/Lt6HiMfzKb3noGsnv44y375jk3fv07ffzqxft6MbMtTVZWr9+LQ6WRAQwhR+CSIE0JkS2NrjzrxFOqrpynf6ilDurVH4fXEVVVu8db10Q9VRnnusmTmSGIS9CtjXaPOGu71ZxsDL77J4JiFAAwKm224Z2kbk9UbNxD/dSt+/+3nh6qXEELkhQRxQogcUdz9UdwqoHH2hFH/wNh/jRZKnKn4tFH+iCYTiOn4QWFXM1tPJf/F3tU/o6oqqsYm2/wnz51n+/Tu/PH7fJN7zfe+SKDmGs9cnGzmSSGEKFiPxIkNBSHziQ1CiAwqNDVJ8mn1DNcdHLC5exS/lzbhaW0LwJUkDZX3vlPYNcxSt5C3WPJdOOXtsv499relvzMkZCz1rIBzB4ERRvfLKg9y9D5VVUnW6rCzzvpINSGEyA0J4iwYP34848ePN6wSEUKYlzp2H0nhl/Cs3Q7P2u1M7lfq+iLh/g3wXtq7CGpn2eC7posjMhsSkvP9JiNik/B0tjN7L/jnn+D6Pga99g0+ro45LlMIIbIiQZwQ4qFYl62NddnaljMoCloPyxsIR9uWxTU5/ei8O3aVKZt0xXB9seow7EmhwqXf8qW+D+Ozeb8Thju+8RcYMmQE5TLcW/jxOBTPavgHtuLJ7h2NnnvpxmuggU0bmtBt0DizZet0KgtXbaRa1Wq0q1fNbJ670YlsPBhCv1b1cHPMfihYCPFokyBOCFEorlV6gkrX/iK0bFfUGr3QhazE/+nZ3L8dhuuf3QC4H/QdZRv2gZnlDc9Ve+oTVFUHHxV9EPf69fSeuSWLrjM4w71J1n9CFLAXTtW5xsmbUbQqbxxoOcTdwJKdu3cw4thgHhx15sPdH+BXqQYpx5aSVL4V/Xv3o6KnI+uCJzMi6RcWnH2T4ePfNjwbn5zKmdsxNPJ3R6NRLL4DYN6ieVhb2zJ0yLNZ5hNCFH8SxFkgc+KEyF/+Q74h+eIT+FfvCLZO0Pl5AFJvpffCOdXrBXbOhutLtcZS1cYBBTgY+DbNQj6iuHjywU9gIV6q+1Ml6gKTksdSuVIAL/+X3u5aMEv3j+TfNYt4gn+4pvqyW1cXK9/a9LE9AkAZJZZ37r4Kd/976MpC1s2ex/IGbzIx6RcAngn7HEgP4l6bOo0XrVexusM39Oti+bSavcdO8fzlVwFITHwSe3t7s/luRibg5mCDs538EyFEcSZ/Qy2QOXFC5DMbB2zr9DJJtssw19/W3gmA6zVHYnP3GFUe/5/hnpVD3k5MSXCqgGLngv39M3l63hIrJfu94WbZfge3jdMGrW/MIEObT/IcWyCb9RE9rQ7Q81T6ebca0s+CPXv9LnNsvwZAu+9d6LLTcO/wpdscPHaCYX26cj8+Gbvlwwx7EmhTkwHjIE6nU7l1P5rLX/XilKYG46bNs1inyPhkNh2/SveGlXFzkKFdIYqCbDEihChS/tXqA5Bs5YTy36kRFZ+ehd/ErSgZeuVsHI1/mUpq9HyOyr/eYRb2r+zjds3nss2rWupaK2asFJXL4bH8ffQGl/auMKQ7qQkAJCRrWX7kBizsw9iTg5j30zdovgykseaiIW9qivGJFPN+/pZ5H43l0q4/aG91knHKXxy8HM7G49fN1mHJDzMZtLEZS+d/aZS+bOVK/j18LH8aKoTIkvTECSGKlq0TTLmOrZVtltnsnNODuFvP/Uu5qnWJrzcYx0XdLT6TqthQrVYDAPye/pYDPyo0v7nIYv7wkQch9AA+m8wvPihOUr9pTn10lFMiDMO6ccmpJKVqWfj7rzxx+R28NdEAjA9732ToNyU5yej6+VD9NjDrTtwypFkt6EFT5S7XfY9Ssay3Uf4xkfrVvaPDPgTeAODcuRAGHx8OgC4o6+7F4xeucObEIQb0HYCtjT54vx2VgI2VBi8Lq3yFEMakJ04IUfTs3cDGIcssPuWrGj7rNPrfPx2rtISJJ9GO3GqUd3/FF+DNa1hPPIaVa9n0G7bOZMXezg7vxn1yWfmiUUNzk6qa2zgoyYa0hprLLFi8gOcvT8Rbic7y+RPXwli46zzn78YYHUXWU00fjm2suYinEkP0xX1ZlhV6P16/F96dsyb34pJSOXT1vsnRZH6/dOSpk6PYunIBALGJyVz7ohMHPumNqqpcuRvJnpMXsnyvOQnJWqP2CPEokyBOCFEiuPulb7tR1ssrw42KWPk34WrF9DljNRt3AAd3cKtgVEb1OumbFEfYlCMzV68KKPau3K/+ZP5VvJCNuTYJWyX7BVl2q8cx7J9mhH/bnY+Xbc8y75bTN/lqYwipWh3r9p3gk++MjxlLnt2YD7+bbxQ8pQ3XLprzAd7zWvDV7yt4a95ajl/Vr9jwUSIB8AjdDMDdS8dpqTlDT6sD6HQ6koLb0vqvpvyzT7/gIzI+Odvg7PL16xyf0YaV8z/Otv1CPAokiLMgODiYwMBAmjVrVtRVEUIAWFnDC9vRPbcKaxcvk9uVBn1Miq07SQFdcG9guoACoEyTAYbP9oN/NHw+VH4ovB8JGv3/Ej0Gfm32+ZOVczYPryRoY3Xa8N+3QvpnmXfi7TeZsLcV7783mebre/HmnUlG96tqbvPu3Ve5/s/3hrR1x66RotXxYtRsKmnCePXCcGZeH0LivL4m5et0Kqo2fY5eSnIStTShAHTZ0IkVGzZw4+Nm/P3rnCzrGfrX/2ipOUP/64UfxCWm6gi5ESG9gKJQSRBnwfjx4wkJCeHgwYNFXRUhRJpyjdBU7WD2luLsg83b17AbvtzoTFfjTApMjYJ3w3Gs3NKQHI+d8TO2TmYfT3QsbzYdQG06Kvv6l3Af2szDK4th2t5W+w2fe61ryZIVK03ytNAYD7laPbjI+an1OLUs/Zzd1FTjRRf99w2mruYqAy6mb6sSm5DE/vdacO69OtyJ0M+/c08MzV2DMti6ZR1/fTaGe5FZD0NbsufX6VT/uRY7dm7NPnM+yEnPpHj0SRAnhCh9rG1RrNLXdfnZxOfoMVUx/7/M6CYvofT+Qh8gTo3KtpxIv3akWDnCmJ3Z5k0e9AeM3Z2j+hUnNoqWLicmmb0397uvDJ+baC5QSxNKf6s9hjRtppWzGT2I088BPLhyDi00Z6mpucHxX98FwF4bY8j3z77Duapv591P80TcHxz5Rb/A4+SlUE5fuWWUR1VVElPMD1UPSfgdOyUVrz3Tc/VeS3YePsHl2/fM3jt99ixhHzfkz7nv5cu78pNOp0pwWYgkiBNClFpRgc+gVWwI6G66GjXezfSosDqBgSZpN/svw7XPh8aJo7cRVX2ASd407qNXYTPlCvg1yLJ+ezwHYBsYBGXrEVM/+6HcJJu87aVXUPyU+2bTX7yTdfBx677l3rCdn+vPybCLvmpI8444AICVmh5gddnQmQ37jrH71CWj53edOMfZ0DCL5Xe7t4gFy1dRY1FDqi2oz7ZdO1j7v8dY9vtPrPpqAqdntCLiQTYb+z2k48eP0H51OzRzW5u9H7PmXWpobjIwzPywf1FJTdWyceZAVn3xQlFXpdSQIE4IUWq5DQzGasoVbPzqmtxzHLkauhkHZ061u8Oof2DQIhKd9IsmytdqafIs5RujbT3R7DtjygTq597Z6DfaVZ/9m/iKnczm9bBJNXx26fcpao+PYdx+s3nvDFyH7csHzN4raWr/0sjivX7qVoI/fYuEiPQjzBor5/lj0y6SUlKN8vbY0IG2fzZGq9Vvjrxl3yHaLW9OrZ+r8/W0cUTFJ7Mr5BqHzl4xem74ieewU1KxU1Lo9E9felkdYOC5yfSLXEgT5RxnN88nRavDHHN7DaatzF2/8jeWf/4isQlJJnkySji1GoAAzV2z9+21sVk+X1SuXTxJUMpm+sUuldOOConsEyeEKL0UBexczN9z9YPWLxFzfgcuVzcR2/YdnBUFKjQFmmJfowekJoG9+d4vd//aRDtWxDXeeLNcl7GbjKtQrTOqdwP4sgoAsY7+OMfr53ZV6Zphnp2VDUrLFy02xca9LIqrH7x5FWLDSP5tCLYP0jf3jX7sC1w3T7b4fEkyPt50gcNTe3pb7Jb4e+Es1FtHGZi6xpD2ivorG2aep4dV7uc9J4dd5Pi0FtxRPfB/bi5V/CuQ/lNkHMSdD73DlZ+Hc0hXg3c0CwHYsqwCXYfqh211OpUbDxLwcbXj4t0Y6pR3M2x6ndHp6+HoVB31KvmiqKkm9zNK0epIStVle2xa2INo9uzYSPvOPfFwNT8PNCvRicbD3hptenCanJiAg1PWW/rodCr345NlX8CHIEGcBXJ2qhACwOWZxXDnBM7lmxjfsLbT/7FEY038s+tx9vZCc2Y1JEZCYD+zQaO1TfqxVTc6fEGtWnUhMQpbX9Ph24xO2jelXuIhAOwd/yvXoQw4lMF2wmFSQw9j/XNnIsp3wrPNKGg0AD6tnKN2P0qevP6B2fS8BHAAne79aggYU39tgrWS3iun/rdAZsXKZSQd/IXB1tupAXTX7DXksb60hQdxrxMZFcn+OaPpaHWcv7UNedp6G0d11Qj36WjIu2zWK1TpMgq/5f2wRkfclHNodMbBU2KKlsOX7tCwsi9Odtas+mwk5RIvUveNTbg4OqCqKqoKGo1xgHnsu5H0T9rE+ouPE/TaAkN6ZFwSNx/EEXL+PC0b1MXf0zQY++efDXjueJsLdSbSqXt/tv/wGnaeFUn76dqwfD7JYRfo+9IswmKTqORpGiSumf8hNa79wfUnf6Nx/fo5+MrnXMiNCAACK3jma7nFjaLKDMQspZ2dGhUVhatr/s830el0hIWF4ePjg0ZTeka3pd2lp92lsc2Qu3brUpLRfKg/EeFs0DJqteiWdeFT9adXXKn8FJWv/KEv463baOwcTfNGXgcnb8Nmysn7fsJ2Q+H2yEVp3HHTRRbqO4vKfk0jakxcS5lZpvsQZrTUui++qTfowJFclR82+hj3FjxLYMopALYPOsX1f35iyL2vWe8zknbPvov7l5UAWOE3kf5jprH0y4nYxofR683FhN6+y79r5tOi53PUmF/PUO4Km550mjiP3YeO4rLlDdpbnQTgb81jPP7enwD8/stPJEWFMfTFt3gwzR9PRb+QZFP58XS7GWy2vn9p21FbuU5o56/p3rGj8c3/fo6PWtWl0f/+5eCpc1w4fYgnBzxlOMUjLxKTkon4qBYAnm+fxd4u69Ngcqug/5+Wm7hDeuKEEKKIaaxzeYD8c3+j3jxCZHIF+C+I09haOPHCvaLRpW2LkSRe3YNq44hV25exndtcf2PoSlIPLsQ66EMIC4Gom9BkGOi0pE73whrjOWBRj/+C29/PApDQYCgxyeBzZhH3m76KxyHj81RTOrwD217PXRtLqBa6o6z67iVMd8MzNih1VZ7KX/vje4zglOG649L/5nMq0Cf8R05c7I/7f/f6357NktmxDI6ar0+Y4UEVoAoQNu8Ho5Hf/inrWLPgPXqH/wAZ4qfHdZtRVZVl6zby9EV98H/2bC9qKekrgctEm57UkeYJq10AeG97jn0+Ozhz8B98/avTs3NHQ55G2lNodSq1lnWgmZLAprWpdOs/LJdfmXRxkWGUV/Q9cfejwrH3sbw1UGZXb4Wxbfn3NO02hHo1qmb/QBGTIE4IIYpahj3qqpU33cjYRNXOKFU7E3A0wz5slvbGM/Mu+6fm6T+rKmrTUShu5aFKR6yrdNSnu2boRdJYEV2+Ix43jfc/c6sbRMymAHQ2jrj1/xoHXSrcHo572frwXxB3seowqrUbjFfFVkQ5uuK2dkzO6pjJ3Xpj8T35XZ6eLQp945cXWNkjyDr4O7bmezIOTA6O/MlsvrQTMzJqFrbU5IxdgBMnjzPo4GDDtRIfjk5V0Cj6gbyA6ENmn8vIW4nGe2kjWgJcgZQOD8j4q0volbMEKAkAuN3YDqQHcVqdys0HCVT0NNPTbIZOTf+FQ6uaX4BiyYX5YxiRspXjvyyH6Udz9WxRKD1jG0IIUZx1fBu10XNYl7e8MjOzMnW7k+JeheSa/fL2TkXR72/Xzvx+bmk8hvyI2vEt7tcYmJ5oZY3LpMO4vbJbH0Ba2UCFJpBhUv4Dt0AIaAMaDW7NnspbHQHfAR9zK8D8li1aJe/Dbo+ioWreevgAfM0EdgDK3RNG19euXSM1Q3edt5L93oiZ6ab7GF1b6dLPAG5xbzmX36tJRJR+Fe7a5YuJ+Kodf67IWXCcmpyU4XOyyf2kVC3HQiNNzvMFaJ+s7zlsoLmco3cVNQnihBCiOOj4Jkq/b3PeowZgY4/NK4exfXpRwdULwMkLpeMUPAbMIrF8K1K6f6pPt7I2CtpMZJ5y/eppGLkZbWB6QBbvUJbzfn2yfr+iUG74fKOkBBt37nm3IOr5f43So3r/kF1rcu2ebYXsMz3Cbu40/vnqdnIStkrWK2SzY6cYL86IybTStYrmDkdWzAag/OnvaaS5yJPHRvDX/oukanXsuXSP2CTjOpy8eI05n03hZMgZQ9reHydy7Mw5o3y//DQbtx+bs2LDBpN66bLrUixmZDhVCCFKssJcLGLviv1o03/4LKngkWn4y60CuFXAyr852n+boBz6CccR6wmIj0L33To0pO8GcL9qfzwurTB6PHnkVmx/7ky8jSeOb4TgYGNPSkL63KyEYZtwq9yCiJQUUs9uxLFBPxJ3z+GeSy1qX/s1T00GcBi5CuY2BuBm248ov/ttkzxnKg+jbKOelFk+2OReSReUx1W8ubLseZNuJZsb+/nx8zd5WnfRMFz7xPomnFnrT2tNKD94vI5N2dqE3b6ONjGGtxO+oB7AvrmGMgZodnLgj2HEvb2b6/fjqenrwsg700EDyoFJ0DPI6J26TJVITNFyOyqRyl5OLF21ivtnd/PUuPw5lSM/SBAnhBAiX4VXfRLne0fxazHQYh6rNi9Bm5cAsHX1g3duknT8L1JPrcCp1Sg8qnWBW+PB3s3wjK1/E3g/Ekf9fhkA2Di4ENHsNdSUBLwqtwDAs9Wz0Eq/6MKl8ZPc3rQIsgniTtvWJ/D5OST93Av7FP3wYILGCbtxu3HyqkJUxxlonDwp33gQty5uodwd4zmCD7yaERDQPJdfqfx13b0FFSPNbwZd3AVqrpmkdUzZSceUnSbz7Wpr9PsovnD/MzB/KIiR5soZjn/YjAaay3yf2osx/0U+ZTAdBtZl6gn/afZ7NI/ZTOQzfzDoyHMArP3bjyaPPZ2DVhU8CeKEEELkK+1jH2Ln7Q1WuZivZuOAXdNnsWv6bHqav5mgSFFMhpw9e/0vy6KtncoYXd91bwTOvjh1eAnnX3vri9VYo5Sth/3Eo/CZfuPllAHzcfDSf3br+LLh+XJj/wZAjX+A8mmA/rNOi62j8R6A57v/So3GHWFm+urIqMErSdn4PxyVZOKTtdgN/hkHbSzaxU9ip9Of4ZvgVAGHuBuEuTfAMe4mzinmz1DN7HalviU2iCtoaXPcxlivNaTZmNk0Wc3QE7fs4FVeivsWNLBjy+eGdI/7RwEJ4oQQQjyqcjO3r4DZW6fXJSJwOD6930dx9AAgwaYMDikP8G32uD6Dk6f+1Iv7l3HNvMFzJopjenDoWUZ/0sK9Vu/itXcGAFZO7mDnTGj5npS/uZ74Z9fhVq011NZPns842Hy08vM0u/QtAA4TD5FyZi0+1bsQdfcaLOhg9N5b9V+i3IlvTduZaZ7Zw4i18cA5xbibK8yzOT4Rj8bRbgCOShIffreAu9cv0NXqMBp0dNKkGnr+Bq5NP9vYQZd+1JlVSkzmooqMBHEWyIkNQgjxaKjcsANxm8uQ4FQer0FfGd1zeHkvhO7Ds3aGnd0cykA2AVyae+1moA07i3tV/Rm6Tq1HwX9BnMZKv4mG/4gFkBCJs4uvxXIC6raG/4I4bBywqf8kAG4B6YGiDgXNm1cp5+DO8cvHaRC7y6iMwA5PwrGphuvwMo14oPGgRsQ/Ft971asDTr5VwdkX7/0zAThe/SUaDJkB2hTufN2FstH61ak+L66BGT4Wy0pzrVwQlW6tzzZfcfDOnQmQg72AY8OuGfbPs9EmFGylckFWp1owfvx4QkJCOHiwECZ0CiGEKDCKnQtOb5zG65Udpjdd/aDO41mvss2CV5eX8R78jaHn0c4+/XgpN+f/jquytoMsAjgA74Y9iW73HokD/zC5lzT2AHcq9SF59G5wcAegfOtMW7a8cQWbMv7cbv0BiVZOxD63Ce8J27nvWsuQJcIj0/Y1z/xFwMhFeA/8Eu+gKdx30m9uW6vLc/r2WNtyu0LP9PzWdtysO9ZiGy67tYAX9xBRvrPZ+9GBz5DkXIFwn9YkvbAni69G8dPZ6pjhsyabs2sLkwRxQgghHn22TvotUQqYxsaW29WHcMc/CI+KWZ99a0RRcO0yGfs6QSa37MrWpOyIX7AvX9eQ5tXqGVIzDqb9Nzzs1+0V7N+5gXNV/SKPyjXThwQ9X1xPfLlW6c9U72oICgE8XtkJE05gVzY98Atsq++hTFu1Wf6Jjw33Ul40nn+nLd8cfOtgZZN+ekioVfr2LK6D5mD32mm8x63HrlwdEp/8lbjuxqd7ACT3CUY74aRJenbuV8l6v8S7rafywLka8Y5ZH4mWnQaJB0lIKR6jdDKcKoQQQuQjv2fmZp/pYSkK91u8js9/Q6BGMmw749tiELGxV7Ct2AxbGwccR60j5eB8rCu1MN0Rzc5Z/ydjUrk6MGYnGmdfw3sZfwBSErDyrmGUt1of/bFc1hl6I21rdIUzC8w2wb6uflFJ7Ia3cVbShyhtm/y3uOWZP+HXJ80+m4oV1hm2pLlXqSdez/xM0u/J2F1cT6K9N/aJ4Yb76qsh+LqVh26vgqqyd/YQWkWtM1t2TuzespJKwyz3ShYWCeKEEEKIEsjnsUmkenhiXc388CUAioJz1ynp1xoNNi1G5u5Ffg2Mr71r6v+rSz/SSocGjYN+/l6FCunn9fr2n0GKly829cyfuAFgT/oJC/cqdMVw8Fz1x4hwr4dn5ElSh63FulwDmKnv2Qv1akflUYtQw8+TeGwZXt3+B1Y22D37x39lAkmxpOz4DOtaQfqj5dIoCvZJEbn6EmRW+9ZyQII4IYQQQuSFtS3WLUYXaRVudPsR913vQ59vSOvDc6vSlOgmL6FxK4+znQs2XUw3R87IWkkPBr2eNT6Zw3P8Fkh4gLWrHwDRfX5Gt2sW5Qd+CvZuKP7NcPBvZr5gO2dsuk0ze8tWG2s2PafKNer+UM/nFwnihBBCCJEn1lXa49jyNJpMJ4e49vkwx2Xc7/0zbmtfILLLLDztXY1v2tiDjV96uU2ehCbmh1hzw6t8Nbh62uL96MbjcKnbndQL/xCPPW57PzW6r6udx/OK85kEcUIIIYQoMh5Nn4SGffC0tiu0d/o+8RmJazXYtRxF3OIhOGv1pzckuQZg13IUrs1fAGs7bKp0xE1VIVMQZ23vYq7YQierU4UQQghRtAoxgAPAxRf7pxagBLTFatACtIoNkR0/xG7ScWj9snF9MmxcHeLRBd3E08VmM2vpiRNCCCFEqeVQszO8cwt36+x3/S3fqDu4loPEsEKoWfYkiBNCCCFE6ZZdADd0JdqL23Br/Ty6rHMWqlIxnLpmzRpq1qxJ9erV+emnn4q6OkIIIYQoSap0xKrbNPjvKLXi4pHviUtNTWXSpEls27YNV1dXGjduzIABA/Dw8CjqqgkhhBBC5Nkj3xN34MAB6tSpQ/ny5XFxcaFnz55s3LixqKslhBBCCPFQin0Qt3PnTvr06UO5cuVQFIUVK1aY5JkzZw6VK1fG3t6eJk2asGvXLsO9W7duUb58+k7NFSpU4ObNm4VRdSGEEEKIAlPsh1Pj4uJo0KABI0aM4IknnjC5v2TJEiZOnMicOXNo06YN33//PUFBQYSEhFCxYkVUVTV5RsliaXBSUhJJSelHgERHRwOg0+nQ6fJ/OqNOp0NV1QIpuziTdpeedpfGNoO0uzS1uzS2GaTdBdXu3JRb7IO4oKAggoKCLN6fNWsWI0eOZNSoUQDMnj2bjRs3MnfuXGbOnEn58uWNet5u3LhBixYtLJY3c+ZMpk0zPaYjPDycxMTEh2iJeTqdjqioKFRVNdnx+lEm7S497S6NbQZpd2lqd2lsM0i7C6rdMTExOc5b7IO4rCQnJ3P48GGmTJlilN6tWzf27NkDQPPmzTl16hQ3b97E1dWVdevW8d5771ks86233mLSpEmG6+joaPz9/fH29sbV1dXic3ml0+lQFAVvb+9S95dA2l062l0a2wzS7tLU7tLYZpB2F1S77e3tc5y3RAdx9+7dQ6vV4uvra5Tu6+vLnTt3ALC2tuaLL76gU6dO6HQ63njjDTw9PS2WaWdnh52d6c7RGo2mwH5IFUUp0PKLK2l36Wl3aWwzSLtLU7tLY5tB2l0Q7c5NmSU6iEuTeY6bqqpGaX379qVv3765KjM4OJjg4GC0Wm2+1FEIIYQQIj+V6NDZy8sLKysrQ69bmrCwMJPeudwaP348ISEhHDx48KHKEUIIIYQoCCU6iLO1taVJkyZs3rzZKH3z5s20bt36ocoODg4mMDCQZs2aPVQ5QgghhBAFodgPp8bGxnLx4kXD9ZUrVzh27BgeHh5UrFiRSZMm8dxzz9G0aVNatWrFDz/8wPXr1xk7duxDvXf8+PGMHz+e6Oho3NzcHrYZQgghhBD5qtgHcYcOHaJTp06G67SVo8OGDWPBggUMHjyYiIgIpk+fzu3bt6lbty7r1q2jUqVKRVVlIYQQQogCl6fh1IULF7J27VrD9RtvvIG7uzutW7fm2rVr+VY5gI4dO6KqqsmfBQsWGPKMGzeOq1evkpSUxOHDh2nfvn2+1kEIIYQQorjJUxD30Ucf4eDgAMDevXv59ttv+fTTT/Hy8uLVV1/N1woWFZkTJ4QQQojiLE/DqaGhoVSrVg2AFStW8OSTT/LCCy/Qpk0bOnbsmJ/1KzIyJ04IIYQQxVmeeuKcnZ2JiIgAYNOmTXTt2hXQ7zKckJCQf7UTQgghhBBm5akn7rHHHmPUqFE0atSI8+fP06tXLwBOnz5NQEBAftavyMhmv0IIIYQozvLUExccHEyrVq0IDw/nr7/+MhxjdfjwYZ5++ul8rWBRkc1+hRBCCFGc5aknzt3dnW+//dYkfdq0aQ9dISGEEEIIkb089cRt2LCB3bt3G66Dg4Np2LAhQ4YM4cGDB/lWOSGEEEIIYV6egrjXX3+d6OhoAE6ePMnkyZPp2bMnly9fNmzGW9LJFiNCCCGEKM7yNJx65coVAgMDAfjrr7/o3bs3H330EUeOHKFnz575WsGiIluMCCGEEKI4y1NPnK2tLfHx8QBs2bKFbt26AeDh4WHooRNCCCGEEAUnTz1xbdu2ZdKkSbRp04YDBw6wZMkSAM6fP0+FChXytYJCCCGEEMJUnnrivv32W6ytrfnzzz+ZO3cu5cuXB2D9+vX06NEjXysohBBCCCFM5aknrmLFiqxZs8Yk/csvv3zoCgkhhBBCiOzlKYgD0Gq1rFixgjNnzqAoCrVr16Zfv35YWVnlZ/2KjJzYIIQQQojiLE9B3MWLF+nZsyc3b96kZs2aqKrK+fPn8ff3Z+3atVStWjW/61noZHWqEEIIIYqzPM2Je+WVV6hatSqhoaEcOXKEo0ePcv36dSpXrswrr7yS33UUQgghhBCZ5KknbseOHezbtw8PDw9DmqenJx9//DFt2rTJt8oJIYQQQgjz8tQTZ2dnR0xMjEl6bGwstra2D10pIYQQQgiRtTwFcb179+aFF15g//79qKqKqqrs27ePsWPH0rdv3/yuoxBCCCGEyCRPQdzXX39N1apVadWqFfb29tjb29O6dWuqVavG7Nmz87mKQgghhBAiszzNiXN3d2flypVcvHiRM2fOoKoqgYGBVKtWLb/rV2RkixEhhBBCFGc5DuImTZqU5f3t27cbPs+aNSvPFSouZIsRIYQQQhRnOQ7ijh49mqN8iqLkuTJCCCGEECJnchzEbdu2rSDrIYQQQgghciFPCxuEEEIIIUTRkiBOCCGEEKIEkiBOCCGEEKIEkiBOCCGEEKIEkiBOCCGEEKIEkiDOguDgYAIDA2nWrFlRV0UIIYQQwoQEcRaMHz+ekJAQDh48WNRVEUIIIYQwIUGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJJEGcEEIIIUQJVCqCuMcff5wyZcrw5JNPFnVVhBBCCCHyRakI4l555RUWLVpU1NUQQgghhMg3pSKI69SpEy4uLkVdDSGEEEKIfFPkQdzOnTvp06cP5cqVQ1EUVqxYYZJnzpw5VK5cGXt7e5o0acKuXbsKv6JCCCGEEMWIdVFXIC4ujgYNGjBixAieeOIJk/tLlixh4sSJzJkzhzZt2vD9998TFBRESEgIFStWBKBJkyYkJSWZPLtp0ybKlSuXq/okJSUZlRUdHQ2ATqdDp9Plqqyc0Ol0qKpaIGUXZ9Lu0tPu0thmkHaXpnaXxjaDtLug2p2bcos8iAsKCiIoKMji/VmzZjFy5EhGjRoFwOzZs9m4cSNz585l5syZABw+fDjf6jNz5kymTZtmkh4eHk5iYmK+vSeNTqcjKioKVVXRaIq8Y7TQSLtLT7tLY5tB2l2a2l0a2wzS7oJqd0xMTI7zFnkQl5Xk5GQOHz7MlClTjNK7devGnj17CuSdb731FpMmTTJcR0dH4+/vj7e3N66urvn+Pp1Oh6IoeHt7l7q/BNLu0tHu0thmkHaXpnaXxjaDtLug2m1vb5/jvMU6iLt37x5arRZfX1+jdF9fX+7cuZPjcrp3786RI0eIi4ujQoUK/P333zRr1sxsXjs7O+zs7EzSNRpNgf2QKopSoOUXV9Lu0tPu0thmkHaXpnaXxjaDtLsg2p2bMot1EJdGURSja1VVTdKysnHjxly/Mzg4mODgYLRaba6fFUIIIYQoaMU6dPby8sLKysqk1y0sLMykdy6/jR8/npCQEA4ePFig7xFCCCGEyItiHcTZ2trSpEkTNm/ebJS+efNmWrduXUS1EkIIIYQoekU+nBobG8vFixcN11euXOHYsWN4eHhQsWJFJk2axHPPPUfTpk1p1aoVP/zwA9evX2fs2LEFWi8ZThVCCCFEcVbkQdyhQ4fo1KmT4TptZeiwYcNYsGABgwcPJiIigunTp3P79m3q1q3LunXrqFSpUoHWa/z48YwfP57o6Gjc3NwK9F1CCCGEELlV5EFcx44dUVU1yzzjxo1j3LhxhVQjIYQQQojir1jPiStKwcHBBAYGWtyKRAghhBCiKEkQZ4GsThVCCCFEcSZBnBBCCCFECSRBnAUynCqEEEKI4kyCOAtkOFUIIYQQxZkEcUIIIYQQJZAEcUIIIYQQJZAEcUIIIYQQJZAEcRbIwgYhhBBCFGcSxFkgCxuEEEIIUZxJECeEEEIIUQJJEFcK3Yi5QWRiZFFXQwghhBAPQYK4UiYyMZJ+K/rR/a/uxCbHFnV1hBBCCJFHEsRZ8KgubLgVd4tkXTLxqfGERIQUdXWEEEIIkUcSxFlQkhc2xCbHcvb+WbP34lLiDJ9vxN4orCoJIYQQIp9JEPeIUVWVJ1Y9wcDVAzl3/5zJ/YxB3LxT8wqzakIIIYTIRxLEPWKik6O5FXcLwGxvXMYg7lr0NS4+uFhodRNCCCFE/pEg7hGTkJpg+Pwg8YHJ/fjUeKPrE/dOFHidhBBCCJH/JIh7xCSmJho+30+6b3I/WZtsdH0v4V6B10kIIYQQ+U+CuEdMojY9iItPiTe5n6pLNbrO3Fun1Wk5dOcQqbpUElMT+eTAJzJ3TgghhCiGrIu6AsVVcHAwwcHBaLXaoq5KrmTsics4tJomc0/c/UTj3rpFIYuYdXgWT9V8inYV2vHLmV8A6FqxKxVdKxZAjYUQQgiRF9ITZ0FJ3WIkY+BmLohL0aUAYKuxBSA2JX3D3/239zPr8CwA/jj3BxEJEYZ7Z+6fKZD6CiGEECJvJIh7xGTsicu8iAHSg7gy9mUAjE5t2HFjh1HeyKRIw+fbsbfzs5pCCCGEeEgSxD1ikrRJhs/m5sSlDaemBXFpW46Ex4ezOGSxUd6r0VcNnw/dPZTfVRVCCCHEQ5Ag7hGT3RYjhp44u/964v4bTn1r11smeU/fO234/O/Nf0lITeDw3cOM2TyGK1FX8rXeQgghhMgdCeIeMRlXp96Nv4uqqkb3TYZT/wvi9t/Zb1LWuQfpJz6kqqlceHCB4RuGs+fWHt799918r7sQQgghck6CuEdM5tWpGYdXwcxwanKcSaAX4BpgdK1R9D8mGXv2zB3pJYQQQojCI0HcIyZjEAeQrDPeUiStJ87dzh3Q97Ddjb9ruN89oDs1PWoaPVPBuQJgPFSbFtgJIYQQomjIv8SPmMzbimTeFy5ts183OzcUFCD9jFUfBx8+a/8ZdT3rGj3j7+IPGK921am6/K24EEIIIXJFgjgLgoODCQwMpFmzZkVdlVzJeMA9YHE41c7KDmcbZwAuR10GwNfJF0VR6FutLw28G9DJvxP7h+zH2VafL+PGwFo1/zdBjk2O5esjX3Mp8lK+ly2EEEI8auTEBgvGjx/P+PHjiY6Oxs3Nrairk2NxqVkHcWnDqTYaG1ztXIlJieFypD6I83TwBMDD3oNfev5ieMbR2hGAu3Hpw646VYeqqiiKkm91n7R9Entv7+XfW/+ypPeSfCtXCCGEeBRJT9wjJnNPXIo2xeg6rSfOxsoGV1tXAEPPl7eDt9kyHawdAIhITD/BQafqTObbPay9t/cCEBIRkq/lCiGEEI8iCeIeMdkNp2bsiXOz0/cwnoo4BYCXg5fZMh1t9D1xGY/hAkhIMT3WSwghhBCFQ4K4R0xUUpTRdVZBnIuti9G9tGHTzMz1xIHxnnRCCCGEKFwSxD1CVFXlRswNAOyt7AHT1alp17ZWtoY8abpU7GK23LQg7l7CPaN0c2ezFoS7cXeZumeqnBIhhBBCZCBB3CMkOjnaEFhVdqsMGPfEqapq2EfORmODvXV6EOdo7Yi/q7/ZctN66DIP1WbeziQ/xSTHEJusP03i5a0v89eFv5iwbUKBvU8IIYQoaWR16iPkTtwdAFxtXQ1DpRl74t7c9SY3YvU9dbYaW6MgLu0EB3PSeuIyy885cZkXYDy+8nEeJD7g4/Yfc+b+GQCuRF0hPiXeMEdPCCGEKM2kJ+4RMmzDMEDfI2djZQMY98Stv7Le8NnGysYoOAv0DLRYrsUgLo89cVqdlkuRlzgYfpAT4ScAeJD0wCjP3fi7JOuSWXR6ET6OPob0sPiwPL1TCCGEeNRIT9wjJONwp53GDtAHcZuubuLzQ58b5bW1sjUKzp4LfM5iuZZ6vvIaxH199GvmnZpnuN4/ZL/RRsIZRSdHG92LTIrM0zuFEEKIR430xD0iMp+ZamelD+KStclM3jGZ23G3je7bamzxdfQ1XGf8nFl+9sSl6FKMAjiA1r+3trho4XLUZcNRYYDFYE8IIYQobR75nrjQ0FCee+45wsLCsLa25n//+x8DBw4s6mrlu9CYUMPnAdUHoNXpj8XKvMVIGjsrO3pU7sGN2Bt42ntSzrmcxbLzM4gbt2WcSZpW1fLGzjdy9HxkUiQJqQmoqipz44QQQpRqj3xPnLW1NbNnzyYkJIQtW7bw6quvEhcXl/2DJczqy6sNn6c0n2LUE2eOrZUtNhobXmzwIoNqDsqybEvBUubVqtnR6rTsu70vV89kFhIRwmN/PsaAVQOMeuiEEEKI0uaRD+L8/Pxo2LAhAD4+Pnh4eHD//qM3JDf/1HzDZwdrB2ytbIGse+JyKnNPnK1GX3ZOhzZvxt4kNCaU3Td3Z5kvwDWAA88coIlvE/pX609jn8YmeZacW0JUUpShTCGEEKK0KvIgbufOnfTp04dy5cqhKAorVqwwyTNnzhwqV66Mvb09TZo0YdeuXXl616FDh9DpdPj7m98P7VEwvuF4AEMQZ+lUhbT7OZE5iKvoWhGA8ITwbJ+9G3eXHn/14Om1T3M95nqWeZ8LfA4HawcW9FjAB20+wNPBM8v8h+8ezvb9QgghxKOqyIO4uLg4GjRowLfffmv2/pIlS5g4cSLvvPMOR48epV27dgQFBXH9enpA0KRJE+rWrWvy59atW4Y8ERERDB06lB9++KHA21TY4lPST04YUnsIkB6kRSdFm31Go+T8W29vZY+CYriu6KIP4jKf4GDO2ftnAf1xYJciLxnde7/h+3zQ+gPDdVmnskb3d97YafjsbONsUvb5B+dzUHshhBDi0VTkCxuCgoIICgqyeH/WrFmMHDmSUaNGATB79mw2btzI3LlzmTlzJgCHD2fdI5OUlMTjjz/OW2+9RevWrbPNm5SUPgQZHa0PgnQ6HTqdLkdtyg2dToeqqg9VdtpRWy62LjhbO6PT6bBR9PvEWdqSI7fvc7B2MJwGYQji4u9lW07GeXOXoy4bPg+oNoA2Pm3QOmpRUFBRqeRSyag8Nzs3w75wVd2rcjz8uFHZDxIfZPn+uJQ4UnWpuNm55bCVBS8/vt8lTWlsM0i7S1O7S2ObQdpdUO3OTblFHsRlJTk5mcOHDzNlyhSj9G7durFnz54claGqKsOHD6dz584895zlvdDSzJw5k2nTppmkh4eHk5iY/we+63Q6oqKiUFUVjSZvHaPH7+qDm7L2ZQkL0wc9SQn6QDQyPtIk/weNPjDkyyk7jR3x6IM4d9wB/ca72ZVz9/5dw+fLD/RB3LNVn+WZgGeIjIzETXXji+ZfoKBgl2BHWEJ6eZMDJ/PmoTcB8LP14zjGQdyGqxto4d6Ctr5tTd6rU3WM2DWCqJQofmn/i9mevKKQH9/vkqY0thmk3aWp3aWxzSDtLqh2x8TE5DhvsQ7i7t27h1arxdfXeA8zX19f7ty5k6My/v33X5YsWUL9+vUN8+0WL15MvXr1zOZ/6623mDRpkuE6Ojoaf39/vL29cXV1zVtDsqDT6VAUBW9v7zz/MNy9rQ+U6njXwcdHf7qBx30PAJJJX51a36s+01tPN5yrmhvOds48SNafqlC/Qn0IgfjUeLy8vbIcmrWOSP8Ri0qJ0tfTrw5lfctipbHC29ubsmXLmn22hWsLOKT/HOgbyIabG0zyTDs2jTX91+DvYjzPcev1rdxK0A+nJ9gnoLHRkJCSQG3P2jlvdAHIj+93SVMa2wzS7tLU7tLYZpB2F1S77e3ts8/0n2IdxKVRFMXoWlVVkzRL2rZtm6uuSTs7O+zs7AgODiY4OBitVr/fmkajKbAfUkVRsi0/LiWOFza9QIBbADPazDBq/7kH5wCo5VnLUEbauahpQ6Bl7Mrwa69f81xHa036j0qAWwAAKipJuiScbJwsPpekM10d6+ngiUajybbdznbpvWflXCzvY9d7RW+erPEk77d635C29spaw+dEbSJD1unnCn7/2Pe0KNsCK42VxfIKWk6+34+a0thmkHaXpnaXxjaDtLsg2p2bMov1V93LywsrKyuTXrewsDCT3rn8Nn78eEJCQjh48GCBvienNl3dxIl7J1h1aRUXIy8a3Qu5FwJAbY/0Xqa0hQ1p23BkDMLyIik1PRjzcvDCStEHQdntFZdx0UWazL1mlqRtZQJQ3b16lnn/PP+n0Z54Gc9YzbgVypjNY/jhxKO3uEUIIUTpU6yDOFtbW5o0acLmzZuN0jdv3pztAoVHzYqLKwyfd9zYYfh8N+4uYQlhWClW1PKoZUi3sbIxet5GY3ydWxm3GVEUxdD7FpsSa0hfdWkVT6x6gl03dnEm4gxgeqqDtcYaPye/HL1TURQ+bf8pbzV/iyruVbLNP23vNFJ1qZwMP8mJeycM6XfijH8JmHN8To7eL4QQQhRnRR7ExcbGcuzYMY4dOwbAlStXOHbsmGELkUmTJvHTTz8xb948zpw5w6uvvsr169cZO3ZsgdYrODiYwMBAmjVrVqDvyYkT4Sc4EnbEcL3nVvqijrQVnxVdKxqdrOBi42JURuagLrcGVB8AwMuNXgYwBHFxyek9ce/sfofzD84z7p9xDFoziNuxt02COA97j1wNZQZVDjJsm5LRjDYzqOpW1Sht1aVV/HbmN3bfMt5UOPO5sQDbQ7eb7SUUQgghSooinxN36NAhOnXqZLhOW1QwbNgwFixYwODBg4mIiGD69Oncvn2bunXrsm7dOipVqlSg9Ro/fjzjx48nOjoaN7ei3aJi+YXlRtfHwo4RlRSFm52bYajQx8HHKE8Z+zJG19bKw32rnw18lscqPYafs74XzRDEpVoeTj0cdtgkiMs4zJlbzjbOhp6/ftX60a9aP+otNF6g8tmhz+gR0MMo7VbsLTJ7eevL9AjowWcdPstzfYQQQoiiVOQ9cR07dkRVVZM/CxYsMOQZN24cV69eJSkpicOHD9O+ffuiq3AROB1xGoBxDcfh4+BDii6Ftn+0JUWbYgjiMgdtHvYeRtep6sOdM6pRNIYADtI3383YE5fZzZibhoUVacrYlbGQO3tfdfoKG40NQQHp+wr+0esPk3wbrhqvYs08nGopX5qY5BgeJD7Icz2FEEKIwlDkQVxxVRTDqT+d/Imn1zxNREKEIe3QnUOGUw8G1hiIVtUa7v1z/R+uRF0BTIM2X0df+lbta7jO3CP2sDLPidOppiuAb8eZDqf+0C3viwqa+zVn05Ob+LDdh4a0Ol51LOZPW+hhKYgzJz4lni7LutB+SXu2Xt+a57oKIYQQBU2COAsKe3WqTtXx1ZGvOBVxynCY/b7b+xixcQQA5Z3L4+XgZfTM6ztfZ9n5ZYBpEKcoCh+2TQ92ElPzd6Niw3Dqf6tTzR3vdS/hHnfj9HvYNfRuyKtNXjVafJEXXg5eJos0praaajZvuwrtAHiQpO9V6+jf0WQ7lFSdcQ/l6YjThsBzwrYJZstVVZX397zPpO2TzAavpV10cjS3Y03nIQohhMhfEsQVE9eirxk+px0svzhksSGtX9V+ALzd4m2zz3s4eJhNT5OkNd2v7WE42/43nPpfEHc/6b5Jnh03dnA1+ioALzZ8kefrPp+vdUjzRI0nOPLsERp6NzSkvd70dco5Ge8t52rranJyQ+btWtJ6PbNyIfICyy8sZ/O1zWYXTZQGWp3WYgAb9FcQ3f7q9lDzH4UQQmRPgrhi4lj4McPn0JhQrkdfZ9eNXQBMbDyR0fVHA9AtoBvlncubPO/rmPW+efndY+RorV8JmxbEZTeHLC1/QbGxsmFej3lsG7SNVf1XMbTOUEOgmcbZxhkXW+NVuyfCTxhdZwziNIrG/DBxhl6m7PbJexSlaFMYsGoAwzcMN7mnqirRyfpe2SN3j5jcF0IIkX8kiLOgsOfEpS1eAP22IUvOLUFFpV35doysN9Jos96Me7alqVGmRpblq6j5V1nSe+LS5sRlF8SZq3N+s9HY4OXgZThWLHOvm4uti0kQ98G+D9Dq9PMMk7XJnLt/znBPp+qISTY9wy7j5sGlMYg7HXGay1GXORp2lBRtitG9jHMgE7X5f9awEEKIdBLEWVDYc+IyDuvFpcSxKGQRAH2r9TXJW72M6ekF2fXE5TfD6tS04dRE0+HUjAojiMss8/w3F1sXk8AO9HVfcnYJTX5pYjjCLOM90M/vm7pnKqcjThOZFGm4H5scS2kTkZi+8Cbz6uO0XjjAbABcEp26d4qrUVcfqox5p+Yxbss42ZtQCJGvJIgrBlRV5VLkJQDc7Iz3pGtRtoVJ/tebvk6Xil1oU74NzjbOvNb0tWzPkq3kkr/76qUFSGsur6HhooZ8fOBjAJr4NmFSk0km+TNuRFxY0nrk0jjZOJk9fiwyKZIZ+2cYpaUFe2k9jLMOzeKvC3/x1JqnjHrfSmNPXMY2Lzu/jBn7Zhh6MzMGcWcizjB1z1RCIkIMaXtu7qHT0k6GqQLm7L65m4WnF6KqD997PP/UfDZd3ZSjvLdib7EjdIdR2r2Eezy99mn6rOjDsbBjjNk8huvR+o3I11xew1NrnjJc/3DiBzos6cDOGztNyvjy8JfsurmLraE5W/F8M/bmQy9G+vfmv2y8uvGhyhBCFG8SxBUDpyNPG/7xa10u/Tix6mWqm+z/BuDt6M3sTrP5rut37B2yl2F1hlkse3HQYtqUb8OsjrPytc4ZhyW1qpYUnX5YbUD1AXSt1NUkf0HPiTPHzc7NaB85Nzs3s4HBgFUDjK41ioYA1wAgPYjLONydcZFITEr+9TZpdVq+P/59rueS3Y27y+s7Xuf0vdPZZ86FpeeWMmj1IMMK4zQZe5O+OvIVS84tYd2VdYBx79vqy6v568JfPL3uaUPamC1juJdwjw/2fWDxvS9ueZHPD31uMQCJTo5m6bmlJkP4ay+v5b1/3yM8Ppy9t/ZyJuIMsw7PYvKOyTkKCJ9e+zQvbX3JEISdiTjD0PVDDfefW/8ce27todffvQB4a9dbnI44zQubX0BVVb45+g33E+8betHTLD6TvkAp4wIm0Ad4S88tJSIhgnP3z5GiTeHCgwv0+KsHYzaPybbOlmh1WsZuGctrO15j3+19eS5HCFG8SRBnQWHOifv2zLeAPsiY3GQy9lb2ALzT4p2HLruhT0O+6/pdjs4ezY3MPYZpnKydTIYxoWiGU0G/j1xD74Z42nvSulxrdKQvVLCzsjP7jKutK54OnoB+zpyqqkbDhhl7SLLa7NgSSwHFyksr+fbYtwzbYDkoN2fM5jFsuLqBl7e+nOu6ZOWDfR9w5v4ZFpxeYJSeeQgV4EbsDcD8VjPmpK1cTdGmGH09Mm75cjz8uNlnX932Kh/s+4D2S9obArk7cXeYsmsKf1/8m87LOvPC5hf47FD6aRwZewiTtcmEx4fzwd4POBF+grnH5rIjdIdh6Hz8P+NRVZWRG0cSGhNqtg5fHfnK8Plm7E1+PfOr4TpFm0J8SjzxKfGk6FKMvn43Ym4Ynhm+YTidlnbig30f0HFpR55c/SRjtowx/FJxJOwI8SnxhMeH88OJH4hIiEBVVWYfns3C0wsJjw/n1zO/kqRNIvhYMPUW1mP1pdWA8fSGDVfMb2othCj5ivzYreKqMI/dSju6qmOFjvg6+XLw2cKZh/cwXG1dzaY72jiaDeKyG+4tSPN7zCdVl4q9tT2BnoGGnpYV/VYQtDzIJL+NxoayTmUB/fyvZeeXGW0YnHHy/r47++hUsROVXHM2XH0i/ATjto5jQpMJDK412OhexmHH3LgUpR+KT9uaJidUVeX8g/MEuAWYDWbTNpEGfdCmqqrhe2huXldCagJn75/lQuSFHL1fq2q5GnWVgasH0i2gG+MajuN69HWjr2NMcgxzjs0hNCaUD9t+iEbR/8554M4BQ56Xt75MsjbZZOgc4OCd9L9H20K3ce7+OWp61GTa3mmGYHHp+aVm6zd179Qse1l/OvmT0fUnBz8xfL7w4AItftNPg+hWrptRvhsxN9CpOhaHLObw3cNZ1hmg7R9t8Xfx53LUZXbf3M0bzd7g51M/A/qg/8KDC3xx6AtDT/jbu9/mwoMLtPRraSgjY+9fbHIsdtZ2hr0W41PimXV4Fj0r96Sxb2OL7RVCFE8SxBUDdhr9P6J9qvYp4prknMWeOBsnbDW2hVybrFlrrA1z4UbXG01McgzNyjajgksFlvddbjKcaqWxoo5n+kkQmYf+0jYPBv28o/4r+7P36b3YW+t7UFVVJUWXgq2V6dfhnX/fISY1hhn7Z5gEcRn3VUvRpmBjZZP58Yd2LfqaYVuVt3e/TccKHfmmyzd8feRrzj84z+xOs7HWWBtto3Iv4R6dlnaiT9U+TG462WxP3Ol7pw2bVJtjbquWPbf2kKhNZNWlVay6tArAaC/BkPshrLy0EtBPM+hTtY9JOWm9dWfun8my3f/7939Z3s8s83nFuZEx+Nt0y3g+3rHwY7T9o22OF32k6FK4HHUZgKNhR3lh0wuGexceXDDkyWj+6fnMP53+vUjby/Ba9DX6r+xPz8o9+bDth0QnR/Pd8e9Ycm4JS84t4eSwk7lopRCiOJDh1GIg7X/C5v7RL64s9cTZaGxMet0GVB9gNm9RsLWyZUrzKXSp2AXQzztsW76tUR4NGotBKmB0LBrohwAz7i/37r/v0nFpR7NDcZa2YknRphhN9o9Kjsq+Mbmk1WkZvGYwb+9+m7d36zeN3n5jO3fi7vDjyR/ZcWMHR8OOolN1HLp7yPDczhs7iUiMMAwLRiWZ1i1j75g55jabvpdwzyRt1830r0FakAIY6rvo9CKTZ4qL3lV6W7w3qMYgw+eHWbWblzmYaUPJH+z7gFRdKqsurWLh6YW0/6O90Ybii04vMts7KIQoviSIKwZS1P+CuGLWg5WVjFt1WCvWTGw8kb5V+1LTo6ZRPi8HL4vHYhUXszvNZnbH2YbrmOQYWpVrZTF/2vyvjJ5b/xx/X/gbnapj1aVVxCTHMHXPVJOgzdI/wpFJkaSq6fPBcjq3LDduxN4wu5r2u+PfGT7rVB2/nvmVH0/+aLaMFG2KSRCbEycfnDR57tS9Uyb5MgZumZ2OOM0Xh7/I9buz8nStp02Os8uNMfXH8MNjP7B/yH5mtptpMV/mvxeFKSY5hv/9+z/2395vSPv80OdG5zADfHboM4ZvGE7/Ff3z5azliIQIvjryFRcfXMw+sxAiTySIs6AwFzak9cSZ2/6iuMrY26ZRNIysN9Jo3lIaa411kc6Hywk7Kzu6VOrCs7WfBWBC4wk4WDuw+cnNDKg+gBX9VnBy2Eka++jnDJnriQJ4b897RsNdB+4coPtf3Y1WtlqSeZ+9jBPxs2OlWBk+J2uTDZ8zL6C4GXPT7PPnH5w3fE6bh2bJ7pu7jfaJy6l3jrxD5z87G6VlPKUkJ55a81Su3wvQp4rlaQpj6o+he0B3w/Wbzd40yeNh70HNMjVp6deSlxq+ZHSvhV8LWpVrZXYLnXeapy9M8nPyo2tF01Xbf/T6w+xxdBMaT6CTfyc87T3NbjOUZkTdEQwNHEqfKn1Y2tv8/D6AFRdXWLyX2aWoS7T8rSX1Ftbj6yNfM3HbRBaeXmi4r9Vpc7S1zocHPuSnkz9luRJZCPFwSk7UUMgKa2HD0nNLiUyOBErWcCrAuAbjmHN8DqPqjbKYJ2OAUdxNbjqZAdUHUM29GgBlncoyrfU0w/20xQ5Z2X9nv9F1QmoC807OY3T90Sw9Z/yPrKqqHA8/zps73zQZ0s1NEKdRNIZelUUhi7CzsqOqW1Xe2PUG77R4h7bl26JRNGbPtwXjHrG78XcNp3CY88q2V3Jcr+zkR29PTvyv1f94vPrjPL/RNFhytXM1mv/YrKzxL217n96LjZWNYfFHii6FSm6VeH3H64DxalqAaa2n8f6e96nnVY+gykF8eOBDAOp716eFXwuOhx831KOOZx3qeNWhtmdtKrhU4FLkJcMq18erPW7092rslrH8e/NfQB9opi2kqO1Rm6DK+sU5+bGvXpq0uYdpPbL/XP+HTdc28UT1J3h/z/uAftug+NR49j691+SIu7RnQL/KVghRMCSIK2JrrqwxfE5bMVZSvNjwRYbWGWp2NWon/05sC93GuIbjiqBmeWOtsTZ7GkYaD3uPPJVrY2XDkLVDTCagx6fG89qO17gbf9dklaS5xQPmpOpSjcpN2/rCz8mPqKQo3tj5huHe/1qan9yf8Ui2baHbcvTeoja361xe3PKiUZqLrYthvllH/4609GtJ/2r9cbB2MAnO0thobAx7AoJpoJ45OLHR2NAjoIchiKvgUsHo/oDqA2hdrjUe9h5YK9Z82vRTvD29DXMsm5VtxucdPif4WLDhFwSNomFgjYGAfng3SZtk2OImzVedviIiIQIrxQpfJ19SdCnsv7OfTv6dDHky93jX8qjF9NbTWXdlnck2MQAbn9hIXEocr2x9xewUgcxOhJ8wOms47Wd08o7JfNzuY1ZdWkXvKr0pY1fGpKfuTtydHP0SJITIHRlOLWIZA7eS1hMHpkdbpfmw7Ycs6LGAvlVNjw0rqTL/w5pT1oq1SQAH+mFZS71eCSk566WyFOylrUjMKCdbkGTcSqWVn+V5gQA1y9Q0CoAyyu4s39x4of4LJmnV3KsxuKbx6t7KrunbjNTzqscztZ8x+vn8o/cfDKoxiOmtpxs9V8+7Hv9r+T+mtpqKm52bYQ5n5nwZ/dX3L37s9iP+Lv4m98o6lTX8XW7k2cgwDJ+me0B3VvVfZXaeXCXXSma/dnZWdpRzLoevk/54vRF1R/Bd1+8MK6LTbHhiA14OXoysO5JlfZZR27M2k5tO5uSwk5wYeoKna6VvvFzOuRzVy1RnYM2BFtuZE3tu7eHlrS/z+aHP6bi0I6qqcjfReIPox/58LNuj+YQQuSc9cUUs4xyyktYTlxUXWxea+DYp6mrkK0/7vAVxadtkZBaZFGlxHqS5w+NPR5zmcuRlo61ocnMWZ3h89kFc2p5inf0781qz1+i5vKfZfC39WvJjN/1QW72F9Uzuz+06lwN3DjB1z1SzK1Nzw9wvCmWdyjK56WTalGvDlutb2Ba6jfdavceN2BscvXuUoYFDTZ6p41mHOq3qoKoq0cnRBHoGGu4Nqpm+evSJGk/wRI0nsqxTfgap+am8c3m2DTLfm6ooChMaTyAqKYpuldL3r3u29rPYWdnRrnw71l9Zz7fHvmVgjYEMqD6AP87+YfHnN6OMGzOHxYdZ3IamXYV2eWiVEMIS6YkrYhnnsZTEnrjSJLvh1A/bfpir8qKSorBWzAdx16OvM/fYXG7Gpi9GeGrNU7y9+2323NpjSMvN2a252QzY29Gbck7lDNcaRcPax9carqu4pZ8A0r5Ce6Nnn671ND6OPvSu0tsQHL0S+IphI18F42E/eyt7o6E2G42N0fCno7Ujn7T7hMwcrB3oVLETH7b9kO2DtlPToyZdKnbhtWavmfRQZaQoCsPqDLM4xPooc7Jx4pP2n9ClUhdDmq2VLc/UfoaKrhUZ02AMe57ew3ut3qOuV11mtJ1hsn9cXc+6Wb5j47WNhlNoMjLXGy2EeDgSxBWxjPORStIWI6VROef0oCZzr5ydlR3V3dPn09XxrMPK/ln3YEQlR1nsifvlzC/MOT6HZ9c9a3LveNhxLkddZtDqQbnaxDbzwexZCfQMxEqTvihFp+qo4FKBau7VUFDoXDF9pemLDdLnpm0btI0pzacYrl9v+jo7B+2kj38fVvVbxbZB2xhRd4Thfi2PWhx45gAz2swwpHk6eDKzbfp2He727vSs0pOKLhUBzAa+8gtQ/sl4LnKatFGClxu9zOKei7MM5L44/AWXYi6ZpGfcFzA/F2EIUZpJEGdBYW4xkqYkbTFSGlVzr8b/Wv6PFn4tWN5vOe3Kt8Peyp4WZVuwsMdCXO3SN0Bu6NOQKm5VGFJriMXyohKjTP4xy7zJ8L2Ee2h1xvt5RSZF8veFvzlz/wwn7z3cLvuW5rSlrdhMm9M4ut5oNIqGxUGL+avvX7TwS9/2oq5XXRYHLWZl/5V4OXgZTRFQFMWoTV4OXkY9fA7WDiiKYtTLeSfuDt6O3obrtBNNvun8DS39WvJTd+Mjr0TBW9J7Ca82eZURdUdgrbHm996/s2/IvlyV8cG+D3h719vUW1iP+ovq02BRA/bf3s/KiytNNkAOiw+zuJWPECKdRA0WFObZqWmK+35qpZ2iKAyqOcgwRDinq/F+ahnnp6UFLkPrDOW3s7+ZLS8qOcrkZAYPew+Tf7zuJ9432pA2WZfMrbhbeW7HM7Wf4dczv6JRNEYBl4e9h2HyeVX3qgBMbT2Vzv6dDZsfO9s6U93WdAVvQ5+GOX7/gOoDmLFf3/OWFiymTdhPo1E0PFH9CU5HnDa8u4p7FcM8PFG4qpepbrJy28nGiVH1RpmcI5uV1ZdXGz7rVB2jNv23jcq/MK/7PBJTEwlwC6D3372p5FqJVf1X5Uv9hXhUSRBXxGRY4dHhaONIs7LNOHL3iGHieHnn8pR1Kmu06jPNrdhbJpP+Pew9jA6fB/3mqxk32P3z/J8PVc+q7lXZ+MRGHKwd+OzgZ4azOdcNWMeIDSPoU7WPoVfYRmNjNH8qP2Q8EzZtxW/GY9zSVoVObT01X98r8t+ExhNoXrY5lyIv0btKb9otyfvChcz7+F2JukKSNgk7KzviU+INvbZCiHQSxBWxjHPiRMn3TedveJD4wGj/MHsr40n2jTwacfT+Uf668JfJ8+ZWwI7eNDrb905pPoWPD3yMg7WDYRNda8WaVDUVFxsXo+O+0rarAHit2WvYWtkyoPoAnGycWNrH8q7/+emjth+x5doWo+Hmrzp9xYnwE/Sr1q9Q6iDyR6tyrQy9pQt6LGD4huEAvFDvBeJT4/nlzC95Lnvt5bVsD93OttBtjKk/hpcavZTtM0KUJhLEFTEJ4h4tTjZOJlti+Dn5cTX6KgBbn9zK1TtXeX636ekBkPcNhVv5teL40ONoVS2NF+v3JetbrS/V3avTyLcRW69v5YcTPwAYnQ7hYe9RJD1efar2MdoqBaBzxc5GCyZEydPEtwnHnzvOtVvXqFSuEigwsOZAZh+enaeNpNNOhwD4/sT3jKk/xtCTG5cSh4O1A/cT76NTdfg4+jx0/VVVZcquKSRrk5nVcZb0/IliT4I4IQrY2y3eZsK2CYyoOwJPB0+0TlralW/Hrpu7TPJ6OOQtiKvoWtFkjptWp+XZQP3q1squlanrWZd2FdrJAhpR4BysHQD93MYqblX4uvPXbLi6gQDXAGqWqUnvv3tzPeZ6rsvdd3sf7Sq0Iyw+jN5/96a+V30uRV0iPiWeLzt9ycarG3m+7vOG7Wxy637ifdZdWQdARGKE0VxUIYoj+b95EdPIAuFHXoBbgGG7EZ1Ofyalj4P5XoO8bChc1a2q2cCskU8jw2dHG0c6VexkkkeIwtIjoIfh8+rHV5OYmsi9hHu42bkxecdkelXuxbxT8wy91uacvX+WA3cOEBIRQkJqgtFZxWM2jwFg/+39zOkyh3d2v0O/av14qtZTOa5jxpNOElISwCEXDRSiCEgQV8R8HX2zzyQeOZZ63DLOpavnVS/bLUQWBy02+Rla9/g6Docdpk+VPhaeEqJoaRQNjjaOVLTR7/33Uzf9CtcWfi3o/ld3i899ffTrbMu+GXuTfiv18ypPRZzi8N3DfNL+E6Ne6jRXoq4QkxxDfe/6hmfTWDoST4jiRLqBitj4huOxt7LnyepPFnVVRCHKuDGwnZWd4XPGszgfq/SY2WfTFgOMrDuShj4N8XP2M7rv7+pP/2r9jTbrFaIkKOdcjiPPHsnXrWQ2XN3AkbtHzN7ru6Ivz6x7hm+OfkNUUhS3YtO37slrELf1+lYmbJ1AZGKk7D7wkG7F3iJFm37Sxzu736Hvir4mJ9Xcjr1NbHLW368kbRI/n/yZCw8uFEhdi4oEcUWsgksF/ur8F++2eLeoqyIKUeeKnanlUQtvB2+G1E5foZlxDk5dL9Nd8QNcA3ip0Ut83/V7JjSeUCh1FaIw2VjZGE03mNJ8Cn/3/fuhysw47Jom476OP5z4gXd2v8Osw7MMaXEpcZwIP0FIREiu3jVh2wS2hm6l3ZJ2tFvSLk9Bg07V5fqZ4iI2OZZDdw6x8PRCVFXlQeID1l1eZ7RpuVanZffN3WbPVdapOi5FXuJ4+HG6/9Wdt3e/DUB0cjSrLq3iStQVDt05BEBCagK3Ym/R6+9ejNg4Aq1Oy8E7B0nSJnEt+prR13H+qfnMPjKbAasGGNJUVTUKEtP8eOJH6i2sx5xjc0zuFTcynGpBcHAwwcHBaLXa7DM/JFuNrayCKmVsNDb82vNXdKoOnarj4oOLtKvQDgdrB+Z1nwekb7gLMKfLHL47/h3vtXoPF1sXWpdvXVRVF6LA+bv6U9+rPs62zgypNeSh//+49vJaxjUYh07VserSKq7HXDfpndtxY4fR9Z24O7y89WUADj5zkJP3TuJk40SgZ6DZd2y9tZXE8ESjtKikKGbsm8HCoIU5ruvhu4d56Z+XeK3pazxR44kcP5ffLkdeZuL2iQyuOZhnaj9DbHIsg9YMolnZZkxrPQ1VVXlp60tExkcyu8tsfJx8WHpuKR/s+8BQhp+TH0vPLWX/nf1ciLxg+MXz74t/M23vNNqVb8eExhP45OAnvNLoFRr6NOTrI1/z86mfDWVsuLqB+t71WXB6gSHtQuQF1l5ey/qr6xlcczApuhTO3j9L81+bk6xLNuR7of4LnIk4Q4/KPTgWfsyQfjfuLvbW9kzePplTEacYUUd/EomKSiOfRoZh+7nH59IjoAcuti48tfYpnqzxpNExg8WBokp/b5bSTmyIiorC1dU1+wdySafTERYWho+PDxpN6ekYlXbnrN17bu3B3sqexr6NC6F2BUO+19Lu/HDwzkGWX1hOB/8OfHbwMxr5NGLj1Y3YW9mTqE3M9vlulbpR27M2Xx35KtfvbuzTmCNh+qDv8w6fExIRwoTGE7ifeB9Ha0cO3znMuK3jzD5bzb0aXg5eJKQm4G7nTv9q/elaqavFdz2+8nEuRl4E4OSw9Dmxqqqiopqd25cmMjGS2Udm08inEX2q9skyb3Y+Pfgpi0MWA7B14FZOR5w2BLX/Pv0vWp2W9kvaG/KXdy5vNKcQIKhyEOuvrM/R+zzsPXit6WuGnrf81tKvJftu5+6oOHNqlqnJd12/IzU6tcD+bucm7pAgLhsSxBUMaXfpaXdpbDNIuwuj3ZcjL5OoTSQqKYr//fs/Ovp35MKDC4aAqyBZOoklJ4YGDuXVJq8arSrff3s/f5z9gxPhJwhLCAPgxNATKIpCfEo8Q9YOwcbKht97/W52NfrlqMv8duY3lpxbYpTe1LcpT9Z4kmru1ajpUTPLei2/sJxVl1bxWfvPeH3n6xy+exjQbxnzVvO3eG/PewAs67MMgIGrB2ZZXh3POpyOOJ3NV6Nkeqn2S4xuOrrIgzgZThVCCFEiVXGvYvi8ZeAWw+czEWcYtGYQ/ar2Y+WllQXy7rwGcACLQhYRHh/OB20/IDQ6lKruVdPPkc2gw5IOjKo3Ci8HLy5FXQLgdMRptlzbQmRSJCPrjuTw3cPciL1h8QzbQ3cPceiufg6ZgoKiKIa5Yg7WDqzot4Jvjn5DRZeKzDmunwPWeZnxptsJqQmGAA7g55M/s+Hqhmzb+agGcABxqXHZZyoE0hOXDemJKxjS7tLT7tLYZpB2F5d211tYz+K9IbWG8NvZ3wgKCCI0JhR/V398HX2N5l8VZwrKQ5/642TjZLLaU2RvQuAEnm/yvPTECSGEEAXlsUqPsfnaZqM0JxsnXm70MkNqDaFTxU7U96qPo40joJ97Zq2xttizVZzkx7GNxS2Ay3jW8+xOs5m4baLZfFOaTyE2OZar0Vcp61TW8P36rut31PWqy84bO5m6ZyrvtnzXqBcxv3QtZ3leY2GSIE4IIcQja0abGdyNu8uJeycMaZ93+NxwhnBLv5ZG+RVF4ZVGr1DZrTLLzi0zWtVoiYutCzHJMSbpwwKHsTAk5ytT81vGALaaezXDgomCMqPNDN7913S7rAbeDZjcdDIVXSrScWnHLMvYM2QPWp2WJG0SjjaOeNh7cD/xvkm+Z2o/Y/h84cEFQxDn5eCFm52b0fnM4QnhfHP0G7Pve6zSYwRVDuKxSo9xPPw4y84t48WGL+Lj4ENUchReDl6svbyWmQdm8lHbj2ji2wR7jT3h4eE5/bIUqKLv6xZCCCEKiKONI7/2+pUP237I/9u7+6Ao7vsP4O8Tj+NBxCAeB6JIjA+NwEXAGHzWREaqwdSMgFLE+tAxQhQ1USM1GrXFJsbpZFSSTo0PbVOSadTmVxgTaARlQEMQEkSiWAmYyoMSngLhQe77+8OwsvKoObxb7v2auRn87vf29r3fXe8zu3u7ADDYejC8nbs+xQrcLeSCRwfjr7/8K37j9RsAwKv+r3ba9+CzB5Eemo7ZunuPtVvptRKxk2Ox5BdLZH1PLTzV6+X+18J/IeLJiF73B4DUkFTEPxcv/dvPxU/6O2zcvcePJf0qCelh6Z3OY8n4JR3aPlrwES4svYCkRUmYrJuM6KeikReZhyXj7vWNmx6HhU8sRNRTUXhq2FOwUt272bizrTMmaidiqO1QzHSfCQC4sLTjvfsGW989dWg1wEo6Mtq+gDv94mlobbWYOnyq7H3tb8eks9d1mO8KrxXY+vRWfBz8MVZ5r8KWSVuwa8ouhI0Lw76Z+6Qbq+uH6bFn2h4MHzQcaiu1dN/O+Y/PR3pYOma4z4C92t6sbgnGa+J6wGvi+gZzW05uS8wMMLc55jYIA5pbm2Ez0OaB3lNcW4xRg0fB57iP1O7j7AO9Vo/NkzbDYDAg+rNonCs/B0B+a5D21+TlReZBCAGVSoW8W3lYmnT3Rt+/cPoFCr4vgO1AW6QsTpGKGeDujzQ2pG7A07qncfKa/KbHYx4bg8KqQjjbOuPfv/o37NX2uGO4g81nN2PC0AlYNmEZ9mXtg/cwbzwx5Aks/r/FcLB2QMaSDABAcnEyNqZulM0zLzIPMWdi8J+S/wAAfun5S/xxxh87XTfFNcUI/lcwgkYFYe+MvbJpf8n7i3Q7l6RfJWHE4LtPoxFCwCAMsBpghezybNS31OMp7VO4dPsSJukmQT1ALZuPzzEf6bRxXmQeWlpbMHDAwA6F1PWa62i809jlffyMqa+3cd5ixIhYxPUN5rac3JaYGWDu/pi7rSDzd/HHkXlHpHaDwYCs61l46/JbWOe7DjPc790/Le1GGl7PeB27p+6WtQsh8ME3H6C5tRkh40Lwj2/+gbkec+Ex2KPLz88qy8IfLvwB16qvYcn4Jdg2+cHuqZZbkQt3B3fZk2Hyb+dL15TN9ZgLXxdfCCHw+wu/x6Xbl3As6Jjs0YDtGQwGfHvzW3i4esDKSv6YPyEESupK4D7I/Wc9AjCrLAs7M3Ziz7Q9mKid+NDzMSYWcY9QXV0d5syZg5aWFrS2tmLdunVYvXp1r9/PIq5vMLfl5LbEzABz98fcX9/6Gkfzj2KT/yYMHzRcau8pc9vRt/6mP491d8ypiOv3P2yws7NDWloa7Ozs0NDQAC8vLyxatAhDhw419aIREZGC+Azzwf5Z+3vueJ/+WMCReej3pbOVlRXs7O5eINnY2IjW1lb084OPREREZAFMXsSdPXsWzz//PNzc3KBSqXDq1KkOfQ4dOgRPT0/Y2NjAz88P586de6DPqK6uhl6vh7u7OzZv3gxnZ+ee30RERERkxkxexNXX10Ov1+PAgQOdTv/www8RExOD2NhY5OTkYPr06QgKCkJJSYnUx8/PD15eXh1eN2/eBAAMGTIEX331FYqKivDBBx+gvLz8kWQjIiIi6ismvyYuKCgIQUFBXU7fv38/Vq5ciVWr7j5X7k9/+hM+/fRTxMfHIy4uDgCQnZ3dq89ycXGBj48Pzp49i8WLO39wb1NTE5qamqR/19bWArh7IaPBYOjV5zwIg8Fw9yfXfTBvc8bclpPbEjMDzG1JuS0xM8DcfZX7QeZr8iKuO83NzcjOzsbWrVtl7YGBgcjIyOjVPMrLy2Fra4vBgwejtrYWZ8+exUsvvdRl/7i4OLzxxhsd2m/duoXGxsYHC9ALBoMBNTU1EEJY3K97mNsycltiZoC5LSm3JWYGmLuvctfVdXz6R1fMuoi7ffs2Wltb4eLiImt3cXFBWVlZr+bx3XffYeXKlRBCQAiB6Oho+Pj4dNn/tddew8aN925+WFtbixEjRmDYsGF9dosRlUqFYcOGWdxOwNyWkdsSMwPMbUm5LTEzwNx9ldvGpvc3ozbrIq7N/T/PfpB77vj5+SE3N7fXn6XRaKDRaHDw4EEcPHgQra2tAIABAwb02UaqUqn6dP7mirktJ7clZgaY25JyW2JmgLn7IveDzNOs17qzszOsrKw6HHWrqKjocHTO2KKionD58mVkZWX16ecQERERPQyzLuKsra3h5+eH5ORkWXtycjKmTJlioqUiIiIiMj2Tn0794YcfcO3aNenfRUVFyM3NhZOTE0aOHImNGzciIiIC/v7+CAgIwJ///GeUlJRgzZo1fbpc959OJSIiIjInJi/ivvzyS8yePVv6d9uPCiIjI3H06FGEhoaisrISu3btQmlpKby8vJCUlAQPj64fEmwMUVFRiIqKkp5hRkRERGROTF7EzZo1q8fHYK1duxZr1659REsk17ZsbfeLMzaDwYC6ujrY2NhY1IWhzG05uS0xM8DclpTbEjMDzN1Xudvqjd48ItTkRZy5a7tfy4gRI0y8JERERGQp6urqejwTqBJ8Gny3DAYDbt68CQcHh17f1uRBtN2H7saNG31yHzpzxdyWk9sSMwPMbUm5LTEzwNx9lVsIgbq6Ori5ufV4pI9H4nowYMAAuLu79/nnDB482KJ2gjbMbTksMTPA3JbEEjMDzN0XenstvuWcxCYiIiLqR1jEERERESkQizgT02g02LFjBzQajakX5ZFibsvJbYmZAea2pNyWmBlgbnPIzR82EBERESkQj8QRERERKRCLOCIiIiIFYhFHREREpEAs4oiIiIgUiEWciR06dAienp6wsbGBn58fzp07Z+pFemhxcXGYNGkSHBwcoNVq8cILL+DKlSuyPsuXL4dKpZK9nnnmGVmfpqYmvPzyy3B2doa9vT2Cg4Px3XffPcoovbZz584OeXQ6nTRdCIGdO3fCzc0Ntra2mDVrFvLz82XzUFLeNqNGjeqQW6VSISoqCkD/GeezZ8/i+eefh5ubG1QqFU6dOiWbbqzxraqqQkREBBwdHeHo6IiIiAhUV1f3cbrOdZe5paUFW7Zsgbe3N+zt7eHm5oZly5bh5s2bsnnMmjWrw/iHhYXJ+phTZqDnsTbWNq203J3t5yqVCm+99ZbUR2nj3ZvvKqXs2yziTOjDDz9ETEwMYmNjkZOTg+nTpyMoKAglJSWmXrSHkpaWhqioKJw/fx7Jycm4c+cOAgMDUV9fL+s3b948lJaWSq+kpCTZ9JiYGJw8eRIJCQlIT0/HDz/8gAULFqC1tfVRxum1CRMmyPLk5eVJ0958803s378fBw4cQFZWFnQ6HebOnSs9kxdQXl4AyMrKkmVOTk4GACxevFjq0x/Gub6+Hnq9HgcOHOh0urHGd+nSpcjNzcXp06dx+vRp5ObmIiIios/zdaa7zA0NDbh48SK2b9+Oixcv4sSJE7h69SqCg4M79F29erVs/N977z3ZdHPKDPQ81oBxtmml5W6ft7S0FO+//z5UKhVefPFFWT8ljXdvvqsUs28LMpmnn35arFmzRtY2fvx4sXXrVhMtkXFVVFQIACItLU1qi4yMFAsXLuzyPdXV1UKtVouEhASp7X//+58YMGCAOH36dF8u7kPZsWOH0Ov1nU4zGAxCp9OJvXv3Sm2NjY3C0dFRvPvuu0II5eXtyvr168Xo0aOFwWAQQvS/cRZCCADi5MmT0r+NNb6XL18WAMT58+elPpmZmQKA+Oabb/o4Vffuz9yZL774QgAQxcXFUtvMmTPF+vXru3yPOWcWovPcxtimlZj7fgsXLhRz5syRtSl9vO//rlLSvs0jcSbS3NyM7OxsBAYGytoDAwORkZFhoqUyrpqaGgCAk5OTrD01NRVarRZjx47F6tWrUVFRIU3Lzs5GS0uLbL24ubnBy8vLbNdLYWEh3Nzc4OnpibCwMFy/fh0AUFRUhLKyMlkWjUaDmTNnSlmUmPd+zc3N+Nvf/oYVK1ZApVJJ7f1tnO9nrPHNzMyEo6MjJk+eLPV55pln4OjoqIh1UVNTA5VKhSFDhsja//73v8PZ2RkTJkzAK6+8IjuCodTMP3ebVmruNuXl5UhMTMTKlSs7TFPyeN//XaWkfXugUeZCD+z27dtobW2Fi4uLrN3FxQVlZWUmWirjEUJg48aNmDZtGry8vKT2oKAgLF68GB4eHigqKsL27dsxZ84cZGdnQ6PRoKysDNbW1njsscdk8zPX9TJ58mQcP34cY8eORXl5Ofbs2YMpU6YgPz9fWt7Oxri4uBgAFJe3M6dOnUJ1dTWWL18utfW3ce6Msca3rKwMWq22w/y1Wq3Zr4vGxkZs3boVS5culT0IPDw8HJ6entDpdLh06RJee+01fPXVV9JpdyVmNsY2rcTc7R07dgwODg5YtGiRrF3J493Zd5WS9m0WcSbW/sgFcHeDur9NiaKjo/H1118jPT1d1h4aGir97eXlBX9/f3h4eCAxMbHDfwztmet6CQoKkv729vZGQEAARo8ejWPHjkkXPT/MGJtr3s4cPnwYQUFBcHNzk9r62zh3xxjj21l/c18XLS0tCAsLg8FgwKFDh2TTVq9eLf3t5eWFMWPGwN/fHxcvXoSvry8A5WU21jattNztvf/++wgPD4eNjY2sXcnj3dV3FaCMfZunU03E2dkZVlZWHarxioqKDtW/0rz88sv45JNPcObMGbi7u3fb19XVFR4eHigsLAQA6HQ6NDc3o6qqStZPKevF3t4e3t7eKCwslH6l2t0YKz1vcXExUlJSsGrVqm779bdxBmC08dXpdCgvL+8w/1u3bpntumhpaUFISAiKioqQnJwsOwrXGV9fX6jVatn4Ky3z/R5mm1Zy7nPnzuHKlSs97uuAcsa7q+8qJe3bLOJMxNraGn5+ftLh5jbJycmYMmWKiZbq5xFCIDo6GidOnMDnn38OT0/PHt9TWVmJGzduwNXVFQDg5+cHtVotWy+lpaW4dOmSItZLU1MTCgoK4OrqKp1eaJ+lubkZaWlpUhal5z1y5Ai0Wi3mz5/fbb/+Ns4AjDa+AQEBqKmpwRdffCH1uXDhAmpqasxyXbQVcIWFhUhJScHQoUN7fE9+fj5aWlqk8Vda5s48zDat5NyHDx+Gn58f9Hp9j33Nfbx7+q5S1L5tlJ9H0ENJSEgQarVaHD58WFy+fFnExMQIe3t78e2335p60R7KSy+9JBwdHUVqaqooLS2VXg0NDUIIIerq6sSmTZtERkaGKCoqEmfOnBEBAQFi+PDhora2VprPmjVrhLu7u0hJSREXL14Uc+bMEXq9Xty5c8dU0bq0adMmkZqaKq5fvy7Onz8vFixYIBwcHKQx3Lt3r3B0dBQnTpwQeXl5YsmSJcLV1VWxedtrbW0VI0eOFFu2bJG196dxrqurEzk5OSInJ0cAEPv37xc5OTnSLzGNNb7z5s0TPj4+IjMzU2RmZgpvb2+xYMGCR55XiO4zt7S0iODgYOHu7i5yc3Nl+3lTU5MQQohr166JN954Q2RlZYmioiKRmJgoxo8fLyZOnGi2mYXoPrcxt2kl5W5TU1Mj7OzsRHx8fIf3K3G8e/quEkI5+zaLOBM7ePCg8PDwENbW1sLX11d2Ow6lAdDp68iRI0IIIRoaGkRgYKAYNmyYUKvVYuTIkSIyMlKUlJTI5vPjjz+K6Oho4eTkJGxtbcWCBQs69DEXoaGhwtXVVajVauHm5iYWLVok8vPzpekGg0Hs2LFD6HQ6odFoxIwZM0ReXp5sHkrK296nn34qAIgrV67I2vvTOJ85c6bTbToyMlIIYbzxraysFOHh4cLBwUE4ODiI8PBwUVVV9YhSynWXuaioqMv9/MyZM0IIIUpKSsSMGTOEk5OTsLa2FqNHjxbr1q0TlZWVss8xp8xCdJ/bmNu0knK3ee+994Stra2orq7u8H4ljndP31VCKGffVv0UiIiIiIgUhNfEERERESkQizgiIiIiBWIRR0RERKRALOKIiIiIFIhFHBEREZECsYgjIiIiUiAWcUREREQKxCKOiIiISIFYxBERmYnU1FSoVCpUV1ebelGISAFYxBEREREpEIs4IiIiIgViEUdE9BMhBN588008/vjjsLW1hV6vxz//+U8A9051JiYmQq/Xw8bGBpMnT0ZeXp5sHh9//DEmTJgAjUaDUaNG4e2335ZNb2pqwubNmzFixAhoNBqMGTMGhw8flvXJzs6Gv78/7OzsMGXKFFy5cqVvgxORIrGIIyL6ye9+9zscOXIE8fHxyM/Px4YNG/DrX/8aaWlpUp9XX30V+/btQ1ZWFrRaLYKDg9HS0gLgbvEVEhKCsLAw5OXlYefOndi+fTuOHj0qvX/ZsmVISEjAO++8g4KCArz77rsYNGiQbDliY2Px9ttv48svv8TAgQOxYsWKR5KfiJRFJYQQpl4IIiJTq6+vh7OzMz7//HMEBARI7atWrUJDQwN++9vfYvbs2UhISEBoaCgA4Pvvv4e7uzuOHj2KkJAQhIeH49atW/jss8+k92/evBmJiYnIz8/H1atXMW7cOCQnJ+O5557rsAypqamYPXs2UlJS8OyzzwIAkpKSMH/+fPz444+wsbHp47VARErCI3FERAAuX76MxsZGzJ07F4MGDZJex48fx3//+1+pX/sCz8nJCePGjUNBQQEAoKCgAFOnTpXNd+rUqSgsLERraytyc3NhZWWFmTNndrssPj4+0t+urq4AgIqKip+dkYj6l4GmXgAiInNgMBgAAImJiRg+fLhsmkajkRVy91OpVADuXlPX9neb9ic7bG1te7UsarW6w7zblo+IqA2PxBERAXjyySeh0WhQUlKCJ554QvYaMWKE1O/8+fPS31VVVbh69SrGjx8vzSM9PV0234yMDIwdOxZWVlbw9vaGwWCQXWNHRPSweCSOiAiAg4MDXnnlFWzYsAEGgwHTpk1DbW0tMjIyMGjQIHh4eAAAdu3ahaFDh8LFxQWxsbFwdnbGCy+8AADYtGkTJk2ahN27dyM0NBSZmZk4cOAADh06BAAYNWoUIiMjsWLFCrzzzjvQ6/UoLi5GRUUFQkJCTBWdiBSKRRwR0U92794NrVaLuLg4XL9+HUOGDIGvry+2bdsmnc7cu3cv1q9fj8LCQuj1enzyySewtrYGAPj6+uKjjz7C66+/jt27d8PV1RW7du3C8uXLpc+Ij4/Htm3bsHbtWlRWVmLkyJHYtm2bKeISkcLx16lERL3Q9svRqqoqDBkyxNSLQ0TEa+KIiIiIlIhFHBEREZEC8XQqERERkQLxSBwRERGRArGIIyIiIlIgFnFERERECsQijoiIiEiBWMQRERERKRCLOCIiIiIFYhFHREREpEAs4oiIiIgU6P8BaQus1EBk7kEAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "fig, ax = plt.subplots(1, 1, figsize=(7, 4))\n", + "ax.semilogy(history[\"total\"], label=\"total\")\n", + "ax.semilogy(history[\"interior\"], label=\"interior\")\n", + "ax.semilogy(history[\"boundary\"], label=\"boundary\")\n", + "ax.set_xlabel(\"epoch\")\n", + "ax.set_ylabel(\"loss\")\n", + "ax.set_title(\"PINN training curves\")\n", + "ax.grid(True, alpha=0.3)\n", + "ax.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "e5e8f24d", + "metadata": {}, + "source": [ + "## 6) Sanity diagnostics (sampled residual and boundary mismatch)" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "8edb551e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "||r_theta||_{L2(Ω), MC} : 1.292829e+00\n", + "||u_theta-g||_{L2(∂Ω), MC} : 4.116012e-02\n", + "||f||_{L2(Ω), MC} : 1.083314e+01\n", + "||f||_{L∞(Ω), MC} : 9.451307e+01\n" + ] + } + ], + "source": [ + "@torch.no_grad()\n", + "def sample_l2_norm(values: torch.Tensor) -> float:\n", + " return float(torch.sqrt(torch.mean(values**2)).cpu())\n", + "\n", + "xi_diag = sample_interior(20000)\n", + "xb_diag = sample_boundary(2000)\n", + "\n", + "r_diag = residual_r(model, xi_diag).detach()\n", + "b_diag = (model(xb_diag) - g_boundary(xb_diag)).detach()\n", + "f_diag = forcing_f(xi_diag).detach()\n", + "\n", + "empirical = {\n", + " \"||r_theta||_{L2(Ω), MC}\": sample_l2_norm(r_diag),\n", + " \"||u_theta-g||_{L2(∂Ω), MC}\": sample_l2_norm(b_diag),\n", + " \"||f||_{L2(Ω), MC}\": sample_l2_norm(f_diag),\n", + " \"||f||_{L∞(Ω), MC}\": float(torch.max(torch.abs(f_diag)).cpu()),\n", + "}\n", + "for k, v in empirical.items():\n", + " print(f\"{k:35s}: {v:.6e}\")\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "f4398b58", + "metadata": {}, + "source": [ + "## 7) Certified interval enclosure of residual $L^2(\\Omega)$ (direct Hessian-based)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "ba604910", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Certified ||r_theta||_L2(Ω) ∈ [0.000000e+00, 6.576903e+01], width=6.577e+01\n" + ] + } + ], + "source": [ + "def interval_abs(iv: Interval) -> Interval:\n", + " lo = float(iv.lower)\n", + " hi = float(iv.upper)\n", + " if lo >= 0.0:\n", + " return Interval.from_bounds(lo, hi)\n", + " if hi <= 0.0:\n", + " return Interval.from_bounds(-hi, -lo)\n", + " return Interval.from_bounds(0.0, max(-lo, hi))\n", + "\n", + "\n", + "def residual_pointwise_power_bounds(model: nn.Module, box: IntervalTensor) -> Interval:\n", + " hess = model.eval_hessian(box)\n", + " u_xx = Interval(hess.lower[0][0][0], hess.upper[0][0][0])\n", + " u_yy = Interval(hess.lower[0][1][1], hess.upper[0][1][1])\n", + " lap = u_xx + u_yy\n", + " f_iv = forcing_interval_on_box(box)\n", + " r_iv = (Interval.point(0.0) - lap) - f_iv\n", + " abs_r = interval_abs(r_iv)\n", + " return abs_r * abs_r\n", + "\n", + "\n", + "def split_box(box: IntervalTensor):\n", + " widths = [float(box.upper[i] - box.lower[i]) for i in range(len(box.lower))]\n", + " dim = int(np.argmax(widths))\n", + " mid = 0.5 * (float(box.lower[dim]) + float(box.upper[dim]))\n", + " lo1 = list(box.lower)\n", + " up1 = list(box.upper)\n", + " lo2 = list(box.lower)\n", + " up2 = list(box.upper)\n", + " up1[dim] = mid\n", + " lo2[dim] = mid\n", + " return IntervalTensor.from_bounds(lo1, up1), IntervalTensor.from_bounds(lo2, up2)\n", + "\n", + "\n", + "def box_volume(box: IntervalTensor) -> float:\n", + " vol = 1.0\n", + " for lo, hi in zip(box.lower, box.upper):\n", + " vol *= float(hi - lo)\n", + " return vol\n", + "\n", + "\n", + "def certified_residual_l2(model: nn.Module, domain: IntervalTensor, iterations: int = 6, theta: float = 0.6, return_boxes: bool = False):\n", + " def box_key(box: IntervalTensor):\n", + " return (\n", + " tuple(float(v) for v in box.lower),\n", + " tuple(float(v) for v in box.upper),\n", + " )\n", + "\n", + " power_cache = {}\n", + "\n", + " def cached_power_iv(box: IntervalTensor) -> Interval:\n", + " key = box_key(box)\n", + " if key not in power_cache:\n", + " power_cache[key] = residual_pointwise_power_bounds(model, box)\n", + " return power_cache[key]\n", + "\n", + " boxes = [domain]\n", + " for _ in range(iterations):\n", + " indicators = []\n", + " for box in boxes:\n", + " iv = cached_power_iv(box)\n", + " indicators.append((float(iv.upper) - float(iv.lower)) * box_volume(box))\n", + " total = sum(indicators)\n", + " order = np.argsort(indicators)[::-1]\n", + " marked = []\n", + " running = 0.0\n", + " target = theta * total\n", + " for idx in order:\n", + " marked.append(int(idx))\n", + " running += indicators[int(idx)]\n", + " if running >= target:\n", + " break\n", + " marked_set = set(marked)\n", + " new_boxes = []\n", + " for i, box in enumerate(boxes):\n", + " if i in marked_set:\n", + " a, b = split_box(box)\n", + " new_boxes.extend([a, b])\n", + " else:\n", + " new_boxes.append(box)\n", + " boxes = new_boxes\n", + "\n", + " integral = Interval.point(0.0)\n", + " final_indicators = []\n", + " for box in boxes:\n", + " power_iv = cached_power_iv(box)\n", + " vol = box_volume(box)\n", + " weighted = Interval.from_bounds(float(power_iv.lower) * vol, float(power_iv.upper) * vol)\n", + " integral = integral + weighted\n", + " final_indicators.append((float(power_iv.upper) - float(power_iv.lower)) * vol)\n", + " result = Interval.from_bounds(math.sqrt(max(0.0, float(integral.lower))), math.sqrt(max(0.0, float(integral.upper))))\n", + "\n", + " if return_boxes:\n", + " return result, boxes, final_indicators\n", + " return result\n", + "\n", + "DOMAIN_2D = IntervalTensor.from_bounds([0.0, 0.0], [1.0, 1.0])\n", + "RES_ITERS = 24\n", + "RES_THETA = 0.5\n", + "residual_l2_iv, residual_boxes, residual_indicators = certified_residual_l2(model, DOMAIN_2D, iterations=RES_ITERS, theta=RES_THETA, return_boxes=True)\n", + "print(f\"Certified ||r_theta||_L2(Ω) ∈ [{float(residual_l2_iv.lower):.6e}, {float(residual_l2_iv.upper):.6e}], width={float(residual_l2_iv.upper-residual_l2_iv.lower):.3e}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "8e53ec26", + "metadata": {}, + "source": [ + "## 8) Certified interval enclosure of boundary trace $L^2(\\partial\\Omega)$" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "ae3340a6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "edge1:(t,0) : [3.506029e-02, 3.660975e-02] width=1.549e-03\n", + "edge2:(t,1) : [3.534749e-02, 3.685463e-02] width=1.507e-03\n", + "edge3:(0,t) : [4.603990e-02, 4.803630e-02] width=1.996e-03\n", + "edge4:(1,t) : [4.541635e-02, 4.722870e-02] width=1.812e-03\n", + "Global certified ||u_theta-g||_L2(∂Ω) ∈ [8.161487e-02, 8.506805e-02]\n" + ] + } + ], + "source": [ + "def make_edge_map(kind: str) -> nn.Linear:\n", + " layer = nn.Linear(1, 2, bias=True).to(device=device, dtype=dtype)\n", + " with torch.no_grad():\n", + " if kind == \"t0\":\n", + " layer.weight[:] = torch.tensor([[1.0], [0.0]], dtype=dtype)\n", + " layer.bias[:] = torch.tensor([0.0, 0.0], dtype=dtype)\n", + " elif kind == \"t1\":\n", + " layer.weight[:] = torch.tensor([[1.0], [0.0]], dtype=dtype)\n", + " layer.bias[:] = torch.tensor([0.0, 1.0], dtype=dtype)\n", + " elif kind == \"0t\":\n", + " layer.weight[:] = torch.tensor([[0.0], [1.0]], dtype=dtype)\n", + " layer.bias[:] = torch.tensor([0.0, 0.0], dtype=dtype)\n", + " elif kind == \"1t\":\n", + " layer.weight[:] = torch.tensor([[0.0], [1.0]], dtype=dtype)\n", + " layer.bias[:] = torch.tensor([1.0, 0.0], dtype=dtype)\n", + " else:\n", + " raise ValueError(kind)\n", + " for p in layer.parameters():\n", + " p.requires_grad_(False)\n", + " return layer\n", + "\n", + "edge_models = {\n", + " \"edge1:(t,0)\": nn.Sequential(make_edge_map(\"t0\"), model),\n", + " \"edge2:(t,1)\": nn.Sequential(make_edge_map(\"t1\"), model),\n", + " \"edge3:(0,t)\": nn.Sequential(make_edge_map(\"0t\"), model),\n", + " \"edge4:(1,t)\": nn.Sequential(make_edge_map(\"1t\"), model),\n", + "}\n", + "\n", + "DOMAIN_1D = IntervalTensor.from_bounds([0.0], [1.0])\n", + "BND_ITERS = 24\n", + "BND_THETA = 0.5\n", + "BND_FORWARD_SPLITS = 3\n", + "\n", + "edge_norm_intervals = {}\n", + "for name, edge_model in edge_models.items():\n", + " iv = edge_model.lpnorm(DOMAIN_1D, p=2.0, iterations=BND_ITERS, theta=BND_THETA, forward_refine_splits=BND_FORWARD_SPLITS)\n", + " edge_norm_intervals[name] = iv\n", + " print(f\"{name:14s}: [{float(iv.lower):.6e}, {float(iv.upper):.6e}] width={float(iv.upper-iv.lower):.3e}\")\n", + "\n", + "sum_lower = 0.0\n", + "sum_upper = 0.0\n", + "for iv in edge_norm_intervals.values():\n", + " lk = max(0.0, float(iv.lower))\n", + " uk = max(0.0, float(iv.upper))\n", + " sum_lower += lk * lk\n", + " sum_upper += uk * uk\n", + "boundary_l2_iv = Interval.from_bounds(math.sqrt(sum_lower), math.sqrt(sum_upper))\n", + "print(f\"Global certified ||u_theta-g||_L2(∂Ω) ∈ [{float(boundary_l2_iv.lower):.6e}, {float(boundary_l2_iv.upper):.6e}]\")" + ] + }, + { + "cell_type": "markdown", + "id": "4b9a7401", + "metadata": {}, + "source": [ + "## 9) Combined a posteriori estimator" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "3fb71bf3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "η interval: [8.161487e-02, 6.585410e+01], width=6.577e+01\n" + ] + } + ], + "source": [ + "eta_iv = residual_l2_iv + boundary_l2_iv\n", + "print(f\"η interval: [{float(eta_iv.lower):.6e}, {float(eta_iv.upper):.6e}], width={float(eta_iv.upper-eta_iv.lower):.3e}\")" + ] + }, + { + "cell_type": "markdown", + "id": "d92a7b0c", + "metadata": {}, + "source": [ + "## 10) PINN field visualizations\n", + "\n", + "We visualize:\n", + "1. PINN solution $u_\\theta$,\n", + "2. absolute error $|u_\\theta-u^*|$,\n", + "3. local certified residual indicators used in adaptive refinement." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0840077a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "saved figure: notebooks\\artifacts\\figure_pinn_results.png\n" + ] + } + ], + "source": [ + "# (a) solution and (b) PDE residual on a regular grid\n", + "N_PLOT = 121\n", + "x = torch.linspace(0.0, 1.0, N_PLOT, dtype=dtype, device=device)\n", + "y = torch.linspace(0.0, 1.0, N_PLOT, dtype=dtype, device=device)\n", + "X, Y = torch.meshgrid(x, y, indexing=\"ij\")\n", + "XY = torch.stack([X.reshape(-1), Y.reshape(-1)], dim=1)\n", + "\n", + "with torch.no_grad():\n", + " U_pred = model(XY).reshape(N_PLOT, N_PLOT).cpu().numpy()\n", + "\n", + "# Compute residual in chunks (no graph retention) to avoid very large autograd graphs in visualization.\n", + "RES_CHUNK = 2048\n", + "r_parts = []\n", + "for XY_chunk in XY.split(RES_CHUNK):\n", + " r_parts.append(residual_r(model, XY_chunk, create_graph=False).detach())\n", + "R_grid = torch.cat(r_parts, dim=0).reshape(N_PLOT, N_PLOT).cpu().numpy()\n", + "R_abs_max = float(np.max(np.abs(R_grid)))\n", + "\n", + "fig, axes = plt.subplots(1, 3, figsize=(15, 4.2), constrained_layout=True)\n", + "\n", + "im0 = axes[0].imshow(U_pred.T, origin=\"lower\", extent=[0, 1, 0, 1], cmap=\"viridis\", aspect=\"equal\")\n", + "axes[0].set_title(\"(a) PINN solution $u_\\theta$\")\n", + "axes[0].set_xlabel(\"x\"); axes[0].set_ylabel(\"y\")\n", + "fig.colorbar(im0, ax=axes[0], fraction=0.046)\n", + "\n", + "res_norm = plt.Normalize(vmin=-R_abs_max, vmax=R_abs_max) if R_abs_max > 0 else plt.Normalize(vmin=-1.0, vmax=1.0)\n", + "im1 = axes[1].imshow(R_grid.T, origin=\"lower\", extent=[0, 1, 0, 1], cmap=\"coolwarm\", norm=res_norm, aspect=\"equal\")\n", + "axes[1].set_title(\"(b) Signed residual $r_\\theta=-\\Delta u_\\theta-f$\")\n", + "axes[1].set_xlabel(\"x\"); axes[1].set_ylabel(\"y\")\n", + "fig.colorbar(im1, ax=axes[1], fraction=0.046)\n", + "\n", + "# (c) local certified residual indicators per adaptive cell\n", + "axes[2].set_title(\"(c) Local residual indicators (final adaptive partition)\")\n", + "axes[2].set_xlim(0, 1); axes[2].set_ylim(0, 1)\n", + "axes[2].set_aspect(\"equal\")\n", + "axes[2].set_xlabel(\"x\"); axes[2].set_ylabel(\"y\")\n", + "\n", + "ind = np.array(residual_indicators, dtype=float)\n", + "if ind.size == 0:\n", + " ind = np.array([0.0])\n", + "ind_min = float(ind.min())\n", + "ind_max = float(ind.max())\n", + "\n", + "for box, val in zip(residual_boxes, residual_indicators):\n", + " x0, y0 = float(box.lower[0]), float(box.lower[1])\n", + " x1, y1 = float(box.upper[0]), float(box.upper[1])\n", + " if ind_max > ind_min:\n", + " alpha = 0.15 + 0.85 * ((float(val) - ind_min) / (ind_max - ind_min))\n", + " else:\n", + " alpha = 0.4\n", + " rect = plt.Rectangle((x0, y0), x1 - x0, y1 - y0, facecolor=(0.1, 0.2, 0.8, alpha), edgecolor=\"black\", linewidth=0.3)\n", + " axes[2].add_patch(rect)\n", + "\n", + "sm = plt.cm.ScalarMappable(cmap=plt.cm.Blues, norm=plt.Normalize(vmin=ind_min, vmax=ind_max if ind_max > ind_min else ind_min + 1.0))\n", + "sm.set_array([])\n", + "fig.colorbar(sm, ax=axes[2], fraction=0.046, label=\"indicator magnitude\")\n", + "finalize_figure(fig, \"figure_pinn_results.png\")\n", + "\n", + "\n", + "print(f\"Grid max |residual|: {R_abs_max:.6e}\")\n", + "print(f\"Adaptive cells in indicator map: {len(residual_boxes)}\")\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "6c63a0ea", + "metadata": {}, + "source": [ + "## 11) Summary table and conclusions" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3beb49c", + "metadata": {}, + "outputs": [], + "source": [ + "rows = [\n", + " {\n", + " \"metric\": \"Residual ||r_theta||_L2(Ω)\",\n", + " \"empirical\": empirical[\"||r_theta||_{L2(Ω), MC}\"],\n", + " \"cert_lower\": float(residual_l2_iv.lower),\n", + " \"cert_upper\": float(residual_l2_iv.upper),\n", + " \"width\": float(residual_l2_iv.upper - residual_l2_iv.lower),\n", + " },\n", + " {\n", + " \"metric\": \"Boundary ||u_theta-g||_L2(∂Ω)\",\n", + " \"empirical\": empirical[\"||u_theta-g||_{L2(∂Ω), MC}\"],\n", + " \"cert_lower\": float(boundary_l2_iv.lower),\n", + " \"cert_upper\": float(boundary_l2_iv.upper),\n", + " \"width\": float(boundary_l2_iv.upper - boundary_l2_iv.lower),\n", + " },\n", + " {\n", + " \"metric\": \"Combined η\",\n", + " \"empirical\": np.nan,\n", + " \"cert_lower\": float(eta_iv.lower),\n", + " \"cert_upper\": float(eta_iv.upper),\n", + " \"width\": float(eta_iv.upper - eta_iv.lower),\n", + " },\n", + "]\n", + "for name, iv in edge_norm_intervals.items():\n", + " rows.append({\"metric\": f\"Per-edge {name}\", \"empirical\": np.nan, \"cert_lower\": float(iv.lower), \"cert_upper\": float(iv.upper), \"width\": float(iv.upper-iv.lower)})\n", + "\n", + "pd.DataFrame(rows)" + ] + }, + { + "cell_type": "markdown", + "id": "37a587e8", + "metadata": {}, + "source": [ + "### Interpretation\n", + "\n", + "- The residual enclosure is now computed directly from certified Hessian bounds and interval forcing bounds on each adaptive cell.\n", + "- The boundary term remains certified via edge-wise `lpnorm` computations.\n", + "- The practical estimator $\\eta_\\theta = \\|r_\\theta\\|_{L^2(\\Omega)} + \\|u_\\theta-g\\|_{L^2(\\partial\\Omega)}$ is therefore fully interval-certified in this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "101ad133-3514-4955-aa02-2858fb5f1485", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.7" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/pz_adaptive_quadrature_comparison.ipynb b/notebooks/pz_adaptive_quadrature_comparison.ipynb new file mode 100644 index 0000000..811da91 --- /dev/null +++ b/notebooks/pz_adaptive_quadrature_comparison.ipynb @@ -0,0 +1,390 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Polynomial-zonotope vs interval adaptive quadrature\n", + "\n", + "This notebook compares certified adaptive quadrature enclosures from the existing interval machinery with the polynomial-zonotope (PZ) two-jet machinery. It uses a small tanh network and a fixed box domain in the same style as the interval norm and PINN notebooks: `IntervalTensor` domains, monkey-patched PyTorch modules, and adaptive `model.lpnorm(...)` / `model.sobolev_norm(...)` calls.\n", + "\n", + "The defaults are intentionally small and reproducible so the notebook can be run quickly in CI-like or laptop environments." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1) Setup and reproducibility" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from __future__ import annotations\n", + "\n", + "import math\n", + "import random\n", + "import sys\n", + "import time\n", + "from pathlib import Path\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "import torch.nn as nn\n", + "\n", + "repo_root = Path.cwd().resolve()\n", + "while not (repo_root / \"src\" / \"intervalnets\").exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "if str(repo_root / \"src\") not in sys.path:\n", + " sys.path.insert(0, str(repo_root / \"src\"))\n", + "\n", + "from intervalnets import Interval, IntervalTensor, PolynomialZonotope, enable_interval_eval\n", + "from intervalnets.pz_integration import _split_box as _pz_split_box, _choose_split_dim_from_jacobian, _dorfler_marking as _pz_dorfler_marking, _integrated_squared_contribution, _interval_add, _interval_width, _sqrt_interval_nonnegative\n", + "from intervalnets.pytorch import _box_volume, _choose_split_dim, _dorfler_marking, _interval_pow_scalar, _jacobian_is_exact_zero, _hessian_is_exact_zero, _interval_tensor_is_exact_constant, _lp_pointwise_power_bounds_refined, _sobolev_pointwise_power_bounds_refined, _split_box\n", + "\n", + "enable_interval_eval()\n", + "SEED = 20260720\n", + "random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)\n", + "torch.set_default_dtype(torch.float64)\n", + "print(f\"repo_root={repo_root}\")\n", + "print(f\"torch={torch.__version__}, seed={SEED}\")" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2) Small model and domain\n", + "\n", + "The network is deliberately tiny, with `Tanh` activations so second derivatives are meaningful. The domain is a flat `IntervalTensor` box, matching the current adaptive norm APIs." + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def make_small_tanh_network(input_dim: int = 2, width: int = 6, hidden_layers: int = 2) -> nn.Sequential:\n", + " layers: list[nn.Module] = []\n", + " in_features = input_dim\n", + " for _ in range(hidden_layers):\n", + " layers += [nn.Linear(in_features, width), nn.Tanh()]\n", + " in_features = width\n", + " layers.append(nn.Linear(in_features, 1))\n", + " return nn.Sequential(*layers).to(dtype=torch.float64)\n", + "\n", + "model = make_small_tanh_network()\n", + "with torch.no_grad():\n", + " for i, param in enumerate(model.parameters()):\n", + " torch.manual_seed(SEED + i)\n", + " param.copy_(0.35 * torch.randn_like(param))\n", + "\n", + "domain = IntervalTensor.from_bounds(torch.tensor([-1.0, -0.75]), torch.tensor([1.0, 0.75]))\n", + "MAX_ITERATIONS = 4\n", + "ITERATIONS = list(range(MAX_ITERATIONS + 1))\n", + "THETA = 0.5\n", + "CHEBYSHEV_DEGREE = 3\n", + "RESIDUAL_SUBDIVISIONS = 32\n", + "FORWARD_REFINE_SPLITS = 1\n", + "FORWARD_REFINE_MAX_CELLS = 256\n", + "model, domain, ITERATIONS" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3) Helpers for timing, cell counts, and diagnostics" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def interval_width(bounds) -> float:\n", + " return float(bounds.upper) - float(bounds.lower)\n", + "\n", + "def time_call(fn):\n", + " t0 = time.perf_counter()\n", + " value = fn()\n", + " return value, time.perf_counter() - t0\n", + "\n", + "def interval_quantity_power_bounds(model, box, quantity: str):\n", + " if quantity == \"L2\":\n", + " return _lp_pointwise_power_bounds_refined(\n", + " model,\n", + " box,\n", + " 2.0,\n", + " forward_refine_splits=FORWARD_REFINE_SPLITS,\n", + " forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS,\n", + " )\n", + " if quantity == \"W12\":\n", + " return _sobolev_pointwise_power_bounds_refined(\n", + " model,\n", + " box,\n", + " 2.0,\n", + " order=1,\n", + " forward_refine_splits=FORWARD_REFINE_SPLITS,\n", + " forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS,\n", + " )\n", + " if quantity == \"W22\":\n", + " return _sobolev_pointwise_power_bounds_refined(\n", + " model,\n", + " box,\n", + " 2.0,\n", + " order=2,\n", + " forward_refine_splits=FORWARD_REFINE_SPLITS,\n", + " forward_refine_max_cells=FORWARD_REFINE_MAX_CELLS,\n", + " )\n", + " raise ValueError(quantity)\n", + "\n", + "def interval_cell_indicator_and_split_dim(model, box, quantity: str) -> tuple[float, int]:\n", + " integrand_bounds = interval_quantity_power_bounds(model, box, quantity)\n", + " width = interval_width(integrand_bounds)\n", + " if quantity == \"L2\":\n", + " jacobian = model.eval_jacobian(box) if len(box.lower) > 1 else None\n", + " return width * _box_volume(box), _choose_split_dim(box, jacobian)\n", + "\n", + " order = 1 if quantity == \"W12\" else 2\n", + " output = model.eval(box)\n", + " jacobian = model.eval_jacobian(box)\n", + " hessian = model.eval_hessian(box) if order == 2 else None\n", + " derivative_zero = _jacobian_is_exact_zero(jacobian)\n", + " second_derivative_zero = True if hessian is None else _hessian_is_exact_zero(hessian)\n", + " indicator = 0.0 if (_interval_tensor_is_exact_constant(output) and derivative_zero and second_derivative_zero) else width * _box_volume(box)\n", + " return indicator, _choose_split_dim(box, jacobian if len(box.lower) > 1 else None)\n", + "\n", + "def interval_aggregate_bounds(model, boxes, quantity: str):\n", + " integral = Interval.point(0.0)\n", + " for box in boxes:\n", + " integrand_bounds = interval_quantity_power_bounds(model, box, quantity)\n", + " weighted = Interval.from_bounds(\n", + " float(integrand_bounds.lower) * _box_volume(box),\n", + " float(integrand_bounds.upper) * _box_volume(box),\n", + " )\n", + " integral = integral + weighted\n", + " non_negative = Interval.from_bounds(max(0.0, float(integral.lower)), max(0.0, float(integral.upper)))\n", + " return _interval_pow_scalar(non_negative, 0.5)\n", + "\n", + "def pz_complexity(model, domain) -> dict[str, int]:\n", + " pz_domain = PolynomialZonotope.from_box(domain.lower, domain.upper)\n", + " jet = model.eval_pz_twojet(pz_domain, chebyshev_degree=CHEBYSHEV_DEGREE, residual_subdivisions=RESIDUAL_SUBDIVISIONS)\n", + " polynomials = [pz for pz in [jet.Y, jet.J, jet.H] if pz is not None]\n", + " max_num_noise = max((pz.num_noise for pz in polynomials), default=pz_domain.num_noise)\n", + " return {\n", + " \"pz_terms\": int(sum(len(pz.terms) for pz in polynomials)),\n", + " \"pz_noise_vars\": int(max_num_noise),\n", + " \"pz_approx_noise_vars\": int(max_num_noise - pz_domain.num_noise),\n", + " }\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4) Certified interval and PZ quadrature comparisons" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def make_trace_row(quantity: str, method: str, iteration: int, bounds, cells: int, seconds: float, meta: dict[str, int]):\n", + " return {\n", + " \"quantity\": quantity,\n", + " \"method\": method,\n", + " \"iteration\": iteration,\n", + " \"lower\": float(bounds.lower),\n", + " \"upper\": float(bounds.upper),\n", + " \"width\": interval_width(bounds),\n", + " \"cells\": cells,\n", + " \"seconds\": seconds,\n", + " **meta,\n", + " }\n", + "\n", + "def run_interval_adaptive_trace(model, domain, quantity: str, max_iterations: int):\n", + " boxes = [domain]\n", + " rows = []\n", + " elapsed = 0.0\n", + " meta = {\"pz_terms\": 0, \"pz_noise_vars\": 0, \"pz_approx_noise_vars\": 0}\n", + " for iteration in range(max_iterations + 1):\n", + " t0 = time.perf_counter()\n", + " bounds = interval_aggregate_bounds(model, boxes, quantity)\n", + " elapsed += time.perf_counter() - t0\n", + " rows.append(make_trace_row(quantity, \"interval\", iteration, bounds, len(boxes), elapsed, meta))\n", + " if iteration == max_iterations:\n", + " break\n", + "\n", + " t0 = time.perf_counter()\n", + " indicators, split_dims = zip(*(interval_cell_indicator_and_split_dim(model, box, quantity) for box in boxes))\n", + " marked = set(_dorfler_marking(list(indicators), THETA))\n", + " boxes = [child for idx, box in enumerate(boxes) for child in (_split_box(box, split_dim=split_dims[idx]) if idx in marked else (box,))]\n", + " elapsed += time.perf_counter() - t0\n", + " return rows\n", + "\n", + "def pz_kind(quantity: str) -> str:\n", + " return {\"L2\": \"l2\", \"W12\": \"w12\", \"W22\": \"w22\"}[quantity]\n", + "\n", + "def pz_aggregate_bounds(model, boxes, quantity: str):\n", + " integral = Interval.point(0.0)\n", + " for box in boxes:\n", + " contribution, _ = _integrated_squared_contribution(\n", + " model,\n", + " box,\n", + " integrand_kind=pz_kind(quantity),\n", + " chebyshev_degree=CHEBYSHEV_DEGREE,\n", + " residual_subdivisions=RESIDUAL_SUBDIVISIONS,\n", + " )\n", + " integral = _interval_add(integral, contribution)\n", + " return _sqrt_interval_nonnegative(integral)\n", + "\n", + "def run_pz_adaptive_trace(model, domain, quantity: str, max_iterations: int):\n", + " boxes = [domain]\n", + " rows = []\n", + " elapsed = 0.0\n", + " meta = pz_complexity(model, domain)\n", + " for iteration in range(max_iterations + 1):\n", + " t0 = time.perf_counter()\n", + " bounds = pz_aggregate_bounds(model, boxes, quantity)\n", + " elapsed += time.perf_counter() - t0\n", + " rows.append(make_trace_row(quantity, \"pz\", iteration, bounds, len(boxes), elapsed, meta))\n", + " if iteration == max_iterations:\n", + " break\n", + "\n", + " t0 = time.perf_counter()\n", + " contributions_and_jacobians = [\n", + " _integrated_squared_contribution(\n", + " model,\n", + " box,\n", + " integrand_kind=pz_kind(quantity),\n", + " chebyshev_degree=CHEBYSHEV_DEGREE,\n", + " residual_subdivisions=RESIDUAL_SUBDIVISIONS,\n", + " )\n", + " for box in boxes\n", + " ]\n", + " indicators = [_interval_width(contribution) for contribution, _ in contributions_and_jacobians]\n", + " split_dims = [_choose_split_dim_from_jacobian(box, jacobian) for box, (_, jacobian) in zip(boxes, contributions_and_jacobians)]\n", + " marked = set(_pz_dorfler_marking(indicators, THETA))\n", + " boxes = [child for idx, box in enumerate(boxes) for child in (_pz_split_box(box, split_dim=split_dims[idx]) if idx in marked else (box,))]\n", + " elapsed += time.perf_counter() - t0\n", + " return rows\n", + "\n", + "rows = []\n", + "for quantity in [\"L2\", \"W12\", \"W22\"]:\n", + " rows.extend(run_interval_adaptive_trace(model, domain, quantity, MAX_ITERATIONS))\n", + " rows.extend(run_pz_adaptive_trace(model, domain, quantity, MAX_ITERATIONS))\n", + "results = pd.DataFrame(rows)\n", + "results\n" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5) Non-certified Monte Carlo/autograd sanity check\n", + "\n", + "The estimates below are not certificates. They simply check that certified lower/upper ranges are plausible for random samples and PyTorch autograd derivatives." + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "def autograd_quantity_values(samples: torch.Tensor) -> dict[str, np.ndarray]:\n", + " samples = samples.clone().detach().requires_grad_(True)\n", + " y = model(samples)[:, 0]\n", + " grad = torch.autograd.grad(y.sum(), samples, create_graph=True)[0]\n", + " hess_sq = torch.zeros_like(y)\n", + " for i in range(samples.shape[1]):\n", + " for j in range(samples.shape[1]):\n", + " hij = torch.autograd.grad(grad[:, i].sum(), samples, retain_graph=True)[0][:, j]\n", + " hess_sq = hess_sq + hij.square()\n", + " grad_sq = grad.square().sum(dim=1)\n", + " return {\"L2\": y.square().detach().numpy(), \"W12\": (y.square() + grad_sq).detach().numpy(), \"W22\": (y.square() + grad_sq + hess_sq).detach().numpy()}\n", + "\n", + "N_MC = 4096\n", + "rng = torch.Generator().manual_seed(SEED)\n", + "lo, hi = domain.lower.to(dtype=torch.float64), domain.upper.to(dtype=torch.float64)\n", + "samples = lo + (hi - lo) * torch.rand((N_MC, len(lo)), generator=rng, dtype=torch.float64)\n", + "volume = float(torch.prod(hi - lo))\n", + "values = autograd_quantity_values(samples)\n", + "mc_rows = []\n", + "for quantity, pointwise in values.items():\n", + " estimate = math.sqrt(max(0.0, volume * float(np.mean(pointwise))))\n", + " final = results[(results.quantity == quantity) & (results.iteration == MAX_ITERATIONS)]\n", + " for method in [\"interval\", \"pz\"]:\n", + " certified = final[final.method == method].iloc[0]\n", + " mc_rows.append({\"quantity\": quantity, \"method\": method, \"mc_estimate\": estimate, \"certified_lower\": certified.lower, \"certified_upper\": certified.upper, \"inside_certified_bounds\": certified.lower <= estimate <= certified.upper})\n", + "pd.DataFrame(mc_rows)" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 6) Width-vs-refinement plot" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "fig, ax = plt.subplots(figsize=(8, 4.5))\n", + "for (quantity, method), group in results.groupby([\"quantity\", \"method\"]):\n", + " ax.plot(group[\"iteration\"], group[\"width\"], marker=\"o\", label=f\"{quantity} / {method}\")\n", + "ax.set_xlabel(\"refinement iteration\")\n", + "ax.set_ylabel(\"certified bound width\")\n", + "ax.set_yscale(\"log\")\n", + "ax.set_title(\"Interval width versus adaptive refinement\")\n", + "ax.grid(True, which=\"both\", alpha=0.3)\n", + "ax.legend(ncol=2)\n", + "plt.tight_layout()" + ], + "outputs": [], + "execution_count": null + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 7) Compact final table" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "final_table = results[results[\"iteration\"] == MAX_ITERATIONS].copy()\n", + "final_table[[\"quantity\", \"method\", \"lower\", \"upper\", \"width\", \"cells\", \"seconds\", \"pz_terms\", \"pz_noise_vars\", \"pz_approx_noise_vars\"]].sort_values([\"quantity\", \"method\"])" + ], + "outputs": [], + "execution_count": null + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/notebooks/pz_l2_value_benchmarks.ipynb b/notebooks/pz_l2_value_benchmarks.ipynb new file mode 100644 index 0000000..0974265 --- /dev/null +++ b/notebooks/pz_l2_value_benchmarks.ipynb @@ -0,0 +1,376 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "overview", + "metadata": {}, + "source": [ + "# Value-only polynomial-zonotope L² benchmarks\n", + "\n", + "This notebook isolates the zero-jet path: it propagates only the certified function-value polynomial zonotope and directly integrates its squared Euclidean norm. Jacobian and Hessian enclosures are never allocated.\n", + "\n", + "The main target is the saved 100-dimensional Poisson PINN with three hidden layers of width 50. It is loaded from the committed checkpoint and is never retrained by this benchmark. Timings are deliberately split into value propagation, direct squared integration, and the public adaptive `pz_l2norm` API." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "imports", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'torch': '2.13.0+cu130', 'threads': 1, 'loaded_checkpoint': 'notebooks/checkpoints/pinn_100d_poisson.pt', 'training_steps': 0}\n" + ] + } + ], + "source": [ + "from collections import Counter\n", + "from pathlib import Path\n", + "from time import perf_counter\n", + "\n", + "import pandas as pd\n", + "import torch\n", + "\n", + "from intervalnets import (\n", + " IntervalTensor,\n", + " PZIntegrationCell,\n", + " enable_interval_eval,\n", + " integrate_pz_value_squared,\n", + " load_tanh_mlp_checkpoint,\n", + ")\n", + "\n", + "torch.set_num_threads(1)\n", + "torch.manual_seed(20260731)\n", + "enable_interval_eval()\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / 'notebooks' / 'checkpoints').exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "CHECKPOINT = repo_root / 'notebooks' / 'checkpoints' / 'pinn_100d_poisson.pt'\n", + "trained_pinn = load_tanh_mlp_checkpoint(CHECKPOINT)\n", + "print({'torch': torch.__version__, 'threads': torch.get_num_threads(),\n", + " 'loaded_checkpoint': str(CHECKPOINT.relative_to(repo_root)), 'training_steps': 0})" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "helpers", + "metadata": {}, + "outputs": [], + "source": [ + "def make_model(input_dim, hidden_widths, output_dim=1, seed=20260731):\n", + " torch.manual_seed(seed)\n", + " layers = []\n", + " previous = input_dim\n", + " for width in hidden_widths:\n", + " layers.extend([\n", + " torch.nn.Linear(previous, width, dtype=torch.float64),\n", + " torch.nn.Tanh(),\n", + " ])\n", + " previous = width\n", + " layers.append(torch.nn.Linear(previous, output_dim, dtype=torch.float64))\n", + " return torch.nn.Sequential(*layers)\n", + "\n", + "\n", + "def benchmark_case(input_dim, hidden_widths, *, half_width=0.1, iterations=0,\n", + " seed=20260731, model=None):\n", + " model = make_model(input_dim, hidden_widths, seed=seed) if model is None else model\n", + " box = IntervalTensor.from_bounds(\n", + " [-half_width] * input_dim,\n", + " [half_width] * input_dim,\n", + " )\n", + " cell = PZIntegrationCell.from_affine_box(box)\n", + "\n", + " start = perf_counter()\n", + " traced = model.eval_pz_value(cell.domain, return_trace=True)\n", + " forward_s = perf_counter() - start\n", + "\n", + " start = perf_counter()\n", + " squared = integrate_pz_value_squared(traced.final, cell)\n", + " integration_s = perf_counter() - start\n", + "\n", + " start = perf_counter()\n", + " norm = model.pz_l2norm(box, iterations=iterations)\n", + " public_s = perf_counter() - start\n", + "\n", + " kinds = Counter(traced.final.noise_kinds)\n", + " row = {\n", + " 'architecture': f\"{input_dim}-{'-'.join(map(str, hidden_widths))}-1\",\n", + " 'iterations': iterations,\n", + " 'forward_s': forward_s,\n", + " 'integration_s': integration_s,\n", + " 'manual_total_s': forward_s + integration_s,\n", + " 'public_norm_s': public_s,\n", + " 'terms': len(traced.final.terms),\n", + " 'num_noise': traced.final.num_noise,\n", + " 'domain_noise': kinds['domain'],\n", + " 'pointwise_noise': kinds['approximation_pointwise'],\n", + " 'max_degree': max((sum(exp) for exp in traced.final.terms), default=0),\n", + " 'norm_lower': float(norm.lower),\n", + " 'norm_upper': float(norm.upper),\n", + " 'norm_width': float(norm.upper) - float(norm.lower),\n", + " }\n", + " layer_rows = [\n", + " {\n", + " 'layer_index': record.layer_index,\n", + " 'layer': record.layer_type,\n", + " 'terms': record.summary['term_count'],\n", + " 'noise': record.summary['num_noise'],\n", + " 'degree': record.summary['max_degree'],\n", + " }\n", + " for record in traced.records\n", + " ]\n", + " return row, pd.DataFrame(layer_rows), model, box" + ] + }, + { + "cell_type": "markdown", + "id": "architecture-sweep", + "metadata": {}, + "source": [ + "## Architecture sweep\n", + "\n", + "The final affine support should contain `input_dim + sum(hidden_widths)` terms: one domain symbol per input coordinate and one pointwise residual symbol per hidden neuron. Degree should remain one." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "architecture-results", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " architecture iterations ... norm_upper norm_width\n", + "0 2-5-1 0 ... 9.970893e-02 2.693051e-04\n", + "1 10-20-1 0 ... 3.640631e-05 5.004868e-06\n", + "2 25-50-1 0 ... 2.876754e-10 7.033868e-11\n", + "3 50-50-50-1 0 ... 4.426063e-19 3.374740e-19\n", + "4 100-50-50-50-1 0 ... 3.013843e-35 3.013843e-35\n", + "\n", + "[5 rows x 14 columns]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "architectures = [\n", + " (2, [5]),\n", + " (10, [20]),\n", + " (25, [50]),\n", + " (50, [50, 50]),\n", + " (100, [50, 50, 50]),\n", + "]\n", + "\n", + "rows = [benchmark_case(input_dim, hidden,\n", + " model=trained_pinn if (input_dim, hidden) == (100, [50, 50, 50]) else None)[0]\n", + " for input_dim, hidden in architectures]\n", + "architecture_results = pd.DataFrame(rows)\n", + "architecture_results" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "largest-case", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " architecture iterations ... norm_upper norm_width\n", + "0 100-50-50-50-1 0 ... 3.013843e-35 3.013843e-35\n", + "\n", + "[1 rows x 14 columns]\n" + ] + }, + { + "data": { + "text/plain": [ + " layer_index layer terms noise degree\n", + "0 -1 Input 100 100 1\n", + "1 0 Linear 100 100 1\n", + "2 1 Tanh 150 150 1\n", + "3 2 Linear 150 150 1\n", + "4 3 Tanh 200 200 1\n", + "5 4 Linear 200 200 1\n", + "6 5 Tanh 250 250 1\n", + "7 6 Linear 250 250 1" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "largest, largest_layers, largest_model, largest_box = benchmark_case(\n", + " 100, [50, 50, 50], model=trained_pinn)\n", + "assert largest['terms'] == 100 + 3 * 50\n", + "assert largest['domain_noise'] == 100\n", + "assert largest['pointwise_noise'] == 3 * 50\n", + "assert largest['max_degree'] == 1\n", + "assert largest['public_norm_s'] < 3.0, largest\n", + "display(pd.DataFrame([largest]))\n", + "largest_layers" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-quality", + "metadata": {}, + "source": [ + "## Adaptive cost and enclosure quality\n", + "\n", + "Each marked cell is replaced by two children. With one marked root, iteration 1 therefore evaluates three cells in total (the discarded parent plus two children). The enclosure width is also sensitive to input-box size: affine residuals grow with wider preactivation intervals even when runtime changes little." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "adaptive-results", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " iterations public_norm_s norm_lower norm_upper norm_width\n", + "0 0 0.065661 -4.940656e-324 3.013843e-35 3.013843e-35\n", + "1 1 0.205508 -4.940656e-324 2.993538e-35 2.993538e-35\n", + "2 2 0.335026 -4.940656e-324 2.982559e-35 2.982559e-35" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "adaptive_rows = [\n", + " benchmark_case(100, [50, 50, 50], iterations=iterations, model=trained_pinn)[0]\n", + " for iterations in (0, 1, 2)\n", + "]\n", + "pd.DataFrame(adaptive_rows)[['iterations', 'public_norm_s', 'norm_lower', 'norm_upper', 'norm_width']]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "scale-results", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + " architecture public_norm_s ... norm_width half_width\n", + "0 100-50-50-50-1 0.064685 ... 1.071259e-87 0.01\n", + "1 100-50-50-50-1 0.066096 ... 8.995507e-51 0.05\n", + "2 100-50-50-50-1 0.068238 ... 3.013843e-35 0.10\n", + "3 100-50-50-50-1 0.068038 ... 4.599380e-15 0.25\n", + "4 100-50-50-50-1 0.069328 ... 5.677901e+00 0.50\n", + "5 100-50-50-50-1 0.073606 ... 6.587481e+15 1.00\n", + "\n", + "[6 rows x 6 columns]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "scale_rows = [\n", + " benchmark_case(100, [50, 50, 50], half_width=half_width, model=trained_pinn)[0]\n", + " for half_width in (0.01, 0.05, 0.1, 0.25, 0.5, 1.0)\n", + "]\n", + "pd.DataFrame(scale_rows)[['architecture', 'public_norm_s', 'norm_lower', 'norm_upper', 'norm_width']].assign(\n", + " half_width=(0.01, 0.05, 0.1, 0.25, 0.5, 1.0)\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "correctness-check", + "metadata": {}, + "source": [ + "## Correctness spot check against the two-jet path\n", + "\n", + "The zero-jet uses fewer noise dimensions because it never introduces the derivative and second-derivative residual symbols. On a small network, its function-value interval should nevertheless match the `Y` component of the full two-jet." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "correctness-results", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "value_terms 7\n", + "twojet_Y_terms 7\n", + "value_noise 7\n", + "twojet_noise 17\n", + "dtype: int64" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "small_model = make_model(2, [5])\n", + "small_box = IntervalTensor.from_bounds([-0.1, -0.1], [0.1, 0.1])\n", + "small_cell = PZIntegrationCell.from_affine_box(small_box)\n", + "value = small_model.eval_pz_value(small_cell.domain)\n", + "jet = small_model.eval_pz_twojet(small_cell.domain)\n", + "value_interval = value.interval_enclosure()\n", + "jet_interval = jet.Y.interval_enclosure()\n", + "\n", + "assert torch.allclose(torch.tensor(value_interval.lower), torch.tensor(jet_interval.lower))\n", + "assert torch.allclose(torch.tensor(value_interval.upper), torch.tensor(jet_interval.upper))\n", + "pd.Series({\n", + " 'value_terms': len(value.terms),\n", + " 'twojet_Y_terms': len(jet.Y.terms),\n", + " 'value_noise': value.num_noise,\n", + " 'twojet_noise': jet.Y.num_noise,\n", + "})" + ] + }, + { + "cell_type": "markdown", + "id": "interpretation", + "metadata": {}, + "source": [ + "## Reading the diagnostics\n", + "\n", + "- Runtime stays low because affine activation enclosures preserve degree one and add terms only linearly: `input_dim + total_hidden_neurons`.\n", + "- Direct L² integration recognizes unit-vector affine support and evaluates the current certified pointwise-residual semantics without constructing pair-exponent rows or a squared PZ.\n", + "- The remaining forward bottleneck is repeated canonical exponent validation and reconstruction inside `PolynomialZonotope`; this is the first target if much wider/deeper zero-jets are needed.\n", + "- Runtime and enclosure quality are different questions. Wide high-dimensional boxes can make affine residual uncertainty dominate and drive the certified lower L² bound to zero even though the computation is fast. Domain partitioning or tighter/non-affine activation enclosures address tightness, not this runtime bottleneck." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/pz_twojet_tests.ipynb b/notebooks/pz_twojet_tests.ipynb new file mode 100644 index 0000000..20b7e0c --- /dev/null +++ b/notebooks/pz_twojet_tests.ipynb @@ -0,0 +1,321 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Polynomial-zonotope two-jet tests\n", + "\n", + "This notebook mirrors the pytest coverage in `tests/test_pz_twojet.py` in an interactive form. Run the cells from top to bottom to validate polynomial-zonotope arithmetic, interval enclosures, two-jet propagation through affine and tanh networks, tanh residual certification, and PyTorch-autograd sample containment.\n", + "\n", + "The cells use plain `assert` statements so failures stop at the failing check and can be inspected interactively.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from pathlib import Path\n", + "import sys\n", + "\n", + "# Allow running this notebook directly from the repository checkout without installing the package.\n", + "repo_root = Path.cwd()\n", + "if not (repo_root / \"src\").exists() and (repo_root.parent / \"src\").exists():\n", + " repo_root = repo_root.parent\n", + "sys.path.insert(0, str(repo_root / \"src\"))\n", + "\n", + "import math\n", + "\n", + "from intervalnets import PolynomialZonotope, enable_interval_eval, pz_twojet_forward\n", + "from intervalnets.pz_tanh import certify_tanh_residual_subdivision, compute_tanh_polynomial\n", + "\n", + "try:\n", + " import torch\n", + " from torch import nn\n", + "except ImportError as exc: # pragma: no cover - for interactive use\n", + " raise ImportError(\"This notebook requires PyTorch to run the two-jet neural-network checks.\") from exc\n", + "\n", + "torch.set_default_dtype(torch.float64)\n", + "print(f\"Using repository root: {repo_root}\")\n", + "print(f\"Using torch {torch.__version__}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def poly(coeffs, x):\n", + " acc = 0.0\n", + " for coeff in reversed(coeffs):\n", + " acc = acc * x + float(coeff)\n", + " return acc\n", + "\n", + "\n", + "def eval_pz(z: PolynomialZonotope, eps):\n", + " value = z.center.clone() if isinstance(z.center, torch.Tensor) else torch.tensor(z.center, dtype=torch.float64)\n", + " eps = torch.as_tensor(eps, dtype=value.dtype, device=value.device)\n", + " for exp, coeff in z.terms.items():\n", + " monomial = torch.ones((), dtype=value.dtype, device=value.device)\n", + " for idx, power in enumerate(exp):\n", + " if power:\n", + " monomial = monomial * eps[idx].pow(power)\n", + " value = value + coeff * monomial\n", + " return value\n", + "\n", + "\n", + "def assert_contains(interval, sample, atol=1e-10):\n", + " lo, hi = interval.to_torch(dtype=torch.float64)\n", + " sample = sample.detach().to(dtype=torch.float64)\n", + " assert torch.all(sample >= lo - atol), f\"sample below lower bound: {sample} < {lo}\"\n", + " assert torch.all(sample <= hi + atol), f\"sample above upper bound: {sample} > {hi}\"\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Arithmetic" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "z1 = PolynomialZonotope(1.0, {(1,): 2.0}, num_noise=1)\n", + "z2 = PolynomialZonotope(3.0, {(1,): -5.0}, num_noise=1)\n", + "\n", + "summed = z1 + z2\n", + "scaled = -2.5 * z1\n", + "product = z1 * z2\n", + "\n", + "assert summed.center == 4.0\n", + "assert summed.terms[(1,)] == -3.0\n", + "assert scaled.center == -2.5\n", + "assert scaled.terms[(1,)] == -5.0\n", + "assert product.center == 3.0\n", + "assert product.terms[(1,)] == 1.0\n", + "assert product.terms[(2,)] == -10.0\n", + "\n", + "scalar = PolynomialZonotope(torch.tensor(2.0), {(1,): torch.tensor(-0.5)}, num_noise=1)\n", + "vector = PolynomialZonotope.constant(torch.tensor([1.0, -3.0]), num_noise=1)\n", + "matrix = PolynomialZonotope.constant(torch.arange(1.0, 5.0).reshape(2, 2), num_noise=1)\n", + "tensor = PolynomialZonotope.constant(torch.arange(1.0, 9.0).reshape(2, 2, 2), num_noise=1)\n", + "\n", + "vector_product = scalar * vector\n", + "matrix_product = scalar * matrix\n", + "tensor_product = scalar * tensor\n", + "\n", + "assert vector_product.shape == (2,)\n", + "assert matrix_product.shape == (2, 2)\n", + "assert tensor_product.shape == (2, 2, 2)\n", + "assert torch.allclose(vector_product.center, torch.tensor([2.0, -6.0]))\n", + "assert torch.allclose(vector_product.terms[(1,)], torch.tensor([-0.5, 1.5]))\n", + "assert torch.allclose(matrix_product.center, 2.0 * matrix.center)\n", + "assert torch.allclose(matrix_product.terms[(1,)], -0.5 * matrix.center)\n", + "assert torch.allclose(tensor_product.center, 2.0 * tensor.center)\n", + "assert torch.allclose(tensor_product.terms[(1,)], -0.5 * tensor.center)\n", + "\n", + "outer = vector.tensor_product(vector)\n", + "assert outer.shape == (2, 2)\n", + "assert torch.allclose(outer.center, torch.tensor([[1.0, -3.0], [-3.0, 9.0]]))\n", + "print(\"Arithmetic checks passed.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Interval enclosure contains sampled noise values" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "torch.manual_seed(0)\n", + "z = PolynomialZonotope(\n", + " torch.tensor([0.5, -1.0]),\n", + " {\n", + " (1, 0, 0): torch.tensor([0.25, -0.5]),\n", + " (0, 2, 0): torch.tensor([-0.1, 0.2]),\n", + " (1, 0, 1): torch.tensor([0.05, 0.15]),\n", + " },\n", + " num_noise=3,\n", + ")\n", + "enclosure = z.interval_enclosure()\n", + "for _ in range(128):\n", + " eps = 2.0 * torch.rand(3) - 1.0\n", + " assert_contains(enclosure, eval_pz(z, eps))\n", + "print(\"Interval enclosure sampling checks passed.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Two-jet shape correctness for neural-network outputs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "d, m = 3, 2\n", + "model = nn.Sequential(nn.Linear(d, 4), nn.Tanh(), nn.Linear(4, m)).double()\n", + "domain = PolynomialZonotope.from_box(torch.full((d,), -0.2), torch.full((d,), 0.3))\n", + "\n", + "out = pz_twojet_forward(model, domain, chebyshev_degree=5, residual_subdivisions=64)\n", + "\n", + "assert out.Y.shape == (m,)\n", + "assert out.J.shape == (m, d)\n", + "assert out.H.shape == (m, d, d)\n", + "print(\"Shape checks passed.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Affine-only network exactness" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "d, h, m = 2, 3, 2\n", + "model = nn.Sequential(nn.Linear(d, h), nn.Linear(h, m)).double()\n", + "with torch.no_grad():\n", + " model[0].weight.copy_(torch.tensor([[1.0, -2.0], [0.5, 3.0], [-1.5, 0.25]]))\n", + " model[0].bias.copy_(torch.tensor([0.1, -0.2, 0.3]))\n", + " model[1].weight.copy_(torch.tensor([[2.0, -1.0, 0.5], [-0.25, 1.5, -2.0]]))\n", + " model[1].bias.copy_(torch.tensor([-0.4, 0.7]))\n", + "domain = PolynomialZonotope.from_box(torch.tensor([-1.0, 0.25]), torch.tensor([0.5, 1.25]))\n", + "\n", + "out = pz_twojet_forward(model, domain)\n", + "expected_weight = model[1].weight.detach().matmul(model[0].weight.detach())\n", + "expected_bias = model[1].weight.detach().matmul(model[0].bias.detach()) + model[1].bias.detach()\n", + "\n", + "assert torch.allclose(out.Y.center, expected_weight.matmul(domain.center) + expected_bias)\n", + "for exp, coeff in domain.terms.items():\n", + " assert torch.allclose(out.Y.terms[exp], expected_weight.matmul(coeff))\n", + "assert torch.allclose(out.J.center, expected_weight)\n", + "assert out.J.terms == {}\n", + "assert torch.equal(out.H.center, torch.zeros(m, d, d))\n", + "assert out.H.terms == {}\n", + "print(\"Affine exactness checks passed.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5. Tanh residual certification" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "approx = compute_tanh_polynomial((-1.25, 0.75), degree=5, subdivisions=96)\n", + "delta, metadata = certify_tanh_residual_subdivision((approx.lower, approx.upper), approx.coeffs, subdivisions=96)\n", + "\n", + "assert approx.delta == delta\n", + "assert metadata[\"method\"] == \"outward-rounded-subdivision\"\n", + "for x in torch.linspace(approx.lower, approx.upper, steps=101):\n", + " residual = math.tanh(float(x)) - poly(approx.coeffs, float(x))\n", + " assert abs(residual) <= approx.delta\n", + "print(f\"Tanh residual checks passed with Delta={approx.delta:.6g}.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 6. Small tanh network: value, Jacobian, and Hessian containment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "enable_interval_eval()\n", + "d, m = 2, 1\n", + "model = nn.Sequential(nn.Linear(d, 2), nn.Tanh(), nn.Linear(2, m)).double()\n", + "with torch.no_grad():\n", + " model[0].weight.copy_(torch.tensor([[0.4, -0.2], [0.1, 0.3]]))\n", + " model[0].bias.copy_(torch.tensor([0.05, -0.1]))\n", + " model[2].weight.copy_(torch.tensor([[0.5, -0.3]]))\n", + " model[2].bias.copy_(torch.tensor([0.02]))\n", + "lower = torch.tensor([-0.4, -0.2])\n", + "upper = torch.tensor([0.5, 0.3])\n", + "domain = PolynomialZonotope.from_box(lower, upper)\n", + "\n", + "out = model.eval_pz_twojet(domain, chebyshev_degree=5, residual_subdivisions=64)\n", + "y_interval = out.Y.interval_enclosure()\n", + "j_interval = out.J.interval_enclosure()\n", + "h_interval = out.H.interval_enclosure()\n", + "\n", + "samples = [lower, upper, (lower + upper) / 2]\n", + "samples.extend(lower + (upper - lower) * torch.tensor(pair) for pair in ((0.2, 0.8), (0.7, 0.1), (0.9, 0.6)))\n", + "for sample in samples:\n", + " x = sample.clone().detach().requires_grad_(True)\n", + " y = model(x)\n", + " jac_rows = []\n", + " hessians = []\n", + " for i in range(m):\n", + " grad = torch.autograd.grad(y[i], x, create_graph=True, retain_graph=True)[0]\n", + " jac_rows.append(grad)\n", + " hess_rows = []\n", + " for j in range(d):\n", + " hess_rows.append(torch.autograd.grad(grad[j], x, retain_graph=True)[0])\n", + " hessians.append(torch.stack(hess_rows))\n", + " jac = torch.stack(jac_rows)\n", + " hess = torch.stack(hessians)\n", + "\n", + " assert_contains(y_interval, y)\n", + " assert_contains(j_interval, jac)\n", + " assert_contains(h_interval, hess)\n", + "print(\"Small tanh network autograd containment checks passed.\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Notes\n", + "\n", + "Passing the sampling cells is a strong regression check, but samples alone are not a proof over the continuum. The certified part is intended to come from the polynomial-zonotope propagation and tanh residual certificate used by `eval_pz_twojet`; these notebook checks make that behavior easy to inspect interactively.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/notebooks/pz_w12_polynomial_reduction_benchmarks.ipynb b/notebooks/pz_w12_polynomial_reduction_benchmarks.ipynb new file mode 100644 index 0000000..e164410 --- /dev/null +++ b/notebooks/pz_w12_polynomial_reduction_benchmarks.ipynb @@ -0,0 +1,4094 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cc424479", + "metadata": {}, + "source": [ + "# Certified polynomial one-jet reduction benchmarks\n", + "\n", + "This notebook compares interval Jacobians, unreduced polynomial Jacobians, and three sound support-reduction policies. The 100-dimensional target is the saved Poisson PINN checkpoint; running the benchmark never retrains it. Reduced terms are not discarded: their componentwise remainder is propagated rigorously and attached as fresh pointwise residual symbols at the output. Both squared integrations return scalar polynomial zonotopes; intervalization is performed only afterward to obtain the final norm bounds. Tightness is assessed by the actual intervals and the absolute and relative widths of both the final certified $L^2$ and $W^{1,2}$ norm intervals. Before integration, we also report the mean, maximum, and relative mean componentwise widths of the full Jacobian enclosure." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "9162c298", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "mkdir -p failed for path /root/.config/matplotlib: [Errno 30] Read-only file system: '/root/.config'\n", + "Matplotlib created a temporary cache directory at /tmp/matplotlib-u9ufjtc0 because there was an issue with the default path ({configdir}); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.\n" + ] + } + ], + "source": [ + "from collections import Counter\n", + "from math import sqrt\n", + "from pathlib import Path\n", + "from time import perf_counter\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "from intervalnets import (IntervalTensor, PZIntegrationCell, affine_tanh_prime_enclosure, enable_interval_eval,\n", + " integrate_pz_onejet_squared, integrate_pz_value_squared,\n", + " load_tanh_mlp_checkpoint)\n", + "torch.set_num_threads(1)\n", + "torch.set_default_dtype(torch.float64)\n", + "enable_interval_eval()\n", + "repo_root = Path.cwd()\n", + "while not (repo_root / 'notebooks' / 'checkpoints').exists() and repo_root != repo_root.parent:\n", + " repo_root = repo_root.parent\n", + "CHECKPOINT = repo_root / 'notebooks' / 'checkpoints' / 'pinn_100d_poisson.pt'" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "3f840a3c", + "metadata": {}, + "outputs": [], + "source": [ + "def make_model(input_dim, hidden=(50, 50, 50), seed=20260731):\n", + " torch.manual_seed(seed)\n", + " layers, previous = [], input_dim\n", + " for width in hidden:\n", + " layers += [torch.nn.Linear(previous, width), torch.nn.Tanh()]\n", + " previous = width\n", + " layers.append(torch.nn.Linear(previous, 1))\n", + " return torch.nn.Sequential(*layers)\n", + "\n", + "def norm_interval(squared):\n", + " return (sqrt(max(0.0, float(squared.lower))), sqrt(max(0.0, float(squared.upper))))\n", + "\n", + "def interval_metrics(bounds, prefix):\n", + " lower, upper = map(float, bounds)\n", + " absolute_width = upper - lower\n", + " scale = max(abs(lower), abs(upper))\n", + " relative_width = absolute_width / scale if scale > 0.0 else 0.0\n", + " return {\n", + " f'{prefix}_lower': lower,\n", + " f'{prefix}_upper': upper,\n", + " f'{prefix}_absolute_width': absolute_width,\n", + " f'{prefix}_relative_width': relative_width,\n", + " }\n", + "\n", + "def jacobian_width_metrics(enclosure):\n", + " lower = torch.as_tensor(enclosure.lower)\n", + " upper = torch.as_tensor(enclosure.upper)\n", + " widths = upper - lower\n", + " scales = torch.maximum(lower.abs(), upper.abs())\n", + " relative_widths = torch.where(scales > 0.0, widths / scales, 0.0)\n", + " return {\n", + " 'J_mean_component_width_before_integration': float(widths.mean()),\n", + " 'J_max_component_width_before_integration': float(widths.max()),\n", + " 'J_relative_mean_component_width_before_integration': float(relative_widths.mean()),\n", + " }\n", + "\n", + "def benchmark(model, box, strategy='topk', **kwargs):\n", + " cell = PZIntegrationCell.from_affine_box(box)\n", + " start = perf_counter()\n", + " traced = model.eval_pz_onejet(cell.domain, return_trace=True,\n", + " reduction_strategy=strategy, **kwargs)\n", + " forward_s = perf_counter() - start\n", + " enclosure = traced.final.J.interval_enclosure()\n", + " jacobian_metrics = jacobian_width_metrics(enclosure)\n", + " start = perf_counter()\n", + " l2_integrated_pz = integrate_pz_value_squared(traced.final.Y, cell, output='pz')\n", + " l2_squared = l2_integrated_pz.interval_enclosure()\n", + " l2_integration_s = perf_counter() - start\n", + " start = perf_counter()\n", + " w12_integrated_pz = integrate_pz_onejet_squared(traced.final, cell, output='pz')\n", + " w12_squared = w12_integrated_pz.interval_enclosure()\n", + " w12_integration_s = perf_counter() - start\n", + " return {\n", + " 'strategy': strategy, **kwargs, 'forward_s': forward_s,\n", + " 'L2_integration_s': l2_integration_s,\n", + " 'W12_integration_s': w12_integration_s,\n", + " 'total_s': forward_s + w12_integration_s,\n", + " 'J_terms': len(traced.final.J.terms),\n", + " 'J_degree': max(map(sum, traced.final.J.terms), default=0),\n", + " 'noise': traced.final.J.num_noise,\n", + " 'L2_integrated_PZ_terms': len(l2_integrated_pz.terms),\n", + " 'W12_integrated_PZ_terms': len(w12_integrated_pz.terms),\n", + " 'W12_integrated_PZ_noise': w12_integrated_pz.num_noise,\n", + " **jacobian_metrics,\n", + " **interval_metrics(norm_interval(l2_squared), 'L2'),\n", + " **interval_metrics(norm_interval(w12_squared), 'W12'),\n", + " 'trace': traced.records,\n", + " }" + ] + }, + { + "cell_type": "markdown", + "id": "5bdd71cb", + "metadata": {}, + "source": [ + "## Small-network exact reference\n", + "The unreduced path is practical here and provides the polynomial reference endpoint." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "3b278dd2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
strategyreduceforward_sL2_integration_sW12_integration_stotal_sJ_termsJ_degreenoiseL2_integrated_PZ_termsW12_integrated_PZ_termsW12_integrated_PZ_noiseJ_mean_component_width_before_integrationJ_max_component_width_before_integrationJ_relative_mean_component_width_before_integrationL2_lowerL2_upperL2_absolute_widthL2_relative_widthW12_lowerW12_upperW12_absolute_widthW12_relative_widthmax_termsmax_degreepca_rankpca_candidates
0noneFalse0.0173180.0021010.0359220.05324014223611290.0238290.0330460.3906040.0021890.0022660.0000770.0338950.0025360.0027750.0002390.086188NaNNaNNaNNaN
1topkNaN0.0096800.0007810.0036500.0133302423611290.0240670.0333270.3938040.0021890.0022660.0000770.0338950.0025320.0027790.0002470.08899424.02.03.024.0
2degreeNaN0.0090420.0006920.0039570.0129992423611290.0240670.0333270.3938040.0021890.0022660.0000770.0338950.0025320.0027790.0002470.08899424.02.03.024.0
3pcaNaN0.0118440.0006770.0040010.0158452723911320.0242460.0335140.3962480.0021890.0022660.0000770.0338950.0025310.0027800.0002490.08939724.02.03.024.0
\n", + "
" + ], + "text/plain": [ + " strategy reduce forward_s ... max_degree pca_rank pca_candidates\n", + "0 none False 0.017318 ... NaN NaN NaN\n", + "1 topk NaN 0.009680 ... 2.0 3.0 24.0\n", + "2 degree NaN 0.009042 ... 2.0 3.0 24.0\n", + "3 pca NaN 0.011844 ... 2.0 3.0 24.0\n", + "\n", + "[4 rows x 27 columns]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "small_model = make_model(8, hidden=(10, 10))\n", + "small_box = IntervalTensor.from_bounds([-0.15] * 8, [0.15] * 8)\n", + "small_exact = benchmark(small_model, small_box, strategy='none', reduce=False)\n", + "small_reduced = [benchmark(small_model, small_box, strategy=s, max_terms=24,\n", + " max_degree=2, pca_rank=3, pca_candidates=24) for s in ('topk', 'degree', 'pca')]\n", + "small_reference_table = pd.DataFrame([\n", + " {k: v for k, v in row.items() if k != 'trace'}\n", + " for row in [small_exact, *small_reduced]\n", + "])\n", + "small_reference_table" + ] + }, + { + "cell_type": "markdown", + "id": "932a04ff", + "metadata": {}, + "source": [ + "## Saved 100D Poisson PINN: 100–50–50–50–1\n", + "The trained candidate is loaded from the committed checkpoint; there is no optimizer or training loop in this notebook. All policies below therefore certify exactly the same saved PINN weights. They preserve a Jacobian polynomial core. The interval result is the speed/looseness baseline. The primary comparison quantities are the absolute and relative widths of both norm intervals; the Jacobian width diagnostics measure tightness before the integration step." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "d6cf3306", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " loaded_checkpoint training_steps\n", + "0 notebooks/checkpoints/pinn_100d_poisson.pt 0\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
total_sJ_mean_component_width_before_integrationJ_max_component_width_before_integrationJ_relative_mean_component_width_before_integrationL2_lowerL2_upperL2_absolute_widthL2_relative_widthW12_lowerW12_upperW12_absolute_widthW12_relative_widthstrategymax_termsforward_sL2_integration_sW12_integration_sJ_termsJ_degreenoiseL2_integrated_PZ_termsW12_integrated_PZ_termsW12_integrated_PZ_noisemax_degreepca_rankpca_candidates
label
interval0.23001417.54881520.8648131.9897600.08.984144e-358.984144e-351.00.01.000326e-331.000326e-331.0NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
topk-321.21893217.11241020.2902201.9814370.03.013843e-353.013843e-351.00.09.760069e-349.760069e-341.0topk32.00.5349720.0187860.683960132.01.0350.01.01.0251.0NaNNaNNaN
topk-641.56424416.97682620.1274671.9812900.03.013843e-353.013843e-351.00.09.683504e-349.683504e-341.0topk64.00.8572760.0120620.706969164.01.0350.01.01.0251.0NaNNaNNaN
topk-962.16471516.84988319.9771301.9811510.03.013843e-353.013843e-351.00.09.611825e-349.611825e-341.0topk96.01.1773930.0120600.987322196.01.0350.01.01.0251.0NaNNaNNaN
topk-1282.37868716.82183419.9450271.9811190.03.013843e-353.013843e-351.00.09.595997e-349.595997e-341.0topk128.01.3800920.0126650.998595226.01.0350.01.01.0251.0NaNNaNNaN
topk-1922.88515716.78986719.9079691.9810840.03.013843e-353.013843e-351.00.09.577961e-349.577961e-341.0topk192.01.6951620.0184511.189995278.01.0350.01.01.0251.0NaNNaNNaN
topk-2562.98298416.76592019.8791981.9810570.03.013843e-353.013843e-351.00.09.564447e-349.564447e-341.0topk256.01.6737530.0327001.309231300.01.0350.01.01.0251.0NaNNaNNaN
degree-641.49566716.97682620.1274671.9812900.03.013843e-353.013843e-351.00.09.683504e-349.683504e-341.0degree64.00.8766340.0122760.619034164.01.0350.01.01.0251.02.0NaNNaN
pca-641.74254116.95561620.1040791.9812670.03.013843e-353.013843e-351.00.09.671521e-349.671521e-341.0pca64.01.0450650.0126650.697476168.01.0362.01.01.0263.0NaN4.032.0
\n", + "
" + ], + "text/plain": [ + " total_s ... pca_candidates\n", + "label ... \n", + "interval 0.230014 ... NaN\n", + "topk-32 1.218932 ... NaN\n", + "topk-64 1.564244 ... NaN\n", + "topk-96 2.164715 ... NaN\n", + "topk-128 2.378687 ... NaN\n", + "topk-192 2.885157 ... NaN\n", + "topk-256 2.982984 ... NaN\n", + "degree-64 1.495667 ... NaN\n", + "pca-64 1.742541 ... 32.0\n", + "\n", + "[9 rows x 26 columns]" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model = load_tanh_mlp_checkpoint(CHECKPOINT)\n", + "assert sum(parameter.numel() for parameter in model.parameters()) == 10201\n", + "display(pd.DataFrame([{\n", + " 'loaded_checkpoint': str(CHECKPOINT.relative_to(repo_root)),\n", + " 'training_steps': 0,\n", + "}]))\n", + "box = IntervalTensor.from_bounds([-0.1] * 100, [0.1] * 100)\n", + "configs = [\n", + " ('topk-32', 'topk', dict(max_terms=32)),\n", + " ('topk-64', 'topk', dict(max_terms=64)),\n", + " ('topk-96', 'topk', dict(max_terms=96)),\n", + " ('topk-128', 'topk', dict(max_terms=128)),\n", + " ('topk-192', 'topk', dict(max_terms=192)),\n", + " ('topk-256', 'topk', dict(max_terms=256)),\n", + " ('degree-64', 'degree', dict(max_terms=64, max_degree=2)),\n", + " ('pca-64', 'pca', dict(max_terms=64, pca_rank=4, pca_candidates=32)),\n", + "]\n", + "rows = []\n", + "for label, strategy, kwargs in configs:\n", + " row = benchmark(model, box, strategy=strategy, **kwargs)\n", + " row['label'] = label\n", + " rows.append(row)\n", + "start = perf_counter()\n", + "interval_bound = model.sobolev_norm(box, p=2.0, order=1, method='interval')\n", + "interval_s = perf_counter() - start\n", + "interval_l2_bound = model.lpnorm(box, p=2.0, method='interval')\n", + "interval_jacobian = model.eval_jacobian(box)\n", + "interval_row = {\n", + " 'label': 'interval',\n", + " 'total_s': interval_s,\n", + " **jacobian_width_metrics(interval_jacobian),\n", + " **interval_metrics((interval_l2_bound.lower, interval_l2_bound.upper), 'L2'),\n", + " **interval_metrics((interval_bound.lower, interval_bound.upper), 'W12'),\n", + "}\n", + "summary = [{k: v for k, v in row.items() if k != 'trace'} for row in rows]\n", + "benchmark_table = pd.DataFrame([interval_row, *summary]).set_index('label')\n", + "benchmark_table" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "23361145", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
public_default_L2_spublic_default_W12_spublic_default_L2_lowerpublic_default_L2_upperpublic_default_L2_absolute_widthpublic_default_L2_relative_widthpublic_default_W12_lowerpublic_default_W12_upperpublic_default_W12_absolute_widthpublic_default_W12_relative_width
00.06359413.161184-4.940656e-3243.013843e-353.013843e-351.0-4.940656e-3249.611825e-349.611825e-341.0
\n", + "
" + ], + "text/plain": [ + " public_default_L2_s ... public_default_W12_relative_width\n", + "0 0.063594 ... 1.0\n", + "\n", + "[1 rows x 10 columns]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "assert all(row['total_s'] > 0.0 for row in rows), summary\n", + "start = perf_counter()\n", + "public_l2_bound = model.pz_l2norm(box)\n", + "public_l2_s = perf_counter() - start\n", + "start = perf_counter()\n", + "public_w12_bound = model.pz_sobolev_norm(box, order=1)\n", + "public_w12_s = perf_counter() - start\n", + "public_default_table = pd.DataFrame([{\n", + " 'public_default_L2_s': public_l2_s,\n", + " 'public_default_W12_s': public_w12_s,\n", + " **interval_metrics((public_l2_bound.lower, public_l2_bound.upper), 'public_default_L2'),\n", + " **interval_metrics((public_w12_bound.lower, public_w12_bound.upper), 'public_default_W12'),\n", + "}])\n", + "public_default_table" + ] + }, + { + "cell_type": "markdown", + "id": "b0edd728", + "metadata": {}, + "source": [ + "## Layer diagnostics\n", + "The activation rows expose where support generation and certified tail growth occur. The benchmark-level Jacobian widths above are computed after the complete one-jet has been constructed but before either squared integral is evaluated. They use the full enclosure $J_{\\mathrm{core}}+[-R,R]$, not merely the reduction remainder. For each entry, the relative width is $(\\overline J_{ij}-\\underline J_{ij})/\\max(|\\underline J_{ij}|,|\\overline J_{ij}|)$, with exact-zero entries assigned zero; the reported relative mean is the mean of these componentwise ratios." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "c9c15a4f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
module_indexlayersecondsY_termsJ_termsJ_degreeremainder_mean_radiusremainder_max_radius
0-1Input0.000000100000.0000000.000000
10Linear0.002670100000.0000000.000000
21Tanh0.0257081509610.0264470.075139
32Linear0.0090181509610.1529480.232838
43Tanh0.4868742008610.1738410.279970
54Linear0.0107502008611.0390351.504241
65Tanh0.6034522509611.0679991.518824
76Linear0.0118962509618.4239169.987156
\n", + "
" + ], + "text/plain": [ + " module_index layer ... remainder_mean_radius remainder_max_radius\n", + "0 -1 Input ... 0.000000 0.000000\n", + "1 0 Linear ... 0.000000 0.000000\n", + "2 1 Tanh ... 0.026447 0.075139\n", + "3 2 Linear ... 0.152948 0.232838\n", + "4 3 Tanh ... 0.173841 0.279970\n", + "5 4 Linear ... 1.039035 1.504241\n", + "6 5 Tanh ... 1.067999 1.518824\n", + "7 6 Linear ... 8.423916 9.987156\n", + "\n", + "[8 rows x 8 columns]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chosen = next(row for row in rows if row['label'] == 'topk-96')\n", + "layer_diagnostics = pd.DataFrame([{\n", + " 'module_index': record.layer_index,\n", + " 'layer': record.layer_type,\n", + " 'seconds': record.elapsed_s,\n", + " 'Y_terms': record.summary['Y']['term_count'],\n", + " 'J_terms': record.summary['J']['term_count'],\n", + " 'J_degree': record.summary['J']['max_degree'],\n", + " 'remainder_mean_radius': record.summary['J']['remainder_mean_radius'],\n", + " 'remainder_max_radius': record.summary['J']['remainder_max_radius'],\n", + "} for record in chosen['trace']])\n", + "layer_diagnostics" + ] + }, + { + "cell_type": "markdown", + "id": "94ff5cf1", + "metadata": {}, + "source": [ + "## Initial activation-approximation errors by neuron\n", + "\n", + "For hidden layer $\\ell$ and neuron $i$, the incoming value PZ is intervalized once to obtain the preactivation interval $I_{\\ell i}$. That same interval is used for both certified affine enclosures\n", + "\n", + "$$\\tanh(z)\\in p_{\\ell i}^{(0)}z+q_{\\ell i}^{(0)}+\\delta_{\\ell i}^{(0)}[-1,1],$$\n", + "\n", + "$$\\tanh'(z)\\in p_{\\ell i}^{(1)}z+q_{\\ell i}^{(1)}+\\delta_{\\ell i}^{(1)}[-1,1].$$\n", + "\n", + "The tables below show the actual certified approximation-noise coefficients $\\delta_{\\ell i}^{(0)}$ and $\\delta_{\\ell i}^{(1)}$ before multiplication by the incoming Jacobian and before Top-$k$/PCA reduction. Rows are one-based hidden-neuron indices; columns are activation layers." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "5cf169a0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layer_1hidden_layer_2hidden_layer_3
neuron
10.0897480.0808180.129496
20.0629940.0801960.147882
30.0877210.0927210.094163
40.0688980.0734580.143326
50.0762350.0811050.123203
60.0710890.0695780.129580
70.0928420.0825950.107198
80.0783790.0630860.121686
90.0793250.0978970.160625
100.0900030.1029290.142696
110.0803200.0834820.146999
120.0860100.1247580.115563
130.0682920.0832790.132988
140.0581660.0784770.107564
150.0701520.0894880.112359
160.0719190.0654930.108481
170.0586770.0817350.116744
180.0859390.0967260.131883
190.0849300.1065170.120021
200.0674990.1083520.147114
210.0671370.0730660.146510
220.0708740.1025140.130191
230.0617230.0996850.117846
240.0665200.0837290.109033
250.0750380.0913070.123356
260.0763780.0797000.100588
270.0853530.1111370.166947
280.0777040.1026360.125910
290.0716000.0934500.140495
300.0758670.0891200.146763
310.0521280.0832450.156356
320.0788060.0951310.202250
330.0863250.0981540.127629
340.0662990.0863660.160058
350.0740240.1175150.141207
360.0770130.0943830.130120
370.0686180.0945660.109205
380.0644630.1128490.151453
390.0804220.0816340.067150
400.1010420.0722130.121613
410.0541610.1283660.136977
420.0731660.1119880.144891
430.0620360.1045530.187602
440.0804020.0820280.109223
450.0790110.1034450.113034
460.0742090.0761160.163690
470.0778400.1085110.121742
480.0727070.0907260.116955
490.0565700.1013360.164305
500.0723970.1089560.178494
\n", + "
" + ], + "text/plain": [ + " hidden_layer_1 hidden_layer_2 hidden_layer_3\n", + "neuron \n", + "1 0.089748 0.080818 0.129496\n", + "2 0.062994 0.080196 0.147882\n", + "3 0.087721 0.092721 0.094163\n", + "4 0.068898 0.073458 0.143326\n", + "5 0.076235 0.081105 0.123203\n", + "6 0.071089 0.069578 0.129580\n", + "7 0.092842 0.082595 0.107198\n", + "8 0.078379 0.063086 0.121686\n", + "9 0.079325 0.097897 0.160625\n", + "10 0.090003 0.102929 0.142696\n", + "11 0.080320 0.083482 0.146999\n", + "12 0.086010 0.124758 0.115563\n", + "13 0.068292 0.083279 0.132988\n", + "14 0.058166 0.078477 0.107564\n", + "15 0.070152 0.089488 0.112359\n", + "16 0.071919 0.065493 0.108481\n", + "17 0.058677 0.081735 0.116744\n", + "18 0.085939 0.096726 0.131883\n", + "19 0.084930 0.106517 0.120021\n", + "20 0.067499 0.108352 0.147114\n", + "21 0.067137 0.073066 0.146510\n", + "22 0.070874 0.102514 0.130191\n", + "23 0.061723 0.099685 0.117846\n", + "24 0.066520 0.083729 0.109033\n", + "25 0.075038 0.091307 0.123356\n", + "26 0.076378 0.079700 0.100588\n", + "27 0.085353 0.111137 0.166947\n", + "28 0.077704 0.102636 0.125910\n", + "29 0.071600 0.093450 0.140495\n", + "30 0.075867 0.089120 0.146763\n", + "31 0.052128 0.083245 0.156356\n", + "32 0.078806 0.095131 0.202250\n", + "33 0.086325 0.098154 0.127629\n", + "34 0.066299 0.086366 0.160058\n", + "35 0.074024 0.117515 0.141207\n", + "36 0.077013 0.094383 0.130120\n", + "37 0.068618 0.094566 0.109205\n", + "38 0.064463 0.112849 0.151453\n", + "39 0.080422 0.081634 0.067150\n", + "40 0.101042 0.072213 0.121613\n", + "41 0.054161 0.128366 0.136977\n", + "42 0.073166 0.111988 0.144891\n", + "43 0.062036 0.104553 0.187602\n", + "44 0.080402 0.082028 0.109223\n", + "45 0.079011 0.103445 0.113034\n", + "46 0.074209 0.076116 0.163690\n", + "47 0.077840 0.108511 0.121742\n", + "48 0.072707 0.090726 0.116955\n", + "49 0.056570 0.101336 0.164305\n", + "50 0.072397 0.108956 0.178494" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "activation_records = [record for record in chosen['trace'] if record.layer_type == 'Tanh']\n", + "assert len(activation_records) == 3\n", + "assert all(record.summary['tanh_approximation_radii'].numel() == 50 for record in activation_records)\n", + "\n", + "def per_neuron_activation_error_table(summary_key):\n", + " columns = {\n", + " f'hidden_layer_{layer_index}': record.summary[summary_key].detach().cpu().numpy()\n", + " for layer_index, record in enumerate(activation_records, start=1)\n", + " }\n", + " frame = pd.DataFrame(columns)\n", + " frame.index = pd.RangeIndex(1, len(frame) + 1, name='neuron')\n", + " return frame\n", + "\n", + "tanh_value_approximation_errors = per_neuron_activation_error_table(\n", + " 'tanh_approximation_radii'\n", + ")\n", + "tanh_value_approximation_errors" + ] + }, + { + "cell_type": "markdown", + "id": "a51cbcc9", + "metadata": {}, + "source": [ + "### Certified $\\tanh'$ approximation errors" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "03ba2fd0", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layer_1hidden_layer_2hidden_layer_3
neuron
10.3035120.2882820.357791
20.2536990.2872700.378499
30.3001220.3071240.310248
40.2659360.2748930.373685
50.2795970.2880770.350153
60.2702160.2668420.357862
70.3084960.2910390.328011
80.2839300.2540790.348642
90.2855940.3163540.390683
100.3038950.3225990.372020
110.2874200.2928740.377478
120.2972550.3527630.341153
130.2647370.2923020.361081
140.2427950.2832490.329571
150.2684740.3023700.336941
160.2718650.2586470.331507
170.2443570.2899700.341591
180.2971470.3145730.361085
190.2954350.3277210.345509
200.2630080.3308360.377610
210.2624100.2738990.376793
220.2692470.3222650.358644
230.2508310.3183510.344017
240.2612000.2933790.332428
250.2777390.3057240.350610
260.2802030.2862930.320055
270.2961430.3352310.396112
280.2826020.3230870.353578
290.2712380.3092510.370583
300.2793110.3022090.376833
310.2294360.2919360.386341
320.2847370.3120130.423880
330.2977790.3164050.355892
340.2607070.2977530.390197
350.2755030.3437670.371019
360.2814250.3109430.358274
370.2653360.3111360.330814
380.2569850.3375860.382161
390.2875880.2895870.261997
400.3210170.2720380.348836
410.2340850.3569970.366765
420.2742620.3360830.375068
430.2518520.3256550.413489
440.2876420.2900440.332618
450.2851440.3244890.335841
460.2762310.2794010.393450
470.2830000.3310030.347976
480.2730660.3041520.341633
490.2395490.3214870.394256
500.2727150.3323660.406266
\n", + "
" + ], + "text/plain": [ + " hidden_layer_1 hidden_layer_2 hidden_layer_3\n", + "neuron \n", + "1 0.303512 0.288282 0.357791\n", + "2 0.253699 0.287270 0.378499\n", + "3 0.300122 0.307124 0.310248\n", + "4 0.265936 0.274893 0.373685\n", + "5 0.279597 0.288077 0.350153\n", + "6 0.270216 0.266842 0.357862\n", + "7 0.308496 0.291039 0.328011\n", + "8 0.283930 0.254079 0.348642\n", + "9 0.285594 0.316354 0.390683\n", + "10 0.303895 0.322599 0.372020\n", + "11 0.287420 0.292874 0.377478\n", + "12 0.297255 0.352763 0.341153\n", + "13 0.264737 0.292302 0.361081\n", + "14 0.242795 0.283249 0.329571\n", + "15 0.268474 0.302370 0.336941\n", + "16 0.271865 0.258647 0.331507\n", + "17 0.244357 0.289970 0.341591\n", + "18 0.297147 0.314573 0.361085\n", + "19 0.295435 0.327721 0.345509\n", + "20 0.263008 0.330836 0.377610\n", + "21 0.262410 0.273899 0.376793\n", + "22 0.269247 0.322265 0.358644\n", + "23 0.250831 0.318351 0.344017\n", + "24 0.261200 0.293379 0.332428\n", + "25 0.277739 0.305724 0.350610\n", + "26 0.280203 0.286293 0.320055\n", + "27 0.296143 0.335231 0.396112\n", + "28 0.282602 0.323087 0.353578\n", + "29 0.271238 0.309251 0.370583\n", + "30 0.279311 0.302209 0.376833\n", + "31 0.229436 0.291936 0.386341\n", + "32 0.284737 0.312013 0.423880\n", + "33 0.297779 0.316405 0.355892\n", + "34 0.260707 0.297753 0.390197\n", + "35 0.275503 0.343767 0.371019\n", + "36 0.281425 0.310943 0.358274\n", + "37 0.265336 0.311136 0.330814\n", + "38 0.256985 0.337586 0.382161\n", + "39 0.287588 0.289587 0.261997\n", + "40 0.321017 0.272038 0.348836\n", + "41 0.234085 0.356997 0.366765\n", + "42 0.274262 0.336083 0.375068\n", + "43 0.251852 0.325655 0.413489\n", + "44 0.287642 0.290044 0.332618\n", + "45 0.285144 0.324489 0.335841\n", + "46 0.276231 0.279401 0.393450\n", + "47 0.283000 0.331003 0.347976\n", + "48 0.273066 0.304152 0.341633\n", + "49 0.239549 0.321487 0.394256\n", + "50 0.272715 0.332366 0.406266" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tanh_prime_approximation_errors = per_neuron_activation_error_table(\n", + " 'tanh_prime_approximation_radii'\n", + ")\n", + "tanh_prime_approximation_errors" + ] + }, + { + "cell_type": "markdown", + "id": "d27a26ed", + "metadata": {}, + "source": [ + "### Per-layer approximation-error summary" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "226381de", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tanh_delta_mintanh_delta_meantanh_delta_maxtanh_prime_delta_mintanh_prime_delta_meantanh_prime_delta_max
hidden_layer
10.0521280.0741790.1010420.2294360.2751290.321017
20.0630860.0924220.1283660.2540790.3059280.356997
30.0671500.1330230.2022500.2619970.3593510.423880
\n", + "
" + ], + "text/plain": [ + " tanh_delta_min ... tanh_prime_delta_max\n", + "hidden_layer ... \n", + "1 0.052128 ... 0.321017\n", + "2 0.063086 ... 0.356997\n", + "3 0.067150 ... 0.423880\n", + "\n", + "[3 rows x 6 columns]" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "activation_error_summary = pd.DataFrame([{\n", + " 'hidden_layer': layer_index,\n", + " 'tanh_delta_min': record.summary['tanh_approximation_radius_min'],\n", + " 'tanh_delta_mean': record.summary['tanh_approximation_radius_mean'],\n", + " 'tanh_delta_max': record.summary['tanh_approximation_radius_max'],\n", + " 'tanh_prime_delta_min': record.summary['tanh_prime_approximation_radius_min'],\n", + " 'tanh_prime_delta_mean': record.summary['tanh_prime_approximation_radius_mean'],\n", + " 'tanh_prime_delta_max': record.summary['tanh_prime_approximation_radius_max'],\n", + "} for layer_index, record in enumerate(activation_records, start=1)]).set_index('hidden_layer')\n", + "activation_error_summary" + ] + }, + { + "cell_type": "markdown", + "id": "interval-radius-intro", + "metadata": {}, + "source": [ + "## Corresponding interval-enclosure radii by neuron\n", + "\n", + "For a direct interval comparison, the same preactivation interval $I_{\\ell i}=[l_{\\ell i},u_{\\ell i}]$ is now mapped through $\\tanh$ or $\\tanh'$, and the radius (half the interval width) is reported. These are radii of the **entire interval images**, whereas the affine quantities $\\delta_{\\ell i}^{(0)}$ and $\\delta_{\\ell i}^{(1)}$ above are only residual radii around dependency-preserving affine functions. Thus the numbers are deliberately related but are not identical error measures.\n", + "\n", + "Since $\\tanh$ is increasing, $\\tanh(I)=[\\tanh(l),\\tanh(u)]$. For $\\tanh'(z)=1-\\tanh^2(z)$, the function increases on $(-\\infty,0]$ and decreases on $[0,\\infty)$. Hence its maximum is $1$ when $0\\in I$; otherwise both extrema are obtained from the endpoints." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "interval-radius-tables", + "metadata": {}, + "outputs": [], + "source": [ + "def interval_activation_radius_tables(records):\n", + " tanh_columns = {}\n", + " tanh_prime_columns = {}\n", + " for layer_index, record in enumerate(records, start=1):\n", + " lower = record.summary['preactivation_lower']\n", + " upper = record.summary['preactivation_upper']\n", + "\n", + " preactivation = IntervalTensor.from_bounds(\n", + " lower.detach().cpu().tolist(), upper.detach().cpu().tolist()\n", + " )\n", + " activation = torch.nn.Tanh()\n", + " value_interval = activation.eval(preactivation)\n", + " value_lower = torch.as_tensor(value_interval.lower)\n", + " value_upper = torch.as_tensor(value_interval.upper)\n", + " tanh_columns[f'hidden_layer_{layer_index}'] = (value_upper - value_lower) / 2\n", + "\n", + " local_jacobian = activation.eval_jacobian(preactivation)\n", + " derivative_lower = torch.diagonal(torch.as_tensor(local_jacobian.lower))\n", + " derivative_upper = torch.diagonal(torch.as_tensor(local_jacobian.upper))\n", + " tanh_prime_columns[f'hidden_layer_{layer_index}'] = (derivative_upper - derivative_lower) / 2\n", + "\n", + " def frame(columns):\n", + " result = pd.DataFrame({key: value.detach().cpu().numpy() for key, value in columns.items()})\n", + " result.index = pd.RangeIndex(1, len(result) + 1, name='neuron')\n", + " return result\n", + "\n", + " return frame(tanh_columns), frame(tanh_prime_columns)\n", + "\n", + "tanh_interval_radii, tanh_prime_interval_radii = interval_activation_radius_tables(\n", + " activation_records\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "tanh-interval-radius-title", + "metadata": {}, + "source": [ + "### Interval radii for $\\tanh(I_{\\ell i})$" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "tanh-interval-radius-table", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layer_1hidden_layer_2hidden_layer_3
neuron
10.7791300.7593520.846162
20.7123950.7579890.870088
30.7747860.7841380.787835
40.7293280.7414760.864571
50.7479570.7593150.837111
60.7351830.7307120.846255
70.7854980.7630930.810587
80.7535940.7128690.835183
90.7558130.7954340.884033
100.7796320.8036520.862971
110.7582090.7653960.868953
120.7710780.8399920.826057
130.7276790.7647210.850288
140.6970540.7529710.812165
150.7327800.7779000.820930
160.7374090.7193940.814329
170.6991560.7615480.826950
180.7709330.7931900.849875
190.7687100.8100080.831767
200.7253430.8136670.869100
210.7244750.7402160.868225
220.7340420.8031460.847157
230.7084090.7981700.829549
240.7227890.7660310.815407
250.7453480.7820670.837569
260.7486540.7567280.800184
270.7696340.8188640.890240
280.7518640.8039570.841152
290.7365610.7865270.861005
300.7474440.7775470.868342
310.6774280.7643430.879217
320.7546530.7899880.920766
330.7717590.7956100.843814
340.7221200.7717570.883470
350.7424490.8291900.861618
360.7502680.7886010.846814
370.7285190.7888760.814036
380.7169270.8217180.874260
390.7584350.7611120.724047
400.8012890.7377710.835321
410.6843050.8450480.856553
420.7406500.8200020.866252
430.7097370.8072120.909383
440.7584770.7617900.815664
450.7551790.8056250.820250
460.7433060.7476860.887167
470.7523430.8138890.834627
480.7391370.7802630.827077
490.6922710.8018640.887993
500.7385830.8153160.901442
\n", + "
" + ], + "text/plain": [ + " hidden_layer_1 hidden_layer_2 hidden_layer_3\n", + "neuron \n", + "1 0.779130 0.759352 0.846162\n", + "2 0.712395 0.757989 0.870088\n", + "3 0.774786 0.784138 0.787835\n", + "4 0.729328 0.741476 0.864571\n", + "5 0.747957 0.759315 0.837111\n", + "6 0.735183 0.730712 0.846255\n", + "7 0.785498 0.763093 0.810587\n", + "8 0.753594 0.712869 0.835183\n", + "9 0.755813 0.795434 0.884033\n", + "10 0.779632 0.803652 0.862971\n", + "11 0.758209 0.765396 0.868953\n", + "12 0.771078 0.839992 0.826057\n", + "13 0.727679 0.764721 0.850288\n", + "14 0.697054 0.752971 0.812165\n", + "15 0.732780 0.777900 0.820930\n", + "16 0.737409 0.719394 0.814329\n", + "17 0.699156 0.761548 0.826950\n", + "18 0.770933 0.793190 0.849875\n", + "19 0.768710 0.810008 0.831767\n", + "20 0.725343 0.813667 0.869100\n", + "21 0.724475 0.740216 0.868225\n", + "22 0.734042 0.803146 0.847157\n", + "23 0.708409 0.798170 0.829549\n", + "24 0.722789 0.766031 0.815407\n", + "25 0.745348 0.782067 0.837569\n", + "26 0.748654 0.756728 0.800184\n", + "27 0.769634 0.818864 0.890240\n", + "28 0.751864 0.803957 0.841152\n", + "29 0.736561 0.786527 0.861005\n", + "30 0.747444 0.777547 0.868342\n", + "31 0.677428 0.764343 0.879217\n", + "32 0.754653 0.789988 0.920766\n", + "33 0.771759 0.795610 0.843814\n", + "34 0.722120 0.771757 0.883470\n", + "35 0.742449 0.829190 0.861618\n", + "36 0.750268 0.788601 0.846814\n", + "37 0.728519 0.788876 0.814036\n", + "38 0.716927 0.821718 0.874260\n", + "39 0.758435 0.761112 0.724047\n", + "40 0.801289 0.737771 0.835321\n", + "41 0.684305 0.845048 0.856553\n", + "42 0.740650 0.820002 0.866252\n", + "43 0.709737 0.807212 0.909383\n", + "44 0.758477 0.761790 0.815664\n", + "45 0.755179 0.805625 0.820250\n", + "46 0.743306 0.747686 0.887167\n", + "47 0.752343 0.813889 0.834627\n", + "48 0.739137 0.780263 0.827077\n", + "49 0.692271 0.801864 0.887993\n", + "50 0.738583 0.815316 0.901442" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tanh_interval_radii" + ] + }, + { + "cell_type": "markdown", + "id": "tanh-prime-interval-radius-title", + "metadata": {}, + "source": [ + "### Interval radii for $\\tanh'(I_{\\ell i})$" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "tanh-prime-interval-radius-table", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layer_1hidden_layer_2hidden_layer_3
neuron
10.3058710.2921450.367426
20.2596190.2885910.381770
30.3037940.3204690.317412
40.2697820.2755600.378471
50.2882570.2991380.360464
60.2745600.2758830.367684
70.3105440.2993210.344604
80.2875500.2567900.356245
90.2899800.3176740.395888
100.3070830.3359120.384057
110.2908770.2977900.382437
120.3010140.3564390.345050
130.2684080.2998490.374814
140.2528410.2952280.340610
150.2708630.3128990.340172
160.2753940.2673610.336934
170.2502800.2920700.354435
180.3006530.3156490.366161
190.2990070.3410300.359694
200.2688300.3407250.382389
210.2661600.2800040.383570
220.2793880.3340030.367985
230.2585810.3283780.349272
240.2639350.2969160.335268
250.2821800.3128140.359002
260.2850290.2901880.327016
270.2999030.3395150.403502
280.2879240.3298050.362966
290.2750070.3150240.376421
300.2832350.3089380.385309
310.2330110.3020930.394441
320.2875430.3158660.426487
330.3016380.3234290.363253
340.2644100.3031150.395005
350.2838470.3459910.379592
360.2853520.3121620.369445
370.2699490.3149090.347299
380.2590930.3410500.383405
390.2913050.2954670.270994
400.3238470.2805290.353387
410.2400690.3619710.372462
420.2776580.3437500.382316
430.2544760.3342200.413503
440.2885930.2983690.336834
450.2867360.3281870.352959
460.2797670.2878460.398902
470.2854480.3412690.360512
480.2807820.3161910.355688
490.2464600.3232680.396093
500.2774930.3339010.409482
\n", + "
" + ], + "text/plain": [ + " hidden_layer_1 hidden_layer_2 hidden_layer_3\n", + "neuron \n", + "1 0.305871 0.292145 0.367426\n", + "2 0.259619 0.288591 0.381770\n", + "3 0.303794 0.320469 0.317412\n", + "4 0.269782 0.275560 0.378471\n", + "5 0.288257 0.299138 0.360464\n", + "6 0.274560 0.275883 0.367684\n", + "7 0.310544 0.299321 0.344604\n", + "8 0.287550 0.256790 0.356245\n", + "9 0.289980 0.317674 0.395888\n", + "10 0.307083 0.335912 0.384057\n", + "11 0.290877 0.297790 0.382437\n", + "12 0.301014 0.356439 0.345050\n", + "13 0.268408 0.299849 0.374814\n", + "14 0.252841 0.295228 0.340610\n", + "15 0.270863 0.312899 0.340172\n", + "16 0.275394 0.267361 0.336934\n", + "17 0.250280 0.292070 0.354435\n", + "18 0.300653 0.315649 0.366161\n", + "19 0.299007 0.341030 0.359694\n", + "20 0.268830 0.340725 0.382389\n", + "21 0.266160 0.280004 0.383570\n", + "22 0.279388 0.334003 0.367985\n", + "23 0.258581 0.328378 0.349272\n", + "24 0.263935 0.296916 0.335268\n", + "25 0.282180 0.312814 0.359002\n", + "26 0.285029 0.290188 0.327016\n", + "27 0.299903 0.339515 0.403502\n", + "28 0.287924 0.329805 0.362966\n", + "29 0.275007 0.315024 0.376421\n", + "30 0.283235 0.308938 0.385309\n", + "31 0.233011 0.302093 0.394441\n", + "32 0.287543 0.315866 0.426487\n", + "33 0.301638 0.323429 0.363253\n", + "34 0.264410 0.303115 0.395005\n", + "35 0.283847 0.345991 0.379592\n", + "36 0.285352 0.312162 0.369445\n", + "37 0.269949 0.314909 0.347299\n", + "38 0.259093 0.341050 0.383405\n", + "39 0.291305 0.295467 0.270994\n", + "40 0.323847 0.280529 0.353387\n", + "41 0.240069 0.361971 0.372462\n", + "42 0.277658 0.343750 0.382316\n", + "43 0.254476 0.334220 0.413503\n", + "44 0.288593 0.298369 0.336834\n", + "45 0.286736 0.328187 0.352959\n", + "46 0.279767 0.287846 0.398902\n", + "47 0.285448 0.341269 0.360512\n", + "48 0.280782 0.316191 0.355688\n", + "49 0.246460 0.323268 0.396093\n", + "50 0.277493 0.333901 0.409482" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tanh_prime_interval_radii" + ] + }, + { + "cell_type": "markdown", + "id": "interval-affine-comparison-title", + "metadata": {}, + "source": [ + "### Layerwise interval-versus-affine comparison" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "interval-affine-comparison", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
mean_tanh_interval_radiusmean_tanh_affine_residualmean_tanh_prime_interval_radiusmean_tanh_prime_affine_residual
hidden_layer
10.7413740.0741790.2794810.275129
20.7817550.0924220.3123940.305928
30.8472760.1330230.3669820.359351
\n", + "
" + ], + "text/plain": [ + " mean_tanh_interval_radius ... mean_tanh_prime_affine_residual\n", + "hidden_layer ... \n", + "1 0.741374 ... 0.275129\n", + "2 0.781755 ... 0.305928\n", + "3 0.847276 ... 0.359351\n", + "\n", + "[3 rows x 4 columns]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "interval_affine_comparison = pd.DataFrame([\n", + " {\n", + " 'hidden_layer': layer_index,\n", + " 'mean_tanh_interval_radius': tanh_interval_radii[column].mean(),\n", + " 'mean_tanh_affine_residual': tanh_value_approximation_errors[column].mean(),\n", + " 'mean_tanh_prime_interval_radius': tanh_prime_interval_radii[column].mean(),\n", + " 'mean_tanh_prime_affine_residual': tanh_prime_approximation_errors[column].mean(),\n", + " }\n", + " for layer_index, column in enumerate(tanh_interval_radii.columns, start=1)\n", + "]).set_index('hidden_layer')\n", + "interval_affine_comparison" + ] + }, + { + "cell_type": "markdown", + "id": "tanh-prime-residual-distribution-title", + "metadata": {}, + "source": [ + "## Distribution and geometry of the affine $\\tanh'$ residuals\n", + "\n", + "The affine enclosure is deterministic and non-iterative. For $f=\\tanh'$ on $[l,u]$, it first takes the secant slope\n", + "\n", + "$$p=\\frac{f(u)-f(l)}{u-l}.$$\n", + "\n", + "It then finds every stationary point of the residual $r(x)=f(x)-px$ by solving $f'(x)=p$ analytically. Together with the endpoints, these finite candidates give $r_{\\min}$ and $r_{\\max}$. Finally,\n", + "\n", + "$$q=\\frac{r_{\\max}+r_{\\min}}2,\\qquad \\delta=\\frac{r_{\\max}-r_{\\min}}2,$$\n", + "\n", + "so $f(x)\\in px+q+\\delta[-1,1]$. Apart from a final outward-rounding inflation, this is the minimax vertical shift for the chosen secant slope. The diagnostics below therefore test whether the difficult cases come from isolated bad intervals or from intervals spanning the central bump of $\\operatorname{sech}^2$." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "tanh-prime-residual-diagnostics", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
delta_mindelta_q25delta_mediandelta_q75delta_maxmean_delta_over_interval_radiuszero_crossing_fraction
hidden_layer
10.2294360.2634400.2758670.2869640.3210170.9842041.0
20.2540790.2896820.3064240.3229650.3569970.9792811.0
30.2619970.3416020.3580680.3773170.4238800.9788481.0
\n", + "
" + ], + "text/plain": [ + " delta_min ... zero_crossing_fraction\n", + "hidden_layer ... \n", + "1 0.229436 ... 1.0\n", + "2 0.254079 ... 1.0\n", + "3 0.261997 ... 1.0\n", + "\n", + "[3 rows x 7 columns]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "derivative_rows = []\n", + "for layer_index, record in enumerate(activation_records, start=1):\n", + " lowers = record.summary['preactivation_lower'].detach().cpu().numpy()\n", + " uppers = record.summary['preactivation_upper'].detach().cpu().numpy()\n", + " interval_radii = tanh_prime_interval_radii[f'hidden_layer_{layer_index}'].to_numpy()\n", + " for neuron_index, (lower, upper, interval_radius) in enumerate(\n", + " zip(lowers, uppers, interval_radii), start=1\n", + " ):\n", + " enclosure = affine_tanh_prime_enclosure((float(lower), float(upper)))\n", + " derivative_rows.append({\n", + " 'hidden_layer': layer_index,\n", + " 'neuron': neuron_index,\n", + " 'preactivation_lower': float(lower),\n", + " 'preactivation_upper': float(upper),\n", + " 'preactivation_width': float(upper - lower),\n", + " 'contains_zero': bool(lower <= 0.0 <= upper),\n", + " 'affine_slope_p': enclosure.p,\n", + " 'affine_intercept_q': enclosure.q,\n", + " 'affine_residual_delta': enclosure.delta,\n", + " 'interval_radius': float(interval_radius),\n", + " 'delta_over_interval_radius': (\n", + " enclosure.delta / float(interval_radius) if interval_radius > 0.0 else 0.0\n", + " ),\n", + " })\n", + "tanh_prime_diagnostics = pd.DataFrame(derivative_rows)\n", + "\n", + "distribution_summary = tanh_prime_diagnostics.groupby('hidden_layer').agg(\n", + " delta_min=('affine_residual_delta', 'min'),\n", + " delta_q25=('affine_residual_delta', lambda x: x.quantile(0.25)),\n", + " delta_median=('affine_residual_delta', 'median'),\n", + " delta_q75=('affine_residual_delta', lambda x: x.quantile(0.75)),\n", + " delta_max=('affine_residual_delta', 'max'),\n", + " mean_delta_over_interval_radius=('delta_over_interval_radius', 'mean'),\n", + " zero_crossing_fraction=('contains_zero', 'mean'),\n", + ")\n", + "distribution_summary" + ] + }, + { + "cell_type": "markdown", + "id": "worst-tanh-prime-intervals-title", + "metadata": {}, + "source": [ + "### Worst derivative residuals and their preactivation intervals\n", + "\n", + "The next table ranks all 150 neuron intervals by $\\delta$. It includes the preactivation interval and the ratio between the affine residual and the full interval radius." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "worst-tanh-prime-intervals", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layerneuronpreactivation_lowerpreactivation_upperpreactivation_widthcontains_zeroaffine_slope_paffine_intercept_qaffine_residual_deltainterval_radiusdelta_over_interval_radius
0332-1.5759401.6127543.188695True-0.0032340.5761220.4238800.4264870.993888
1343-1.5240391.5238543.047893True0.0000190.5865110.4134890.4135030.999965
2350-1.4613631.4989922.960355True-0.0042930.5937380.4062660.4094820.992148
3327-1.3853631.4635172.848880True-0.0101160.6039140.3961120.4035020.981685
4349-1.4221581.4027182.824875True0.0025840.6057460.3942560.3960930.995363
5346-1.4375271.3808222.818349True0.0075950.6065640.3934500.3989020.986333
639-1.4210501.3680522.789102True0.0073350.6093300.3906830.3958880.986853
7334-1.4163011.3674772.783777True0.0067970.6098150.3901970.3950050.987829
8331-1.3340871.4132852.747372True-0.0114860.6136920.3863410.3944410.979465
9338-1.3569221.3448922.701814True0.0018350.6178400.3821610.3834050.996754
1032-1.3489601.3183332.667293True0.0048530.6215060.3784990.3817700.991433
11320-1.3519641.3076692.659633True0.0070790.6224020.3776100.3823890.987502
12311-1.3063001.3521952.658495True-0.0073440.6225350.3774780.3824370.987034
13330-1.3663111.2888892.655201True0.0124360.6232060.3768330.3853090.978001
14321-1.3577271.2955942.653322True0.0100000.6232320.3767930.3835700.982334
\n", + "
" + ], + "text/plain": [ + " hidden_layer neuron ... interval_radius delta_over_interval_radius\n", + "0 3 32 ... 0.426487 0.993888\n", + "1 3 43 ... 0.413503 0.999965\n", + "2 3 50 ... 0.409482 0.992148\n", + "3 3 27 ... 0.403502 0.981685\n", + "4 3 49 ... 0.396093 0.995363\n", + "5 3 46 ... 0.398902 0.986333\n", + "6 3 9 ... 0.395888 0.986853\n", + "7 3 34 ... 0.395005 0.987829\n", + "8 3 31 ... 0.394441 0.979465\n", + "9 3 38 ... 0.383405 0.996754\n", + "10 3 2 ... 0.381770 0.991433\n", + "11 3 20 ... 0.382389 0.987502\n", + "12 3 11 ... 0.382437 0.987034\n", + "13 3 30 ... 0.385309 0.978001\n", + "14 3 21 ... 0.383570 0.982334\n", + "\n", + "[15 rows x 11 columns]" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worst_tanh_prime_intervals = (\n", + " tanh_prime_diagnostics\n", + " .sort_values('affine_residual_delta', ascending=False)\n", + " .head(15)\n", + " .reset_index(drop=True)\n", + ")\n", + "worst_tanh_prime_intervals" + ] + }, + { + "cell_type": "markdown", + "id": "tanh-prime-distribution-plots-title", + "metadata": {}, + "source": [ + "### Residual distributions\n", + "\n", + "The left panel shows the spread of $\\delta$ in every layer. The right panel compares $\\delta$ with the full interval radius; points near the diagonal retain almost no advantage over the constant interval enclosure." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "tanh-prime-distribution-plots", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABZQAAAIQCAYAAAAW1etxAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAASdAAAEnQB3mYfeAABAABJREFUeJzs3Xd4FFXbwOHfbEuvpBF676E3qQIWBAUFC9JEVBAVVETwtSAiCooooiAqgoBiQ4r4IQgiIiC9hE4oUkIJpPct5/sj7sq6G1JISAjP7cX1vpk5c84zZ2Z2Z589e0ZTSimEEEIIIYQQQgghhBBCiDzoSjoAIYQQQgghhBBCCCGEEDcGSSgLIYQQQgghhBBCCCGEyBdJKAshhBBCCCGEEEIIIYTIF0koCyGEEEIIIYQQQgghhMgXSSgLIYQQQgghhBBCCCGEyBdJKAshhBBCCCGEEEIIIYTIF0koCyGEEEIIIYQQQgghhMgXSSgLIYQQQgghhBBCCCGEyBdJKAshhBBCCCGEEEIIIYTIF0koCyGEEEIIIYQQQgghhMgXSSgLIYQQQgghhBBCCCGEyBdJKAtRRNq3b4+maY5/BoOBcuXK0bBhQwYNGsTixYsxm81ut/3ggw/QNI2FCxde56hzfPLJJ2iaxueff+60PCQkhIoVK5ZITHaPPPIImqaxb9++Eo3jWh07doy+ffsSGRmJTqfD09OzpEO6Ji1atEDTNJflmZmZvPzyy9SqVQuTyYSmafzyyy9s374dTdMYPnx4scV06NAhNE1jwIABxdZGURs+fDiapvHXX3+VdCjFLrdzJjepqalomkbnzp2LL6hcPPbYY2iaxu7du69720IIURhHjx5l+/btnDp1Kl/llVLExsayc+dOtm/f7rQuMzOTQ4cOsWPHDg4fPgzA4cOH2b59O1artchjv5Hs2rWL/fv3F2gbe98ppYopqtwVJl4hShM5h3NXmvsmt9e90hyzKDhJKAtRTKxWK/Hx8ezfv58FCxbQt29fGjVqxM6dO4u8rddffx1N01ixYkWR113cbuTY88tqtXLvvfeyePFizp07VyIfKAqqb9++aJrGyZMnC7TdxIkTeeutt4iJicn1C5TS4ocffkDTNCZPnlzSoQghhBCFlp2dTdu2bWnZsiW9e/fOs/xvv/1GjRo1qFChAs2bN6dly5ZYLBYA3n33XcqVK0e9evVo0aIFw4YNA2DYsGG0bNmSlJSU4tyVUq9Dhw706dOnQNsMHTqUli1bkpWVVUxR5a4w8YqyZceOHRw4cOCGjaEoz+HS0BdFqTRf37m97pXmmEXBGUo6ACHKmm3bttGiRQuUUqSkpHDs2DFWr17N9OnTOXz4MLfddhtbtmyhZs2ajm2effZZnn322RKLefjw4cU6cvRazJs3j3nz5pV0GNckJiaG6Oho2rZty48//khERERJh3TN/juaye7HH3/Ew8ODjRs30qRJE/R6vWPdjZBIF8Unt3NGCCHEtVmxYgWXL18GckZ/RUdH06hRI7dlzWYz/fv35/z580RGRhIREeH4dd3evXsZO3YsSilq166Nn58fderUAaBOnTqkpqZiMNzcHx+bNWtGWFhYSYchRL61bt2ahg0bluivrq4lhqK85kpDX9zs5DW0bLm57wiEKEaapuHv70/Tpk1p2rQpjz32GF26dGHv3r28+OKL/PjjjyUdorhOTp8+DeR8I1sWkslXc/r0aapUqULz5s1LOpSbypkzZyhfvrxTAl8IIcTN4csvvwRyRhHPnj2bL7/8kqlTp7ote/z4cc6fP0+3bt1YvXq101REmzdvRinF22+/zbhx45y2mz17dvHtwA3kjz/+KOkQhLipyDVXtsjxLFtkygshrpNy5co5RtouXbqUS5cuOdblNofy3r176devH1WrVsXLy4uqVavSq1cv1qxZ4yjTsGFDJkyYAMDdd9/tNI+zXc+ePdE0jTNnzvDJJ58QFRWFl5cXPXv2BHKfQ9kuPT2d0aNHU6lSJTw9PWnQoAEfffSRy4jTvKYQ8PT0pGHDhgWK/WpzKC9evJhOnToREBCAl5cXDRo04M033yQjI8OlrH0+6OzsbMaPH0+1atXw8PCgZs2aTJkypcCjZ6Ojo3nwwQcJDw/HZDJRqVIlHnvsMZe5CzVN47bbbgPgnXfecezfm2++ma92fv/9d+69914iIiLw9PSkTp06PPPMM44k9ZXWrVvHPffcQ2hoKCaTicqVK/PEE09w5swZl7K+vr7UrFmT9PR0xowZQ9WqVTEYDEydOhVN01i8eDEA1apVc8R85bH773y4vXv3RtM00tLSOHLkiGObFi1aAFx1DuXk5GRef/11GjZsiLe3N76+vrRt25YFCxa47ZPk5GSeffZZKlas6DinPvvss3z1p91DDz3E/fffD8BLL73kdO7ZzzWz2cwPP/zA3XffTdWqVfHw8KBChQoMGDCAI0eOuNS5Zs0aNE3jhRde4Pjx4/Tt25fg4GC8vb1p3759njdQy5Yto1WrVnh7exMSEsLAgQO5ePFinvvyzTffUKlSJcfPkq+mSZMmaJqG1WplypQp1K1bFw8PD6fjkp2dzfTp02nZsiV+fn54e3vTpEkTpk2b5nbuzEWLFtGhQwfCw8Px8fGhQYMGjBw5khMnTjiVy20O5aysLF577TWqVauGp6cntWrV4t1338Vms7ndh27duqFpGomJiS7r5s2bh6ZpfPTRR07Lz5w5w8SJE2nVqhVBQUF4e3vTuHFjJk+eXKCpWdatW0ePHj2oWLEiXl5e1KpVi4EDB7Jr16581yGEEEUpLi6OlStXUrFiRaZMmYKHhwdfffWV29frXbt28eeffwIQFBTEjh072L59u+Pfnj17ADAajU7Lr/x3Zb2nTp1i+/btpKWlAZCRkcGBAwfyPV1WZmYmhw8fZv/+/aSnpxdov921ffDgQbc/Z79w4QJ79+7l77//zrPeuLg49u/fT3x8vNv1ec3/mZ6ezv79+93ep10pKyuL7du3c/z4cbfrDx06dNVf9mRnZ3PkyBEOHTpUJFNppKWlsX37dmJiYnItc+bMGbZv355r3+TVz0optm/fzqFDhxzLzp8/z65duxwj7O2udhyute/yOsb5cenSJaKjo/O8T1NKcfz4caKjo0lISMi1zH/75dy5c0RHR5OcnJxnLLntT2JioqMfMjIynK7j2NhYl3ryc04VNNaCxuCOu2vuesRxLeez/fWpsNdTUV/fecX73+uvoO3n93UP3B/P0nBNi0JSQogi0a5dOwWobdu2XbVcvXr1FKCWLVvmWPb+++8rQC1YsMCxbM+ePcrT01MBbv/FxcUppZRq0KBBrmXsevTooQA1ePBgp/U9evRQSik1a9YsBajPPvvMKdZy5cqp8uXLq65du7qtf+TIkU7lv//+ewWot99+2+2+e3h4qAYNGjj+zk/s9pijo6Od6nr11Vdz3bZt27YqIyPDZV8iIyPV3Xff7XabCRMm5H7Q/mPNmjW5HpuQkBC1f/9+R9ncYpw4cWKe7UyZMkVpmuZ2+169ejmVnTp1aq5lw8LC1LFjx5zK+/j4qKpVq6qOHTs6lX333XdzjfnKY9e8eXOn49SrVy+32zRv3lwppdS2bdsUoIYNG+YUx/nz51XdunVzbXPUqFFO5TMyMhxt//ffwIEDFaD69++fZ98++OCDubZpP9cWLVqUaxl/f3918OBBpzp//fVXBag+ffqokJAQl21MJpPau3ev0zbDhg1TgHriiSfcttO0aVNlsViuui89e/ZUgBoxYkSe+924cWO3rwX245Kenu5yTlz575577lFWq9VR36RJk3It26lTJ6e2/3vOKKWU1WpVd95551WP53/rsb8eJSQkuOzf3LlzFaBmzJjhtLxTp05X3af/Gjp0qALUrl27HMuWLl2a6zVWrly5PPteCCGKwwcffKAA9fzzzyullOrdu7cC1M8//+xS1sfHJ9fXwvz+u/K11/4etnHjRvXqq6861V+vXj21fft2tzHHxsaqAQMGON1LGY1G1a9fP8f9bV7sbf/xxx9qzJgxjrYbN27sKLNkyRLVsGFDp/grVaqkvvjiC5f6PvvsM1WlShWnslWqVFHvv/++Sx/WqVPHZXuz2axGjx6tvLy8HNs3adJE7dq1y/EZ4cp706NHjypAPfjgg273r3Xr1gpQZrPZafnSpUvVbbfdpnQ6naMdDw8PNWTIEJWYmOhST27x/ldaWpry8/NT5cuXz/W+o3379krTNHX8+HGn5fnt54yMDAWo1q1bq+joaMc+Amru3LlKqfwdh8L2XX6P8dUsWbLEcS9l/1e9enU1Z84cp3I2m01NnTpVhYWFOcppmqY6d+7sci94Zb8cOXLEcb4AymAwqGHDhqns7GyXWPLanyVLluR6Hb/88suOegpyThU01vzGcDXuzuHijKMozudPPvmkUNdTcV3fecVrv/4K2n5BX/dyi7kkr2lxbSShLEQRyW9C+YEHHlCA+vjjjx3L3CWUn3/+eQWofv36qcOHD6vMzEx15swZtWLFCtWtWzd16dIlR9nx48crQP30009u27QnlA0Gg5owYYI6ceKEstlsjvVXSygDys/PT33++efq4sWL6uzZs2rq1KnKZDI5PkTYFTShnJ/Y3SWUt2zZojRNUwaDQb3xxhvq5MmTKj4+Xn333XcqNDRUAerVV191uy+hoaFqwYIF6sKFC+ry5cvqvffeU4AKDAx0eZNyJzMzU0VGRipA9e7dW0VHR6uUlBT1119/qVatWilAtWzZ0mmblStXKkCNHTs2z/rt/vzzT6VpmtLr9eqZZ55R+/btU2lpaerIkSNqxowZ6plnnnHqD51Op6pUqaIWLVqkzp8/rzIzM9WRI0fUk08+qQDVs2dPp/rtH7xCQkLUwoUL1YULF5zW9+nTRwHqxIkTbuNzlxxUKucYu7uxyS2hbE9EP/bYY2r37t0qNTVVxcfHq+XLl6uqVasqQP3111+O8hMnTlSAqlixolq+fLlKTExUx44dU6NGjXLcSOQnoaxU3ufrjz/+qAYNGqTWrVunzp07pzIyMtSBAwccfXrfffc5lbcnlAHVoUMHtXnzZpWamqqOHj2qunfvrgA1dOhQp23sH4gBNXr0aHXs2DGVmpqq1qxZo8qXL68AtXLlylz3ITs7W/n5+SlA7dmzJ899tn8I8vb2VtOnT1dnzpxxWj969GgFOUnWTZs2qaSkJJWSkqLWrVunmjVrpgC1cOFCR/nKlSsrvV6vZs6cqS5cuKAyMzPVwYMH1ccff6yGDBniVLe7c8aeAA4KClILFy5Uly9fVqdOnVJvvPGG0uv1CoomoTx48GA1depUtXfvXpWamqouX76sVq5cqerXr68AtXr1aqfy7hLKd911l+M6PnXqlMrKylLHjx9XX3/9tbrjjjvy6nohhCgW9tfmLVu2KKWU+vrrr3P9YN6hQwdVp04dx/1Q8+bNnf7Z729q1Kjhss7X19fltdf+Hta5c2fHa3ndunWVh4eHAlT58uVVamqqUwznzp1TlSpVciQqatWqperUqePYpkGDBiotLS3P/ba3bb//DggIUE2aNFEPP/ywUkqpGTNmON5fQ0NDVcOGDR3vq4D64IMPHHV9++23juXlypVTjRo1ctxP/jeJkVsC57HHHnOUDwkJUdWrV1eapqnQ0FBVu3Ztl8RKYRMo9uPn7e2t6tevr6pUqeJI/vz3/fJq8brz6KOPKkD93//9n8u6Y8eOKUB17NjRaXlB+tme0KpZs6YKDw9XOp1OVatWTTVv3lz99NNP+T4Ohem7ghzj3Ng/r0HOFyC1a9dW4eHhClDh4eFOZa+8Lw0MDFS1atVy3Nf4+fk5DT65sl8qVqyodDqdql69uqpYsaKjjtdee82p/vzsz++//+649/Ly8nK6nmfOnOmoqyDnVEFjzW8MV3O1hHJRx1GU53NhrqfivL7zircw7Rf0dS+3mEvqmhbXThLKQhSR/CaU7YmKyZMnO5ZdLaF8ZUIjN/lNKP93tKddXgnl/y5XSqnXXnvNkQi0u14J5eHDhytAvfDCCy7l7Um98uXLu90Xd2/qt956qwLUvn373MZwpaVLlyrIGQFz5UhNpZRKSEhwjEy9MrlXmITyQw89pAD10ksv5Vl2wIABCnAZ8WDXrl07ZTAYnD7U2RPKS5YscbvN9UgonzhxQoHraGu7devWKUCNGTPGsaxGjRoKUJs2bXIp37dvXwVFl1C+miZNmig/Pz+nL2bs516FChVUcnKyU/nz588rQDVs2NBpuf0D8aOPPurSxscff6wA9frrr+caxx9//KEgJ4GdH/aE8rvvvuuyLisrS/n4+KhGjRo57ZfdqVOnFKDuvfdex7LKlSurGjVq5Kttd+eMfeTw999/71L+2WefdXsDW5iEcm7++usvBTnJ/CtdLaGcnp6er7qFEKK4RUdHK0BVq1bNsSwlJUV5eXkpT09Pt6+TGzZsUIB66qmnXNbZv7R1d29gf712l1D28PBQixYtciw/f/684/3mm2++carHPrDi9ddfd3qvvHTpkurXr58C1LRp0/Lcd3vbRqPR6f5ZKaX+/vtvZTQaVdWqVdWGDRuc1q1fv16FhIQoX19flZSUpJRS6uGHH1aAmj59utO93alTp9Rzzz3nNMLQXTJk586djiTMjz/+6Fh+7Ngxp9GsRZFQfvnll9XWrVtd9rd9+/YKUOvWrXNaV5CE0/r16xWgHnroIZd19vv1K0fiFrSf7QktyBklefLkSadt8nscCtN3BTnG7hw7dkwZDAYFqPHjxzsdy+PHj6v//e9/jr/37NmjIOeXafPmzXMsv3DhguMepmvXro7lV/ZLhw4d1N9//+1Y98033yhARUREFKqvlFJKr9c7jdz/r4KcU4WJNT8xXM3VEspFGUdRn88FvZ6UKt7rO694C9p+YV73cou5JK5pUTRkDmUhrjP7vE7+/v5XLde3b190Oh1Dhw5l4cKFLnPzFoZ9ztiC0DSN/v37uywfNGgQQInMHbpz504ABg8e7LKuW7duVK5cmXPnznHhwgWndV5eXtxxxx0u29jnBs7PvEv2tgcMGIBO5/wSGhgYyH333Qdce7/89ddfADz99NN5lt28eTMATZs2xWAwoNfr0ev16HQ6NE1j48aNWCwWzp4967SdyWTinnvuuaY4r4V9H3/66ScMBoMjdp1Oh06n49ZbbwVwzMWYkpLCsWPHqFmzJm3btnWpz935cK2WL1/OXXfdRUREBEaj0THP8u7du0lJSSEpKcllm65du+Ln5+e0LDw8nNDQ0FzPsd69e7ssy895aZ9P/amnnsrvLgHuXwv27t1LWloa+/btw2g0uhyPypUrAzjNjfnAAw9w7NgxHnnkEVavXu22P65m165deHl50adPH5d1RXk87fN0N23aFD8/P8dxbNOmDYDbecb/64EHHgByjtWSJUvyNb+1EEIUJ/vD+B588EHHMl9fX3r06EFmZibffffddYljzJgxPPTQQ46/w8PDGTNmDJDz3mKXnJzM4sWLadSoEb169eLYsWPs2rWLnTt38vfff/PYY48BsGrVqny3PXz4cAYMGOC0bOHChZjNZoYPH46vry+7du1ytOPj40OvXr1ITU1l06ZNQM79G+TcQ155b1epUiWmTZuW5wNvf/jhB0c/3HvvvY7l1atXZ+HChW6fH1BYb775Ji1btiQtLY2DBw+yY8cOLl686HiP2rBhQ6Hr7tChA9WrV2fp0qVO7+dKKebPn4+3t7fT/UNB+9lOp9OxcOFCqlSp4rT8Wo/D1Vxr3QsXLsRisTBkyBBef/11PD09HeuqVavGpEmTHH9/++23AIwYMcLpXiYsLIxFixbh7+/P2rVrnZ6nAzlzl3/33XeO+y3IubZbtGjB+fPniYuLK7L9uVJhzqmCxJqXffv2uczV/t++yU1RxlHU53NBryco3us7r3gL2v71fN1zpzhfL0T+SUJZiOssOjoagIoVK161XNu2bVm3bh1hYWE8/vjjVKlShfDwcAYPHux0c14QlSpVKvA2wcHBeHl5uSyPjIwEKHACqSjY26xQoYLb9fa+/e9Du4KDg12SwJCTWAXcPsCmqNouqMTEREwmk6Ofr8b+IAWr1YrVasVms2Gz2VweNJidne30d4UKFdz2x/Vij9tmsznFrnJ+PeMoZ487r77PbXlhTZs2jV69erFy5UouXLiAxWJxKZOZmemyLDQ01G19JpMp13PM3Tb5OS9//fVXIiIiHF9k5Je71wL78VBK5et4AEyePJnp06eze/du7rjjDoKCgmjQoAHjx4/P12tDSkoKkZGRbm86i+p4ZmZm0r59eyZMmMDu3btJTU11WyYvgwcP5rvvviM9PZ3777+f8PBwqlatysiRI4vkCz8hhCgIq9XKV199BTgnlOHfL8DsCefi1rFjR5dlVatWBXJe5+327duH1WolOjqapk2b0rRpU5o1a0bz5s1p3rw5Xbt2BXIeFJVfXbp0cVm2e/duAMaNG+dow95OixYtmDNnjlM7w4cPx8fHh65duzJmzBiWLVuW7weGAY4HAd55550u6xo2bFio++/cbN++nVtvvRV/f3/q169PixYtaNmyJSNHjgS4pi87NU1j0KBBLl9GbNiwgRMnTnDfffc5fWFe0H62q1SpEjVr1nRp/1qPw9Vca932B1bmZ3DOwYMHAejRo4fLutDQUFq3bu1Uzq5WrVpERES4bOPuWirKvirMOVWQWPPSu3dvWrZs6fRv6dKl+dq2KOMo6vO5oNcTFO/1nVe8BW3/er7uuVOcrxci/wwlHYAQN5OdO3dy6NAhdDod7dq1y7N8x44d6dixI1lZWRw4cIA///yT6dOn8+2337J582aaNm1aoPYLkzyMj48nIyPDJalsf8EOCAhwLLN/E/jfxCXkvKEX1VNq7W2ePXuWoKAgl/X20YZXxlZUrmzbnaJqOzAwkMTERGJjY/NMKgcEBJCSksKlS5cc39bmR0kmk+HfPho5ciTTp0/Ps7x9VH9ufZ/b8sIwm82OESjvvfcePXr0IDw8HA8PDzRNo3Pnzqxfv77I2iuMpKQktm7dyssvv4zRaCzQtu6Ovf143HXXXfz888/5qkev1zNy5EhGjhxJQkICO3fuZPXq1UyZMoXly5ezZcsWR2LcHT8/P86dO4dSyiWpnNvxvNrrjLsRLd988w3R0dG0aNGCqVOn0rBhQwIDA9Hr9Rw6dIh69erla18h54Pk/fffT1paGnv37uX3339n2rRpfPvtt+zZs8fthxohhCgOq1ev5ty5c4SHh2OxWNi+fbtjXXh4OAaDgU2bNnH06FFq1apVrLH8NykC/75W22w2xzL7F3ohISFuR8fZ5ZbscMfd6669nQYNGjiNJP0v+z1ko0aNOHDgADNnzmT16tVMnz4ds9lMkyZNeO2115xG37lj/1KyXLlybteXK1euSL54jImJoVOnTqSnp2M0GqlatSr+/v7o9XrS09M5ePAgZrP5mtoYPHgwEyZM4Msvv+Txxx8H/v1i4r+/HCpoP9vl9l55rcfhaq617oyMDCBnlHFe7GWDg4PdrrefJ/Zydu6uI3B/LRVVXxX2nCpIrHlp1KiRy+eX3AZnFGccRX0+Q8Gup+txfV8t3oK2f71e93JTnK8XIv9khLIQ10l8fDxDhgwB4L777sv1JsMdDw8PmjZtyjPPPMOnn35KVlYW8+bNc6w3GHK+GyqqhO2VlFIsWrTIZfnChQsBnJLa9jfY/fv3u5Rfvny52/oLE7u9TXcjb9atW8epU6coX758sSR37G0vXLjQ5SYlKSmJJUuWANCsWbNrasf+U/yPPvooX2WtViuffvrpNbV5peI8p+zs+/jDDz/k66dt/v7+1KhRg5iYGLZs2eKyfv78+QVq/2r7eOHCBVJSUujQoQMjRoygSpUqeHp6omka586dY8eOHQVqqzisW7cOTdN44okniqS+xo0b4+Xlxbp16zh8+HCBtw8KCqJr165MmTKFRx99lN27d7s9Tldq2rQp6enpjuvmSrkdz9xeZ5RSbl9nYmJiAHjppZfo1KkT5cqVc3zQ+Omnn/LeMTd8fHxo27YtL730EhMmTODixYtu90EIIYqL/R7owoULLqP7OnXq5PhVTUHfG4uTPRlXt25dl5+4X/nvm2++KZJ2JkyYcNV2evXq5dimcuXKTJ48mZ07d5Kamsr69evx9fWlb9++/Pnnn1dtLyQkBPj3/eZKNpuN48ePuyy334OkpaW5rdNdImbu3Lmkp6czbNgwEhMTHVOGbN++nffee++qMeZX1apV6dSpExs3biQmJoaMjAy+//57KlWq5DIavDD9nJf8HIfC9F1+685NeHg48O+ozKuxJ0PdnQ8AR48edSpXWNeyP3bX45zKy5IlS67pnCkqxXE+F+R6KuljUdD2C/O6l5uSuKZF0ZCEshDFRClFSkoKe/bs4Z133qFRo0bs3buXcuXKMWXKlDy3f/zxx3nxxRfZtGkTFy9eJCsri0OHDjkSjFcmM+03OcuWLbvmqRbcee6555g7dy6XLl3i3LlzfPDBB7z99tsAjiQ55CSHTCYTixcv5osvviApKYkLFy7w+eef5zrHa2Fit7f5wQcfMHHiRE6dOkVCQgKLFy+mX79+AAwdOrSwu3tV3bt3p3z58uzZs4e+ffuyb98+0tLS2Lp1K927dycuLo4WLVoQFRV1Te08/fTTaJrGO++8w6hRozhw4ADp6enExMTw8ccfO356BPDss8+iaRr/+9//eOGFF9i/fz/p6enExcWxdetWXnvtNUe/5Jf9uHz//fekp6df077kpmbNmtx9993ExsbSuXNnFi9ezLlz58jMzCQmJobFixdzzz33OOYJhn+/zb///vv5+eefSU5O5sSJE4wePZrvv/++QO3b9/HXX391jJK1i4iIwMPDg82bN7N48WJSU1NJSEjg//7v/7j99tvdTptwva1Zs4bevXsX2dQQXl5eDB8+nIyMDG677Ta++OILTp8+TVZWFidPnmTlypUMGDDA8ZO/rKwsWrZsycyZM9m3bx+pqamkpqby22+/OUY45zUyxH48n3jiCRYtWkR8fDxnzpxh0qRJzJgxw+029p+KPv/88+zatYuMjAwOHDjAwIED3c4tZx8F9/HHH3PgwAGysrL4+++/eeutt3jllVfy3T/du3dn0qRJ7Nixg4SEBDIyMtixY4cjWVOQUTBCCHEtkpKSWLZsGQBRUVGOKSOu/Fe3bl0AFixY4DIFVklp1KgRoaGhbN68mdWrV+dariA/U3fHnqiZMmVKrlMaXdnGf9szmUx07NiR5557DpvNxooVK67a3i233ALg9tdW9vvh/woNDUWn07Fz506XX9z83//9H+fOnXPZxj4f7KBBg/D29nZaZ39vLgr29+b58+fz448/kpKSwsCBA11+3VTQfs5Lfo9DYfruWo+x/bkekyZNcntffOUy+/nw/vvvu9wbrFu3jh07duDv7+94VkZhFGR/PDw8cr1vvV7n1NViuJ6uFkdRn892+b2ertexyE1B2y/M615uSuKaFkWkJJ4EKERZ1K5dO8fTTHP7V7duXbVr1y6Xbd9//30FOD2lukePHrnW4+Pjo/bu3esoe+LECceTh6/899+6Tp8+7Tb2WbNmKUB99tlnTsvLlSunypcvr7p06eI2jpEjR7rU9cwzz7gtO2jQIOXh4aEaNGjgVD6v2AcPHqwAFR0d7bTd//73v1z7p3Xr1io9Pd1lXypUqOB2/0ePHu32ybm5WbVqlfLw8HDbdnBwsEusK1euVIAaO3Zsvuq3mzx5cq772KtXL6eyH374odLpdLmWb9eunVN5Hx8fVaNGjVzb/uWXX1zquPLYNW/e3Ok42Xl4eLh92vC2bdsUoIYNG+a0PC4uTkVFRV31ulm5cqWjfEZGhmrWrJnbcgMHDlSA6t+//1X71S4rK0uFhYW51GM/fqNGjXLbTq1atVSHDh0UoM6dO+eo79dff1WAGj16tNv2KlSooMLDw52W2Z9Sv3nzZpfymzdvVoB66qmn3NbXtGlT9dtvv+VrX+3sT13OTWZmprrtttuuejxmzJihlHJ+WrS7f61atXJ6GrO7c8Zqtao777zzqsezU6dOTttcvnxZhYeHu5Q3Go1qyJAhTjHay5cvX95tG0OHDnV7PdmXX/l6XaNGjVz3tUKFCiouLq5Ax0IIIQpr9uzZClD16tVTNpvNbZmMjAwVEhKiAKf3ig0bNuT63jJx4kQFqCVLlris69SpkwJUQkKCY1lh3sM+/PBDBSgPDw/15JNPqh9//FH9+eef6vvvv1dvvPGGqlOnjtNreG6u1nZWVpaqW7euAlSdOnXUhx9+qH799Ve1du1aNWfOHDV48GDl7+/vKN+xY0d1xx13qI8++kitXLlS/fnnn2ru3Lmqdu3aClCTJ092lPXx8XG5z0lISFDBwcEKUN27d1eLFy9Wv/76q3rppZeUwWBQer1eASojI8NpuzZt2ihA9ejRQ/3888/q119/Va+99pry9vZ23Btf+T46bdo0BaioqCi1aNEi9eeff6pFixapbt26Ocr/9z7LXbx5SUlJUT4+Pqpq1aqqW7duClCHDh265n623ze0bt3abbsFOQ4F7buC1O1OZmamqlmzpgJU9erV1UcffaR+++03tXTpUvXiiy869XFiYqLj2mvfvr365ptv1G+//abeeust5ePjowA1bty4fPfLgw8+qAB19OjRQu1P3bp1laZpasaMGWrTpk1q27Zt6uzZs0qpgp9ThYk1rxjy4u4cLo44ivp8tsvv9VTc13de8Ra0/cK+7uUW8/W+pkXRkISyEEXkvwllvV6vgoKCVP369dXAgQPVDz/84PQieCV3CeVz586pd999V7Vr106FhoYqb29vVbNmTTVs2DB1+PBhlzoWL16smjdvrry8vBwx2F1LQrlChQoqNTVVjRw5UkVGRiqTyaTq1q2rPvzwQ7cfYsxms3rllVdU5cqVldFoVNWrV1eTJk1SFovFbUI5r9hzSygrpdS3336rOnTooPz8/JSHh4eqV6+emjBhgkpLS3MpW5QJZaWU2rVrl+rbt68KDQ1VRqNRVahQQT366KPqxIkTLmULm1BWSqk1a9aoHj16qHLlyjmStSNHjlSnTp1yKbtlyxbVr18/VaFCBWU0GlV4eLhq06aNevPNN12OfV4JZaWU+uCDD1TdunWVyWRSUDwJZaWUSk9PV1OnTlWtWrVSvr6+ysvLS9WuXVs98MAD6qefflIWi8WpfGJioho5cqQqX7688vDwUPXr11ezZs1SBw8eLFBCWSml/vrrL9W5c2fl5+fnklDOyspSkyZNUrVr11YeHh4qMjJSPfHEEyouLs5xTZVkQjklJSXf+2mXV0JZqZwk75w5c1SnTp1UYGCg8vDwUNWrV1c9e/ZUX3/9tdON4bZt29TTTz+tGjZsqHx8fFRwcLBq1qyZevfdd13iy+2cycjIUK+++qqqUqWKMplMqkaNGuqtt95SSUlJbhPKSil18OBB1bNnT+Xr66t8fHxU586d1caNG9XcuXNdEspKKXXkyBF13333qbCwMOXt7a2aNm2qPv/8c3X69Ol8J5RjYmLUK6+8olq0aKECAwOVn5+fql+/vho3bpzTeSCEEMXtlltuUYCaPXv2Vcu98sorClCDBw92LCvphLJSSr344otX/ULyyy+/vOp+5dW2UkodO3ZM1atXL9c2goODHWXvuuuuXMs1aNDAaZ9zS4b8/PPPjvulK//dc889juP138TK2rVrldFodNlm0KBBqnXr1i4JlOTkZFW9enWX8l5eXo5BCEWRUFbq3/tvQLVp0ybXcgXp57wSWgU5DgXtu4LUnZuDBw+qqlWruq3jv/fUv/32m9O95ZX/7rjjDpWZmZnvfnGXHC3I/rzxxhsuZV5++WWlVMHPqcImcq8WQ16KMqGcVxxFeT5fKT/XU3Ff33nFW5j2C/O6l1vMJXFNi2unKVVKfgMlhBBCCCGEECJXcXFx3HXXXXh6erJ69WqXhyZf6fz589xzzz34+Pjw66+/YjAY2LNnD0OHDuXBBx9kzJgxTuW/+OILZs6cyXvvvUenTp2c1g0bNowdO3bw+++/4+vrC8Dbb7/N4sWLmTdvnsvP9/ft28cjjzzith2APXv2sGDBAvbs2UNGRgYVKlSgfv36PPzww/l6iODV2rbLzs5m0aJFrFq1ilOnTuHl5UWVKlXo3Lkzffr0cfSdzWZjzZo1/PTTTxw9epTMzEwqVqzIHXfcwYMPPuj0cNmOHTsSFhbGDz/84NLe/v37mT59OgcOHMDf358ePXowfPhwhg0bxu7du9m0aZPLg2q3bt3KRx99xLFjxwgODqZPnz4MHjyYRx99lOjoaLZs2eKY9x8gMTGR9957j23btpGZmUnDhg0d08oNHDiQPn368NJLL+Ur3qvZuHEjo0aNAnKmmHr44YdzLZvffs7OzuaWW26hUaNGzJ0716WeghyHgvZdQevOTVpaGnPnzmXNmjXExcVRvnx52rZty+OPP+54eLTdmTNnmDVrFlu3biUtLY1KlSrRu3dvHnzwQafpDvLql5dffplVq1axdOlSKlasWOD9sdlsfPrpp/z888/ExcVhsVgYOnQoTz75JFCwc6owseYnhqtxdw4XZxxFdT5fKb/XU3Fe3/mJt6DtQ8Ff964Wc0lc0+LaSEJZCCGEEEIIIYQQQgghRL7IQ/mEEEIIIYQQQgghhBBC5IsklIUQQgghhBBCCCGEEELkiySUhRBCCCGEEEIIIYQQQuSLJJSFEEIIIYQQQgghhBBC5IsklIUQQgghhBBCCCGEEELkiySUhRBCCCGEEEIIIYQQQuSLJJSFEEIIIYQQQgghhBBC5IsklIUQQgghhBBCCCGEEELki6GkA7iZJSYmsn79eipVqoSHh0dJhyOEEEIIcVPLysri9OnTdOrUicDAwJIOp0yT+2AhhBBCiNKjoPfBklAuQevXr6d3794lHYYQQgghhLjC0qVL6dWrV0mHUabJfbAQQgghROmT3/tgSSiXoEqVKgE5B6tmzZolHI0QQgghxM0tJiaG3r17O+7RRPGx9/EPP/xA3bp1Szia0sNisZCSkoKfnx8Gg3xUs5N+cU/6xT3pF/ekX9yTfnFP+sW9stQv27Zt46WXXuL8+fO0b9+eRx99lEcffTTf98E39t7f4Ow/76tZsyYNGjQo4WiEEEIIIQQgUzBcB3If7J7FYiExMZHAwMAb/oNqUZJ+cU/6xT3pF/ekX9yTfnFP+sW9stIvn3zyCSNGjMDLy4tZs2YxbNgwDhw4AOT/PvjG3XshhBBCCCGEEEIIIYQQ+da1a1e6dOnCrFmzqFWrVqHqkISyEEIIIYQQQgghhBBClEFWq5WpU6dSuXJl+vXrR61atVizZs011akrotiEEEIIIYQQQgghhBBClBLHjx+nc+fOjBs3jhkzZqCUKpJ6JaEshBBCCCGEEEIIIYQQZYRSijlz5tC4cWP+/PNPRo0axdq1a9E0rUjqlykvhBBCCCGEEKWOUoqUlBSSk5Mxm81FNqKmNLPZbJjNZhITE9HpZOyPnfSLe4XpF51Oh4eHB+Hh4dKXQghRhg0aNIiFCxdSsWJFlixZQrdu3Yq0fkkoCyGEEEIIIUoVi8XC2bNnSU9PB8BgMKDT6YpsVE1ppWkaRqOxzO9nQUm/uFfQflFKkZ2dTUZGBllZWVSuXFmSykIIUUZ16dIFm83GRx99RFBQUJHXLwllIYQQQgghRKmSkJBAeno6AQEBhIWFYTDcHB9blFJYLBYMBoMkT68g/eJeYfpFKcXFixeJj4/nwoULlC9fvpijFEIIcT2kpKQwbtw4XnjhBapVq8YjjzzCkCFDiq29m+POTAghhBBCCHHDSE1NRa/XU758eUkgClGENE0jLCyMpKQksrKySjocIYQQhaSUIvpSNJtjN7Nnyx6+f+N7Lp29hJ+fH5MnTy72+6dSm1DOzs7m008/5ffff8disdCiRQueeeYZAgICClzXBx98wIoVK2jUqBHvv/++0zqz2czy5ctZt24dp06dIjIyki5dunD//fe7dP78+fOZP3++2za+//77YhlCLoQQQgghxM1GKSWjUYUoJpqmodfrsdlsJR2KEEKIQkjOTmb6julsP7ud3Qt2c+qnU+iMOho/0Rjve7xJzk7G3+RfrDGUyoRyRkYGXbt25dy5c4wfPx4fHx/eeecd5s6dy6ZNmwgPD893XVu3buXFF19EKUVmZqbL+ooVK2KxWHjmmWe4/fbbOXDgAMOGDWPGjBmsWrUKb29vR9njx4+zdu1afvrpJzw9PZ3q8fHxKfwOCyGEEEIIIYQQ14l8WSOEEDcmpRTTd0zntyO/sWP8DlL+TiGkdghtR7fFGmLlz9g/0XZovNLmlWJ9rS+VCeVp06bx119/sXfvXho2bAhA165dqVGjBmPHjmXevHn5qicjI4NBgwbx2GOP8eOPP7otU6NGDRYvXuyYO+qee+6hWbNm3HHHHUyfPp2XXnrJZZvOnTvj6+tbuJ0TQgghhBBCCCGEEEKIAoq+FM2ui7vw8vUitHooNTrUoPFDjdEZch6yeiHtArsu7iL6UjRRoVHFFkepfKTr3LlzadWqlSOZDBAcHMy9997Ld99953jac17GjBlDWloaU6ZMybXM2rVrXR5E0KVLFzRNY8uWLYXbASGEEEIIIYTIxaFDhzhy5EhJhyGEEEKIG8jJkyd5dtSzJKQnUM6rHB1f7EjTAU0dyWSAYM9gkrKS2By7uVhjKXUJ5YSEBI4dO0bTpk1d1jVt2pSMjAz27t2bZz2//vorM2fO5JNPPsHPzy/Xcl5eXi7L9u3bh1KK0NBQt9s89thjdO/enYEDBzJnzhx5mIEQQgghhBAi3x555BFGjBhR0mFcswMHDrB9+3ays7NLOhQhhBCizFJKMW/ePKKioli3aB2Xoy9j1BvdTmlh1BuxYSMlO6VYYyp1U17ExsYCuE3mhoWFAXDu3Lmr1pGQkMCQIUPo168fPXr0KFD7NpuNF154Aci50buSpmn07t2bLl26EBoaytatW3nmmWeYMWMGa9eupVy5crnWe/HiReLi4pyWxcTEAGCxWLBYLAWKUwghhBBCFC25HxMib0eOHOGzzz7jhx9+ID4+nuTkZE6cOEHVqlVLOjQhhBCizImLi2PYsGEsWbKE8uXL88iUR9hfbj9mqxmj3uhS3mw1o0OHnyn3wbVFodQllM1mMwAGg2toRmNOR+X1DfhTTz1FVlYW06dPL3D7L7/8MmvXrmXkyJG0a9fOad3zzz/vNNr53nvvpXv37nTu3JmxY8fy+eef51rvzJkzmTBhgtt1KSkpJCYmFjhWIYQQQghRdFJSinckhxBlwZYtWwgLC+P3339nxowZvPfeeyUdkhBCCFEmbd++nR49enDx4kUefPBBZs6cyRnrGV7f9DrxmfGE+4S7bBOfGU+ARwBtI9sWa2ylLqHs7+8PQFpamss6+02+vYw7S5cuZdGiRXz11VeEhIQUqO13332XyZMn8+CDDzJt2jSX9e6mzujYsSNNmzZl+fLlV617xIgR3H///U7LYmJi6N27N35+fgQGBhYoViGEEEKUnHvvvZfjx48XevtTp04BULly5UJtX716dZYsWVLo9oV7V5smTZQNSil2n07kz6OXSM404+9ppH2tEJpUCizWJ6Hnx7Fjx4iPj8dqtWIymQgJCXF5jdi3bx8Gg4G6deu6bH/u3DnOnj1L48aNHQNxIGfk/bFjx7BYLNSqVQuTyeS03f79+/H09KRGjRpkZ2dz7NgxfHx8cn19GjhwYBHsrRBCCCHyUrNmTSIiIvjggw/o168fAEEqiKZhTdlwdgMX0i4Q7BmMUW/EbDUTnxmP2WamTfk2NAppVKyxlbqEcuXKlfHw8ODEiRMu6+zLateunev2u3fvRq/X88UXX/DFF184lickJLBv3z66detGv379GDp0qNN2H330ES+++CJ9+vRh4cKF6PX6fMccEhLC/v37r1omLCzMMWXHfxkMBrcjsoUQQghROl1r4sn+i6xraV/uHYqe9GnZlpRh5p1fDrHj7wQS0rOx2UCng5+jz9G8ShAv3lmXAC/Xn45eL7NmzeL3339HKYXZbObkyZMEBwcza9YsunfvDsCMGTNYuHAhZ8+edRmQ8vDDD3PixAnHZyar1crEiRN5//338fLywtPTk/j4eF5++WXGjh3r2K5Pnz7UrVuXQYMG8fzzz2Oz2bjrrrv45JNPrtu+CyGEECLHxo0b2bVrF08//TSBgYHs2rULne7fR+Bpmsao5qMA2HVxF7GpsdiwoUNHgEcAbcq3YVTzUcX+RXmpu2s2GAx06dKFP/74A4vF4nRjv2bNGmrWrEmNGjVy3X7QoEG0b9/eZfnOnTupUKEC48aNc5nf67PPPmPkyJHcd999fPPNNwX6MJGZmcmePXuoU6dOvrcRQgghxI0tr18m5aVBgwYAeX4hLYQoGkop3vnlEL8fjsOo16gY5IVRr8NstXE5NZvfD+c86+TN3g1LbKTy1KlTUUo5PgNlZWUxZswYHnroIQ4cOECFChV46qmn+PTTT/nyyy8ZNWqUY9tDhw7x+++/M2HCBEf8I0eOZM6cOXz33Xfcc889AKxcuZJevXrh4+PD008/7dg+JiaG5cuXc/jwYTw8PNi6dev13XkhhBDiJqGUIvpSNJtjN5OSnYKfyY+2kW2p41+HCRMmMGXKFLy9vXnooYcICQlxSibb+Zv8eaXNK27raRTS6Lrcy5S6hDLASy+9ROfOnZk0aRLjx48H4LvvvmPjxo18+eWXTmXnzZvHwoUL+eCDD2jYsCHVq1enevXqLnWaTCYCAgLo1q2b0/KvvvqK4cOH07t376smk202G2+//TZPP/00AQEBQM60HE899RQXLlzg/fffL4pdF0IIIYQQQhSx3acT2fF3Aka9RkSAp2O5Ua8jIsCTc0mZ7Pg7gd2nE2laOagEI4ULFy5w6dIlLBYLPXr04KOPPuL333+nf//+REVF0a5dO2bNmuWUUJ45cyY6nc7xUPETJ07wySef8OKLLzqSyQDdu3dn4MCBjs81didOnOCPP/7Aw8MDgFatWl2fnRVCCCFuIsnZyUzfMZ1dF3eRlJXkGFn8/frv2f/Rfs4fPU/z5s1ZsGBBntP4appGVGgUUaFR1yl6Z6UyodyhQwfmz5/PM888wzfffIO3tzcHDx7k7bffZtCgQU5lY2JiWLt2baEeapeUlMQjjzyCzWbj8uXLjp+S2dWvX58PP/wQyDlQFouFhg0b4uvrS0BAAAcPHsTPz4+5c+c65jIRQgghhBBClC5/Hr1EQno2FYO83K4v52PibGIGfx69VGIJZfuDwWNiYqhWrRo+Pj4opQD4+++/HeVGjBhB//79+e233+jSpQvp6enMnz+f2267zTHv8fr167HZbFSpUoXdu3ejlHLUFRISQmxsLOfOnaN8+fJAzq8mgoODr/MeCyGEEDcPpRTTd0xnw9kNGHVGIn0jMeqNHPrlEFs+3oKyKjoN6cTqT1a7PO+gNCqVCWWA/v3707dvX/bu3etI5Lp7UMqQIUPo3LkzjRpdfbLp77//Hi8v5xtIb29vVq5cmes29pHIkJNQHj9+PK+88grHjh3j/PnzhIWFUbt2bbfDz4UQQgghhBClQ3KmGZstZ0SyOyaDDpvKKVcSzp8/zz333MNdd93Fxo0bCQgIQNM0zp8/T/ny5bHZbI6yffv25fnnn2fmzJl06dKFr776iqSkJKdnxCQnJwM5z4n5/PPPXdpr3rw52dnZjr8jIiKKce+EEEIIEX0pml0Xd2HUGQn3CXcsD6wYiF+EHw1GNCC4XjCHkg6V2Kjjgii1CWUADw8PWrZsedUyNWrUuOqcynYdOnRwWWY0Gl2mwMiLXq+ndu3aV30woBBCCCGEEKL08Pc0otOB2Wpzm1TOttjQaTnlSsKGDRtIT0/nueeew8fHx7H80KFDLmVNJhNDhw7lnXfeITY2llmzZlGuXDl69erlKFOpUiUA3nnnHe66664825cBMkIIIUTx2hy7maSsJMr7lCdmTQx+EX6ENwwnomEEvT/pjRUrsamxbI7dfEMklOXOQQghhBBCCFGmta8VQpC3icup2W7XX07LJsjbRPtaV5+vsLgEBgYCuEzj9/HHH7stP3z4cJRSPPHEE+zatYuBAwc6/Tz2tttuIzQ0lJkzZ7rdPi0trUjiFkIIIUT+pGSnkJmcyZ+T/2TD1A1s/2K7YzoqnV6HUW/Eho2U7JQSjjR/SvUIZSGEEEIIIYS4Vk0qBdK8ShC/H47jXFIm5XxMmAw6si02LqdlY7EqmtcMokmlwBKJr3379tSpU4dRo0bx5ptv4u/vz/z586lTp47b8pUqVeLuu+9m6dKlADz66KNO6319ffn666+57777uO2223jssceoUKECp06dYt26dZw8eZJff/21ULEmJydz5MgRIOcBggDR0dFcunQJT09PGjZsWKh6hRBCiLLs761/s/2V7WQnZlO1fVXajmyLpmmO9WarGR06/Eyu0/2WRpJQFkIIIYQQQpRpmqbx4p11AdjxdwJnEzOwKdBpEORtonnNIF68s67TB7viVK9ePby9vR1/e3l5sX79et566y1mzpyJn58fDz/8MPfeey+//PILkZGRLnUMHz6cpUuX0rJlS7fPk+nWrRv79+/n008/Zc6cOaSnp1OtWjW6dOnCRx995CjXsGFDqlatmu/Y9+/fzzPPPOP4u3nz5kyYMAHISXQvWbIk33UJIYQQZZVSiuhL0Ww6u4kFby1g0/ebMHgbiBoZRbPuzVzuOeIz4wnwCKBtZNsSirhgJKEshBBCCCGEKPMCvIy82bshu08n8ufRSyRnmvH3NNK+VghNKgVet2QywNy5c12WhYeH88EHH2CxWDAYDI54tm/f7raO8+fPAzg9jO+/KlWqxMSJE68ayw8//JDfsAFo27ZtrjEJIYQQApKzk5m+Yzq7Lu4iKSuJs7azBDUMosawGmhBGufTzhPiFYJRb8RsNROfGY/ZZqZN+TY0CnH9krg0koSyEEIIIYQQ4qagaRpNKwfRtHJQSYdyzb755hv8/Pzo169fSYcihBBCiH8opZi2ZRpfzf6KandUI7JcJJUerYTFZuFy1mXiM+NJzk4my5KF0hQ6dAR4BNCmfBtGNR91Xb/gvhaSUBZCCCGEEEKIG4DNZmPnzp1s2bKFVatWMWHCBPz9/Us6LCGEEEL8Y9mmZUwfMp3EY4n42HyoOLgiACa9ifLG8ug0HQpF64jW+Bh98DP50TayLY1CGt0wyWSQhLIQQgghhBBC3BCysrIYPnw4Pj4+TJw4kXHjxpV0SEIIIYQg50vfGTNmMGbsGMzZZho+0JDGDzd2KRfsGUxsaiwRPhEMazysBCItGpJQFkIIIYQQ4gZ14MABZs2axdGjRwkODqZv377cd999Ba4nJSWFfv36kZqayssvv8xtt91WbG2JwvPy8pL5i4UQQohSJj4+ngceeIC1a9dSrmI5Kj9RmSatmrgta9QbsWEjJTvl+gZZxHQlHYAQQgghhBCi4NauXUuzZs1ITExk1KhRtGzZkoEDB/Lss88WuK5nn32WNWvWsH79ei5cuFCsbQkhhBBClCX+/v6kpaXxxBNP8Op3rxJUJwiz1ey2rNlqRocOP5PfdY6yaMkIZSGEEEIIIW4wFouFRx99lLZt27JgwQIAunfvjpeXF08++SQPPPAAt9xyS77qWr58OfPnz+fll19mwoQJxdqWEEIIIURZEB8fz1dffcXTTz+NwWDgt99+w8vLi71xe1l/cT3xmfGE+4S7bpcZT4BHAG0j25ZA1EVHRigLIYQQQghxg1m3bh2nTp1iyJAhTssHDhyI0Whk7ty5+aonLi6Oxx9/nDFjxtCkSZNibUsIIYQQoixYvXo1DRs2ZOTIkaxfvx7ImZYKoFFII5qGNcVsM3Mh7YJjpLLZ+s/fNjNNw5rSKKRRicVfFCShLIQQQgghxA1m27ZtADRu7PywFx8fH2rVquVYn5dhw4YRFBTE+PHji70tIYQQQogbWXp6OmPHjqVHjx6kpqYyb948OnXq5FRG0zRGNR9Fhwod8NB7EJsay4mkE8SmxuKh96BDhQ6Maj4KTdNKaC+Khkx5IYQQQgghxA3m3LlzAISGhrqsCwsL48CBA3nWMW/ePJYuXcr69evx8PAotrYuXrxIXFyc07KYmBgArFYrFovFZRubzYamaSil8tyPssS+vzfbfudF+sW9a+kXpRRKKbfX343OYrHk+tpyM5N+cU/6xT3pF1d79+7loYce4ujRo7Rv354vvviCNL80Ptv9GSnZKfiZ/GhVvhUNyjXAW+fNuBbj2H95P1vPbXVZr2laqevbgsYjCWUhhBBCCCFuMGZzzs8nDQbX23mDwUB2dvZVt//7778ZNWoUTz75JB06dCjWtmbOnOl2bmaAlJQUEhMT3bZpNBpL3Yet68FqtZZ0CKWS9It7he0XpRRms9nt9Xejs1qtpKamAqDX60s4mtJD+sU96Rf3btZ+UUpxPOk4By4dIN2SjrfBm/oh9akeUB2Ay5cvM27cOIYOH8ryk8s5mniUtOw0FAoNjV2ndlErsBZ9avfBx+hDRUNFKlaq6NRGUlJSSexanlJSUgpUXhLKQgghhBBC3GD8/f0BSE1NJSwszGldamqqY31uHn/8cQICApg8eXKxtzVixAjuv/9+p2UxMTH07t0bPz8/AgMDXbZJTExE0zS3SeyyzD7SVK/X3/A/hS1K0i/uXUu/aJqG0Wh0e/3d6OxfRAUEBNx0ryFXI/3invSLezdjv6Rkp/Dx7o/ZG7eXpKwkbNjIiM0g+0I2tW+pTYOQBoxfOp6WYS359vS3bIzbiFFnJMg7CKPeiNlqJjYrlr/j/ibdmM7YlmNvqPcsPz+/ApW/Oc4KIYQQQgghypDatWsDcOLECapXr+607sSJE0RFRV11+71792Iymbj77rsdyy5dugTApEmT+Pzzz/nmm2+IiIi45rbCwsJcEtF2er3e7QdVnS7nUS830gexgrj//vvx9vbmyy+/dLte07Qyu+/XQvrFvcL0i32bspoosr+2lNX9KyzpF/ekX9y7mfpFKcVHez5iQ+wGjDojod6hHPm/I+yYswPNoJFeNZ1DSYcINgazJ24Px7KO4WXyopxPOQCsWNEZdJQzlONC2gV2xu3kYOJBokKvfo9UmhT0OJf9s0IIIYQQQogy5vbbb0fTNNauXUvXrl0dy6Ojo7lw4QJ33nnnVbf/8ccfXaaq+PPPP3n11Ve57777uO222xwjF6+1LeHq9OnT+Pr6lnQYhXLu3DnmzZvHhg0bOHPmDBUrVuSOO+5g+PDhV52LWwghhCitoi9Fs+viLow6I74Zvqx7cx2xu2IxhZqoNqwaHt4e6DQdgZ6BpGemk2pOBV1OIvq/X+gFewYTmxrL5tjNN1RCuaAkoSyEEEIIIcQNpnLlygwePJiZM2cyYMAA6tevT1ZWFqNHj6Z8+fI89thjTuV79epFcHAwc+fOBeCWW25xqdM+l2q9evXo3LlzodsSZVdKSgqVK1emTp06/O9//6Nhw4bs3r2bsWPH8sUXX7Bx48YbNlEuhBDi5rU5djNJWUmk7khl9cersaRZCOoYRIWHK+Dt642GRrYtm3RLOiH6EADSLemkmdPwNTm/7xn1RmzYSMku2JzENxpJKAshhBBCCHEDmjlzJmazmebNmxMVFcXJkycJDQ1l1apVLvMab9y4kYiIiOvSlii7rFYr/fv3Z/bs2Y7RyFFRUURERHDHHXcwa9YsxowZU8JRCiGEEAVzOfMySdlJxCXFoXSKyqMq49805/4m3ZKOUWdEobDYLOj0OnTosNgsJGcnuySUzVYzOnT4mQo2J/GNRhLKQgghhBBC3IC8vLxYuHAh06ZN49ixYwQHB1OnTh23ZZctW4bJZLpqfe3bt2fdunXUq1fvmtoq1ZSCszvg2DrITATPQKhxK1RoDiU8N++TTz7Jr7/+CuTMW1muXDk6duzICy+8QEhIzmioxx57jAMHDvDnn3865pm2e/HFF1m1ahVbtmzB09MTyJkre/r06ezcuROLxUKzZs0YO3Ys9evXd2x3xx13UL16dcaPH8/rr7/O5s2b6dWrF2+88YZLjAEBAcybN89leYsWLQDYv39/kfSFEEIIcb2sXr2ajSc3ku2RjV8bP+o0roPeW49COcqYbTlJYoNmwNvgjVFvJMuahcVmcakvPjOeAI8A2ka2vZ67cd1JQlkIIYQQQogb2NUeemfXrl27POsJCQlxmuqisG2VWhmJsHYCnN4C6fFgs4JODweWQKXW0HU8eAWWWHjjx4/n+eefx2KxYLPZOHToEK+//jqrV69my5YtGI1Gunfvzpw5c1i5ciU9evRwbJuUlMTMmTPp27evI5m8dOlSHnzwQfr06cP777+Pp6cnn3zyCa1ateK3336jVatWAPz9998opRgwYADDhg1j4MCB7N27122MuT34bePGjQBUrFixKLtECCGEKDbp6emMGzeOGTNmEBQVRIXnK+Q8rNTHgE3ZXMorFEa9ES+DF14GL1KtqWRaMjFbzRj1RsxWM/GZ8ZhtZtqUb0OjkEYlsFfXjySUhRBCCCGEEGWbUjnJ5KO/gt4IgZVBbwJrNqTF5SwH6DGtxEYqR0REEB4ejsViwWAw0KBBA+rVq0eDBg1YtWoVPXv2pFevXlSoUIFZs2Y5JZTnz59PWloaQ4cOBSA1NZVHH32Ubt268fXXXzvKtWnThmPHjvHss8+yadMmx/K1a9eyY8cOmjRpArifYzs3qampjBkzBg8PD0f7QgghRGm2fft2Bg4cyKFDh6jZrCY+/X2wYgXAqqxut1Eosq05DzT2NHjiZ/PD3+RPbGosNmzo0BHgEUCb8m0Y1XxUrl/ClhWSUBZCCCGEEEKUbWd35IxM1hvBv8K/y/WmnL+Tz+asP7sDKrYokRCTkpKYPn06v/32GxcuXMBsNqNUzs9tDx06RM+ePTEYDDzxxBNMmDCBkydPUrVqVQA++eQTatWqRYcOHQBYs2YNCQkJPProoy7t3HPPPYwdO5akpCQCAgIAqFOnjiOZDLmPRP4vq9XKgAEDOHz4MJ988gnVqlW7hh4QQgghipdSijfffJM33ngDTdOYMmUK2e2z+SHmB66Y4cItDQ2zzUxCZgI2ZeP2qrfTq0Yv/jr3FynZKfiZ/Ggb2ZZGIY3KfDIZJKEshBBCCCGEKOuOrcuZ5iKwsvv1PqGQeCqnXAkklLOzs+nYsSNJSUm8/vrrNG7cGF9fX5KTk2nRogWZmZmOsk888QRvvvkmn3zyCZMnT2bdunUcOHCAyZMnO8qcOnUKgNGjR/Pyyy+jlHIkp5OSklBKcfHiRUdCuXLlXPrlKmw2G0OGDGHZsmW88cYbDBs27Fq6QAghhCh2mqZx5MgR6tWrx4IFC4iKiuLhnx92THGh/fOf+ue/KykUGZYMjB5Gbom4hZHNR+Jv8qdxWOOS2JUSJwllIYQQQgghRNmWmZgzZ7I+lwcT6k2gbDnlSsC6devYu3cvy5Yto3v37hgMBjRNczuXcUREBPfeey9ffPEFb7zxBrNmzcJgMDBo0CBHGT+/nCfLT5kyhebNm7tts0qVKo7/b593Ob+UUgwfPpwFCxbw2muv8eqrrxZoeyGEEOJ6UUrx7bff0qdPH4xGIzNnzsRkMuHh4cHeuL0kZSWh1/SOqS40LSepDP9Of6FDh4/Rh9ur3s7tYbfTrEozjEZjie1TaSAJZSGEEEIIIUTZ5hmY8wA+a7b7pLI1GzRdTrkSkJaWBkB4eLjT8h9//NFt+REjRvDdd98xY8YMli5dSvfu3Slfvrxj/a233oper2ffvn08+OCDRR7vqFGj+Oyzz3j11VeZMGFCkdcvhBBCFJRSiuhL0WyO3eyYgqKmvibvvfgeK1euZMqUKbz44ouOL10BNsduJtOSSYBHAAlZOVNZWJUVvaZHKeUYrazTdNxZ9U7GtRxHUlLSTTGlRV4koSyEEEIIIYQo22rcCgeW5DyA78o5lO3S4sA7OKdcCWjdujVeXl5Mnz6dTz/9FL1ez08//cTWrVvdlu/UqRMNGzZk7NixWK1Wl4fhVa1aldGjR/POO+9QpUoVHnnkEQwGA9nZ2WzYsIFffvmFd999t1CxvvTSS8yYMYNXXnmFN954o1B1CCGEEEUpOTuZ6Tums+viLpKykrBh49LmSxyecxhzipn+g/ozfPhwl+1SslNQmqKyX2VIhsSsRKzK6vJgvir+VXi2xbOSSL6CrqQDEEIIIYQQQohiVaE5VGoNVnPOA/j+eUo71ux//jbnrK/gfnqIYg+vQgW++uor1q1bR0REBCEhIXz++ed88sknuW4zYsQIrFYr4eHh9OjRw2X9lClTmD59OlOmTMHb25uKFSsSHBzMO++8w5133lmoOE+cOMHkyZPRNI1FixZRs2ZNp3/uHgIohBBCFCelFNN3TGfD2Q1kWbMI0YVwevZp9n2wD03TaPxCY+oMr+M0MtnOz+SHDh1KKaoFVKN2UG1CvULxNfria/Ql2COYQFMg3at1x9/kXwJ7V3rJCGUhhBBCCCFE2aZp0HV8zv8/vSXnAXzKljPNhXcwVO+cs/46jTz64Ycf0Omcx/bce++99O7dmzNnzhAQEIC/vz9KKY4ePUq5cuVc6qhTpw4AgwcPxmBw/7Fu+PDhDB8+nPj4eNLT04mMjHRpd/Xq1ZhMucwt/R8VK1bk6NGjua738fHJVz1CCCFEUYm+FM2ui7sw6oyE+4STnZbNhf0XqNS6EreMuoUUjxR2XdxF9KVookKjnLZtG9mWVSdXEZ8ZT7hPOL4mX3xNvo71F9Iu4KH3oG1k2+u9W6WeJJSFEEIIIYQQZZ9XIPSYBmd3wLF1OQ/g8wzMmeaiQvPrlkyGnMRsbiIiIhwJYk3TqFmzptty33//PZqmuUx34U5wcDDBwcFu11WuXDkfEecwGo25xiOEEEJcb0oplsQs4UzCGTIPZmJtZcXf5E+PaT3wCvZC0zSMViOxqbFsjt3sklBuFNKIpmFN2XB2AxfSLhDsGYxRb8RsNROfGY/ZZqZN+TY0CmlUQntYeklCWQghhBBCCHFz0DSo2CLn3w3szJkzLFq0iF69elG7du2SDkcIIYS47uzzJi/5fQmHPjpE1rks0sen41XVC4NmwDPeE5PehJfBiyxbFsnZyS51aJrGqOajANh1cRexqbHYsKFDR4BHAG3Kt2FU81Eyd7IbklAWQgghhBBCiBtAeno6UVFRnD17llq1ajFjxoySDkkIIYS47pRSTNsyjfkfz+fEDycACO8TjmdlTwAsykKqORXMoKGh03TsvLCT5Oxkl7mQ/U3+vNLmFaIvRbM5djMp2Sn4mfxoG9mWRiGNJJmcC0koCyGEEEIIIcQNwMvLi19++QUfHx8iIiLkQ64QQoib0v9t+z8+fPxDEg4n4BHpQcVhFfGq4uW2rEKhlOJc2jmm75jOK21ecXn/1DSNqNAolykxRO4koSyEEEIIIYQQN4CrzakshBBClHVKKaIvRTN782wSYhIod0c5wvuEozPprrqdpmnoNX2uD+cTBScJZSGEEEIIIYQQQgghRKl19NRRpq2dxoXgC5wPOU/tKbUxhZjyta192oukrCS3D+cTBXf1FL4QQgghhBBCCCGEEEKUkMWLF9Mkqglzx8wlPT2dAFNAvpPJgGOKCxs2UrJTiivMm0qpTSifO3eO4cOHU7duXWrWrMlDDz3EoUOHClXX8OHDiYiIoFevXtfcVlHGJYQQQgghhBBCCCGEcJWUlMTgwYPp27cv2dZsqvSvgjIqNAr2DAGrzYoNGzp0+Jn8iinam0upTCjHxcXRtm1boqOjWbhwIT/99BM2m402bdpw8ODBAtW1dOlSvvzyS5KTk7l8+fI1tVWUcQkhhBBCCCGEEEIIIVz9/vvvNIpqxPz58wluEkytN2vh1cyLc2nnOJ9+vkB1KRQpWSkEeATQNrJtMUV8cymVCeVJkyZx7tw5vv/+e1q0aEG9evVYsGABvr6+jBkzJt/1XLx4kSeeeILXXnsNf3//a26rqOISQgghhBBCCCGEEEK4t2HDBs5fPE+9J+pR+4XamAJNaLqckckKVeD6zDYzlf0r0yikUVGHelMqlQnlb7/9lvbt2xMZGelY5uHhwb333ssvv/xCQkJCvup5/PHHqVChwlWTvQVpq6jiEkIIIYQQQgghhBBC/Gv37t2O2QV6PNaDW6ffSrU7qhHgGQAa2Gw2APSaPt91amiY9CYMOgPV/as75lMW16bUJZTPnz/P+fPnadCggcu6Bg0aYLVaiY6OzrOeOXPm8PPPPzNnzhwMBsM1t1VUcQkhhBBCCCFubjNnzmTOnDklHYYQQghRKlitVt5++21atmpJ9wHdmbptKjP3ziTDP4MgjyD8Tf5omoYNm2P+5KvNo6z985+HzoMI7wiqB1THx+hDti37eu1Smec+01qC4uLiAAgODnZZZ19mL5ObkydP8txzzzF69GiaNWtWJG1da1wXL150WR8TEwOAxWLBYrHkuq0QQgghyiZ5/y9d5HiI62X+/Pn4+voydOjQkg6lUC5fvszq1auJiYkhICCAJk2a0LFjx5IOSwghxA3o+PHj9B/Yn782/YVfRT9UJ8XPJ34mzZxGtjWb2LRYKvpWxKgzYrFZUOQ8lE+n6bAqq0t9Ok2Hh94DX6MvFf0qYtAZMFvN6DW9PJCvCJW6hLJSOfOguBuCrtPlDKi2Wl1PGDubzcbgwYOJiIhgwoQJRdbWtcY1c+bMXONJSUkhMTHxqrEKIYQQouyw3zPI+3/pkpKSUtIhCFHqTZw4kS+//JJbbrmFatWqsXv3bl566SUaNmzITz/9RFhYWEmHKIQQ4gaglOKLL77g2WefJTU1lcp3VabegHqEBoRi1Bs5nXyaixkXScxKBCDAFECWJQuFwqZsjnr0mh4vgxcZlgxsyoaP0YeKvhXxMfo4cnjxmfHyQL4iVuoSyoGBgQAkJye7rEtKSgIgKCgo1+2/+OILNmzYwLp16/D09Cyytq41rhEjRnD//fc7LYuJiaF37974+fk56hdCCCFE2afX58z7Ju//pYufn4xaESIvd955J//73/8cr2MAw4cPp3Xr1rz66qvMnj27BKMTQghxo1BK8eWXX+Lj70PjMY0JaBhAuE+4Y32QZxBJ2UlkW7NJNacS7h2Oh8EDi82CpmlYbVY0TaOib0VCvEI4GH+QLEsW3gZvPPQeaJqG2WomPjMes81Mm/Jt5IF8RajUJZQrVaqEn58fR48edVl35MgRALfzGNudOXMGTdN48MEHnZbHxcURHx9PREQEzzzzDC+//HKB2rrWuMLCwnL9tt5gMOQ6z7MQQgghyi55/y9d5HiUfUopoi9Fszl2MynZKfiZ/Ggb2ZZGIY1K/CE9CxYsYN++fdhsNoxGIyEhIXTs2JEWLVo4ysycOZPMzEyef/55l+1XrFjBxo0bGT9+vGNgjdVqZfXq1ezcuROLxUKzZs3o0aOH4xeWANOmTSM0NJSBAweyZs0aNm/eTKNGjejdu7fbOFu2bOmyrFWrVgQEBDim9BNCCCFys3nzZtq0aYNOp2PRokV8d/w7VpxbQbCn8xSzvkZffI2+JFoTybZmk2nJzPk7KxGrzYoOHYEegQSaArmYfhE/ox8R3hFoaMSmxmLDhg4dAR4BtCnfhlHNR5X4e31ZUurumjVN4+6772bFihWkpaXh4+MD5Nz8/fzzz7Rs2ZLIyMhct3/hhRcYPny4y/KoqCiqVavGsmXL8PX1LXBb1xqXEEIIIYQQouQkZyczfcd0dl3cRVJWkuOD5qqTq2ga1pRRzUfhb/Ivsfh8fHwIDAzEZrNhsVjYtm0bL730EiNGjOD9998HIDU1lbFjx9K9e3fq1avn2NZmszFy5EgqVarkSCafOnWKu+++m4sXL9KnTx88PT15+umnmTRpEitXrnT8uvLTTz+lbt26rF+/npSUFCIjI9m4cWOuCWV3tm7dSlJSEl26dCm6DhFCCFGmJCcn89xzz/HFF18wd+5cHnnkESpUqIA11ooNG0a90XkDDSr6VQRypqxIyk7Cy+CFXtOjoaHX9FiVldi0WEfSeGSzkfyd/Hep/OK4rCl1CWWAV199lWXLljF8+HBmz56NyWTizTff5NChQ6xatcqp7JQpU3j//fdZvnw5rVq1wtfX15EwvpJOp8NoNBIREVHotgpSVgghhBBCCFE6KKWYvmM6G85uwKgzEukbiVFvdPwUdsPZDQC80uaVEvvAed9996GUwmKxYDAY0DSNH3/8kT59+tC/f39atGjB0KFDGT9+PLNmzeLDDz90bLty5UpOnDjB66+/DuTsb58+fUhOTmb//v2Oh4j/73//o2HDhowePZovvvjCsf0ff/zBO++8w2OPPQbkb3738ePHk5GRwalTpxwjo8eOHVt0HSKEEKLM+OOPP+g3sB+xp2Kp3bo25yPPszduL41CGuFn8kOHDrPV7JxUVpBpyUSv6THoDJTzLEdUaBTV/KsR5hNGXHqc26RxVGgUUaFRJbezN4lSmVCuW7cua9asYeTIkQQHB6PX66lYsSKLFy+ma9euTmVTUlK4cOEC2dnZxd5WQcoKIYQQQgghSofoS9HsurgLo87oND+jUZ/z94W0C+y6uIvoS9El+iH0wIEDrF27losXL2KxWMjMzARg27ZttGjRgnLlyvHAAw8wf/583n77bcevJmfNmoW/vz99+/YFYNOmTWzfvp1Zs2Y5kskAwcHBDB06lKlTp/LJJ59gMpkA8PDwYMiQIY5y+ZnfPSAgAKPRiJeXF8nJyezdu5fExERCQkKKqjuEEELc4LKyshj78lg+nPYhOqOO2o/Wpvzt5fk18Ve2btpK07CmdKvSjVUnVxGfGe94j7bYLJxJOUOqOZUsaxYaGmnmNI4mHMXX6MvD9R8u0V8ViVKaUAZo06YNW7duJT09HavVmutDUsaNG8fTTz9NuXLlrlpfdHS004MjCtNWQcsKIYQQQgghSt7m2M0kZSUR6et+irpgz2BiU2PZHLu5xBLKL7zwAtOnT6d79+40atSIgIAAvLy8ALh8+bKj3IgRI5g/fz5ff/01jz/+OCdPnmTlypU8/vjjeHt7A7B3714gZ57KM2fOoJRCKQXArl27yMzM5OzZs1SrVg2A2rVr5/pZKTdXzuM8btw4WrRowSOPPMKKFSsK3wlCCCHKlIsXLzJ79mz8qvvRZFQTKteojEFnICkriQtpF1h+bDnrTq3Dy+hFQmYCWdYsIrwjiE2LJTEzERs29JqeQI9AIn0iSchKKBW/KhKlOKFsZ78pyk1uU1z8V2ho6DW3VdiyQgghhBBCiJKTkp3ifn7Gfxj1RmzYSMlOuc6R5di1axfvvfceU6dOZeTIkY4pL2JiYhzTWNi1bt2a5s2bM2vWLB5//HFmz56NzWZj6NChjjL25HFAQIDLZ6VOnTrRqVMnAgICHMvs8ykXVp06dejatSs///wzZrMZo9F9PwshhCj7rFYr+/bto3HjxiR4JnDLxFswRZooH1Aei9VCTGIMyVnJ2LABkGnNRMvU0NDIsmaRmJWIxWYBwKQ34Wv0paJfRQw6A+GG0vOroptdqU8oCyGEEEIIIcS1yHV+xn+YrWZ06PAzlcyvDw8fPgxA586dnZZv2bLFbfkRI0YwdOhQ/vjjD+bMmUOjRo1o2bKlY33jxo0BaNasGY888kixxPxfmZmZ6PX6Ao90FkIIUXacPHmSwYMHs33bVvZ8NpLNnschJIUQYwXMVjMHLx8ky5blsp365z+dpsOmchLNoV6hBHkG4WP0cRqJXBp+VSRAV9IBCCGEEEIIIURxahvZlgCPAOIz492uj8+MJ8AjgLaRba9zZDlq164N5Mx9bBcXF8dnn33mtny/fv0IDg6mf//+xMXFOY1OBmjXrh3t2rXj9ddf5+zZs07rsrKyCv1A8dTUVDZs2OCyfMOGDaxbt477778fnU4+YgohxM1GKcW8efOIiorijz/+4NHWgVyOXcqf8ftJMacTl3iMmLhot8nkK5ltZgy6nLGvQZ5B+Jp8Xaa1KOlfFYkcMkJZCCGEEEIIUaY1CmlE07CmbDi7gQtpFwj2DMaoN2K2monPjMdsM9OmfBsahTQqkfiaNWvG0KFDef755/nzzz/x9fVl06ZNvP/++6xfv96lvJeXF0OGDOG9997DZDIxYMAAlzI//vgjDz/8MHXq1KF79+5UqFCBU6dOsXPnTvr168cdd9xR4Dg1TeONN94gPj6eRo0a4e/vz+HDh1mzZg333HMPM2fOLNT+CyGEuHHFxcXxxBNPsHTpUiKDfZg/vArRTf2ZYtITi5UsFOdRWJUF8pjz2Kqsjnn/EzIT8DW5TnFb0r8qEjkkoSyEEEIIIYQo0zRNY1TzUQDsuriL2NRYbNjQoSPAI4A25dswqvmo6/Zwn6eeesplnuHPP/+cwYMHs3PnTvz9/ZkyZQoBAQG8/fbbdOzY0aWOPn368N5779G7d2+3DygPCwtjzZo17Nmzhy1btpCens6tt97Kp59+SkhIiKPc6NGj83zAuZ2Pjw+//vorhw8fZtu2bVy4cIEWLVrw0UcfUatWrQL2ghBCiLJgw4YNLF26lAfvvo2POyYw0z+LDSYdNhSeaGShyJkRWQOV8z9XpeW8b6eZ09yuLulfFYkcklAWQgghhBBClHn+Jn9eafMK0Zei2Ry7mZTsFPxMfrSNbEujkEbX9UnxAwcOdLu8ffv2tGnTxvFQPoBx48a5Lfvbb78BuEx38V+NGzd2zKnszuOPP56fkJ3UqVOHOnXqFHg7IYQQZYBSpB7ZQPyeX6gcoHFfuUA2LvmcW4Ius3f3XHaYPEnBihWwoP559J5jY/LKKGtomHQmFKpU/qpI5JCEshBCCCGEEDewuLg4jh07RnBwsGMu3vxSSnHq1CkuXLhAZGQkFStWdFtu06ZNZGdnuywPCQmhYcOGhYq7JGiaRlRo1A3/EB+bzcaCBQuoU6cO3bp1K+lwhBBC3CwyEtk4YxiD3l1GkCdsfqoiRqOBW7yD4ZzGZoMiFisWFDo0PP5JHqe7JJZzp2ka5X3KU9W/KqdSTpX4r4qEe5JQFkIIIYQQ4gaUmZnJE088wffff09UVBQnT54kPDycRYsW0aBBg6tua7FYmDx5Mp999hlGo5HAwEAOHz5M1apVeeedd+jevbtT+fvuuw+lFPXq1XNa3r59e958880i3zfhXnZ2Nq+99hrbt28nJiaG//u//5OH4AkhhLgusrOyGD/oNt5ZvB2TQePZ22qjC6vOXpXOZks8x21ZbPHUk4lCBxj5dyyyJxoZKFQebWhoeOm9aB7enJdbv8y+y/tK/FdFwj1JKAshhBBCCHEDeuqpp1i+fDk7duygfv36ZGVlcffdd3PHHXdw6NAhfH1dH2Rjl5WVhV6vZ8eOHY75dJOSkujbty+9evVi//79LnPi3nbbbSxcuLBY90lcnaZpBAYG0qtXL6ZPn57nFwdCCCFEUdi3bx8DH+rD7v1HaF7JiwXDW1Ih0oe3SGAHWcTqIRsjVgAUVjQyUegBDzT05CQgzXlMd+Fn8uPWyrcyqvkodDpdmfhVUVklCWUhhBBCCCFuMKdPn2bu3LmMGzeO+vXrA+Dh4cHUqVNp3Lgxn332Gc8991yu2/v4+PDSSy85LQsICGDUqFGsWbOGdevWyUPWSiGj0ZjrnMpCCCFEcVBK8fo7r7P34FG63RXCvb2qc8AEEzjHSayYUVgBPRpKKWz/5Ixt//xTKLwUeKCw6jS3U18YNAM1g2ryfPPnaVO+jYxAvgFIQlkIIYQQQogbzKpVq1BK0bVrV6flUVFRhIeHs3LlyqsmlHNz6tQpAGrWrOmyLiMjg+3bt2OxWKhRowahoaGFC14IIYQQpd7Zs2ex6C18dugzsu7Koln1OmRX1jNPl4GFdMz/KW8DlAb/TnSRM8GFBTBrGkrTo9fpMaBRM7AmJr0JgJpBNeldozdRoVGSSL6BSEJZCCGEEEKIG8yRI0cAqFatmsu6atWqcfjw4XzVc/nyZaKjo8nMzGTnzp1MnTqV0aNH06VLF5eyP//8M8eOHSMlJYWTJ0/SrVs3Zs+eTdWqVa/axsWLF4mLi3NaFhMTA4DVasVisbhsY7PZ0LSckU43E/v+3mz7nRfpF/eupV+UUiil3F5/NzqLxZLra8vNTPrFPekXV0op5s+fz3PPPUeTe5rg29sXo8lIgzqViM2IIxMNjX/nSHZ+BVLgJilsBTz1nug1PWHeYbzU8iUahjg/1NdqtRbbPhWVsny+FHSfJKEshBBCCCHEDSY5ORnA7TzJvr6+jvV5OXLkCK+//jqpqakcOnSIZs2a8cADD7iUe/XVV3nkkUfw8fEBYOvWrfTu3Ztbb72V6Ojoq87XPHPmTCZMmOB2XUpKComJiS7LzWYzRqOxTH5gy8uN8IG6JEi/uFfYflFKYTab3V5/Nzqr1UpqaioAer2+hKMpPaRf3JN+cXb58mVeeOEFVqxYQbnQcgTUCCBcF06QZxAZJOJv9cSPnNHIub765DLI2M/oh1FnJCo4igr6Cjfk609ZPl9SUlIKVF4SykIIIYQQQtxgjEYj4H40icViwWQy5auetm3b8vvvvwM5D+UbMmQIHTp04M8//6Rly5aOck899ZTTdq1ateLdd99lwIABfPXVVwwbNizXNkaMGMH999/vtCwmJobevXvj5+dHYGCgyzaJiYlomobBcHN9XLGPNNXr9fKz3ytIv7h3Lf2iaRpGo9Ht9Xejs78uBgQE3HSvIVcj/eLeTdsvSkHsLjjxB2QmgWcAK0/qeXTsW1y6eInGXRrT+bHOHNAOYPGwkKhL5Jw6R5z1HB42G9maRrZrpW5HJwNoaAQZgugc2ZkhTYbgZ/Ir7j0sFmX5fPHzK9gxKVt7L4QQQgghxE2gfPnyAMTFxREZGem07uLFi0RERBS4zoCAAGbNmkVERASffPKJU0LZnVtuuQWAgwcPXrVcWFgYYWFhbtfp9Xq3H8h0Oh3ATZs81DTtpt33q5F+ca8w/WLfpqwlROzsry1ldf8KS/rFvZuuXzISYe0EOL0F0uPBZuUyOoa89zfJqRbqP12PkPYhnFFnSMhOwJJmoaJfRbJsWZh1NvQaKJsZK+qKB+xpbkcma2joNT06nY7m5ZvzctuXb/jX8bJ6vhR0f8rW3gshhBBCCHETsCd79+zZQ+PGjR3L09LSOHr0KIMHDy5UvYGBgeh0unz9DPXy5ctAwUe0CCGEEKKEKJWTTD76K+iNnLaFUTHUj5kqjlpPVsLLW6NC+WA8/athTjNDNiRmJgJg0pkADWXwQG+zolkz8m4OhbfRG0+9J9UDqt/wyWTxL11JByCEEEIIIYQomM6dO1OpUiXmzp3rtHzBggWYzWaXhPLGjRvZtm2b4++kpCS39S5ZsgSbzUanTp3yLPvxxx8D0KNHj0Ltg/hXYmIip0+fJjMzs0TanzFjBl9//XWJtH29vfvuu/z4448lGsPs2bOZM2eO4++DBw/yyiuvEBsbW4JRCSFuCmd3oE7/xQ69nh5rLlH1pT955MBxNmtmgqv4UjXYE2N2KmSn4GvyxaQ3oVCkmlOxYcOmbKSb00m3Zec+h/I/NDR0/6QdAzwCaBvZtvj3T1w3klAWQgghhBDiBmM0GpkzZw6bNm1i4MCBrFy5kvfff58XXniBp59+mvbt2zuV79WrF0OGDHH8vWzZMtq0acP777/PihUr+OGHH3j22Wd59NFH6d69O8OHD3eUXbFiBS1btmTatGmsWLGCr776il69ejFv3jxeffVV2rRpc932uyyx2Wy8/vrrlC9fnqCgICpXroy3tzedOnW67g8jnD17Nt99912htv3uu+947733ijii4jN9+nSWL19eojHMnTuXBQsWOP4+fPgwkyZNkoSyEKLYJR9ZxcjzF+k27Sj/99MZvCt4sCvASiwW0rBh0RnAaobMRLwMXvgYfdA0jUxLJhfSLmBVVmzYsKp/08naf+a60KPHpDPhqfcEDazKStOwpjQKaXS9d1cUI5nyQgghhBBCiBvQbbfdxs6dO5k1axYffPABwcHBzJs3j759+7qUbdeuHcHBwY6/Bw0aRPPmzVm4cCG///472dnZVK1alW+//ZaePXs6/SS1f//+NGnShAULFrB+/XrMZjN16tThr7/+onXr1tdlX8uiqVOn8uabbzJt2jQ6deqEh4cH58+fZ+PGjdd9XsaRI0cSFBRUqG1//PFH/vrrL0aPHl3EUd086tevz8SJE6lQoUJJhyKEKMOsVisPzf+aVV+dwGZVNOwZQbNe5TltsHIZGynYOKNpVAWw5XyxWcGnAgnZCZgxo1Bu67Uv16HDqDPmjGTGhlIKDY1q/tUY1XyUTHdRxkhCWQghhBBCiBtUgwYN+Oijj/Ist2zZMrfbvv322/luZ/LkyQWOT+Ru9erVBAQE0LNnT6pXrw5AnTp1aNeu3XWP5YknnrjubYp/1a5dm1deeaWkwxBClDVKwdkdcGwdZCby0+nz/LLwIF7BBjo/Xo3wWjnPQDBhQ//PJqnYSNUgQJeTLsy0ZjoSw/aEsEEzoNN0WJUVs83sWGbUGQn3CcdsM2OxWci0ZOJv8mdcq3H4m/yv++6L4iUJZSGEEEIIIYS4zoYNG0b//v2pVasWTZo0oWfPnjz66KNERkZe91hmzJhBuXLlePjhhwE4ceIEc+bMYfDgwURERPD9998TGxtL48aNnUawz549m+joaBITE50Soq+//rpjlHVSUhIrVqzg+PHj+Pr6ctttt9GwYUOn9t99912qVatGnz59WLNmDdu2baNJkyZs27aN+vXrux11P3fuXBISEnj++eex2Wy89tprjnUmk4lKlSrRo0cPwsLCCtwf9vo6d+5M586dWbFiBdHR0dx+++20bNmyQG2dPXuWJUuWkJKSQvv27enQoYNLmYMHD/LVV18xYsQIx/GfPn06oaGhjmNit2LFCnbt2sWrr77qWJaamsqKFSs4efIkgYGBtGnThqZNmxZ4v4UQZYdKTyB61Wg2xe0mNj6VCn46zut1NHqmMvWqeeDtbXKU9UdHPDasKCwokvV6AjwDAUgxp2BWZvRaTsrZw+CBTsuZPdeojCgUFpsFm7JhURbMNjOhXqHEZ8aj03S0jWxLVGjUdd9/UfwkoSyEEEIIIYQQ11FWVhZ//PEHDRo0YMiQIezcuZP33nuPDz/8kOXLl7vMgQ1w6dIlPvjgg3zVf9tttzk9WDEvs2fPpmbNmo7k5d9//82kSZOoVq0aX375JY0bNyY9PZ0JEybQr18/5s+fD4CHhwd6vR5N0/D09HTUZ084//zzzwwYMIBq1arRtWtXYmJiGDduHKNHj+att95ylJ8+fTqdO3dm2bJlZGdnExISQnp6Ovv37+fjjz+mV69eTj+VTkxMZMSIETz++OOO9q5sPykpiU8++YRnnnmG5cuX06VLl3z3BeQklCdNmkR2djbvvfce4eHh6HQ6duzYQatWrfLd1sqVK+nbty/16tWjS5cuTJkyhV9++cWlPfscyr1793YklGfNmkXdunVdEsq//PKLY/5ygAMHDtC1a1dHH58/f565c+fStGlTZs+eXaD9FkKUDclZSUxf3o8tF06y7atzJB5Np/XEOihPHcYoX0zZ2WDNypnWwuCBj06PLxpJKKwosvUmMPlhtVpJzkxGKQUa2JQNs82MQTPkJJg18NR7kmZLyxmxbLMSnxmP2WomwCOANuXbyFQXZZgklIUQQgghhBDiOho6dChbt25l586d+Pr6Ajkjlm+55RbGjx/P2rVrXba5dOkSkyZNylf9np6eBUoo5+ajjz5i9erVhIaGAhAVFcWzzz7L888/T5MmTXjkkUf45ZdfSE5Odpmy4dixY9x///307duXefPmodPljGi7//77ufvuu7nlllvo2bOno/zKlSuZPn06AwYMACAhIYGNGzeydOlSfvrpJ+655x5H2a+++orMzEyGDh0K5CSU3U0Z8dBDD/HYY49x/PjxQu3/F198wZIlSxyjihMSEvLdVmJiIg8//DCdOnVi+fLljhHb48eP58iRI0RFFc2Ivffff5/AwED++usv9Hq9Y/m2bduKpH4hxI1FKcX0P15hydZDHPjiLFlJFqo08yfCpohVVsyaximDgWpmM5qygjkdDY2KOgM2g45EnY50vYGTKSeJ0CIcU1rYVM68yNnWbCyaBb2mx0PvAYBBZ8Df5E+mNZNagbVoX6E9bSPb0iikkSSTyzBJKAshhBBCCCFuKA0aNHC7fNmyZdSsWZOYmBh69erltsz+/fsBWLVqFc8//7zL+ho1arB8+XIAPv74Y2bOnOlS5vbbb+f9998vVOyrV6/mq6++4qeffnIkkwHatm1LVFQUMTExbrcLDQ1l4sSJ+Wqjc+fOhYrtv/r27etIJtv/fvbZZ9m8eTNNmjS56razZ88mIyODKVOmOJLJAD179qRBgwbMmzfPKaEcEBBA//79HX8HBQXRvXt3KlSowNy5c50SynPmzKF58+Y0btzYscxms7F+/Xp27dpFYmIiNpuNlJQUTpw4waVLlwgJCSnw/jdq1Mhpigr7gwvz09bSpUtJTEzkf//7n9NDFl988UXefffdAseSm4yMDFJTU4mLiyMiIsKxvGXLlkXWhhDixrHl6Frmvr2IE+suY/TU0eGRcGq09kPDTKRZI8ZoIFmnI02nw1fT58yzjMJg8MDbOxBvgyetI1vjo/lwOu40nlZPsrKzHA/eU0qhlMKich7ap9N0GHQGDDoDkZ6RjGk5Rqa4uElIQlkIIYQQQgghrpN58+bh7+/P7bff7rLOYrEQGBjodrty5cpd9we31atXz+lve8Ly7NmzeW67Z88evLy8HNMu2JMQSimysrJcEuf16tVzGcmm1+sZPHgwU6ZMITY2lsqVK7N792527drllOhPSEjgjjvu4Pjx4/Ts2ZPKlSvj6emJh0fO6LnCJpTr16/vsiy/bR0+fNhtHT4+PlSpUqXAseRm1KhRrFmzhurVq9OlSxc6duzInXfeWWQjoIUQpZdSiuhL0Ww6u4kTSSe4nHmZHd+s5cS6y4TU8qLzkPL4lTOBsoGy4WtT+NsUCXqNWIOeajYdRqMnZmsW8ViwYKNDZFteafMKey7sYfaF2fib/NF0GomZifyTU85p+5+5k3WaDk+DJzpNR9OwpjQKaVRi/SGuL0koCyGEEEIIIW4o9lHGualZs2aeZe644448yzz11FM89dRTBY7vanbv3k2NGjUwmUxOyxMTEzl69CjDhg1zu11xzqGcG29vb6e/7VMqWCyWPLe12WyYTCan0bl2gwcPdknw2kf//tfQoUN5++23mT9/Pq+88gpz5szBy8vLaW7hadOmsXPnTg4dOkTNmjUdy6dMmcKSJUvyjDU37mIqaFtKqXwt+y+DwYDVanVZnpaW5vR3y5YtOXr0KCtWrOD3339n1qxZjB07lgEDBjB//nz5ubkQZZFSJJ/cwPSd09meEcuZrHSyzVY0Lx22Dj5UNFWgXKsAzmvgabFhvGLTyhYLaToTGhCr07BhRqfXEWC10MZQzjHn8dZzW0nLTiPIO4gArwAAUrJTyLJeMVoZhYZGhHcEzcOby3zJNxlJKAshhBBCCCHEdZKeno6Xl5fL8vnz52M2m7n//vvdblcScyjnh06nc5sgbdy4Mb/99htPPfVUrsni/KhevTq33nor8+fP54UXXuDrr7+mT58+BAQEOMocPHiQSpUqOSV4AbZu3VrodnOT37bso7v3799Px44dHctTU1P5+++/naancKdChQqcP3/eZfm+fftclgUFBTFo0CAGDRoEwEsvvcTkyZMZMWIEbdu2zd+OCSFuDBmJqDWvM/3Cev7Qmbl4PouYz85gCDJS+ZnKaDoIbBuEFUUicMioo26WxZFUVkrhb7PSMdNMhEcQyToj6djAnISnzcKig4toG9mW5OxkFAqj3ohVs1LVvyqp5lSSspLItGRiVmayrFnUD67PmJZjZL7km5AklIUQQgghhBDiOqlfvz6rV6/mwIEDjukQNm3axMsvv0z//v1zTQCWxBzK+REREUF8fDwWi8VpNPKTTz7JrFmzGDVqFHPmzMFo/HeM3MWLFzl9+jTNmzfPVxuPPvooAwYM4LnnniM+Pt7xMD67WrVq8dNPP3HmzBkqVqwIwO+//86mTZuKYA+d5bet3r17ExwczFtvvUXbtm0d+z9lyhSXkd/udOjQgfHjx3P06FFq1aoF5MwRfvr0aadya9eu5dZbb3VMuQE584ADTnNXCyHKAKVg7QSiT6xhpzec+i2Bo9+dQ1kUofV9wabQ6TVsV2ySqWkcNBkJVAqDUlg0jQCrjXvTMqnqGch0UtlDFkkmPbbMWHSHv2PVyVV467zR0DBbzegMOtDA1+SLryln7n+z1UxsaiztK7SXOZNvUpJQFkIIIYQQQojr5OWXX2bNmjV06NCB7t27k5CQwOrVq7nllluYNWtWrtuVxBzK+fHQQw/x4Ycf0rNnT5o1a4ZOp+P111+nRo0a/PTTTwwYMID69evTuXNnvL29OXr0KIcOHeLDDz/Mdxv33XcfQUFBzJ49mxo1ariMvn7uuef49ttvadOmDX369OHcuXOcPn2asWPH8txzzxXp/ua3LX9/fxYtWkSfPn1o0aIFnTp14uDBg7Rr186RIL6aZ555hnnz5tGpUyfuu+8+Lly4gI+PD3379mXevHmOcuvWrePJJ5+kZcuWVKlShdjYWJYuXcqwYcNo3bp1ke67EKKEnd0Bp7fwf+k21n96moT9qZhCjVR8rCI+tX3Q/TMVhYbTdMdk6XRcUjlTVOjQ8LXZqJyVznTzWTboTRhtViLRY/SvitnoSXxmPMmZyYQSSnJWMuUM5VxCic+MJ8AjgLaR8iuIm5UklIUQQgghhBDiOmnXrh1bt27ls88+4/Tp04SHh7No0SLuu+8+NE3L1/zERW3kyJFO01JUq1aNiRMnUrt2bZeyEydOpF27do6/W7Vqxb59+/jtt99ISEhAKeX42XOXLl04ceIEa9eu5fDhwxiNRnr06EHnzp2d5pB+8cUXr/qgOk9PT2bNmsWRI0do1aqVy8+qw8LC2L9/P8uWLePs2bPceuut9OjRg4MHDzJx4kRCQ0Od2qpWrdpV+0Ov1zNx4kQ6dOjgsq4gbd1+++3ExMSwfPlyUlJS6N+/P61bt6Zq1aqO+aghZ9T6xIkTqVChgmNZQEAAe/bsYcmSJZw/f57evXvTrVs3Vq1aReXKlR3l3nzzTUaPHs0ff/zB33//TYMGDZgwYYLLlBxCiDLg2DpIj+frHy6SsD+VyA7BBPULR+elJ+f3CDmpZB2K/87ArlCYAL2ykaDTMTg8lHgDKGUhUoHR0x88/DBqEO4TzqXUSyiLIt2SjiXNQrBnMEa9EbPVTHxmPGabmTbl28hD+G5imsrPEwFEsdi/fz8NGzZk3759NGjQoKTDEUIIIcR1Yn/fz+uBYOL6knuz68fe17t376Zx48Yu648fPw7kzJ97M1FKOaaOkLko/yX94t619EtZvsYsFguJiYkEBga6fSjkzUr6xb0bpV+Sk5Px2zQZLfp73tYFsujUJcKbBnARKzbg36+och6Zd+W0FxrgY7MRabEQr9eTpNNh+ec1Qw8YNQO+nkFU9K+EQZfTBzaLDWu6FeWpSLelk5SVhA0bOnQEeATQNKwpo5qPwt/kf726oFS4Uc6XwijofXDZ2nshhBBCCCGEEEIIIcqIX3/9lSFDhvDqg60ZFqmnR6AP28pZSVYWdBrYUOSkjXPYk8n2JTqlCLDZSNDrSdbp0K5Y56npMesNJGUnQ8oZqvpXBY2ch/FhpUloE26vcTubYzeTkp2Cn8mPtpFt5SF8QhLKQgghhBBCCCGEEEKUCkrB2R2k71/FuJlLmbF8O74+3nhXigKPczRKjaeph4U/jGDUaVg0DSvq35SypuXUQU7i2AgYFcTrc5LJpn8ezgegNB0mvYlsazap5lRSzan4mnwxW81oaPh7+BMVGiUP3hMuJKEshBBCCCGEEEIIIURJy0iEtRPY/udaBs49zKGL2XSo5sWXQ+pQrUYCWJugHVjKqOQsCApku6eJvw16rPz7ID5N4RizrFcKP5siW9OwAB4qZwSz9s989xZNwwQYdUayrFkkZyfja/IlISuBSFMkrcq3Kpl+EKWeJJSFEEIIIYQQQgghhChJSsHaCWQeWMXds44Sn25lyv11GN0xEH3aeYj+AXzDAQ1/Ba9cTiDaZGSdtyerfLy5pNNh0zTAhlnT0ClFoM1GRbOFc0YDoKH9s84IeCqNdDSyrdmOuZOzrdlcSLuAsilqlatFg3LyTAnhniSUhRBCCCGEEEIIIYS4Xv6Z1oJj6yAzETwDSTME4XPqLzw9PZj3WDPK++uJ8r0MSX+DzZLzL+HYP9NZ6NA0iMrKJCork5EJiUR7eLDZ24tkTWOnpwfn9AY8bTmjlfVKoVBk/TPVhb9NUcEzmLNGPanmVLKt2VhtVtIt6QR6BNIsohl9qvWReZJFrkp9QjkjIwOr1Yqvr2+htk9NTcXLywu9Xu92/cWLF7HZbG7XAURERDjVlZqa6rZcWFgYOp2uUDEKIYQQQgghhBDXi/rn5+5CiBLwz7QW6tQWzKmXyM428+nWVCatjef3oUHYoupyshFEZ8SzOTubtkYdjaweaHhAVso/ldgntci5jnOSy9lEmW2grCRrGtPLlWOXtzexej1ZKJSmoYBAG1TUDBh8w6ji4UuaOY3Y1Fg0TaNblW7cW/Ne6gXWIykpqWT6R9wQSm1CeevWrYwcOZLdu3ej1+upVKkSU6dOpWfPnnlue+TIET788EN+/PFH0tLSyMjIoE6dOjz77LMMHTrUqWyHDh3cXiQXLlzAaDSSlpaG0WgEYOrUqUyYMIGwsDCXN9/o6GhCQ0OvYY+FEEIIcT298cYbfPvttyXS9rFjxwBo0OD6/4zwwQcf5LXXXrvu7QpREJqmYTabJeklRDFQSmG1WjGZTCUdihA3n3+mtbAe+ZWETEV0gi+vLD3BpphEKgbqmWEycVHFk6IM2DxAZ/JilY8XTbMtjEpKw//fiv4ZqayBY3CjyhnFrNPjr+lypsSwGtjsG0iyTmOnyuCcsuCJQpn8wMMPi81MmjkNb6M3HSp04JU2r+TMrWyxlEj3iBtHqRxSe+TIEbp27UqtWrWIj48nKSmJhx56iN69e7Nu3bo8t//2229p1aoVe/bsISkpifj4eLp3785jjz3G+++/71T28OHDnD9/3unfsmXLAOjVq5cjmXylY8eOuWwjyWQhhBDixvLtt98SExNTIm0bjUa39xjFLSYmpsSS6EIUhK+vL1arlXPnzsmHWiGKkFKKixcvYrVa8fDwKOlwhLj5nN2BOrWFhEzFnD0W7v04mk0xifRuFsb9/6vByboBZKMIy7ZQzWwh0mIlCxsbPPRM9/P458F7uX3RquEYuRxcHU1nICrLwrCEBMZcusTshAy62gx4mvyINRk5kXyC2NRYPPQedKjQgVHNR8mXuCLfSuUI5TfeeANN05g1axbe3t4AjB8/nh9++IFx48axZcuWq27/6quvOv3t6+vLO++8w7x58/j666957rnnrrr9F198AcATTzxxDXshhBBCiNKuZs2a7N+/v6TDuG5KYkS0EIURFBREeno6SUlJJCUlYTAY0Ol0Zf6DrlLKMSq7rO9rQUi/uFfQfrGPTLZarXh5eREeHn4dohRCODm2DnPqJX477cG4744Q5G1gev96VG4cwOeBaRiUmXCrBXty2Igi3AIX9Hp2mYxEe5iIyraCsv5Tofrnf9S//19vBO8QSI+Hah0htC5kJuLvGcgr1TsT7WFi87m/SMlOwc/kR9vItjQKaSSvr6JASl1CWSnFTz/9ROfOnZ3mTdY0jR49evDOO+8QGxtLZGRkgeq1WCyYzWZCQkKuWi49PZ1vvvmGatWq0a1bt1zrunz5Mv7+/iUyukgIIYQQQoiyzGAwULlyZVJSUkhOTnZMf1HWKaUwm80YjUb5YH8F6Rf3CtovmqZhMpnw8PAgPDxcngEkRHFz8+C9zNj9WM0WakSE8PLd1bmngQ9VvDP52pRAms5AhNn+Xuf8nhdstRJrNLDZy5OorDRyJhz453lg9ueCaf/MqexVDmxm0Olzksmdxjjq0YAoICqscbHuuij7Sl1C+fTp0yQnJ1OrVi2XdbVr1wbgwIED+Uoox8XFYTabOXv2LFOnTsVgMPDmm29edZsffviB5ORkxo4dm+ubcnh4OEajkezsbNq1a8ekSZO45ZZb8rF3QgghhBBCiPzQNA1/f3/8/f3zLlxGWCwWEhMTCQwMxGAodR/VSoz0i3vSL0KUYv88eI/TWyA9nowsM//7JYHfj6Xxx9AAPHUZPN/Chre6gN5qwaz5o9Dhgc0xccWVjIANjRTdP4lkTQ/qn//V6XOSyUrl/O0TDGlx4B0MNW697rsubg6l7l0nMTERwO2NY0BAAADx8fH5quuOO+7g9OnTXLp0iaCgIGbPnk3z5s3/n737Do+yWPs4/t3dbHoHQuglCUUIvTdpAhYUBQQRqWIJCpYjIDaQo4LiQVQ4KKJSFEGUY0VRmiAQ6b0FadIJ6XXL8/4RyWtMQEICm/L7XFfEnZln5n6GJTzcmZ254jVz5szBzc2NYcOG5aqrUaMGS5YsoUePHvj4+LBnzx4eeeQROnXqxIoVK2jXrt1l+z137hznz5/PUXZp30a73a694URERFykNP4dXBrv+WpoXkRERKRADAP+2AzLxsCFQ2BxY2ucPw/MP8neU8m0qe5FUpqNCJ/j4DRhYCLT5I6X04wZsGPgDjgxYzKB2TDABDZMmE0m/JyQlW52gpvnn6uTTX9moE3g4QfpiVmH89XsCJWunAMTuVZFLqF8aVVwXh9pc/65jN9isVxVX1u3bgUgOTmZqVOncu+99zJ58mTGjh2bZ/uYmBjWrl1Lr169CA0NzVU/ePDgHK/r1avH0qVLCQ8P57nnnmPNmjWXjWXmzJlMnDgxz7qkpKTsRLqIiIjcGA5H1t5zpenv4NJ4z/mRlJTk6hBERESkuLq0KvnwKkg4gd3uZMr6TCasOobJBK/dE8Ezt4XDqa2YHGk4MZOBJ2CiRVomK72tXLRYKO9wZK1StniCYQennYsWMwEOB60zMgETmK1ZyeOM5KztLSCrzGQCd2+o0hK6vPTnNhgiha/IJZTLlSsH5L0K+VLZpTZXy9fXlwkTJrB27VomTJjA008/nedHgj788EMMw8jXYXxly5alSZMmbNy48YrtoqKi6Nu3b46ymJgYevXqhZ+fH4GBgVc9poiIiBTcpR9Ql6a/g0vjPeeHn5+fq0MQERGR4sgwspLJB5dDRgIYTqZFZ/L8ilRuKmdhQW8/GoeZwGzC8A6EpDQMTFjJxGRA4wxokOHOei8Pzlog2OHEHSc2qycXDTs2w0krm5NIazCY0qBOTwionLU3c2ZK1gpld2/wCsra5qJSUyWT5boqcgnl0NBQQkND8zxxfc+ePVgsFiIjI6+p7/DwcFauXElsbGyuE20dDgdz586lWrVqdOvWLV/9pqen/+PhfCEhIYSEhORZ5+bmpj2vREREXKQ0/h1cGu/5amheRERE5Kr99dC9CwcgZiVGRhJ2WyZWC0Q1tZBp8+DpNh54WoH0BMhMwmQyY2Ai3eJPsuGOyWnHgYWecd6kmAwOe2Ry2mrBaTIw4yDAZKGVyZvRHkGYMk6DVwA0eQAqN3P1DEgpViSfmu+9915mzZrFqVOnsg/fy8jIYOnSpXTr1o2goKDstsnJySQnJ1OmTJnspK5hGLkO1LPZbPz6669UqlQpz8TuDz/8wKlTp5g0adJlT7vNq9+DBw+ydetWevToUaB7FhERERERERGRYuBvh+6RlsCZ+GSGf51OrWAz/+nhyWE/D8reGcC7ZjN+Tiet0zKJTI3DhAmTyYSvjw94VyYhzYbdaRBqNvGimxtHko+w0ZRJkrsnfp5+tDb7EukAU8ppcNi0N7IUCUUyofzcc8+xdOlS+vbty9tvv423tzcvvfQSSUlJvPHGGznaTp48mVdeeYW1a9fSrl07MjIyaNGiBaNHj6ZRo0YEBQURExPD1KlTOXDgAJ9//nmupDBkHcZnsVgYOnRonjE5HA4aNWrEqFGjaNKkCQEBAWzatInx48fj7+/P66+/fl3mQkREREREREREiohL21sc+gksVgiowhebT/PwVynEphkENrLyclAgO7w8SbCYcWLCjMGP3k4aO+yMTgd/ixUTTvw8rfh55vzEe8MMLxqm28DsBXFxYMSCyQzewVnJZO2NLEVAkUwoh4SEsHHjRiZMmMCAAQOw2+00a9aMDRs2cNNNN+Vo6+fnR/ny5XF3dwfAw8ODzz//nBkzZvDee+9x6tQpgoODadmyJVu2bKFBgwa5xktISGDz5s3079+fSpUq5RmTxWLhyy+/5N1332X27NmcOXOGkJAQ+vbty5NPPkmFChUKfyJERERERERERKTIMP7YTPrh9aRnwAm7L6/M28biLYmU8TLxeV8v9rUPYa2XF1bDoKLNjhWwARctFta6m8Bs5nn36pgyUyHxJPiUA4s7ODIh5Tw4nVDvbmg4AH5fnbVPsmeg9kaWIqVIJpQBKlasyPvvv/+P7caOHcvYsWNzlNWqVYvp06df9VgBAQEcP378H9tFRETkq18RERERERERESkZEtJs/PbdZzSOP8dJoxyPfrKbDTHx3Brhxgc9PTlf1pNPPTywGgblHY7s66xAeYeDsyYT26wWdlVsRoOURLj4O8QfB8OZexWyVyBUae6qWxW5oiKbUBYRERERERERESkKDMPg9R/2U/fsWUyGE4vVgye7VadfnSM83sSJyWTimz+3uahos+fZR7DDySmznQ0nVtOAAAiuCeV6gD1Dq5ClWFFCWURERERERERE5Aq2n4hn1frfmDt/GR63WGlYJ4Mu5eMJKGfCRFYCONFsxoEJ62X6sFq9cJoNkry8ICkdTu8A3xC4/T9KIkuxooSyiIiIiIiIiIjI3xiGwfYT8fyy/yyzZ05nzzcfYMbJvrhQbrOfwGrYMDCy2/s7nVgwsEHupLLJhM1swYwDP5Mb+JfN2kP5RDSc3AKVm93IWxMpECWURURERERERERE/mQYBr/GXOCNHw8QE3OYo1++TuqJvVjLVKHS3f+iXaVvcDe2YAKcmLGQtV9yq7R0fvT25qLFkmMP5SxmLhoOAkxmWuOVVeRTLmsP5cOrlFCWYkUJZREREREREREREbIO3puybB/f7zpNfGISJ94fhTMtkfAWNzP85uqU89yItzMJh9kMmHHDln1tZEYmjTIyWOflxVmLhWCHAytgAy5aTNgMO61MvkTinnWBxT3rQL70eBfcqci1U0JZRERERKSYOnDgALNmzeLQoUMEBwfTp08f7rzzzqu6Ni0tjc8++4z169dz9uxZKlasSI8ePbjzzjsxm82FOpaIiEhxcOngveW7z5CS6cDdy4caXe+nf8BuBtaIJZCjmE1OfEjHzbDjMLlhYMIATH9+jY6LxwRs8/DglNUNJybMQIDTSau0dEa7BWLy+HO/ZEcmmMxZB/KJFCO5nxRFRERERKTIW716NY0aNeLMmTM8/PDDREZG0r9/f/71r3/947UpKSnUr1+fTZs20bZtW4YNG0ZQUBD33Xcft9xyC3Z7ztPpCzKWiIhIcWAYBos3n2DR51+w692HyEyKw2IyeKXJOR4NO4mXOZM/KMdhZyXiDV/MGFixYcYgFU8cf6bY/J0Gz8fGMSH2Ivcmp3J7Shr3pqQxISGN5y8m4J+W+P+DppwH72AI6+Siuxa5NlqhLCIiIiJSzNjtdoYOHUqLFi1YuHAhAD179sTHx4eRI0fSp08fWrVqddnrPTw82LZtG/7+/tllvXr1omzZsvzrX//i22+/pVevXoUyloiISFGXmGbj1a838fF/JhK79UfMnr5kxP5BE98LNDQdwG5y46ypDJiyDuBLxx0DEybDAJOBGcj6rxOTyQSGQYOMTBp4lgeHDdLi4M9ynPaslckp57PqanaESk1defsi+aYVyiIiIiIixczq1as5evQoQ4cOzVE+ePBgrFYrH3300RWvd3Nzy5FMvqROnToAxMXFFdpYIiIiRZVhGBw8k0SvFz5g1mN3ELv1R+rVDOWFh++gdVVP2pl2EmhK4rzx59+ZJhMmwMP854F7JjBj4GF24uZmwYQJjD8rTOasr+Dq4B2UVWY4svZLjj8OVi+IuAW6vJSVbBYpRrRCWURERESkmNm0aRMAjRs3zlHu4+NDREQE0dHR+e4zLS2N2bNnExwczG233XZdxxIREXG1+JQMoj7ZwqltK1k36wU83ExMvi2YIU3dMEwbiDN2YWDCggMbbpgNIythDFhw4DSZsZjNmJw2LIYdHH9JClus/78a2WyFMuFw8Xdw2KFGByhXJ2ubi0pNlUyWYkkJZRERERGRYubUqVMAlCtXLlddSEgIe/fuvap+oqOjee6550hPT2fv3r00atSITZs2Ub58+UIb69y5c5w/fz5HWUxMDAAOhyPXfs2lmd1u15zkQfOSN81L3jQvedO8ZDEMg51/JLBs1yk+iT6G2WTwbLXThLUuw31Ng/ALLsdh3LBgp6w5kSBTIlacmJ3gNFmzOjGBzeyFw5QObu44HRlg9QGrN2QkgSMDMIPJCRYvcDgh9QKYPSGiM/SY/P9JZIfDZXNxJXq/5K0kz0t+70kJZRERERGRYsZmswFZW1f8ndVqJTMz86r6CQsLY9y4cSQnJxMdHc0777zDmDFj+OSTT/Dw8CiUsWbOnMnEiRPzrEtKSiI+Pv6qYi0NHA4HycnJAFgsFhdHU3RoXvKmecmb5iVvmhdIzrDz2W/H2Xsyjg3ffUbC8f30GTiYGj42Iu9pTbzTm/Q/29qAPwAbF/E3pRJgeHOWYAAsZkjxCCHJcR6Tw5a1GjmgUlZS2emApNNZiWXDCWYfyAC8a0DITdBkMCQkuGgGrp7eL3kryfOSlJSUr/ZKKIuIiIiIFDOX9j9OSUnJVZeUlJTn/sh5KVu2LF27dgWyDuVr2bIld999N61ateJf//pXoYwVFRVF3759c5TFxMTQq1cv/Pz8CAwMvKpYS4NLq4MCAgLyTOCXVpqXvGle8qZ5yVtpnxfD6WTJ4i9xbvuOjUuWE3P8HL7BIZSP3Uiw2xniU9OwOnMnehNNNiqZTuBpWLCafElzC6JCGX/83QxMF06ALTVrZbKvN5hSsw7bc1wAZyr4V4SqrcAzIGubi4qNi832FqX9/XI5JXle/Pz88tW+ZN29iIiIiEgpEBERAcCRI0eoUaNGjrojR44QGRl5Tf3efvvtmM1mNmzYUGhjhYSEEBISkmedxWIpcf8gK6hLc6J5yUnzkjfNS940L3krtfOSFs+5/z1L4hff8tr3J0nONHikhRfPdzHj7rGaDCrjaVwk2bCSbHgBf0n6mk1kYOYiAYT4uhPABUzJ57IO2/P7y1ZQCUeyViSbzOAdDDW7ZR225xV4o++20JTa98s/KKnzkt/7KVl3LyIiIiJSCnTr1g2TycSKFSvo3Llzdvnu3bs5e/YszzzzzDX1e/bsWZxOZ479kq/XWCIiItedYcCKifz348+ZsDyWCn5mFvX1oXuYGROpOLATj40ypgtYTHZsJjfiDD/i8CXZ8MJqcuDEwrHytxBxR39Mv6+G9HjwDMw6VK9iEzi1FQ6vylmuw/akhFNCWURERESkmKlWrRoDBw5k5syZDBo0iNq1a5OZmckzzzxDaGgoI0aMyNG+d+/eBAcHM3v2bACWL18OZCWLLzl//jwPPfQQVquVhx566JrHEhERKSqMPzZjOh7NoAYWjl/wYFJXH8p6GpjIOgzP+LOdGwYWHFhx4GnKJIgkkvDCYfLA4RFE6259MVVpDlWa5x6kcrOsL5FSRAllEREREZFiaNasWWRmZtKoUSOaNGnCkSNHCAgIYNmyZbn2NV6zZg2hoaHZr6tWrcr48eMZOHAgNWvWJD09nQMHDlC7dm1+/PFHmjRpcs1jiYiIuIRhwMktcHgVSRfP8tSctURW8GRUzT+o4mPnvZ5eODFwMxzZm1r8fQ2x889f3bBT1pyIyeKFud5tmGq2vIE3IlL0KaEsIiIiIlIMeXt789lnn3Hq1CkOHz5McHAwN910E6Y8PmL75Zdf4u7unv26Tp06fPnllyQmJnLgwAEyMzOpXr06lSpVKvBYIiIiN1xaPKyYCIdX8evekzzweRxH4hz0rG3l8YqeWEwmDMByhS5MXEowG1gwsJgtmKzucFMvbV8h8jdKKIuIiIiIFGMVK1akYsWKV2zToUOHPMv9/f1p3jyPj+8WYCwREZHrzXA6Obh1NRd3/Qhp8VRJ3EpAylEmrUrmzfXpuFtgeg9PRrawYvozmXw1KWEDMDBh8vDBFFgVUmPhj00Q1vH63pBIMaOEsoiIiIiIiIiIFAsJcRfYv+BpysVtJ8yZgIeRhh/p3PppCst/d9C4ghuz7/ajUTknJozsZPKl/ZJzvvgbE5hNZkzeweAdDKkXsg7bE5EclFAWEREREREREZEizTAMth2L48zCx2iYvgmHyYwJJ95kYgKimrvTsrKD59p7YLE4MRkGTkxYTJdJKl/69S9fJsjazskrEByZYDKDZ+CNvVGRYsDs6gBEREREREREREQuJyHNxvP/2837Cz+nRtoeMg0LZ+NSuX/BGXaftQPQs7aViR09cbeYsGD8mTM24fhL6st0FV9YrODhBynns1Yph3W6gXcqUjzkO6EcHR3NsmXLcpT98ccffPvtt6xYsYKLFy8WWnAiIiIiIsXRxx9/zJEjR7JfO51OoqOj+frrr9myZQtOp/MKV4uIiMglTqeTMUt28NX2U0SkbCLASOSzHal0fe80Px22sfywPbut8bdfTWStVM4Xqw8kngKHDaq0hEpNC+U+REqSfCeUn3vuOX744Yfs1//973+pXr06PXv2pGvXroSGhnLbbbexYcOGQg1URERERKQ4iImJYejQoaSnpwOQnp5O+/btadWqFXfddRfNmjWjUqVKPPXUUyQnJ7s4WhERkaIrIc3Go59sZfWB86Rk2DGlxvHQkvOM//okPlb4YkAgT7TxBvI+dM8MmC+7YfJl2NLA6gURt0CXl8B0Ncf5iZQu+d5D+cCBA9x///0AHDlyhNGjR1OzZk1GjhyJl5cXe/fu5csvv6RDhw689dZbjBw5stCDFhEREREpqg4cOICHhwd16tQB4JVXXmH9+vX069ePHj16kJCQwKpVq5gxYwbfffcd3377LRERES6OWkREpGgxDIPXf9jP+sOxZDqc2NOTmfDBMpJS0uhZ15v3brfi7e2JDQdW7P+/ZcVlXek0PsDklrXdReVm0OXFrJXJSiaL5CnfCeW0tDS8vbN++vPFF1/g7u7OL7/8QmhoaHabKVOm8Mwzz/D444/TqFEj2rZtW3gRi4iIiBSSr7qdgRktXR3GDfNVtzPctTz0nxtKgaSlpeHl5ZV1qA+wcOFC7r//fhYsWJDdZvTo0ezdu5fevXtz7733snXr1uz2IiIiAttPxPPbkYuk2xwYBlg8fSkf2YbnQ/fzYH0nPuYMMjBwYMGOBTccOc7ay2YCMGclhw3HXyrMWYfumdzAswxYreDpl5VMrtzsRt2mSLGU7y0vypUrx9mzZwH46aef6NixY45kMoCHhwfTp0+nY8eOTJ8+vXAiFREREREpBsqVK0dSUhJpaWkcOXKEw4cP079//1ztbrrpJr788kt27NjBqlWrXBCpiIhI0WQYBos3n2Df9s2cXDwRpy0DANvNT1ChXnMwgQUnXmTggQ0bFpyYMMi5BjkDM6lmfwioCH6hWfsjYwFM4O4N/pUhoHJWIhlDeyaLXKV8r1Du1KkT77//PgEBAaxYsYI77rgjz3Ymk4mePXsyY8aMAgcpIiLX15133snhw4ev+frjx48DULVq1Wu6PiwsjK+//vqaxxe5VnctD2XPtGhXh3HD3FWvnqtDKBWaN2+Ol5cXL7zwQvZCDDe3vB+769atS0REBFu3bqVz5843MkwREZEiKSHNxmvf7OLDd9/g/LrFYLaQefognlUj8ScVdxyYTGDCwIKBG04MwIGZBLzwIhN37KTgSax7JSqHhmKy/Lme0mmDuKOQcgEMA+zpkJmUlVyu3E57JotcpXwnlJ999lm+/vprhgwZQnBwMCdPnuS1115j3LhxuT6md+LEiUILVEREii6bzebqEEREigxvb28mT57M448/jmEYdO7cmaioKJYuXUrDhg1ztLXZbNlJZxERkdLOMAyeef87Pnl9DKmnDuFePowydzyNe9mqgMEzbp/R1ryHeMOXcwRS0RSLJ5mYMTBj4I6dRLzxJZ3zHtWoUaFczlyV2QqBVcFhh/L1oHxDcCsPES2ganMlk0WuUr4TylWqVOHgwYOsWLGCRo0akZGRQcuWLfniiy8YMGAAzZo1w9vbmxUrVvDuu+/Sr1+/6xG3iIgUooKuDq7356rHPXv2FEY4IiLF3siRI+nWrRsHDx6ke/fuvP766zRv3pz+/fvTs2dPqlWrRmJiIv/5z39ISEigdevWrg5ZRETEZQzDYPuJeF778AuW/DsKw+kgoHU/Atr2x2SxAtDIdJhm5oPYcOOMEQzARcMfX1MaAaQQQAoZWPmfuQs9PbZTzTsz7/MJUi6AX/msvZJDG0F8PAQGKpkskg/5TihD1qqLnj17Zr/euHEjI0aM4Omnn87Rrlq1akyaNKlgEYqIiIiIFEMRERFEREQAMH78eEJDQxk7dizz58/P0e7hhx/WIdYiIlI6GAac3AKHV0F6PHgGkly5Pa/t8GbL8XgOppTBs0ZjAlr1xaNS3RyXtjPvIpBk/jDK/aXURLLhTTLenCOIyubzhAT6ExreEdOhnyDxJPiUA4s7ODIh5Tw4bFCzY9ZeyQ4HIpJ/15RQ/ruIiAhWr17Nrl27iI6O5uLFi1StWpU777wTb2/vwhhCRERERKRYGzZsGPfddx9r1qxh//79ALRu3ZqWLVu6ODIREZEbIC0eVkyEE9GQehHD6SDTCbM3vcPuM15U6vIgt7gfwK9fGEnGftY6rWw3woCslcP+plQsJid2I+9Ult3khhmD8h4ZmLpMzio8EQ3xx8FwgskM3sFZyWTtlSxSIIWSUL4kMjKSyMjIwuxSRERERKTE8PLyokePHvTo0cPVoYiIiNw4hpGVTD70E1is2Pwqs+VECi9+eYCf9sZR3tfM+jZv4uXpicXkxGGYuc2ykc3OWrxh708iPiQa3jgMM27Ysf81nWUCswk8TA7ADJ4B4BUIt/8n12powjplrUxWMlmkQAo1oSwiIiIiIiIiIpLDyS0Yx6PJcFo45wzkf7+dZNJXh7mQbOO2Ol58cIeVYK9MThl+WIysw/VCiaO7eTO4GbxgH846ZyS3WzZS1pSYvYey2QRWNzNuZhNBjjiSTf4ER3bPGtNkgsrNsr5EpFApoSwiIiIiIiIiItdN+oGfybh4lmOOsry/9ndmrDiOt7uFf99ZhdEN07CYnLjhoJIpFgMTJgyMP7e6uMMSzQ+Olvxq1GezsxadzDsINcVykQA8PDyxGDb8HPFYDDunglvSoklH196sSCmghLKIiIiIiIiIiFwXhmEQved3brLZSHGaaVkzgE1HAhh7W02aBSViNSVjAswYODFIxz07qWzFjh9pPOO2iF9t9XnD3h83dxONOEAl03ncbAZOzCSb/Tkf3JI6A9/EZDa7+pZFSrwinVBOT09n+/bt2O12IiMjCQgIyNf1586d4+DBg/j6+hIWFoafn1+uNjExMcTExOR5fefOnXF3dy/0uERERERERERESoNVe08x+fNN3F8pnZsi7ISX9+GtAXUBcMOBGQMzBgbgwJK9MtnARCZWrNipYj5PY/NhTvrU4/PAJ1kWu4OePgeo4p2J4RlAcGR3WjTpqGSyyA1SZBPKCxYs4LHHHqNKlSp4e3uzc+dOXnzxRZ599tl/vHb58uVMmjSJ33//nfDwcE6fPs0ff/zB8OHDeeONN/D09MwxzsSJE+natSsWiyVHPy1btsyVUC5IXCIiIiIiIiIiJZlhGGw/Ec+6QxeIOXSA2ZOeIu3UIVJr+jC/1v/vfwxgx4IJ/tziIuv1X5kwcGDGHRsd3XaztkxzzidlkOofSfg9A2hcNejG3pyIAPlMKMfGxrJv3758DVC2bFnq1KmTr2vWrl3L4MGDefnll3nuuecA+OKLL+jTpw+VKlVi0KBBV7z+7NmzzJgxgwYNGmSXffLJJwwcOBAPDw+mTp2a65qlS5fi6+t7XeMSERERkZJv9+7dxMfH5+safepNRERKgoQ0G6//sJ/NR2LZv2oJJ36YjWG34d+qD8NuzsTKLqqZzmRvaeGGHdOfa5Jtf65X/isrduy4YceNctZ0zidlYHcYNA0PolGVQJfco4jkM6G8atUq+vbtm68BevfuzZIlS/J1zWuvvUZISAjjxo3L0U+7du2YNGnSPyZuH3jggVxl999/P2PHjmXFihX5iqUw4xIRERGRku+JJ57I9zPnTz/9RNeuXa9TRCIiItefYRi8vmw/P+45w6FPXiTxwEasgaEE3/4knpXrMYfztOUQ1UznsGK/dNX/Xw/Zh/Fd2j/ZBKTggRMLcU5vvKwWmoYHMaZHHUwmkytuU0TIZ0K5ffv2LFu2LF8DhIaG5qu93W5n5cqV3HXXXbm2oOjatSsTJkzg8OHDhIWF5avfhIQELl68SPv27fOs37lzJ2lpaZQtW5Z69erh5pZzaq5XXCIiIiJSskyePJkLFy7k65rGjRtfp2hERESuP8Mw+Hj9UZZsOUGmw8CtciS+Hv4Ed34Qk4c3YPCI29f4kMFFww8npj8P4TPhSQZBphSsOLK3vwATdiwk4EWm4Y7dzZuykd15rWkkjaoEKpks4mL5SiiXL1+eHj16XK9YADh27BgZGRlUr149V12NGjUAOHjw4FUlblevXk1qaionT57kv//9L5GRkbzxxhu52plMJoYMGUK5cuXYu3cvVquVV199lQcffLDQ4jp37hznz5/PUXbpMEC73Y7dbs/rMhGRYkXfy6Q4Ko3v29J4z1ejsOalWbNmhdKPiIhIcZCQZmPi4mg+fOvf+La9HzffYHyb3pkj6dvIdJhm5oPYcMuxh3IWg3qmI3iRSYZhJdXkgQ0ryYYXXmTgZTEo36AjNe+6G5RIFikSityhfElJSQD4+Pjkqru0x3FiYuJV9TVnzhzOnDnD0aNHiY2NZezYsVSoUCFHm9atW3PgwAEiIiIASElJYeTIkYwYMQIfHx/uu+++Qolr5syZTJw4Mc+6pKSkfO+zJyJSlDgcDgB9L5NipTS+b0vjPefHpec9ERERubJLB++tPXSB+Uu+ZvuC17Anx2LyL09A63tzrSBuZ95FIMn8YZTLozcTvxuVqG3+A7vZA7NhxgsbHiYH8fiSWL455btPVDJZpAgpcELZMAw++OADfv75Z06dOoVhGDnqO3XqxKRJk666P6vVCuS9QsRmswHg7u5+VX3Nnz8/+///97//0adPHw4ePMicOXOyy7t3757jGh8fH9577z1+/PFHJk+enJ1QLmhcUVFRufafjomJoVevXvj5+REYGHhV9yQiUhRd2gpI38ukOCmN79vSeM/54efnd936Pn36NFOnTmXPnj0kJyfnqn/rrbe0sllERIqFSwfv/XboNNu/nMmF6K8wuXtR5tbR+ER2BQzMnidw8zmEyZKO4fAkNf0CZrsTu/FnGspE9vbJJhO4efiQbvdir0djTlqr4ONM4UymB7s8mjDk1t7gFeiiuxWRvBQ4oTxs2DA+/fRTKlWqRGJiIuXLl+fIkSMANGnSJN/9VaxYEcjaIuLvzp49m6NNfvTq1Yu77rqLjz/+mHfffRcvL6/LtvXw8KBBgwasWbOm0OIKCQkhJCQkzzo3N7dcezaLiBRH+l4mxVFpfN+Wxnu+GtdrXk6ePEmjRo1wc3MjLS2NihUrkpyczB9//EH16tWv6dlWRETkRjMMg63H4hj7xU5+P3GaPxaMwRb7Bx6V61Hm9iexBoaCOQ2PkB+weB3DZEkFnICZnxw20jK86BFrw5OsBXsWswmz2YRhGBj2TJwmCyetVVjk1Z/YlEzsFoOOYeVoVDXIpfctIrkV6Kn51KlTzJ8/nw0bNnDs2DE+++wzlixZQkpKCm+99Rb79++/7DYPlxMUFER4eDjbtm3LVbdt2za8vLyIjIy8pngDAwNxOp2kpKRcMaEMcP78efz9/W9IXCIiIiJScs2ePZvatWuzYsUKbr/9dsaNG0fXrl3Zs2cPY8eO5ZFHHtHqZBERKdIS0mz8+9u9LNt9muQMB4aHH27BlfGJvAX/5r0wmS2AgUfID7j5HMAwLDgzgwAL4CDZLY6N3lYcpPKoLQSr2UyAlxUvq4XjF1PxSDvDRcOXZal1OZmRRpC3O03DgxjTo44O4BMpgswFuXjnzp00b96c5s2bYzKZsre78PHx4bnnnsPPz48FCxbku98hQ4awadMmdu7cmV0WGxvL0qVLuffee/H29s4uP3ToED/88EP2XoBOp5OLFy/m6jMuLo7ly5cTGRlJ2bJlgayfrsXGxuZq+8svv7B9+3buvPPOa45LRERERARg+/btjBgxAg8PjxzPzPXq1ePTTz9l9OjRpKenuzhKERGRvBmGwZRl+/hy9SZOLJuFYTixmEyE3vMcAS3vobHlCI9ZljLM5wNCvHbibjgx7AFkJZMBLKTby+Iw3DngZSLJM57K/m74eVpxw04NjwTKeplJK9+MiMY3M7BlNV67J5J/96pPgJfVlbcuIpdRoBXKKSkp2YfceXh45ErktmnThlWrVjFo0KB89fvUU0/x3Xff0bNnT1588UW8vb2ZOnUqZcuWZcqUKTnazp07l1deeYW1a9fSrl07HA4HzZo144477qBRo0YEBQURExPDrFmzyMzM5OOPP86+1jAMmjdvzq233kqTJk0ICAhg06ZNvPPOOzRu3DjXWPmJS0REREQErvzM7O/vT4UKFdi1axfNmzd3VYgiIiKXte14HJ/P/5Aj387CsGfiXasN3lXr40cq/zIvpKn5IIEks9DXisXNnTCbiVSTjRNGCPbspDIkOkKxeJ1lA9Ag/jgYTjCZMXkH41GzIw26vEQD7ZUsUiwUKKFsGAZmc9Yi55o1a7J161aSkpKyDzTZvHnzVR+g91deXl6sXr2aDz74gOXLl2O32+nTpw8jR47MsQ0FQK1atejevTtBQVl76litVvbu3cvChQvZuHEjp06dIjg4mKeffpoBAwbkOITGbDZnt92wYQNnzpwhJCSEDz74gL59+2YfxHctcYmIiIiIQO5n5jVr1tCvXz8AkpOTOXDgwDU9M4uIiFwPhmGw/UQ86w5d4I+TJ5n/+jhO74nGLaA8oT2fwq1SPTAMnnH7jJtNO7Dhxh9GOf4wObBhwx0nHqasA2iPGFk/UDWZIMDHF9zTSKraFMzlIT0ePAMhrBNUaprVSESKhUI7eeSmm26iUqVKtGvXjj59+nDkyBHmzp3Ll19+eU39ubu7ExUVRVRU1BXbDRo0KNcKaE9PT4YOHcrQoUP/cZz8tM1PXCIiIiIif9enTx86duxISkoKtWrV4ssvv8Tb25u6deu6OjQRERES0my8/sN+thyL4/jeLRxc8BKOtCR8I7sS3OUhfHx9Sbc7aWCKoYnpIDbcOGMEA+DtdGI2TKQYbviY7PibUvE10kgxeRHk7U7FIHfOpZrxC46Ahg+7+E5FpCAKlFC+tE3EJZ999hkDBgzgxRdfxMfHh+eff5677rqrwEGKiIiIiBRXTzzxRHbCuEOHDkyePJlJkyaRnJxMnTp1+Pzzz7VCWUREXMcw4OQWjMMr2bn7MNXizFx0a4QlLJxTQeUI7f0vMio1xQAyHU7MJmhn3kWQKYk/jHLZ3dRNtbDZ106ym4G7ww0PMgkyp+DuFUCt8r6cSz1HgEcArSu2dt29ikihKFBCuWbNmtSsWTP7dYMGDdi9ezfx8fH4+/tnf7RPRERERKS0uuOOO3K8HjNmDE899RRpaWnZW8WJiIi4RFo8rJgIJ6KxJV3gwp4EUuIcjGm+nu3UJuPRN0k0+RKfmkmmw8DhBC93CwFGKmac2Iz/TyvVyDATnm5mt7eDixaDcg7wtUKZMh6cSz2HzWmjVYVWRJaNdN39ikihuC4Z38DAQCWTRUREREQuw83NTclkERFxLcPISiYf+om0lBSivk3hnnnnePnnBFJSUmlu38LQjPnEpWbgMIysS4AMm5NUsw9OzFhNduDPPZK9rIx0BNDC7o6P2cQ5dwtx7pmcSz2Nh8WD9pXaM7rpaEzaK1mk2CvQCuUdO3bw0UcfXbFNo0aNGDJkSEGGEREREREptv773/9y4MCBK7aJioqiVq1a19R/fHw8R48eJTg4mKpVq+brWsMwOHbsGOnp6dSoUQMPD488223dupXMzMxc5cHBwdcct4iIuI5hGBzcsooye39h5/FUHllylkNnU2lQxY8X7wojxd0TPy7S3HKQZuYjbHWEYRhOnAY4DYNVtvrc4raBMiRywVKGIB93qpfxxs1sZgo+7Eo5yQZ3D5Jq3IxfcAStK7YmsmykkskiJUSBEsq///47s2bNylFmt9txOByYzWasViv33HOPEsoiIiIiUmp98803rFy5MkdZRkYGAFarFbPZzF133ZXvxGxmZiaPPfYY8+fPp1atWpw4cYLw8HDmz59P7dq1r3jtsWPHmDp1KgsWLMDX1xcfHx+OHDnCwIEDmTx5MuXKlcvR/rbbbsPhcBAWFpaj/Oabb2bKlCn5iltERFzr0sF7tQ58welfjvLCj3EAjOxSlf4tK+BmyUr6XjD8qcR5Whk72O0WgWEYOJwGNcr6YPZqyrGkjbS0b6WCZzIegeUxmc3gyMSUcp4GDjsNqnWCjm9kLV8WkRKlQAnlu+++m/T09BxlTqeTo0eP8v7771OlShVGjhxZoABFRERERIqz77//PldZSkoK0dHRzJkzh5deeumaVvmOGjWKhQsXsnHjRho2bEhqaiq333473bt3Z8+ePfj4+Fz22mXLlvG///2PBQsWcPvttwNZnz7s2rUru3btYuPGjbm2sOvevTsLFizId5wiIlJ0GIbB6z/sZ/WB8zSwJxFWxo0a5bx4tU8tqpbxxOHM2gnDADJxw2xy4u1MId3hwASYTCZa1Ajm1XsaQFr97P2XiT8BhhNMZvAOhpodoctLSiaLlFCFvtGx2WymZs2aTJ48mY0bN/Lzzz8X9hAiIiIiIsWaj48PnTt35j//+Q/9+/fH4XDk6/qTJ08ye/ZsoqKiaNiwIQDe3t5MmzaNY8eOMWfOnCtef9NNN7Fp06bsZDJAw4YNGTVqFJs2bWLLli35vykRESnydpyI538L55J++hB4BdAlwocvoyKpW9EX429trSY7TswkGt7YHAZ2p4HFbMLb3ZLVwCsQbv8P9Hwbmg2HyL5Zv/Z8O6vcK/AG352I3CjX9eS8Tp06sWjRous5hIiIiIhIsVW+fHnc3NzYvn17vq778ccfcTqd3HLLLTnKGzVqRLly5fJcFf1XHTp0IDQ0NFf5pVXNFy9ezFWXkZHB7t272b59OwkJCfmKV0REXO/MmTMMvq8vez9/k5M/f8QujyYkmf0pY0rA4TQw/pZRLkMCcYYv65yRXFpn7GU1E+Dl/v+NTCao3Axufga6v5L1a+VmWpksUsIVaMuLf7JlyxacTuf1HEJERETkmuzduxeAevXq3fCxjx8/DpDvA9QKKiYmhvDw8Bs6plxZQkICv//+e76fmS8d8lezZs1cdTVr1mT//v35jsXhcDBv3jzc3d1p0qRJrvqlS5eya9cukpKSOHPmDHfccQczZsygcuXKV+z33LlznD9/PkdZTExM9ph2uz3fsZZUl86j0ZzkpHnJm+Ylb5qXvH3++ec89thjXLx4kXKNb6H94H9xxN2HAx6RNMjYir+RQIbZDxtuuBl2ypgSsJgsbHPW45A1HD+TiUybEy83E21qBpWY+dX7JW+al7yV5HnJ7z0VKKF85swZNm/enKPMMAwuXLjATz/9xMKFC/n8888LMoSIiIhIiWOz2Vwybnh4OP369XPJ2KVZdHR0roRqRkYGR48eZc6cObi7u2dvW3G1Lq0Q9vX1zVXn5+eXnXDOj5dffpkdO3YwduzYXIfyjRkzhmHDhhEYGAjA2rVr6dOnD506dWLHjh14e3tftt+ZM2cyceLEPOuSkpKIj4/Pd6wllcPhIDk5GQCLxeLiaIoOzUveNC9507zk5HQ6GTVqFIsWLSIoKIhHnn+DC2UiKedrxc1s56cy9+NMCSY08wgBzlRMOHEa7sRTlsOmKnxt6kINw4TdmbXlRdUgC1V9jRLzvVvvl7xpXvJWkuclKSkpX+0LlFBet24dffv2zbMuODiYqVOn0qdPn4IMISIiInJdGH//XOcNdGlV9J49e1wWg9w4zz33HCtWrMizrlWrVixYsAB3d/c86y/HarUC5Ln3st1uz66/Wh999BGTJk2ia9euTJo0KVf9U089leN1+/bteeONNxg8eDCffvopDz744GX7joqKyvVvhpiYGHr16oWfn192klr+f3VQQEAAbm7X9cOkxYrmJW+al7xpXnIyDAOzVwD1WnSgX9Q4bH7lOXg4nrg4g/L+VsDKfI97CUjZTa20rfiRSqLhTTT12WXUAExA1t7J4Eb36hUJCgpy7U0VIr1f8qZ5yVtJnhc/P798tS/Q3Xfq1Im1a9fmKDObzZQvX56qVavm+0FWRERERKSkeeutt3Kt5PLy8qJKlSqEhIRcU5+X9j8+f/48FSpUyFF34cKFPPdHvpxFixYxYsQIOnTowFdffXXVz/Dt2rUDYPfu3VdsFxISctn7tFgsJe4fZAV1aU40LzlpXvKmeclbaZ+X9PR03nzzTYY+PJIZa09woX4/AiLs7LhocPrkBRIynGTYnZxNysTP00qwjzuxHnX4KakKTgPK+LhTzted4HQ7dqeBm9mE3eEkwNudLjeFlrh5Le3vl8vRvOStpM5Lfu+nQHdfpkyZ7AdJERERERHJrX79+oXeZ7NmzQDYuXMnDRo0yC5PS0vj4MGDDBw48Kr6Wbp0KQMHDqRNmzZ89913V9y64u/i4uKA/z/IT0REXG/btm088MAD7Nmzh5UHY0mvewdWi4nKwd6UdbdzKiWTdJsDuxPsDgcpmQ7OJmXgbjHj7W7B5jDwcDPjYbVQyctKpt1JbEomZsNM02pBNKoS6OpbFJEiwOzqAEREREREJH86depEpUqVmDt3bo7yTz75hMzMTAYNGpSjfMuWLezcuTNH2bJly+jfvz+tWrXi+++/v2xi+NJegX/33nvvAXDbbbdd622IiEghcTgcvPbaa7Rs2ZIDBw4Q9a/nsDTISiaHBnjiZjZzLjGDhEw7bmYzVrMJdzczwT7u+LhbcLOYaFotiF4NK+Lj4cbJ+DQOn0/hZHwaXlYLHWuXY0yPOphMJlffqogUAflaofzll19y77335muA3r17s2jRonxdIyIiIiJSXHXv3v2yeyZfzvLly+ncufNVt3d3d+f999/nrrvuYsSIEQwYMID9+/czduxYRowYwc0335wrptDQ0OztKdavX0/v3r0JCQnhpZdeyrVtRVhYWPbBfF9++SXvv/8+AwYMoFatWiQkJLB48WIWL17MM888Q9u2bfN1ryIiUrjOnDlD7969Wb9+PXXr1mXBggX8GufH5uhjVA7yAiAlw44t04EZsLiZcRoGGXYnnlYL4SG+nE5I51R8Oo91Cqdv8yqsO3SBxHQb/p5W2kWUpVGVQCWTRSRbvhLKERERPPHEE7nK//e//+Hh4UGzZs3w9vYmJiaGDRs2ZH98TkRERESktOjduzeRkZE5yuLi4vjss8/o0KEDNWrUICkpiejoaKxWK7feeitVqlTJ9zi33XYbv/32GzNmzGDChAkEBwczc+ZM7r///lxtmzVrRpkyZbJfHzx4MHurjOeffz5X+xdeeIHbb78dgEGDBlG/fn3mzZvHV199hc1mo3bt2qxZs4YOHTrkO24RESlcQUFBpKamMnr0aPpFjeHXEyks33uGpHQ76TYnbmYTiWk23J1O3CxuOAww/5kcdjizDiku4+POyfg0fo2J5fEuETSuWnIO3hORwpevhHJkZCRTp07NUfb444/z5JNPMnLkyBzlu3fvJioqitdee63gUYqIiIiIFBMPPfRQjtenTp2iX79+7Nq1i5o1a2aXO51OXn31VWw2GxEREdc0VuPGjfnggw/+sd0PP/yQ4/WQIUMYMmTIVY/TpEkTmjRpkt/wRETkOjl79ixffPEFUVFReHh48MPKX3h79VEmfh9DXGrmn8lkB7+fT8bP04q72cDdBGYTOAxwGlmJZIs5K7Hs7mbGaUBius2VtyUixUSB9lA+dOgQ69aty5VMhqzDR2677TZmzpxZkCFERERERIq1N998k4EDB+ZIJgOYzWaef/55PvnkE86ePeui6EREpLhZunQp9evXZ+TIkWzZsgXDMHh79VFWHzhPus1B5SAvapb1wcPNjM1hEJ9mIznDjmEY/LkgGZvDwM1iJsDLCkCm3YnZBP6eVhfemYgUFwVKKB84cICAgIDL1gcGBrJv376CDCEiIiIiUqz90zOzr68vhw4duoERiYhIcZSYmMjQoUO55557cDqdLF68mKZNm7L9RDxbjsVlH8BntZjx9XTD39OK2WzC4TSwOw3MZjM2u4MMuxMD8PNww9fDAkBsSiZB3u60iyjr2psUkWKhQAnlypUrEx0dnesQD4DMzEzmzp17TfvBiYiIiIiUFJUrV+ajjz7C6XTmqtuwYQO7d+/WM7OIiFzRb7/9RoMGDfj444/p0aMHu3btom/fvgCsO3SBuNRMyvi657imShlvgrysuFtM2B1OnH8mlp2Ggb+HG1WDvbA5DE4npGN3GDStFkSjKoEuuDsRKW7ytYfy3zVq1IjWrVvTvn17Hn744exD+Q4fPsz777/P0aNH+fTTTwsrVhERERGRYicqKooWLVrQsmVLhg0bRvXq1UlKSmLjxo3Mnj2bXr16Ua1aNVeHKSIiRZiPjw8JiUncO3oikV1788W+FNrZ42hUJZDEdBtOJ1gtOdcMuplN1CjnQ3K6nT8uJlM52AMffyuxKXbSbHaOXUzDbIIgb3eahgcxpkcdTH8e1iciciUFSigDfPHFFzz88MO8/vrrGH9u6g5Qt25dfv75Z2rUqFHQIUREREREiq0GDRrw3Xff8dBDDxEVFZVdbrFYeOCBB5gxY4YLoxMRkaJq586dnDlzhpbtO/HpQSdNx37KUYeF37efxmyG73adpmm1IAK93TGbweZw5koqQ9aBe74ebrSsHszQzvXYfTqZdYcukJhuw9/TSruIsjSqEqhksohctQInlIOCgli8eDGnT59mx44dZGZmUqNGDerXr69vRiIiIiIiQJcuXTh06BA7duzg+PHjeHl50bhxY8qVK+fq0EREpIhxOBy8+eabvPDCCwQFBTHorW/49WgSVos7lf3dsVrM2BxOYpMzWX3gPPUr+RPk7U5sciahAZ65+otNySTQ253IygGYTCYaVw2icdUgF9yZiJQUBU4oX1KhQgUqVKhQWN2JiIiIiJQoZrOZxo0b07hxY1eHIiIiRdSRI0cYPHgwa9eupXbt2rwwdSbzY9KyD9y7xGoxExrgyemEdI5eSKF6WR92n0zkdEI6ZXzccXczk2l3EpuSid1h0LhqIOEhvi68MxEpSfKVUI6NjWXfvn2ULVuWOnXqZL++kkttRURERERKg927dxMfH09kZCQBAQHZr6/kUlsRESldDMNg+4l41h26wJrvv+Db//6bjLQUHnvsMaZMmcKcDSeJSz1G5SCvPK8v4+POyfg0wsv5UtbXgy3H4jgZn4bTIMf+yE92DceZkXKD705ESqp8JZRXrVpF37596d27N0uWLMl+fSWX2oqIiIiIlAZPPPEEK1as4KeffqJr167Zr6/kUlsRESk9EtJsvP7DfrYciyMuNZMzh07jdPOk6SMT8e/cDZvJetkD9y5xdzPjNCDT4eTfvepnJ6f/vj+yw+EgPuMG36CIlFj5Sii3b9+eZcuWERoamuP1lVxqKyIiIiJSGkyePJkLFy5kb21x6fWVaBsMEZHSxTAMXv9hP1/+7ysCK9eicuVKVL+9H2kdbyPR4c7qA+cBKO/vecUD9zLtTswm8Pe0an9kEblh8pVQLl++PD169LjsaxERERGR0q5Zs2ZXfC0iIvLrvhPMnfIsJ6O/o3KTjlR7+FUAvH398QZOJ6Sz5VgcQ9tW/8cD94K83WkXUfYG34GIlGZ5f2aiAAzD4H//+x+zZ88mLi6usLsXERERESn2Tp06xezZs//x034iIlLyrFu3jl5d2nIy+jtC6rag8b1P5mpTxseduNRMziak07RaEDaHwemEdDLtTiBrZfLphHTsDoOm1YJoVCXwBt+FiJRmBU4oP/nkk3z44YfZr0eNGsXdd9/NQw89RKtWrcjI0CY9IiIiIlJ6JSQk0KBBg+yD+eLi4mjcuDEPPfQQt912G6+99pprAxQRkRvC4XDw7LPP0qFDBxLjLlD1jsfoOHoa3kHlcrW9tDdyUoadMT3q0LF2ObysFk7Gp3H4fAon49PwslroWLscY3rUwWQyueCORKS0KlBCOTY2ls8//5xBgwYBWQ/Ls2bN4l//+hdbtmzBbDazaNGiQglURERERKQ4eu+997jjjjsIDAwE4MMPPyQ9PZ2VK1fy4YcfMnXqVNLT010bpIiIXHdms5m9e/fSrFkzxsz6H6Gt78LuNPJs+9e9kQO8rPy7V31euyeSgS2r0atRRQa2rMZr90Ty7171CfCy3uA7EZHSLl97KP/drl27qFu3Lm5uWd2sXr0ai8XChAkT8PHx4dFHH2XTpk3ZCWcRERERkdJm69at9O3bN/v1zz//TN++fenUqRMAb7zxBocPH6ZevXquClFERK4Tp9PJwoUL6d+/PxaLhXnz5uHt7c3u08ls/nLXVe+NrAP3RKQoKdAK5czMTAzj/3+atnr1apo2bYqPjw8AQUFBJCQkFCxCEREREZFi7K/PzDabjfXr13PzzTdn1+uZWUSkZDp27BidO3dm4MCBzJ49G4CAgACsViuNqgRqb2QRKbYKlFCOiIhgw4YNHD16lPj4eBYvXkznzp2z6w8fPkx4eHiBgxQRERERKa4iIiJYtGgRhmGwcOFCEhMTs1cng56ZRURKGsMwmDt3LpGRkaxZs4aoqCgeeOCBHG1MJpP2RhaRYqtAW17UqFGDrl27EhERgbu7Ow6HgyFDhmTXf/PNN3z00UcFjVFEREREpNh68MEHady4Mf7+/qSkpHDHHXdQuXJlAH777TfCwsIICQlxcZQiIlIYYmNjGTFiBEuXLqVChQp8/vnndO/ePc+2l/ZG3n4innWHLpCYbsPf00q7iLI0qhKoZLKIFFkFSigDLFq0iPfff5+TJ0/St29fwsLCADh+/Di33HILDRo0KHCQIiIiIiLF1aVP9S1atAh/f3+ioqKy63bv3s3EiRNdGJ2IiBS2jRs3cu+99zJz5kzKlClzxbbaG1lEiqMCJ5Q9PT0ZNWpUrvKqVasyefLkAvW9b98+1qxZg91up1mzZrRq1eqqr01PT+eXX37h4MGD+Pr6EhkZSdOmTfNse/78eX799VeOHz9OxYoVadOmDRUrVszVbt26daxbty7PPh5//PHsvaNFRERERP4qMjKSyMjIXOXDhg1zQTQiIlKYkpOTWbt2LbfeeitlypRh69atlC9fXiuMRaTEKtAeypccP36cCRMm0Lt3b1555RUAzp49yw8//HDNfY4dO5bGjRuzbt06du3aRY8ePejXrx92u/0fr509ezY33XQT06dP59ChQ3z//fe0bduW9u3bc/bs2Rxt7733XipXrszUqVOJiYlh1qxZhIWFMWHChFz9/vzzzzz77LOcP3+e+Pj4HF9/PZxQREREROTvli1bxiOPPELPnj3Zvn07AL/88gtHjhxxbWAiInLN1q9fT6NGjbjzzjvZt28fAKGhoUomi0iJVuAVysuXL+eee+7B4XAQHByc/U0zMDCQqKgooqOjKVeuXL76/Oyzz3j99ddZvHgxffv2BeCRRx6hRYsWTJ48meeff/6K19euXZtdu3blWDG8ZcsWWrRowTPPPMO8efOyy1euXMmXX37J7bffnl32n//8h6effppatWoxYMCAXP1PnDgRX1/ffN2TiIiIiJROhmEwZMgQ5s2bR0hICKmpqVy4cAGAM2fO8N577/HJJ5+4OEoREcmPzMxMJk6cyOTJk3F3d2fq1KnUrl3b1WGJiNwQBVqhnJGRwaBBgxg6dCgXLlxg+vTp2XUeHh706tXrmh6Op0+fTnh4eHYyGaBx48b06NGDd955B4fDccXrO3TokGv7iaZNm1KlShV2796do/zXX3/NkUyGrOS1yWTiq6++ynfsIiIiIiJ/tXjxYr766iuWL1/O2bNnadmyZXZd7969+eGHH0hKSnJhhCIikh979+6lVatWvPrqqzRq1IitW7cyevRozOZC+RC4iEiRV6Dvdr/99hsBAQG8/fbb+Pj45PpIR926dbM/zne10tPT+e2332jTpk2uunbt2nHu3Lnsj5Hkx++//87Jkydp27ZtjvK8foJ4aQuLy31E5ZNPPuHNN99k7ty5+oiiiIiIiFzR119/zQsvvMAtt9wCkOMZ02KxUL16dfbs2eOq8EREJJ9Onz7Nrl27eP7559mwYQN169Z1dUgiIjdUgba8uHjxIjVq1Mh+KP57AtbhcPzjauK/O3bsGE6nk8qVK+equ1R25MgR6tev/499zZw5k/j4eE6ePMm3337LsGHDruqgwClTpgDQs2fPXHXlypVjxYoVlCtXjkWLFjFs2DAeffRR3n777Sv+NPLcuXOcP38+R1lMTAwAdrv9qvaGFhEp6vS9TCR/9GemaLlevx8XL14kLCws+3VhPDOLiMiNdeLECeLi4mjQoAFdunTh8OHDVK1a1dVhiYi4RIESypUrV2bHjh2kp6fj6emZ6+H4119/zfHwfDVSU1MB8PT0zFXn5eUFQEpKylX1lZCQQFxcHOfOnSM+Pp7ExETS09NzbYfxV1999RXvvPMOHTp04L777stRd/fddzN27NjsOCAr+Txu3DgqVarEs88+e9l+Z86cycSJE/OsS0pKIj4+/qruSUSkKLqUCNH3MpGroz8zRdP12naicuXKbNy4kV69egE5E8oXLlzg4MGD+X5mFhGRG8MwDD755BMee+wxypcvz+7du7FarUomi0ipVqCEcuPGjfHx8WHo0KH897//zVG3YMECPvvsM7Zt25avPi8lazMyMnLVpaenA+Dt7X1Vff01wbt//35atmxJ//79+emnn/Js/8svv3DfffdRu3ZtPv/881wrjhs2bJjrmjFjxvD+++8zZ86cKyaUo6KicuwJDVkrlHv16oWfnx+BgYFXdU8iIkWRxWIB0PcykaukPzNFk5+f33Xpt0+fPtx11100btyYfv36ZZcnJCQwbNgwmjVrRmho6HUZW0RErl1sbCyPPvoon3/+OaGhoUybNg2r1erqsEREXK5ACWWz2cxHH33ErbfeypdffkmFChVITU2latWqnDhxghdffPGqtqb4q6pVq2IymTh58mSuulOnTgFQrVq1fMdap04d7rzzThYsWEBCQgIBAQE56qOjo7njjjuoXLkyK1euJCQk5Kr6NZlM1KpVi5UrV16xXUhIyGX7dHNzw82tQL8VIiJFgr6XieSP/swULdfr96N79+4MGjSI/v378/jjj5OZmcmoUaM4fvw47u7urF279rqMKyIi1+6HH35g2LBhnD59mt69ezNr1izKli3r6rBERIqEAj81t2/fni1btjB16lQ2bNiAxWIhLCyM//znP/Tp0yff/Xl7e9OkSROio6Nz1a1fv57g4GBuuumma4rV6XRiMplybc2xdetWevToQfny5Vm1ahUVKlTIV7+///67VpWIiMu9/PLLLFq0yCVjHz58GIB69eq5ZPx+/frx4osvumRsEZGr8f7779OxY0fmzp1LTEwMJpOJ+++/n3HjxlGjRg1XhyciIn+zcuVKUlJSmDdvHgMHDsyVRxARKc0uf4rcVThz5gzffvstHh4ezJ49m927d3P48GGWL19+TcnkS6Kioti3bx/ff/99dtn+/fv57rvveOihh3J8xGTt2rVMnjyZP/74A8jak3DTpk25+jxw4ADffvstXbp0wd/fP7t89+7ddOvWjbJly7J69WoqVaqUZ0yGYbBx48Zc5bNnz+bgwYM88MAD13y/IiKFYdGiRdmHfd5oVqvVZR//i4mJcVkiXUTkakRHR/Pjjz/St29ffvzxRw4fPsyePXt47733lEwWESlCNm3alL2f/qRJk9i1axcPPPCAkskiIn9ToBXKGzdu5LnnnmPHjh2FFQ8AQ4cOZf369fTt25dhw4bh7e3Nxx9/zM0338xLL72Uo+2PP/7IK6+8Qrt27ahcuTKGYfDkk09iNptp1KgRQUFBxMTE8OWXX9KwYUPmzp2bfW1qaiq33HILsbGx3HfffcyfPz9H35UqVcpOFBuGwTPPPIPT6aRJkyYEBASwadMmli9fzsCBA3PFJSLiCuHh4ezZs8fVYdxQrloVLSJytV566SW6d+9O9+7dXR2KiIjkwWazMWnSJF599VUefPBBZs2ahYeHhw7eExG5jAIllCMiIvD19S30/eZMJhMffPABw4YNY82aNdjtdj7++GN69OiR6yeDHTp0wG63U6VKFSBr77t169axfft2NmzYwKlTp2jWrBkjR46kdevWua4fPHhw9v///aT1v65kNpvNrF27lm3btrFhwwbOnDlDz549mTZt2jVvwSEiIiIiJV9ERIQOYBQRKaL27dvHAw88wJYtW2jUqBGPP/64q0MSESnyCpQJrlevHlWqVOHHH3+8Lisu2rRpQ5s2ba7Yplu3bnTr1i1XeaNGjWjUqNEVr/X29mby5Mn5iqlx48Y0btw4X9fIjXfnnXdm7+l6LY4fPw5wzT+RDgsL4+uvv77m8UVERKTkePzxx3nooYfo27cvvr6+rg5HRETIOmPp3XffZezYsWRmZvLss88yYcIE3N3dXR2aiEiRV6CE8tGjR+nUqROjRo3ipptuonnz5pQrVy7HKuAaNWrQpUuXAgcqciPZbDZXhyAiIiIlxOHDh2nWrBl169ald+/e1KhRAx8fnxxtbrvtNipWrOiiCEVESh+Hw8G8efOoWLEi8+bNo23btq4OSUSk2ChQQnnz5s089thjABw8eJD//e9/udr07t1bCWW54Qq6OvjSnqylbS9aERERKXxvvvkmK1asAGD69Ol5tvnpp5+UUBYRuQHWr19PmzZtsFqtfPHFFwQHB+Pn5+fqsEREipUCJZRvvfVWDh06dMU2+lifiIiIiJRm8+bNIzU19YptKlWqdE19x8TE8N5773Ho0CGCg4Pp06cPt91221Vde+bMGRYuXMiOHTtIT08nPDycoUOHEhYWVuhjiYi42sWLF4mKimLRokV88cUX3HPPPVSrVs3VYYmIFEsFSij7+PgQHh5eWLGIiIiIiJQ412vl8S+//EKPHj24/fbbGTx4MPv376d3796MGjWKKVOmXPHaL774gv79+9OkSRNGjBiBr68vS5YsoW7durzzzjs8/PDDhTaWiIir/fjjjwwbNoxTp07Rq1cv2rdv7+qQRESKtQIllEVERERE5MZzOBwMHTqUpk2bsnjx4uwzTPz8/Hj88ce55557aNmy5WWvj42N5aWXXmL8+PGYzWYA+vfvT69evRg1ahR33303ISEhhTKWiIirpKamMmbMGGbMmIGfnx8fffQRgwcPznHuk4iI5J/Z1QGIiIiIiEj+rFmzht9//51hw4blSIwMGTIENzc3Pvrooytef//99/P8889nJ5MvueWWW8jMzGTbtm2FNpaIiKucOnWKjz76iA4dOrBz506GDBmiZLKISCFQQllEREREpJj57bffAGjcuHGOcl9fXyIiIoiOjr7i9T4+PnmW7969G4AyZcoU2lgiIjeSzWbLPlw9PDycX3/9lZUrV1K9enXXBiYiUoJoywsRERERkWLm5MmTANnbUvxV+fLl2bt3b7773Lt3Lx999BH169enadOmhTbWuXPnOH/+fI6ymJgYIGs7Dbvdnu9YSyq73a45yYPmJW+al9wOHDjA4MGDiYmJYfPmzVSvXp369etjGEapnye9X/Kmecmb5iVvJXle8ntPSiiLiIiIiBQzNpsNADe33I/zVquVzMzMfPUXGxvLPffcA8BHH32U4yPhBR1r5syZTJw4Mc+6pKQk4uPj8xVrSeZwOEhOTgbAYrG4OJqiQ/OSN83L/zMMg48++ogJEyaQnp7OiBEjcHd31/eXv9D7JW+al7xpXvJWkuclKSkpX+0LLaF87tw59uzZg7+/f44VDSIiIiIiAk6nk71793Ly5EmaNm1K2bJlr7kvPz8/AFJSUnLVJScn4+/vf9V9JSYm0qNHD37//XeWLFlCs2bNCnWsqKgo+vbtm6MsJiaGXr164efnR2Bg4FXHWtJdWh0UEBCQZwK/tNK85E3zkuXUqVOMGDGC5cuXU716dWbPnk1kZGSpn5e/0/slb5qXvGle8laS5+XS897VKvDdJyYm8sgjj7Bo0SKcTie9e/dmyZIlHD9+nJ49e7Jt27Zch32IiIiIiJQma9asYfjw4Rw+fBiAn376ia5du/Lyyy/j4+PD008/na/+atWqBcDRo0epUaNGjrojR45Qv379q+onJSWF2267je3bt7N48WLuvPPOQh8rJCQkz+0yIGt1T0n7B1lBXZoTzUtOmpe8aV5g5cqVLF++nGHDhjFt2jS8vb2Jj48v9fOSF71f8qZ5yZvmJW8ldV7yez8FzvTef//9rFu3jmnTpvHCCy9kl1etWpWwsDCWLVtW0CFERERERIqtQ4cOcdttt1GvXj0WL16c49N8I0aMYMaMGRiGka8+b7nlFiArkfJXe/bs4cyZM3Tv3v0f+0hLS6Nnz55ER0fz2Wefcffdd1+3sUREClNcXFz2/u6DBw9m/fr1zJkzJ1+fzhARkWtXoITywYMHWbNmDevWrWPUqFE0bNgwR32rVq1Yvnx5gQIUERERESnO/vvf/9KzZ0+++uor+vbtS1BQUHZdhQoVANi/f3+++qxevToDBgxgxowZ2Qfc2Ww2xowZQ0hICA8++GCO9v369eORRx7Jfp2Zmck999zD2rVrWbhwIb179y60sURErqeff/6ZyMhI7rvvPhwOByaTidatW7s6LBGRUqVACeUDBw7QsmVLqlatCpDj8A7IekA+d+5cQYYQERERESnWDhw4kGMP4cJ6Zn7//ffp3LkzDRo04Oabb6ZmzZocOnSIZcuW5dqXeMWKFaxbty779bvvvssPP/xAmTJl+OCDD+jRo0eOr59++umaxxIRuR7S0tIYPXo0t9xyCwkJCQwePFjba4qIuEiBNvzw8vLi4sWLl63//fffKVOmTEGGEBEREREp1v7pmfnIkSPX9Mzs4+PDkiVLOHHiBDExMQQHB9OgQYNcCWuAxYsX4+Hhkf26V69e3HTTTZft++91+RlLRKSwbdmyhYEDB7J//37atWvH3LlzqVmzpqvDEhEptQqUUG7atCl79uzhhx9+oEePHjkeKOPj43n//feZNm1agYMUERERESmu2rRpw/Tp0xkwYAA+Pj45npnnz5+P3W6nbt2619x/lSpVqFKlyhXbdO7cOcfrmjVrXlMy5mrGEhEpbG+++SaHDx9m8uTJ/Otf/8Jisbg6JBGRUq1Anw8JCgpi9OjR9OzZk+HDh/PTTz9x6tQpXn/9dRo1akS5cuW45557CitWEREREZFiZ/jw4SQmJlK/fn1efvllTp8+zYoVK3jkkUcYOnQoL7zwgpIjIiJ/c+TIEex2OwDvvPMOmzZtYuzYsfp+KSJSBBRohTLAq6++itlsZtq0aWRkZACwYcMGbr75Zj799FPc3Ao8hIiIiIhIsRUQEMDKlSsZNGgQL730EgC7du3C09OTiRMn8vjjj7s4QhGRosMwDN577z2efvppnnvuOcaPH0+ZMmW0naaISBFS4GyvxWLhtdde49lnn2Xr1q2kpaURFhZGrVq1CiM+EREREZFiLzw8nPXr13Po0CFiYmLw9PSkadOm+Pv7uzo0EZEi4/Tp0wwfPpxly5ZRrVo12rZt6+qQREQkD4W2fNjf35+OHTsWVnciIiIiIiVOREQEERERrg5DRKTI+eKLL3j44YeJjY1l8ODBvP322/qhm4hIEVUoCeUTJ06wcuVKTpw4QXp6eo66+vXr079//8IYRkRERESk2Fq9ejW7d+/m3LlzOJ3OHHVDhw4lLCzMRZGJiLhWamoqTz75JJCVWNZZTCIiRVuBE8pvvPEGzz//PJmZmXnW9+7dWwllERERKVHuvPNODh8+fM3XX7q2Xr1613R9WFgYX3/99TWPLzdWXFwcd9xxB+vXr79sm44dOyqhLCKlzvHjx6latSre3t4sXbqUSpUqERoa6uqwRETkH5gLcvGxY8cYN24cjz76KPv37yclJQWbzZbja/HixYUVq4iIiEiJYLVasVqtrg5DbpA333yTmJgYFi9ezOnTp8nMzMz1zNylSxdXhykicsOkp6fz1FNPERYWRnR0NABNmzZVMllEpJgo0ArlnTt30qhRI956661CCkdERESk6NPqYMmPbdu28e9//5u+ffu6OhQREZfbtm0bDzzwAHv27KF169aUKVPG1SGJiEg+FWiFcoUKFfD19S2sWEREREREShw9M4uIgN1u59VXX6Vly5YcOHCAV155hV9++YXw8HBXhyYiIvlUoIRys2bN8PX1ZefOnYUVj4iIiIhIifL444+zcOHCXAfxiYiUJtu3b+f5558nIiKC6Ohoxo8fj5tbgY91EhERFyjQd+9z587x4IMP8tBDD9G5c2eaNm1KQEBAjjbly5cnMjKyQEGKiIiIiBRXTqeTFi1a0L59ex544AGqVauWaw/tJk2aEBwc7KIIRUSuD8MwOHv2LKGhoTRr1oyvvvqKrl274uXl5erQRESkAAqUUP7ll1+y94K7tJH+3/Xu3ZslS5YUZBgRERERkWLrmWeeYcWKFQCsX78+zzY//fQTXbt2vZFhiYhcV2fOnGHEiBHs2bOHHTt24OfnR8+ePV0dloiIFIICJZQ7dOjATz/9dMU25cuXL8gQIiKSD191OwMzWro6jBvqq25nuGu5TgQXkaLrjTfeIDY29optmjRpcoOiERG5/pYuXcpDDz3EhQsXGDhwoLb8EREpYQqUUA4JCdFKChERERGRK2jcuLGrQxARuSESExMZPXo0H3/8McHBwXz++ef06dPH1WGJiEgh0w74IiIlyF3LQ9kzLe8tiEqqu+rVc3UIIiIiIgLMmjWLjz/+mB49evDhhx9SoUIFV4ckIiLXQb4Syhs2bGDKlCm0adOGMWPGZL++kkttRURERERKg5deeokdO3bw8ssv06BBg+zXV3KprYhIcZOeno7NZsPPz48nn3ySKlWq0L9/f0wmk6tDExGR6yRfCeXk5GRiYmKoWbNmjtdXcqmtiIiIiEhpcOrUKWJiYkhNTc3x+koutRURKU527NjBwIEDadSoEfPnz8dqtXLfffe5OiwREbnO8pVQvuWWW9i9e/dlX4uIiIiIlHazZ8++4msRkeLO4XAwdepUXnjhBQzD4N5778UwDK1KFhEpJcz5abxkyZIcG+pv27aN999/v9CDEhEREREprrp27crPP/+c/Xr69Ons37/fhRGJiBSeI0eO0LFjR8aNG0dYWBgbNmzghRdeUDJZRKQUyVdC2d3dHZvNlv368OHDLF++vNCDEhEREREprv7+zPzNN9/wxx9/uDAiEZHC8/DDD7Nu3TpGjRrF1q1badasmatDEhGRGyxfW17UrFmTdevWsWbNGmrXrk1SUhKZmZlcuHDhstd4eHjg5+eX78CcTieLFy9m9erV2O12mjVrxpAhQ/D09Lyq6w8dOsTSpUs5ePAgvr6+REZGct999+Ht7V2gsQoal4iIiIiUbDVr1uSjjz4iLCyM4OBgbDYbiYmJV3xmDggIwGq13sAoRUSuXkJCAgEBAQDMmDGDY8eO0bVrVxdHJSIirpKvFcr169enffv2dOzYkQoVKjBs2DC++eYbypUrd9mvoUOH5jsom83G7bffzujRowkPD6dp06a89dZbtGrVivj4+H+8/rnnnuO+++4jISGBtm3bEhISwqRJk6hZsya7du265rEKGpeIiIiIlHyPPfYYy5cvp3bt2pQrV45ffvmF3r17X/GZec2aNa4OW0QkT19//TUREREsXLgQgIiICCWTRURKuXytUAZYunQpK1euZN++fWzcuJEdO3Zw//33X7Z9nTp18h3Uu+++yw8//EB0dDQtWrQAoFevXkRERDB+/Hhmzpx5xesffvhhXnnllRxlgwcPpkaNGowfP55vvvnmmsYqaFwiIiIiUvLVqVMne2u4kydP8t5773HzzTcTHh5+2WsiIiJuYIQiIv8sKSmJJ598kjlz5hAUFISHh4erQxIRkSIiXwnl1NRUUlJS6NKlC126dKFatWq4u7szbty4Qg1q9uzZNGnSJDtpC1ChQgV69erF/Pnz+c9//nPFLSaqVq2aq6xSpUpUqFCBM2fOXPNYBY1LREREREq+s2fPEhAQwH333QfAhg0bGDRoEB06dHBxZCIiV2fdunUMGjSII0eO0K1bNz788EMqVark6rBERKSIyNeWF99//z2PPvpo9uugoCDCwsIKNaDExET27dtH06ZNc9U1a9aM5OTkXNtWXI3Vq1dz4sQJ7r777msa63rFJSIiIiIly/3338+6deuyX9eqVQt/f38XRiQicvUSExO54447OHPmTPandJVMFhGRv8rXCmVPT0/S09OzX585c4Zt27YVakCXTsAuX758rrrQ0FAATp48SfPmzf+xr5EjR3L+/HlOnjzJgQMHePfdd3MkxPMzVkHjOnfuHOfPn89RFhMTA4Ddbsdut//j/ciNp98XKY5K6/u2tN63iBSewvo+8vdn5k2bNtGlS5dC6VtE5HpJSUnBx8cHf39/5s+fT0RExDVtYSkiIiVfvhLKtWvXZvXq1cydO5fatWvz+++/ExcXx+bNmy97TXBwMDVr1rzqMTIzMwFwd3fPVXep7K8P6FfSvXt3kpKSOHr0KKdOneLdd9+lY8eO3HTTTfkeq6BxzZw5k4kTJ+ZZl5SUpEP9ihiHwwGg3xcpVkrr+7a03reIFL6kpKRC6adOnTq8+eabuLu7ExwcTFJSEgcPHiQwMPCy19SuXRs/P79CGV9EJD8cDgfTpk1jypQpbNq0ierVq9OzZ09XhyUiIkVYvhLKERERDB8+nCFDhuQov9Jq4d69e7NkyZKrHsPX1xfI+uno310qu9qH7TvvvDP7/6OiomjQoAH9+vXL3poiP2MVNK6oqCj69u2boywmJoZevXrh5+d3xX9gyI1nsVgA9PsixUppfd+W1vsWkcJXWAndJ598kg4dOtC9e/fsst9+++2K1/z000907dq1UMYXEblax44dY/DgwaxZs4ZatWqRkJDg6pBERKQYyFdCGWD69Ok8/vjj7N+/nxUrVrBhwwaeeOKJy7avUqVKvvqvVq0aVquV48eP56o7duwYwBVPyL6coKAgunfvzpw5c7hw4QJly5bN11gFjSskJISQkJA869zc3HBzy/dvhdwA+n2R4qi0vm9L632LSOEprO8jlSpVYt++fWzZsoWTJ08yceJE7rrrLurXr3/ZayIjIwtlbBGRq2EYBnPnzmXUqFEkJSUxcuRIXn/9dby9vV0dmoiIFAPX9NQcHh5OeHg43t7euLu7079//0ILyGq10r59e9auXYvT6cRs/v9zA1etWkXVqlWpVavWNfUdGxuLm5sbPj4++R7resYlIiIiIiWLu7s7rVu3BrKeFe+8805atGjh4qhERLKsXr2aoUOHUqFCBT7//PMcn6gQERH5J+Z/bnJ5nTt3ZsqUKYUVS7YxY8bwxx9/MGPGjOyy5cuXs2rVKsaMGYPJZMouX7x4Mf379+fAgQMA2Gw2Zs+ejc1my9Hn119/zbfffsuAAQPw8vK6prHy01ZEREREBGDGjBlKJotIkXDp3J+OHTsyffp0du/erWSyiIjkW6F8ri8mJoa1a9dy6tQpDMPIUVe3bl169+6dr/66d+/O22+/zZgxY1i8eDHe3t6sXr2ap556ipEjR+Zou3PnThYtWsRjjz1G7dq1sVgs7Nixg/DwcKpVq0ZQUBAxMTHExMQwbNgwpk+ffs1j5aetiIiIiMglhmHw888/s2fPHpKTk3PVDxw4kOrVq9/4wESkVEhOTuapp55i9+7d/PLLL7i5uTFq1ChXhyUiIsVUgRPKH374IY8++iiZmZl51vfu3TvfCWWAxx9/nPvvv5/o6GjsdjsffvghlSpVytWuX79+1K9fnzp16gBgNpt59913mTx5Mrt27eLUqVMEBwfTpEkTAgICCjRWftuKiIiIiGRkZHDHHXfw888/X7ZNq1atlFAWketi/fr1PPDAA/z+++906dKFxMREgoODXR2WiIgUYwVKKCcnJzNq1CgmTZpEmTJl+Pbbb3n77bfZvXs3b7/9NnfccQfDhw+/5v6Dg4O59dZbr9gmMjIyz0NMfH19s/etK6yxrqWtiIiIiJRuH374Ifv37yc6OpqxY8fyxBNPEBERwQ8//JD9/Fy3bl1XhykiJUxmZiYTJkxgypQpuLu7M336dB577LEc5wGJiIhciwIllLds2UKlSpUYM2YMS5YswWKxUKVKFapUqUL37t3p0aMHjRo1om3btoUVr4iIiIhIsfLLL7/w1FNP0aJFCywWCz4+Ptx0003cdNNNdO/encGDB7N58+Zr7j8pKYljx44RHBxMxYoV8319amoq+/btw8fHJ/tTf3+3Y8eOXGeUAAQFBREWFpbvMUXk+ps0aRKvvfYaTZo0YcGCBfrBlYiIFJoCJZRjY2OpXbt2VkdubqSkpGTXmc1mBgwYwGeffaaEsoiIiIiUWld6Zq5Xrx5ubm7s2bOHevXq5atfm83G6NGj+eijj6hZsyZ//PEH9erVY+7cuURERPzj9d9++y3z589n1apVxMfH06RJEzZu3Jhn2+7du2Oz2ahRo0aO8o4dOzJ16tR8xS0i14/T6cTpdOLm5sbTTz+Nj48PTz31FO7u7q4OTURESpACfdbF6XRm/8UUGhrKnj17chzKd+HChTwPHRERERERKS3+/sy8a9euHPWxsbHX9Mz8xBNPMG/ePH799Vf27NnDyZMnsVqtdO/endTU1H+8fsWKFdxzzz0cOXKEkJCQf2x/6623snnz5hxfSiaLFB3Hjx+na9euvPzyywAEBgYybtw4JZNFRKTQFdrmSY0bNyY5OZnHHnuMTZs2sXjxYqZMmULjxo0LawgRERERkWKtS5cuTJs2jYULF7JlyxZGjx7NyZMnL7vVxOWcOnWKWbNm8eijj9KkSRMg6wyRadOmceTIET788MN/7GPatGn069cPHx+fa7oXESkaDMNgwYIFREZGsmrVKuLi4nIs9BIRESlsBdryolKlSrRo0QIADw8Ppk+fzrBhw5g5cyYAbdu25aGHHip4lCIiIiIixVTbtm2zVwDfd999zJ07lwEDBgBgsVh45513CAgIyFefP/74I06nk27duuUob9KkCeXKleO7777jscceK5wb+JPNZmP//v3Y7XaqV6+Or69vofYvIvl38eJFHnnkEb744gtCQ0NZuHAht912m6vDEhGREq5ACeXWrVvTunXr7NcPPPAA7du3Z+PGjVSoUIF27dphsVgKHKSIiIiISHE1ceLE7P93c3Nj+fLl/PLLL5w/f54mTZpc06F2+/fvB6BmzZq56mrWrJldX5iWLFnCli1bSEpKIjY2lrvuuot33nnnHw8CPHfuHOfPn89RFhMTA4DD4cButxd6rMWV3W7XnORB85I3m83GwIED2bRpE7179+bdd9+lbNmypX6e9H7Jm+Ylb5qXvGle8laS5yW/91SghHJeqlevTvXq1Qu7WxERERGREsFsNtOxY8cC9ZGQkACAn59frjo/Pz8OHDhQoP7/7qmnnuLBBx8kODgYgFWrVtGvXz86derE9u3b8fLyuuy1M2fOzJFU/6ukpCTi4+MLNdbizOFwZO+nrYU5/0/zkpPD4cBiseBwOHj66ae5cOEC9957LyaTSX+e0PvlcjQvedO85E3zkreSPC9JSUn5al/oCWUREREREbm+3NyyHuMdDkeuOofDgdVqLdTxxowZk+N1p06deP311xk6dCiffvopw4cPv+y1UVFR9O3bN0dZTEwMvXr1ws/Pj8DAwEKNtTi7tDooICAg+/dYNC9/FR0dzfDhw5k1axatWrWiY8eOmpe/0fslb5qXvGle8qZ5yVtJnpe8FilcScm6exERERGRUiA0NBSACxcuUKFChRx158+fz66/njp06ADA7t27r9guJCQkew/pv7NYLCXuH2QFdWlONC85lfZ5sdlsTJo0iVdffRU3NzeOHj2avcVkaZ6Xy9G85E3zkjfNS940L3krqfOS3/sxX6c4RERERETkOmnWrBkAO3fuzFGelpbGwYMHadq06XWPIS4uDgBvb+/rPpZIabZv3z5at27NpEmTiIyMZMuWLQwZMsTVYYmISCmmhLKIiIiISDHTuXNnKlSowLx583KUL1y4kMzMTAYNGpSjfMeOHezZs+eaxkpNTc2z/IMPPgCgR48e19SviPyz2NhYWrRowbZt23j22WeJjo6mXr16rg5LRERKuZK1PltEREREpBRwd3fnvffe4+677+aRRx7h/vvvZ//+/fzrX/9i6NChdOrUKUf7Ll26EBoammN7iqNHj3LhwgUg6+P0KSkpbN68GYCKFStSsWJFABYvXszHH3/MgAEDqFWrFgkJCSxevJhPP/2UJ598kvbt29+guxYpPZxOJ2azmTJlyvDqq6/SpEkT2rZt6+qwREREACWURURKjL179wK4ZNXK8ePHAahateoNHzsmJobw8PAbPq6IiKv17NmTjRs38u677zJu3DiCg4OZPn06gwcPztW2UaNGlC1bNkfZ3Llz+eabbwCoUqUKAI888ggAw4cP59FHHwVgyJAh3HTTTcybN49FixZhs9moXbs2K1asoHPnztfzFkVKpYULFzJ58mTWrFlDYGAgjz/+uKtDEhERyUEJZRERKTCbzeayscPDw+nXr5/LxhcRcaVmzZrx8ccf/2O7n3/+OVfZSy+9xEsvvXRV47Ro0YIWLVrkNzwRyYeLFy8SFRXFokWLCAkJISYmJnu/dBERkaJECWURkRLCMAyXjX1pVfS17s8pIiIiUpr9+OOPDBs2jFOnTnH33Xfz3nvvUa5cOVeHJSIikicllEVERERERERc5IcffuDWW2/Fz8+Pjz/+mEGDBmEymVwdloiIyGWZXR2AiIiIiIiISGlz6dNlXbt25fHHH2fnzp0MHjxYyWQRESnytEJZiqyXX36ZRYsWuWTsw4cPA6453AygX79+vPjiiy4ZW0RERERErh+bzcYrr7xCTEwM8+fPx83NjbffftvVYYmIiFw1rVCWImvRokXExMS4ZGyr1YrVanXJ2DExMS5LpIuIiIiIyPVz4MAB2rZty8SJE9m1axeJiYmuDklERCTftEJZirTw8PBSd8iXq1ZFi4iIiIjI9WEYBjNmzGDMmDGkp6czduxYJk6ciIeHh6tDExERyTcllEVERERERESuo/HjxzN58mSqV6/OvHnzaN++vatDEhERuWZKKIuIiIiIiIhcB4ZhYDKZeOihh0hISGDy5Mn4+/u7OiwREZEC0R7KIiIiIiIiIoUoLi6OAQMG8PHHHwNQo0YNZs6cqWSyiIiUCEooi4iIiIiIiBSSn3/+mcjISBYuXMiKFStcHY6IiEihU0JZREREREREpIDS0tIYPXo0t9xyCwkJCcyZM4f58+e7OiwREZFCpz2URURERERERArAMAy6dOnChg0baNeuHXPnzqVmzZquDktEROS6UEJZREREREREpABMJhOPP/44vXr14umnn8Zisbg6JBERketGW16IiIiIiIiI5NOhQ4fo2rUrMTExANx3332MGTNGyWQRESnxlFAWERERERERuUqGYTBr1iwaNWrEypUr+fnnn10dkoiIyA2lLS9ERERERERErsLp06cZPnw4y5Yto1q1asydO5ebb77Z1WGJiIjcUEooi4iIiIiIiPyD06dPExkZSWxsLEOGDGH69On4+/u7OiwREZEbTgllERERERERkX9QoUIF7r//fm6++WbuueceV4cjIiLiMtpDWURERERERCQPq1at4p577sFmswEwffp0JZNFRKTUU0JZRERERERE5C/S09N56qmn6Ny5M8uXL2fHjh2uDklERKTIUEJZRERERERE5E/btm2jadOmTJs2jTZt2rBjxw6aNWvm6rBERESKDCWURURERERERIDvv/+eFi1acPDgQV599VV++eUXwsLCXB2WiIhIkVJkD+WLi4tjypQprF69GrvdTrNmzRg/fjxVq1b9x2svXrzI/PnzWbVqFcePH6dixYp07tyZqKgoPD09c7Rt0aIFiYmJefZjMpnYuXMnVqsVgHfffZd33303z7br1q2jbNmy+bxLERERERERKSratWtH9+7dmTRpEo0bN3Z1OCIiIkVSkUwox8fH07p1a7y8vHjjjTfw8fHh5ZdfplmzZmzcuJGaNWte8fqKFStSsWJFxo8fT5MmTdi7dy/jx49n9uzZ/PrrrwQHB2e3/eSTT3A4HDmuP3r0KLfeeitt2rTJTiYDXLhwgQMHDrBlyxa8vb1zXBMUFFQIdy4iIiIiIiI3imEYzJ49mzNnzvDiiy/i7+/Pt99+6+qwREREirQimVB+7bXXOHz4MIcOHaJ69eoALFmyhLCwMJ555hm++OKLK17fo0cPFixYgK+vLwBNmjQhPDyc1q1bM23aNCZNmpTdNiIiItf1CxcuBOChhx7Ks/9atWpl9y0iIiIiIiLFz5kzZ3jwwQf57rvvqF69Ok8//TQ+Pj6uDktERKTIK5IJ5U8//ZQ2bdpkJ5MBfHx8uPvuu5kzZw4JCQkEBARc9vovvvgCi8WSo6x58+bZW1hcidPpZO7cuQQGBnLvvfcW6D6k4L7qdgZmtHR1GDfUV93OcNfyUFeHISIiIiJSYn355Zc89NBDxMbG8sADD/D2228rmSwiInKVityhfOfPn+ePP/4gMjIyV11kZCQ2m41du3ZdsY+/J5MBNmzYgGEYVK5c+YrX/vzzzxw7doyBAwfi5eWVZ5tbb72VyMhIOnXqxMSJE7l48eIV+xQREREREZGi4cUXX6R3794YhsHnn3/OvHnzCAwMdHVYIiIixUaRW6F89uxZAMqUKZOr7tKhd+fOnctXn5mZmTz55JNYLJbLbmNxyYcffgjkvd2Fu7s7jz32GLfeeivlypXjt99+4+WXX2bOnDmsW7fuigcGnjt3jvPnz+coi4mJAcBut2O32/N1T6XFXctD2fHGr64O44a6q2FDAL0npFjS+1ZEijN9DxMpHW6//Xa2bdvG+++/T4UKFVwdjoiISLFT5BLKlw7IM5tzL56+tPI4vw/7Dz/8MJs3b+aVV16h4Z/JurxcvHiR//3vf7Rq1SrPFdJjxozBze3/p6x58+a0a9eO5s2bM2bMGD777LPL9j1z5kwmTpyYZ11SUhLx8fFXf0OlxKX3Qmmbm9J631K86X0rIiVBUlKSq0MQkesgPT2dF154gVtvvZXOnTvTsmVLvvnmG1eHJSIiUmwVuYTypb2R83qgT0xMBMjXx5GefPJJPv74Y0aPHs348eOv2PaTTz4hIyODESNG5Fn/12TyJQ0bNqRx48YsX778in1HRUXRt2/fHGUxMTH06tULPz8/fcQqD5d+gFDa5qa03rcUb3rfikhJ4Ofn5+oQRKSQ7dixg4EDB7J7926OHj1K586dXR2SiIhIsVfkEspVq1bFx8eHw4cP56q7tEVE3bp1r6qvZ599lrfeeouRI0fy1ltv/WP7Dz/8EH9/f/r375+vmP38/EhLS7tim5CQEEJCQvKsc3NzyzNZLVlK69yU1vuW4k3vWxEpzvQ9TKTkcDgcTJ06lRdeeAHDMJg0aRLjxo1zdVgiIiIlQpF7ajabzfTo0YOVK1eSnp6Op6dndt2yZcto0KABVapU+cd+Jk6cyOTJk4mKiuLdd9/9x/Zbt25l+/btPProo3h7e191vAkJCWzbtu2KW2mIiIiIiIjIjeF0OunWrRsrV66kTp06LFiwgKZNm7o6LBERkRIj90bFRcALL7xAamoqTz31VPZ+ydOmTWP79u38+9//ztF2+vTp1KlTh61bt2aXTZ06lQkTJvDoo49eVTIZYM6cOUDeh/FB1k+4H330UQ4dOpRddurUKe677z7i4+OZMGFCfm5RRERERERErgOz2Uznzp0ZNWoUW7duVTJZRESkkBW5FcqQtS/xt99+y8iRIwkJCcHd3R2z2czcuXPp2bNnjrbnz5/nwIEDpKamAhAXF8czzzwDwIoVK3Jtj9GoUaNch+elp6fz6aef0rx5cxo1apRnTBaLhWbNmtGnTx+OHj2Kn58fp0+fpmnTpixbtoxu3boV0t2LiIiIiIhIfpw7d47x48fz+uuvExwczPjx4zGZTK4OS0REpEQqkgllgK5du3LgwAFOnz6N3W6nUqVKmM25F1Q/8cQTDBw4kGrVqgHg7+/Pvn37Ltuvl5dXrjKTycSGDRsIDg6+YkzDhw9n+PDhpKSkcPbsWcqVK6fDW0RERERERFzo66+/5sEHH+T8+fM0btyYkSNHKpksIiJyHRXZhPIlFSpUuGJ92bJlKVu2bPZri8VCnTp18jWGh4dHvq7x8fGhZs2a+RpDRERERKSwHTt2jA8++IBDhw4RHBxM79696dKly1Vff+jQIRYvXszmzZupVasWU6ZMuW5jiRS2pKQknnjiCT788EOCgoL47LPP6Nevn6vDEhERKfGK5B7KIiIiIiJyZRs2bKB+/frs2LGDu+66i3LlynHbbbfx0ksvXdX1t956K7fffjupqan88ssvrFmz5rqNJVLYjh8/TsOGDfnwww/p1q0bu3btUjJZRETkBinyK5RFRERERCQnh8PB4MGDqV+/Pl999VX2x/sDAwN5+umnufPOO//xILJ33nmH8PBwAObOnXtdxxIpbJUqVaJWrVo8/fTTREVFaYsLERGRG0grlEVEREREipm1a9dy6NAhHnzwwRyJtOHDh2M2m5kzZ84/9nEpmXwjxhIpDLt27WLs2LEYhoHFYmHZsmXaL1lERMQFlFAWERERESlmoqOjAWjSpEmOcn9/fyIiIti4cWOxHEskLw6HgxkzZtCqVSvefPNNduzYAaBEsoiIiItoywsRERERkWLm5MmTAJQvXz5XXWhoKPv27SsyY507d47z58/nKIuJiQGyEoV2u72QIi3+7Ha75uRvjh49ytChQ1m3bh0RERHMnTuX+vXra47Q++VyNC9507zkTfOSN81L3kryvOT3npRQFhEREREpZjIzMwFwc8v9OG+1WsnIyCgyY82cOZOJEyfmWZeUlER8fHyBYywpHA4HycnJAFgsFhdH43orV65k+PDhJCcn88ADDzBx4kT8/Pz0nvmT3i9507zkTfOSN81L3jQveSvJ85KUlJSv9kooi4iIiIgUM76+vgCkpqbmqktOTsbPz6/IjBUVFUXfvn1zlMXExNCrVy/8/PwIDAwstFiLu0urgwICAvJM4Jc2zZs3p2rVqkyZMoXmzZtrXv5G75e8aV7ypnnJm+Ylb5qXvJXkecnvs2PJunsRERERkVIgIiICyNoOoHr16jnqjh07Rp06dYrMWCEhIYSEhORZZ7FYStw/yArq0pyU1nn59ttviY+PZ+DAgdSoUYNdu3bhdDqJj48v1fNyOaX9/XI5mpe8aV7ypnnJm+YlbyV1XvJ7PzqUT0RERESkmOnatSsAq1evzlG+f/9+Tp06Rbdu3YrlWFJ6JScn89BDD9GzZ0/GjRuXvZWK2ax/soqIiBQ1+ttZRERERKSYCQsLo2/fvrz77rscPXoUyNrXb9y4cZQpU4YRI0bkaD9o0CBGjx59Q8YSya9ff/2Vhg0bMnv2bLp06cLGjRvx8PBwdVgiIiJyGSVrfbaIiIiISCnxwQcfMGDAACIjI2nTpg0xMTE4HA6+++47goODc7T9/vvvCQ0NzVH29ttvs3LlSgBiY2NJTU2lV69eANx7770MGDDgmsYSyY8pU6Ywfvx43N3dmT59Oo899phWJYuIiBRxSiiLiIiIiBRD/v7+fPvtt8TExHDo0CGCg4Np1qxZnqeOz58/P9eKz3bt2lG1alUAhgwZkqOudu3a1zyWSH6EhYXRuHFj5s+fT926dV0djoiIiFwFJZRFREREbiCbzcavv/7KxYsXCQ4Opm3btlitVleHJcVYeHg44eHhV2xz66235ipr0qQJTZo0KfSxRK7E6XTy9ttv0717d+rWrUufPn24++679cMJERGRYkSfJRIRERG5AWw2G5MmTaJq1ap06tSJ3r1706lTJ6pVq8akSZOw2WyuDlFE5Lo6fvw4Xbt25cknn2TChAnZ5Uomi4iIFC9aoSxF1t69ewGoV6/eDR/7+PHjANkfA72RYmJitPJHRKSEsdls9OrVi++//56qVavy8ssvExYWxuHDh/nggw948cUXiY6OZunSpVqtLCIljmEYLFiwgMcee4zExEQeffRR3njjDVeHJSIiItdICWWRPLhylVh4eDj9+vVz2fgiIlL4Jk+ezPfff8+gQYP44IMPciSNx40bx4MPPsi8efOYMmUKzz//vAsjFREpXDabjQEDBrBkyRJCQ0P57LPP8tyCRURERIoPJZSlyDIMw2VjX1oVvWfPHpfFICIiJYPNZmPmzJlUrVo1VzIZwGq18sEHH7B69WpmzpzJ2LFjtUpZREoMq9WKv78/ffr0YdasWZQpU8bVIYmIiEgBaQ9lERERkevo119/5cyZMzz44IOXTRRbrVYefPBBTp8+za+//nqDIxQRKVzJyclMmjSJjIwMAGbNmsXixYuVTBYRESkhtEJZRERE5Dq6ePEiAGFhYVdsV7NmzRztRUSKo40bN/LAAw8QExNDSEgIDz/8sD51ISIiUsJohbKIiIjIdRQcHAzA4cOHr9ju999/z9FeRKQ4sdlsvPDCC7Rt25YTJ04wbdo0RowY4eqwRERE5DpQQllERETkOmrbti2hoaF88MEHlz301Waz8cEHH1ChQgXatm17gyMUESmYEydO0Lp1a/7973/TsGFDtmzZwhNPPIHZrH9uioiIlET6G15ERETkOrJarURFRXH8+HEefPDBXEllm83G8OHDOX78OFFRUfpouIgUO8HBwaSkpPDcc8+xcePG7AOuRUREpGTSHsoiIiIi19m4ceOIjo5m3rx5rF69mgcffJCaNWty+PBh5syZw/Hjx7n99tsZO3asq0MVEbkqf/zxB8uWLWPEiBH4+Piwbds2PD09XR2WiIiI3ABKKIuIiIhcZ1arlaVLlzJlyhRmzpzJiy++mF1XoUIFJk2axNixY7U6WUSKPMMwWLhwISNHjiQhIYH27dtTp04dJZNFRERKESWURURERG4Aq9XK888/z9ixY/n111+5ePEiwcHBtG3bVolkESkWLl68yKOPPsrixYspX748X3/9NXXq1HF1WCIiInKDKaEsIiIicgNZrVY6duzo6jBERPJlzZo1DBgwgFOnTnH33Xfz3nvvUa5cOVeHJSIiIi6ghLKIiIiIiIhcka+vLxkZGXz88ccMGjQIk8nk6pBERETERZRQFhERERERkVx+++03UlNT6dixI02bNuXYsWP4+Pi4OiwRERFxMbOrAxAREREREZGiw2azMWHCBNq0acOwYcOw2+0ASiaLiIgIoBXKIiIiIiIi8qcDBw4wcOBANm/eTMOGDZk/fz5ubvpno4iIiPw/rVAWERERERER3nvvPRo3bsyWLVsYO3Ys0dHRREZGujosERERKWL0o2YREREREREhMTGR8uXLM2/ePNq3b+/qcERERKSI0gplERERERGRUurzzz/n3LlzADz11FPs3LlTyWQRERG5Iq1QFhER7rzzTg4fPnzN11+6tl69etd0fVhYGF9//fU1jy8iIiL5ExcXx8iRI1m4cCFDhgzho48+wmKx4Ofn5+rQREREpIhTQllERArMarW6OgQRERG5Sj///DNDhgzh5MmT3HXXXUyZMsXVIYmIiEgxUuoTyk6nE6fTmWedTjMWkdJCq4NFRERKvszMTJ555hnefvttfH19mTNnDkOHDsVkMrk6NBERESlGiuweyvv27eOOO+7A19cXLy8v2rVrx7p1667q2t9++40RI0YQHh6Oh4cH1atXZ9iwYRw7dixX25dffhmr1Yqnp2eur7NnzxZqXCIiIiIiIq7i5ubG7t27ad++PTt37mTYsGFKJouIiEi+FcmE8vHjx2nfvj1ms5mYmBjOnj1LkyZN6Nq1K7/99ts/Xt+y5f+1d+dxNab//8Bf7bSoSEX2SkqRZG8GUxiMNUlja5S1MYvlY4wxPrZhxhgMY18+1jAlS3YjEsq+lZqyVEpkEkXUOef6/eHX+TpzDlpOneL1fDw8Hrru67qv932dc9d93ue6r7sNbt++jS1btiAnJwe7d+/G9evX4ebmhjt37qhsk52dDYlEovDPyspKrXERERERERGVJ4lEgi1btkAIAW1tbYSEhCAiIgINGzbUdGhERERUSVXIhPKsWbOQl5eHjRs3wtraGtWqVcOiRYtQt25dTJ48+Z3tp0yZgmPHjqFt27bQ19eHq6srtm7diqysLCxevFhjcREREREREZWXv//+Gx4eHhg6dCiCg4MBAObm5tDR0dFwZERERFSZVciEclhYGD7++GOYm5vLy3R0dNC7d2+cOnUKDx8+fGv7+fPnK9261bBhQ2hpaeHu3bsai4uIiIiIiKisCSGwYsUKtGjRAufOncPkyZPh7e2t6bCIiIjoPVHhEsppaWnIyspCkyZNlLY1adIEQgjcuHGj2PsNDw+HEELlfgGgfv360NbWRs2aNeHj44PY2NhyiYuIiIiIiEhdMjIy0LNnT4wbNw41a9bEiRMn8Msvv8DAwEDTodUiZ0gAAEpiSURBVBEREdF7QlfTAfzbP//8AwAwMzNT2lY4M7iwTlFlZWXh22+/hbGxMb788kuFbVZWVli5ciW6d+8OCwsLnD9/Hl9//TXatGmDyMhIuLm5qSWuhw8fIjMzU6EsKSkJAORrNlPFw9eFiIjow8G/+/Q+EEIgJiYG/v7+WLJkCapVq6bpkIiIiOg9U+ESyoWEEG8sK86TiF+8eAFvb2+kpKRg27ZtqFu3rsL2sWPHKvzcsWNH7N+/Hw4ODvjuu+9w5MgRtcS1fPlyzJw5U+W2nJwcZGdnF+VwqJxIpVIA4OtCRET0AcnJydF0CEQlkp2djfPnz6NLly6oVasWYmNjYW1tremwiIiI6D1V4RLKFhYWAFQn8grLatSoUaR9FRQUYMCAATh58iRWrFiBQYMGFamdjY0NWrZsiTNnzqgtrnHjxsHHx0ehLCkpCX379oWJiYnKmc+kOYUPKuHrQkRE9OEwMTHRdAhExXb8+HH4+/sjMzMT8fHxqF+/PpPJREREVKYqXEK5du3aqFGjBuLj45W2xcXFQUtLC87Ozu/cj0QiwaBBg7B//3788ccfGD16dLHikMlkCjOOSxuXpaUlLC0tVW7T1dWFrm6FeykI4OtCRET0AeHffapM8vLy8P3332Px4sUwMjLC0qVLUa9ePU2HRURERB+ACvdQPgDo378/IiMjFdYklkgk2Lt3Lzp27IiaNWvKy2UymdJ6dzKZDMOGDcOuXbuwdOlSjBs3rlj9p6am4tKlS+jQoUOJ4yIiIiIiIioLly9fhru7OxYvXoz27dvj6tWrCAwMLNbSgEREREQlVSETytOnT4exsTEGDx6M5ORkZGZmIigoCOnp6ViwYIFC3R9//BF6enqIiooC8Go945EjRyI4OBiLFi3CmDFj5A+9k0gk8rVxgVfr5H700UcICwtDcnIysrOzcfToUfTo0QM6OjqYP39+ieMiIiIiIiIqCykpKUhMTMRPP/2EyMhI2NraajokIiIi+oBUyIRy3bp1ERUVBUNDQ7i5ucHOzg5JSUmIiIiAu7u7Ql0dHR3o6OjIv43PysrCxo0boaOjg0mTJqFKlSoK/zp27KjQ9tdff0VISAg6d+6M2rVrY+TIkWjVqhUuXrwIV1fXEsdFRERERFQenj17hvj4eDx48KDM2sfGxuLKlStK/+7cuVPSsKmYkpKS5Mvv9enTB0lJSZg6dar82R9ERERE5aXCLhTXuHFj7Nq16531Zs6ciZkzZ8p/rlGjhtISGG/Tpk0bbN26Ve1xERERERGVpYKCAkycOBFr165F3bp1cf/+fbi6umLDhg1FmrFanPaenp7Iz89XWqO3c+fOWLRokVqPixQJIbBmzRpMmDAB9vb2uHjxIrS1tbleMhEREWlMhZyhTEREREREbzdx4kSsX78eJ06cQEJCAu7duweZTIauXbsiLy9P7e179OihNEOZyeSylZGRgV69emH06NGoXr06fvvtN2hr8yMcERERaRavRoiIiIiIKpn79+9j+fLlGDNmDFq3bg0AqFatGpYsWYLbt29j/fr1Zdqeyt6uXbvg7OyM/fv3Y+jQobh27Ro6d+6s6bCIiIiImFAmIiIiIqpsDh8+DKlUiq5duyqUt2zZEhYWFggPD1d7e4lEIl/H9/nz56U/CHqrY8eOQQiBP//8E5s2bYKZmZmmQyIiIiICUIHXUCYiIiIiItUKH86maq1kW1tb+XZ1tt+5cyeio6ORm5uLJ0+eoH///liyZAmsra3f2tfDhw+RmZmpUJaUlAQAkEqlxXr+yfsuMjIStra2MDY2xrx58zB16lTUqlXrgx8jiUTC94oKHBfVOC6qcVxU47ioxnFR7X0el+IeExPKRERERESVzJMnTwAAJiYmSttMTEyQkJCg1vZff/01AgMDUbNmTQDA0aNH4efnh86dO+Py5cuoUqXKG/tavny5wkO0X5eTk4Ps7Oy3xvohePHiBebNm4cVK1YgICAAc+bMgY6ODqpWrcrxwasvHnJzcwEAOjo6Go6m4uC4qMZxUY3johrHRTWOi2rv87jk5OQUqz4TykRERERElUzhhxipVKq0TSqVQk9PT63tp06dqvBzly5d8MsvvyAgIADbtm3DiBEj3tjXuHHj4OPjo1CWlJSEvn37wsTE5INfyuHKlSvw9/dHbGwsWrVqhREjRsDU1BS6uvyoVqhw1hTHRRHHRTWOi2ocF9U4LqpxXFR7n8dF1SSDt3m/jp6IiIiI6ANgZWUFAHj06BFq1aqlsC0zM1O+vazaA0DHjh0BANevX39rPUtLS1haWqrcpqOj8959ICsqqVSKX3/9FdOnT4cQArNnz8akSZOQm5sLXV3dD3Zc3qTwvcJxUcRxUY3johrHRTWOi2ocF9Xe13Ep7vHwoXxERERERJWMu7s7AOVk7osXL5CYmIiWLVuWaXvg/5bNMDQ0LHLc9H/y8/OxceNG2NraIjo6Gj/88MN79+GUiIiI3k9MKBMRERERVTKffPIJrK2tsWnTJoXy7du34+XLlxg6dKhC+Y0bNxTWRS5O+7y8PJUxrFu3DgDQrVu3Uh3Lh0QIgTNnzgAAqlativDwcFy6dKlICXwiIiKiioJfgRMRERERVTIGBgZYsWIFvL298eWXX2Lw4MGIj4/HhAkTMGzYMHh6eirU79SpE6ytrXHjxo1it9++fTu2bNmCwYMHo3Hjxnjy5Al27tyJTZs2Yfz48fj444/L9dgrqwcPHmDkyJHYt28fjhw5gi5duqBRo0aaDouIiIio2JhQJiIiIiKqhPr27YvTp09j2bJl+Oabb1C9enUsWLBA5QPyXFxcUKNGjRK1/+KLL+Dg4IBNmzZh48aNKCgogIODAw4fPoyuXbuW6TG+L3bv3o1Ro0YhMzMTgwcPRqtWrTQdEhEREVGJMaFMRERERFRJtW3bFm3btn1nvYiIiFK1b9++Pdq3b1/s+D50T58+xbfffov169fD3NwcO3bswMCBAzUdFhEREVGpMKFMRERERERUBpKTk7FlyxZ069YN69atg42NjaZDIiIiIio1JpSJiIiIiIjU5OXLl0hMTISzszNcXFwQExOD5s2bQ0tLS9OhEREREamFtqYDICIiIiIieh9cu3YNrVu3hqenJzIzMwEArq6uTCYTERHRe4UJZSIiIiIiolKQSqVYsGABWrVqhdjYWIwbNw5mZmaaDouIiIioTHDJCyIiIiIiohK6e/cuhg8fjsjISDg4OGDz5s1o1aqVpsMiIiIiKjOcoUxERERERFRC+/btQ2RkJL788ktcunSJyWQiIiJ673GGMhERERERUTFkZmZCJpPBysoKQUFBaNWqFdq2bavpsIiIiIjKBWcoExERERERFVF4eDicnZ0xYsQICCGgra3NZDIRERF9UJhQJiIiIiIieoecnByMHDkSvXr1Qn5+PoYMGQItLS1Nh0VERERU7rjkBRERERER0VucPn0aQ4cOxZ07d+Dl5YUNGzagTp06mg6LiIiISCM4Q5mIiIiIiOgt5s2bh/v37+P333/H4cOHmUwmIiKiDxpnKBMREREREf1LUlISGjVqBG1tbaxZswbZ2dlwdHTUdFhEREREGscZykRERERERP+fTCbDb7/9hqZNm+KPP/4AANSqVYvJZCIiIqL/jzOUiYiIiIiIACQnJ8Pf3x8nTpyAvb09WrduremQiIiIiCoczlAmIiIiIqIPmhACmzZtQrNmzXDixAmMHTsWly9fRps2bTQdGhEREVGFwxnKRERERET0QXvy5AkmTZoEIyMj7NixA59++qmmQyIiIiKqsJhQJiIiIiKiD1JycjLq168PMzMz7N27F/b29qhRo4amwyIiIiKq0LjkBRERERERfVByc3MxZswYODg44Pr16wCAtm3bMplMREREVAScoUxERERERB+Ms2fPYujQobh16xY6d+4MMzMzTYdEREREVKlwhjIREREREb338vPz8cMPP8DDwwP37t3DokWLcOzYMdStW1fToRERERFVKpyhTERERERE770zZ85g7ty5aNGiBbZs2QInJydNh0RERERUKTGhTO+l3r1749atWyVuX9i2adOmJWpva2uLvXv3lrh/IiIiIio9mUyGzMxMWFlZoVOnTtizZw8+/fRT6Ovrazo0IiIiokqLS14QqaCnpwc9PT1Nh0FEREREJZSamoquXbvik08+wYsXLwC8mnTAZDIRERFR6XCGMr2XODuYiIiI6MMkhMC2bdsQFBSEJ0+eYPTo0ZDJZJoOi4iIiOi9UaETykII3Lp1CxKJBLa2tsWeMZqbm4uUlBTUrl37nU9vLkrd9PR0pKenq9zm6uoKXd0KPZxERERERO+1rKwsjB07Fjt37oSVlRW2bt2Knj17ajosIiIiovdKhc2A7t+/H2PHjoVUKkWVKlXw+PFj/Pzzzxg5cuQ7265evRpr1qxBbGwsGjRogJSUFDRu3BgLFy5E586dS1V35syZaNGiBbS1FVcLOXz4MGrUqFH6AyciIiIiohJZtGgRdu7ciX79+mHVqlWoWbOmpkMiIiIieu9UyDWUL168iH79+sHHxwepqam4desWfv75Z4waNQphYWHvbD9mzBh89NFHSE9PR1xcHDIyMmBra4uuXbsiJiamxHULRUZG4sKFCwr/mEwmIiIiIip/z549w/PnzwEA33//PXbu3InQ0FAmk4mIiIjKSIVMKM+ZMwfVqlXDvHnz5DOBR44ciVatWmH69OnvbL9161b89ttv8qUrjI2N8ccff0AikWDNmjUlrktERERERBVHTEwMWrRogcmTJwMAqlatCh8fH2hpaWk4MiIiIqL3V4VLKEulUhw+fBgdO3ZUegJzt27dEBsbi+Tk5Lfuw8/PT6nMyMgIWlpaePToUYnrFkpPT8fVq1eRlpb2rsMhIiIiIiI1KygowIwZM9ChQwckJyejUaNGmg6JiIiI6INR4dZQTklJQV5ensqLQltbWwBAfHw86tevX6z9bty4EUIItG3bttR127VrBwsLC9y+fRsNGjTAzz//jP79+791nw8fPkRmZqZCWVJSEgBAIpFAIpEU8UiIiIiIqCzweqxyiI+Px9ChQ3HhwgU0b94cmzdvhouLi6bDIiIiIvpgVLiE8pMnTwAAJiYmStsKy7Kzs4u1z8TERHz//fewsbHBuHHjSly3efPmiI6ORps2bQAAjx49QmBgIAYMGIA9e/agV69eb9zv8uXLMXPmTJXbcnJyin1MRERERKReOTk5mg6B3kEIAX9/f1y8eBHfffcd/vvf/8LAwEDTYRERERF9UCpcQllX91VIUqlUaVthmZ6eXpH3l5GRgR49ekAikSAkJATVqlUrcd1+/fop/GxhYYEtW7agYcOGmDVr1lsTyuPGjYOPj49CWVJSEvr27QsTExP5Gs5EREREpBmqJjRQxfDkyROYmppCS0sLa9aswZMnT+Dh4aHpsIiIiIg+SBUuoWxtbQ0AKtcvLlwyorDOuzx69AheXl5IS0vDgQMH3rrcRXHqvs7Y2BgtWrTAqVOn3lrP0tISlpaWKrfp6urKE+lEREREpBm8HquYtm/fjnHjxmHjxo3o1asXl7cgIiIi0rAK91A+CwsL1K9fH9euXVPadvXqVejr6xfpIvLx48fo0qULbt++jX379qFTp05qqfum9oaGhsVqQ0REREREb5aVlQU/Pz/4+flBT0+PCX8iIiKiCqLCJZQBYPDgwThz5oz8oXUA8PTpU+zatUu+RESh9PR0XLhwAbm5ufKynJwcdOvWDQkJCdi7dy88PT3f2Fdx6j5//lyp7OLFi7hy5Qq6d+9e3MMkIiIiIiIVjh49ChcXF2zfvh19+vTB9evXeb1NREREVEFUyK/5p0yZgr1796J3796YN28eDA0N8dNPP0FPTw8LFixQqLt8+XLMnTsXp06dgoeHByQSCXr27Inz589j3rx5MDMzw4ULF+T1jY2N0aRJEwAoVl2pVApnZ2cMHToUbm5uMDU1xfnz5zF//nw0aNAAv/zySzmMDBERERHR/0lNTcX69euRmJiI6tWrw9vbGx07diyT9qXtq6gePnyIPn36QFdXF+vXr4e/vz+0tLTU3g8RERERlUyFTChXq1YNZ86cwaJFi/D7779DIpHA3d0dwcHBSusn29jYoGXLlvJZy8+ePcPz58/RsmVLhISEICQkRKG+i4sLNmzYUOy6Ojo6uHDhAtatW4eNGzciIyMDlpaWmDFjBkaMGAFjY+OyGg4iIiIiIiUxMTHo2rUrOnToAD8/P8THx6NLly6YPn06pk+frtb2pe2rKHJzc2FsbAxLS0ts3rwZbm5uaNiwoVr2TURERETqUyETysCrp2z/+OOP76w3duxYjB07Vv6zqampwizjtylOXQCoXr06Jk+eXOT6RERERERlQSaTYfjw4WjSpAnCw8Ohrf1qJbvC69WePXvCzc1NLe1L29e7SCQS/PTTT1i+fDkuX76MWrVqwdvbu8T7IyIiIqKyVSHXUCYiIiIiojc7deoUEhISMHLkSHmCFwACAwOhra2NdevWqa19aft6m+TkZHTo0AEzZsyApaUlnj59WuJ9EREREVH5qLAzlImIiIiISLWYmBgAQMuWLRXKTU1NYW9vj+joaLW1L21fbzNw4EDk5+dj8uTJmD17NgwMDEq8LyIiIiIqH0woa9DLly8BAElJSRqOhIiIiIgKr8kKr9Eqsnv37gEArKyslLZZWVkhPj5ebe1L29fDhw+RmZmpUBYXFwfg1UOwf/nlF7Rs2fKd+/kQSKVS5OTkwMTEBDo6OpoOp8LguKjGcVGN46Iax0U1jotqHBfV3udxKe51MBPKGpSamgoA6Nu3r2YDISIiIiK51NTUUq0JXB7y8/MBALq6ypfzenp6ePHihdral7av5cuXY+bMmSq3/fPPPwgICHhreyIiIiIqH0W9DmZCWYM6duyI3bt3o27dury9r4JJSkpC3759sXv3btjZ2Wk6HKIKj+cMUfHwnKmYXr58idTUVHTs2FHTobyTsbExAOD58+dK2549ewYTExO1tS9tX+PGjYOPj49C2ZUrVzBkyBDs3LkTTk5Ob23/IeHvBtU4LqpxXFTjuKjGcVGN46Iax0W193lcinsdzISyBpmZmaFPnz6aDoPews7ODk2bNtV0GESVBs8ZouLhOVPxVPSZyYXs7e0BvHqoXYMGDRS2JScnw8HBQW3tS9uXpaUlLC0tVW5zcnLiOaACfzeoxnFRjeOiGsdFNY6LahwX1Tguqr2v41Kc62Dtd1chIiIiIqKKxNPTEwBw4sQJhfK///4b6enp6NKli9ral7YvIiIiInq/MKFMRERERFTJ2NnZoX///vjjjz+QkpIC4NWDYqZOnYrq1atj1KhRCvVHjBiBCRMmlKh9cfsiIiIiovcbl7wgIiIiIqqE1q9fD19fX7i4uMDDwwOJiYnIy8vDvn37UKNGDYW6e/fuhbW1dYnbF6cuEREREb3fmFAmUqFmzZqYMWMGatasqelQiCoFnjNExcNzhtTB1NQUhw4dQnx8PBITE1G9enW0bt0aenp6SnU3bNiAKlWqlLh9ceoWBc8B1TguqnFcVOO4qMZxUY3johrHRTWOi2ocl/+jJYQQmg6CiIiIiIiIiIiIiCo+rqFMREREREREREREREXChDIRERERERERERERFQkTykRERERERERERERUJEwoU6Vx6dIl7Nq1S9NhEFUaMTEx2Lt3r6bDIKo0Tp8+jQMHDmg6DCIiIiIiogqND+WjSuPLL7/Eli1bkJ2drelQSqygoAAnTpzAP//8Aw8PD9SpU0fTIdF7zN/fH8eOHcO9e/c0HUqJSKVS3Lx5E3fu3IGlpSWcnJxgYmKi6bDoPTZgwADcuHED8fHxmg6lxJKSkpCQkABDQ0M4ODigdu3amg6J3mORkZEICQlBVlYW7OzsEBAQgLp165ZJ+9L2VZ5OnjyJ0NBQZGVloXHjxhgxYkSxrvmK0j42NhYzZsxQ2f7bb79Fhw4dSnUMZSEiIgK7du3C48eP4eDggBEjRsDGxqZIbWUyGSIjIxEaGor79+8jMDAQn376aZn0Vd7++usvhIWFITs7G02aNEFAQABq1apVpLYymUz+fsnIyMDo0aPRpUsXpXpXrlzBnDlzVO5jypQpaNWqVamOoSwcPXoUe/bsQXZ2NhwdHREQEABra+sitb169SrCw8ORmJgIc3NzuLu7Y9CgQdDR0VF7X+Xt8OHD2Lt3L548eQInJycEBATAysqqSG0vX76M/fv3IykpCdWrV4e7uzt8fX2VxuX8+fP4+eefVe7jhx9+gKura2kPQ+0OHjyIffv2IScnB05OTggMDETNmjWL1DY9PR07d+7EjRs3oKOjA3t7ewwbNgyWlpZq76u87d+/H+Hh4cjNzUXTpk0RGBgICwuLIrVNS0vDjh07EBcXB11dXfm4/PtYT58+jUWLFqncx8yZM9G0adNSH4e6hYeHY//+/cjNzYWzszMCAwNRo0aNYu/nwIEDWL9+PczNzbFmzZoy7asi4gxlonKQnJws/+Azfvx4+Pn5ITo6WtNhEVVY8+bNQ4MGDdClSxesWbMGgYGBqFev3hsvVog+dAcPHoSbmxt69eqFVatW4fvvv0e9evXg6+tbqb+IpYpr9uzZ8PLygqGhIbp164ZLly6hadOmiImJUXv70vZVnmbMmIEuXbrA2NgY3bp1Q0xMDJo2bYoLFy6otX1mZiZCQ0Nhb2+PQYMGKfyrX79+WRxaqUybNg2ffvopTE1N0a1bN5w5cwbOzs64fPnyO9smJCSgXr16+O9//wuJRILQ0FAkJSWVSV/lbcqUKejZsyeqV6+Obt264dSpU2jatCmuXr36zrZxcXGoW7cuZs2ahYKCAoSGhuLWrVsq62ZkZCA0NBRNmzZVer9UxET7xIkT0atXL9SoUQNdu3bFiRMn4OzsjBs3bryz7aBBg+Dm5oabN2+iU6dOsLa2xn/+8x80b94caWlpau2rvH399dfo27cvLC0t0bVrVxw7dgzOzs6Ii4t7Z1tvb2+0atUKCQkJ6NSpEywtLTFx4kS0aNEC9+/fV6iblpaG0NBQuLq6Kr1fivplR3kKCgqCt7c3rK2t4eXlhcOHD8PZ2RkJCQnvbLtx40Z4eXkhPT0dHh4ecHZ2xo4dO1C/fn2Vd3qWpq/yNmbMGPj6+sLGxgZeXl44cOAAnJ2dkZiY+M6269atQ7du3fDgwQN4eHjAyckJ27ZtQ/369bF//36FuqmpqQgNDUXr1q2V3i9vSsprUmBgIPz8/FCnTh14eXlh3759cHFxeePvzzd58OABhg8fjj179mDfvn1l2leFJYgqiaCgIGFqaqrpMErk8uXLYt26dSIrK0sEBwcLAOLPP//UdFj0nhs+fLiwsbHRdBglYmNjI+bOnSvy8/PlZfPmzRMAxIYNGzQXGL3XvL29hYODg6bDKJE9e/aIxMREhbK//vpL6OjoiOHDh2smKHpvXbp0SWhpaYn58+fLy2QymWjfvr1o3LixkEgkamtf2r7K0/nz54WWlpb49ddf5WVSqVS0bt1aODo6CqlUqrb2ERERAoDYt2+f+g9Ezc6ePSsAiMWLF8vLpFKpaNmypXB2dhYymeyt7R8/fizu378vhPi/4166dGmZ9FWeTp06JQCIZcuWycskEolwdXUVrq6u72yflZUlMjIyhBBCHD16VAAQK1asUFn34MGDAoA4evSoeoIvQydOnBAAxMqVK+VlBQUFwtnZWbRs2fKd7bt37y4iIyMVypKSkoSBgYH4/PPP1dpXeTp27JgAINauXSsvy8/PF46OjqJNmzbvbN+tWzcRFRWlUPb3338LfX19MWzYMIXysLAwAUCcOnVKPcGXoUOHDgkA4n//+5+87OXLl6Jx48aiQ4cO72yfnp6u9Ls5Ly9PNGzYUNSvX1+tfZWn8PBwAUBs2bJFXvbixQthZ2cnOnbs+M72aWlpSr8vnz9/LurVqydsbW0VygvzG+fPn1dL7GVp9+7dAoDYvn27vKzw9fb09CzWvj777DPh4eEh2rdvL6ysrMq0r4qKCWWqNFQllGNjY0VwcLAIDg4W27dvFwcOHJBfcBaKiYkRO3bsUHkBmZqaKoKDg0V6erpC+ePHj8WRI0fErl27xLVr15TanT17VuzZs0cIIcTTp0/FoUOHxOHDh4t0HEwoU3lRlVC+evWq/JzZsWOHOHTokHjw4IFCndOnT7/x/Xnnzh0RHBwsHj58qFD+zz//iMOHD4uwsDARGxur1O7UqVNi//79QohX59fBgwfFX3/99cbYVV3ASqVSYWxsLLy8vN7Yjqg0VCWUL168qHDOHD58WDx69EihzsmTJ0VYWJjKfSYmJorg4GCRlZWlUJ6ZmSkOHjwowsLCREJCglK7iIgI+d+VR48eif379yt9QC4KR0dHYWdnV+x2RG/z5ZdfCi0tLfHPP/8olP/vf/8TAMTx48fV1r60fZWnMWPGCG1tbZGdna1QvnbtWgHgnedwcdpXpoRyYGCg0NXVFTk5OQrlK1euFADEmTNniryvdyWU1dlXWfP39xf6+vri2bNnCuXLli0TAMS5c+eKvK/3KaE8ZMgQYWBgIPLy8hTKFy9eLACIS5cuvbX9v8+fQq6urqJOnTpq7as8DRo0SBgaGoqXL18qlP/6668CgMrPrK9707g4OzuLBg0aKJRVpoTygAEDhLGxscIkFCGEmD9/vgAg4uLiSrTfHj16iKpVq5ZLX2Whb9++wtTUVOlL1zlz5ggA4u+//y7Rfrt27SqMjY0VyipTQvmzzz4T1atXV/oS4b///a8AIG7fvl2k/axevVpUqVJFxMfHiw4dOqhMKKurr4pMt6xmPhOVh4SEBOzevRvAq/VWU1JScPHiRQQFBWHJkiUAgLt378LX1xfVqlVTWmtt9uzZ2LRpk/w2HyEE/vvf/+KXX36Bo6MjrK2tERMTg2bNmuHPP/+Urze0bNkynDhxAgYGBvj2229Rt25d5OXloWvXruV38EQlEBsbiz179gB4dc7cuXMHV69exaRJkzBv3jwAQHx8PAICAnDq1Cl4eHgotJ82bRrCw8Pl54xUKsX333+PxYsXo1mzZrCwsEB0dDTatm2L4OBgmJmZAQB+/fVXxMfHo6CgAFOnTkWdOnWgra2NTz75RGWc/+63sC8hBJ4/f66u4SB6p6tXr+Lw4cMAAIlEglu3biE2NhY//vgjfvjhBwCv1qb8+uuvcfnyZaV1BSdOnIizZ8/Kb7MtKCjApEmTsHLlSri6usLc3BxnzpyBp6cnNm/eDGNjYwDAnDlzkJ2djX/++QezZ8+GjY0NTExM8NFHHxU59uzsbKSlpcHLy0sNI0H0f2JiYtCwYUNUr15dodzd3R0AEB0djc6dO6ulfWn7Kk8xMTGws7ODqampQvnrsb7tHC5J+7Vr12Lbtm0wNDSEu7s7Pv/8c1SrVk0dh6M2MTExaNy4sfz3W6HXj6tdu3aVrq/SiomJQZMmTWBoaKhQ/nqs6l7bePny5Vi/fj2MjIzQqlUrfP7550pjpWmFy7xUqVJFofz1cWnRosUb2//7/AGAly9fIjk5WWl5j9L2VZ5iYmLg7OwMfX19hfLXY3VxcXlje1XjkpeXh5SUFDRq1EhlmyVLlmD58uUwNjZGmzZtMGjQIBgZGZXiKNSv8LO6np6eQvnr4+Lo6FisfSYmJuL06dPo379/mfdVVmJiYtC8eXOl9bFfj9Xe3r5Y+0xISMDZs2fh7e2tcvuvv/4KLS0tVKtWDW3btsWgQYNQtWrVkh1AGYmJiYGrqyu0tRVX/319XBo2bPjWfdy5cwcTJkzAjz/+CAcHhzLtq6LjGspUqfXr1w/bt2/H9u3b8eeffyImJgZHjhzB0qVLERoaKq9Tq1YtLF++XKHt06dPsXXrVnh7e8uTXvPmzcPs2bOxefNmXLp0CQcOHEB8fDwyMjIwdOhQhfa5ubnYtGkTLl68iMOHD2PDhg3lcsxEpeHn56dwzly4cAG7du3C/Pnz5UkzPz8/mJubK50zmZmZCAkJgZ+fn/zDz/Tp07Fw4UKEhYXh/PnzOHjwIGJjY5GQkICRI0cqtM/KysKuXbtw5coVHDlyBH/88UexYv/f//6HZ8+ewdPTsxQjQFQ8X3zxhfycCQkJweXLl7Fx40b8+OOPiIqKAgAMHz4cRkZGSudMSkoK9u/fj2HDhsk/fEycOBErV67EoUOHEBMTg0OHDuH69es4d+4cxo8fr9A+PT0dx48fx7Vr13D06FEsWLDgnfEeOHAAwcHB+O233/DRRx+hRYsW+P3339U0GkSv3Lt3T+VDoArLVK1TWtL2pe2rPJXnuACAvb09mjVrhm7dusHGxgbTp0+Ho6Mjrl+/XtJDKBPl+Rp+SO+X4mrSpIn8/WJlZYXvvvsOTk5OFe5BtGUxLvPnz8fjx48xZMiQMu+rrJRFrD/99BOePn2qNC4A0LRpU7i4uKBbt26wsLDAxIkT4eLi8tb1yzUhLS1NLeMyd+5cDBgwAJ06dYKbmxu++uorrF+/vkz6KmsymQz3799XS6yzZs3CgAED0LFjR7i7u2PixIkqHz7XvHlzNG3aFJ9++inMzc3x9ddfo1mzZkhOTi7dwahRQUEBMjMzSzUuMpkMw4YNg52dHSZPnlymfVUGnKFMlZ5MJsOVK1eQkpKCly9fQgiB6tWr4/jx4/D29oaenh5GjhyJOXPmIDk5Wf6gko0bN+LZs2cYMWIEACA/Px8///wz+vfvjwEDBsj3X7NmTXz//fcYNmwYEhMT5d/kPXnyBFOmTJF/62Zra1vOR05UMlKpFJcuXcK9e/fw8uVLAICRkRGOHz+Obt26oWrVqvjiiy+wbNkyPHz4UP4whbVr1yI/P19+zuTm5mLRokUYMmQIevToId9/7dq1MXnyZAQFBSE9PR21a9cG8CohPWPGDPnMiuKcMzdv3sTEiRNRt25dTJgwQS3jQFRUEokEFy9eRFpaGvLz8wEAurq6iIiIgIeHB0xNTTF48GBs3boVCxYskM8CWrVqFaRSqfycefToEVasWIHRo0crzKisX78+vvnmG0ydOhWLFi2Sf8n58OFDzJo1C7q6ry7XinLO/PXXX0hNTUVKSgpSU1PRtWtXmJubq3M4iJCfny9/X76u8IuTFy9eqK19afsqT+U5Li1atEBsbKzCTLlRo0ahWbNm8PPzq1APFCvP1/BDer8UR5s2bXD9+nWF/kaOHIlmzZphyJAhRX5oZHlQ97gcOnQIs2fPhru7O7799tsy7ausCCFQUFCg1ljDw8Px008/oU2bNvjqq68Utn388ce4evWqwuzWESNGwNXVFf7+/vIv1DVNIpFAJpOpZVw6duyIxo0bIz09Hc+fP8fKlSvRvn17+R3O6uyrrL1+rfpvxY21c+fOcHR0RFpaGp49e4bly5ejffv26NKli7xOly5dMHDgQIWZuMOHD4ebmxsCAgJw7Nix0hyO2hR+7i3NuCxYsABnz57FuXPnVO5HnX1VBkwoU6UWFRWFIUOG4NmzZ3Bzc4OZmRm0tLSQn5+P9PR0eb1Ro0bhp59+wurVqzF37lwAwMqVK9GwYUP5h/q4uDg8ffoUVatWRUhICIQQAF79AU9NTQUA3LhxQ55Q1tfXf+ttRUQV0bFjx+Dv74+CggK0aNECpqam0NLSgkwmUzhnxo4di0WLFmHdunWYOnUqZDIZVq9eDWdnZ7Ru3RrAq9v8X7x4AT09PaVzJiMjA0IIxMbGyhPK5ubmb7yl7m1SU1PRvXt3aGlpISwsTJ5sIyoP4eHhCAwMhLa2Npo3b45q1apBS0sLABTOmaCgIKxevRqbNm3C+PHjUVBQgHXr1qFt27ZwcnICAFy4cAESiQRaWlpK58yjR48glUpx8+ZN+e3YtWvXLvbT1BcuXCj/f3R0NDw9PZGQkIDw8PBSjQPR64yNjVUuP/Ts2TMAgImJidral7av8lSe46Lq9nUbGxsMHToUS5YsQUJCwltvxS1P5fkafkjvl+JQ9cVi/fr14efnh1WrVilMutE0dY5LVFQUvL29YWdnh/DwcKXlIirL+0VLSwtGRkZqi/XkyZMYOHAgHBwcsG/fPqUlHP69xBAA2NnZYcCAAdi4cSMePHigcuZledPV1YWBgYFaxuX15fbGjx+Pjh07YtCgQbh79y7MzMzU2ldZq1KlCnR1ddUS6+vLLI0fPx4fffQRfH19cffuXfnySjVq1FBq5+joiD59+mDnzp3Izs6uEJ/fDA0Noa2tXeJxiYuLw48//ohJkybBzc2tTPuqLJhQpkpLKpVi4MCBaNy4MY4cOaJwgWBpaSn/oA68usDu06cP1q1bhxkzZuDMmTOIi4vD7Nmz5YmBvLw8AMDff/+NkJAQpf58fX1Rs2ZN+c8WFhbytkSVwYsXL+Dr64s2bdpg7969Ct+YGhkZKZwzdnZ26Nq1K1atWoUpU6bgwIEDuHv3LhYtWiSvU3jOxMXFIScnR6k/X19fhQ8wr58/RXX//n14enoiKysLhw8fRsuWLYu9D6KSevr0Kfz8/NCtWzfs2LFDPlNHIpGgSpUqCudMs2bN0KFDB6xYsQLjx49HaGgoHjx4gDlz5sjrFJ4z165dQ2ZmplJ/vr6+CheXJTlnXle4ft369euRmZlZ6v0RFbK3t8eVK1eUygtvbX3XuozFaV/avsqTvb29yuUDijMupWkPANbW1gBeLTNVUdjb2+Pu3btK5WXxGpZnX6Vlb2+v8nbw8oz19fdLRUkoq2tczp07hx49esDGxgbHjx9XmQCtCK9BUakr1rNnz+Kzzz5DvXr1cPz48WJdG7z+fqkICWWgbF5DbW1teHt7IyoqCleuXEGnTp3KrK+yYmdnp/ZYdXR00L9/f0yePBnXrl1T+cyb11lbW0MIgcePH1eIhLK2tjYaNWpU4nG5dOkS8vPzcePGDYU72uPj45Gbm4sBAwagXbt2mDhxYqn7qiyYUKZK686dO7h//z6mT5+ukEx+8OCByg/q48aNQ2hoKHbt2oWwsDBoa2vD399fvt3W1hZaWlr45JNP5A8nexsmk6myiY+PR1ZWFvz8/BSSybdv31b57em4cePQp08fHDhwACtWrIC+vr7CGmt2dnYAgB49emDatGnv7L+450xmZia8vLxw//59HDp0qMI8RIc+HNeuXUNubi4GDx6scNtnXFwcpFKpUv1x48Zh8ODBOHHiBFasWAEjIyP4+vrKtxeeM/3798fXX3/9zv7V8XemcL3zwmQ2kTp4enri2LFjiI2NRdOmTeXlx48fBwCFW2FL2760fZUnT09PnDhxQml2cHHGpTTtgVcfeLW0tEp0R1BZ8fT0xMyZM3Hr1i2FpXuOHz8OLS0ttT44tDz7Ki1PT0/MnTtXaXZwYazl8cyIS5cuQUdHBw0aNCjzvorK09MTP//8M+7du4c6derIy48fP/7WBzq/7vLly/K1oiMiIuR3y5VFX+XF09MTv/32G+7fv69w99Lx48eho6NTpIeTXrx4Ed27d0ft2rUREREhTxAX1aVLl6Cvr4969eoVO/6y4unpiaVLlyos0we8Ghc9PT15Mri4Hj16BEBxdn9Z9VUWPD09sWrVKvzzzz8KM4iPHz8OfX19dOzYsUT7VTUub3Lp0iUYGhq+8fzTBE9PT6xfvx6PHz9WOIbjx4+jSpUqb31w7scff4w///xTqTw2NhZSqRSDBg1S+D1Smr4qDUFUSQQFBQlTU1P5zzk5OUJLS0tMnjxZod748eNFlSpVRJ8+fZT24ejoKJo1ayb09PRE9+7dlbb37dtX1KxZU6Snpyttu3v3rpDJZEIIIQYPHixsbGxKdBzBwcECgPjzzz9L1J6oqIYPH67wPn3w4IEAIGbOnKlQLyAgQBgYGIjBgwcrlEulUlG/fn3h5uYmtLW1hY+Pj1IfXl5ewsbGRjx69Ehp2+3bt+X/79Onj3BwcChy7FlZWaJ58+bC2NhYnDp1qsjtiErD29tb4X1669YtAUAsXLhQXiaTycTgwYOFrq6uGD16tEL7ly9fCisrK9GyZUsBQHzxxRdKfbRt21Y0atRIPHnyRGnb6+eMp6enaNmyZZFjT05OVirLysoS9erVE02aNCnyfoiK4tGjR6J69eqid+/eQiKRCCGEuHfvnrCyshLe3t4KdW/cuCG8vb3Fjh07StS+OHU17eHDh8LMzEz0799fHmtKSoqoWbOm8PX1Vah77do14e3tLUJCQkrUfuPGjSIjI0OhbMuWLUJLS0vp77mmZWRkiGrVqgkfHx8hlUqFEK+uq2vUqKEU6+XLl4W3t7cICwtTua+IiAgBQCxdurTUfWlaWlqaMDExEX5+fvJYb9++LczNzcXw4cMV6l64cEF4e3uLvXv3qtzX0aNHBQCxYsUKldvXr18vHj58qFC2bt06AUAEBASU/mDUKDU1VRgZGYkhQ4bIP3slJiYKU1NTpVjPnTsnvL29RXh4uLwsNjZWWFhYCDs7O3Hv3j219aVpd+/eFYaGhsLf318ea0JCgjAxMVG6Hjl79qzw9vYWBw8elJddu3ZNVK9eXTRu3FikpaW9ta+1a9cqXdsvX75cABBBQUFqOiL1uHXrlqhSpYrC63Xz5k1hbGysFGtUVJTw9vYWR44ckZetXr1aPH78WKHe+fPnhZmZmWjVqpV8rIvbl6YlJiYKAwMDhffGjRs3hKGhofj6668V6kZGRgpvb29x7NgxedmqVatEdna2Qr2YmBhhamoq2rVrp1C+evVqkZWVJf9ZJpOJ3377TQAQEydOVONRlV58fLzQ19dXeL2uXbsmqlatqhRrRESE8Pb2FhEREW/dZ4cOHYSVlVWp+qqsOEOZKi1jY2OMGzcOS5YsgZaWFmxtbXHkyBHUq1fvjbdsjRs3DuPHjwcABAQEKG1fu3YtevbsCRcXF4wcORL29vZ49OgRzp07h6tXryIhIaFEM8by8/Oxa9cuAK9uMwKA06dPQyKRAID84YFEZcnS0hLDhw/H/Pnz8fLlS9SrVw/79++Hq6srLCwslOpra2tjzJgxmDp1KgDV58zmzZvRo0cPNG3aFCNHjkSjRo3w4MEDREdH49atWyV+0nyPHj1w9epVBAUF4d69e9i+fbt8m5GREXr16lWi/RIVR6NGjTBgwADMmDEDT548Qa1atbB792588sknKtck1tfXR2BgoHytflXnzI4dO9C9e3c4OTkhMDAQDRo0QHp6Os6cOYNHjx4hOjq6RLH26NEDzZo1g5ubGywsLHD79m1s2LABQghs27atRPskepMaNWogPDwcvr6+aNKkCRwcHBAVFYXWrVtj3bp1CnUzMzMRGhoKZ2fnErUvTl1Nq1mzJvbt24dBgwbB0dER9vb2iIqKQvv27bF69WqFug8ePEBoaChcXV1L1F4ikeCjjz6CoaEhbGxscOvWLdy+fRsBAQH4/fffy+Nwi8zKygr79u2Dn58fHB0dYWdnh1OnTuHjjz/GypUrFepmZGQgNDQU7u7uCuW+vr6QSqXyuxDXrVuHEydOAAB+//13+Qy44vSlabVr18aePXswePBgODk5wdbWFpGRkejcuTP++OMPhbr3799HaGgo2rZtq3ANVHjb9cOHDwEAq1evlj8Aa9myZfIZqPn5+WjXrh1MTExQu3ZtJCYmIjk5Wf7MjIqkTp062L17N4YMGQInJyc0bNgQp06dgpeXl9J7Oy0tDaGhoQq33vv6+uLRo0do3Lix0t1AOjo62LFjR4n60rT69esjLCwMQ4cOhbOzM+rXr4/IyEh8+umnSq/hvXv3EBoaqjBjduDAgcjKyoKjo6PSQ/j09fUVrhWeP3+O1q1bw9TUFLVq1UJCQgLu3buHr776Cr/++muZHmdxNWrUCLt27cKwYcMQHR2NevXq4eTJk+jZs6fCcyWAV89lCQ0NhZeXl/yOj9zcXLi7u8PMzAzW1tZIS0vD9evX0bt3b6xcuVLhs39x+tI0Ozs7hISEwN/fH6dPn0adOnUQGRmJ3r1745dfflGom5ycjNDQUPkDCIFXy765ubnB3NxcYVz69u2r9Ls0JycHLVu2hLm5OaysrJCQkID79+9j0qRJmD9/frkcb1E5ODjgzz//xBdffIHIyEjY2Njg5MmT6N+/v9Jd6nfv3kVoaCg+++yzMu+rstIS4rUFAIkqsA0bNiAqKkrhw4MQAjt37pRfUHbt2hX9+vXDxIkTUbt2bUycOFFhH0+fPkX16tVRvXp1pKWlqUziymQyhIeHIyIiAllZWahduzbatGmDXr16yW95/uOPPxAXF6d0sfcmOTk5GDly5Bu3r1+/Xn5bMpG6rFixAjdu3FB4n8pkMmzbtg1RUVHQ1tZGz5490bNnTwQFBcHJyQlBQUEK+3jw4AGsra1Rt25d3L17V+HpvYUkEgn27t2LkydPIjs7G3Xq1EG7du3Qo0cPef2FCxciPT29yBdbgwcPVrmkAPDqA/fSpUuLOgxERbZ48WKkpqYqvE+lUik2bdqE6Oho6OnpoU+fPujSpQsCAgLQvn17paTx7du3YWtrCwcHB5VroQJAQUEBwsLCEBUVhSdPnqBevXrw8PBA165d5R9c5s6di+fPn8uT0+8ik8lw4MABnD59GmlpaTAzM0OrVq3Qv39/GBkZlXBEiN5OIpEgJiYGWVlZsLOzg6Ojo1KdzMxMnDx5Ek5OTvIHVBanfUnqalpBQQFiYmLw+PFj2Nvbo0mTJkp1Hj58iMjISDg7OyttL0p7APIHed65cwfGxsZwdXUt0m3ImlJQUIDo6GhkZ2ejcePGKh8a+ODBA5w6dQouLi4K23ft2gWZTKZyv926dVN6uFFR+qoo8vPzERMTgydPnqBx48Zo3LixUp2MjAxERUWhWbNmCttVPfelUPfu3RV+/0skEsTFxSE5ORkmJibyhzNXVPn5+YiOjsaTJ0/QpEkTleuNFn4h6+rqKl9W6uDBg/KHXv2btrY2+vfvX6K+KoqXL18iOjoaOTk5aNKkify4X5eWloazZ8+iRYsW8qVfDhw4oHKJO+BVor1fv34KZQUFBfL3i6mpKVq0aCF/CFtF9Pq4ODo6Kix5U+jevXuIjo6Gm5ubwrJABQUFuHnzJu7evQtjY2M0a9ZM5WSb4vRVUbx48QLR0dHIzc2Fk5OTyuWQUlJScO7cObi7uyssf/P6e8DExAQuLi5vHJf8/HzExcUhJSUFZmZmaNGiRYV+6FxeXh5iYmKQm5sLZ2dnlcv+JCcn4/z582jVqtVb15g/ceIEcnNz35h4LkpflRUTyvRBiYmJQdu2bTFhwoQK9y0iUUX0119/wcvLCz/++CNmzpyp6XCIKrw9e/agb9+++OWXXzB58mRNh0NERERERKR2ylPNiN5j27Ztg7a2NkaPHq3pUIgqhW3btkFXVxeBgYGaDoWoUti2bRuqVKmi8NBXIiIiIiKi9wnXUKYPwoEDB3Djxg2sXr0agYGBKm8lI6L/s3fvXty4cQObN2/GV199hbp162o6JKIKbdeuXbh27RpCQkLwww8/oGbNmpoOiYiIiIiIqExwyQv6IEycOBGZmZlwd3fHmDFjoK+vr+mQiCq0oKAgPHv2DK1bt8aoUaOgq8vvH4neJiAgAFKpFB06dEBAQIDK9caJiIiIiIjeB0woExEREREREREREVGRcPoMERERERERERERERUJE8pEREREREREREREVCRMKBMRERERERERERFRkTChTERERERERERERERFwoQyERERERERUQUVGxuLkJAQTYdRLi5fvox9+/ZpNIb4+HiEhIRAKpXKy0JCQhAbG6vBqN4tISEB4eHhCAkJQX5+vqbDwePHjxESEoK0tDS17E8ikSA0NBQZGRlvrCOVShEaGor09HS19ElEb8aEMhEREREREVEFFRwcDB8fnxK1TUlJQUhICJ4+farmqMrGqlWr8MUXX2g0hpCQEPj4+CAvL09e5uPjg+DgYA1G9XajRo1Cq1atsGrVKmzfvh0vX75UWU9Vsrys3Lp1Cz4+Pjh79qxa9rdixQoEBQWhWrVqCuXXrl1DSEgInj17Bh0dHWzZsgXjxo1TS59E9GZMKBMRERERERFVUM7OzvD29i5R28jISPj4+CAlJUXNUX1YvL294ezsrOkwVLp69SrWrFmDDRs2YN++fQgJCYGJiYnKuqqS5ZVBTk4OZs6cif/85z8wNDRU2DZjxgx8/vnn0NXVlf+8Z88eREZGaiJUog+GrqYDICIiIiIiIiLVBg0ahEGDBmk6jA9aRV5y5ObNmwAABwcHDUdSdjZt2oSnT59i2LBhStsuXLgAFxcXGBgYAABcXV3RokULLF68GB9//HF5h0r0wWBCmYiIiIiIiKiCio2Nxc2bNzFgwAB52e7du9GgQQO4uroiJSUF169fh7W1NVq2bCmvEx8fj/PnzwMAjh49ivj4eABAu3btYGNjI6/34MEDXLlyBRKJBK6urgrbgFfrGqempqJ3797Iy8vD2bNnIZFIYGhoiOzsbHz22WdKMd++fRuXLl2Cl5cXzMzMEBUVJV/7VkdHBxYWFnBzc4ORkVGJxiQqKgovXryAl5cXsrOzcf78eVStWhUeHh7F7is2NhZ37tyBnZ0dmjRporJOSEgIHB0d0bRpUwDA9evXkZiYiP79+yvUe/DgAU6dOoWOHTuiZs2a8vL79+8jPj4eUqkUTk5OqF27dpGP9ebNm7h16xaMjIzg7u6uMPv4yJEjOHnyJADg+PHjiI+PR506ddC2bVul/Vy8eFG+DvTu3btRpUoVAICnpyfMzc1x9OhRPHnyBACgq6sLKysruLm5yRO1hU6cOAEA6NSpE/755x+cP38eRkZGaNu2LfT09N54HFevXkVqaiqcnJzQqFGjtx7z77//DkNDQwQGBgIA1q9fDy8vL1hYWCjUe/jwIe7du4cePXoolPv6+uKHH35AZmamwutAROrDhDIRERERERFRBRUcHIy5c+dCCCEvGzJkCPz9/WFlZYXQ0FDUqFEDp0+fxkcffYT9+/dDX18fsbGx8vVrDx48KF97tk6dOrCxscGzZ88QFBSE4OBgtGjRAlWqVMG5c+cwfPhwLF26VL6EQOG6vOHh4Rg9ejSsra2RnZ0NX19fTJkyBXFxcXB0dFSI+bvvvkNERIT8gWwnT57E5cuXAQAvX77EzZs38fDhQ6xYsQKDBw8u9pjMmTMHGRkZmD59OqZNm4ZatWrB2NgYHh4eRe7r+fPn8PPzw6FDh9C6dWvk5eWhefPmqF+/vlJ/Pj4+mDZtGubMmQMA2LhxIxYvXgyJRKJQ7/Lly/Dx8UFERAQ6deoEABg/fjzWrl0Ld3d3GBsbIy4uDu3atcP69euVlm943b179zBo0CBcvXoVrVq1QkZGBlJSUjBnzhx88803AID9+/fLvzTYv38/TExM0K5dO5UJ5fPnz+P69esAgNDQUOjo6AAA3NzcYG5ujoMHD8qXRsnLy8O1a9fw4sULbN26FV27dpXv54cffgAAfPPNN5gxYwbq1KmDc+fOwcrKChEREahVq5ZCvy9evMCAAQNw//59SCQSnD9/HtOmTcPs2bNVHndmZia++eYbuLi4IDAwEFlZWbh8+TJmzZqlVPfChQsAoPBFCgB4eHhAIpEgIiICAwcOfOMYE1EpCCIiIiIiIiKqkKZNmyb+/dHdyMhIODg4iCVLlsjLjh07JgCIVatWycs2b94sAIjr168r7XfAgAHC1NRUXLx4UV52+fJlYWhoKGbMmCEvGz16tKhatarw9fUVT58+FUII8ffff4uMjAyhq6srJk2apLDfR48eCQMDA/Htt9++8ZhkMpmYMWOGqFq1qkhOTlboq0aNGu8YESG6desmatWqJfz9/UV+fr48puL2VbVqVREdHS0v27hxo7C1tRUARE5OjrwcgJg2bZr854kTJwodHR2lvg4ePCgAiIiICCGEEIcPHxYAxNGjRxXq7dy5U2RlZb3x+GQymWjRooWoW7euuH37trx85syZAoDYuXOnvGzNmjUCgLh58+Yb91do9uzZSsf2JhKJRIwaNUpYWFjIX3chhOjQoYOoW7euGD9+vJBIJEIIIe7evSuMjY3F2LFj5fXOnz8vAAhXV1eFMf7hhx+Etra2iIuLU9nvtm3bBACxcOFCIcT/jeHu3buV6haOx+vvYSGEyMnJEQDEhAkT3nmcRFQyfCgfERERERERUSWjo6ODr776Sv6zp6cnHBwccPDgwXe2jY2NRUhICP7zn//Azc1NXu7q6oqAgAAsXboUMplMXp6Xl4fx48fLl1uwt7eHlZUVPvvsM2zatAkFBQXyulu2bMHLly8REBCg0OeLFy8QHR2NPXv2IDQ0FNbW1sjLy8Pp06dLdPwZGRmYNWuWfJkFe3v7IveVk5ODDRs2YNiwYWjTpo283bBhw2BpaVmieFQpnKH97336+PjA3Nz8je0iIiJw+fJlfP/992jYsKG8fNq0aWjYsCEWLlyothhfl5ubi6ioKOzevRthYWGoV68eHj16hCtXrijUe/ToEWbPni2f5Vy/fn307NlT5XuvXbt2CmMcFBQEmUyGI0eOqIzh6NGjqFq1Kr744gsAkC9f8u/lLoBXy3gYGBjAxcVFodzY2BgGBgbytkSkflzygoiIiIiIiKiSeT0RXKhevXpITU19Z9vCpTCkUil2794NIYR8SY0XL14gKysLaWlpqFu3LgBAS0sL7u7uSvsJDAzE7t27ER4ejn79+gF4td5tmzZt5OsNA8C6deswceJEmJqawsnJCUZGRsjPzwcApKenF/PIX7G2tpbH97qi9BUbG4v8/HyVx9SqVSv5+JTWp59+CgsLC3z88cfw9fVF586d8cknn7wzaV24ZMfriVjg1ZcI7u7u2LNnj1rie938+fMxa9Ys1KlTB/b29qhatap8TeV/v0a2trYwNTVVKKtXrx5CQ0MhhICWlpa8/N/LUVhbW0NfX/+N79Njx47Bz89PnnAvfO1Urc9c+EA+Vdv09fXlbYlI/ZhQJiIiIiIiIqpk/p3QA14l0V68ePHOtnl5eQCAU6dO4erVq0rbvb295bNPAcDExETp4WzAq4SpjY0N1q9fj379+uHChQu4du0aVq9eLa+TkJCAUaNG4ZtvvsGvv/4qTzYmJiZi3759CmtDF4eqh60Vta/CRKOqNYzftq5xIW1tbZVxv3z5UuHnWrVqIS4uDuvWrcOxY8ewdetWPH/+HN7e3ti4ceMb+yqMT9WDBI2MjFBQUACZTAZtbfXcdH7ixAlMnToVCxcuxIQJE+TlEREROHbsmNKxvum9J5FIIJVK5etvv6munp6eyvdpQkICUlNTMW7cOHlZ4czkrKwshboZGRlIT09H7969lfZTUFCAnJwcPpCPqAwxoUxERERERET0Hnp9pujrbG1tAQBjxoxB//79S7wfHR0d+Pv7Y/78+bh//z7WrVsHIyMjDBo0SF7n7NmzkMlk8Pf3V9jPjRs3inMoRYqpqH0VLiNx69YtpX0kJSW9s28rKyvIZDI8fvxYYekKVfurWbMmvvvuO3z33XcoKCjA2rVrMW7cOLi7u2PKlCkq99+gQQMArxLhjRs3VtiWmJiIevXqlSiZ/KbXMSoqCgDky0wUKu1rVFzHjh1D69atFWY1Ozk5AQBu376tULfwgXyqZpkXvg6FbYlI/biGMhEREREREdF7qEaNGgAgX7qgkJeXFxo0aIAFCxZAIpEotUtJSSlyHyNGjIBMJsPKlSsRHByMgQMHytdaBl7N0gWAu3fvyssKCgqwbNmy4hxKkRS1r7p166JNmzbYsGGDfLY28Oq4jx49+s5+XF1dAQB//fWXvOzly5fYsmWLQr20tDSFtaj19PQwbNgwaGlp4eHDh2/cf/fu3WFiYoJly5YpzA4+d+4cTp8+rZCwL443vR9UjVtubi7WrFlTon5K6ujRowgKClIoa9y4MWrVqoVz584plBcmlP+9pAYAREdHAwA6d+5cRpESEWcoExEREREREb2HWrduDTMzM8ycORPDhw+HgYEB2rVrBxsbG4SFheGzzz6Di4sL/P39YWNjg5SUFERERMDIyAi7d+8uUh+NGjVC586d8dNPP0EikSg9jO+TTz6Bm5sbRo8ejaSkJBgYGCA4OBi+vr44fvy4Wo+3OH2tWrUKnTp1goeHBwICAvD8+XPs3r0b/v7+WLJkyVv78fT0hIeHB8aMGYOUlBQYGBhg165d+Pzzz+XrHwPAwYMHsWDBAvTv3x8ODg7Iz8/Hli1bYG5ujpEjR75x/2ZmZli/fj0GDx6M7t27w9vbGxkZGVi4cCHatWuH6dOnl2h8OnfuDF1dXUyaNAm9e/eGnp4ePD094ePjg9mzZ2PgwIGYMGGCPE5/f39MnDixRH2VxJMnTzBw4ECl8kGDBmHr1q2QSCTy5TQuXLiAKlWqwNnZWan+3r174eLiorCONxGpFxPKRERERERERBWUs7MzvL29Fcr69euHFi1aKNX18PBQWCKhevXqiIiIwKZNm7B3715IpVLUqVMHNjY2cHV1RXx8PHbs2IGYmBhcv34dDRs2xPfff68ws9PNzU1hFq8qU6ZMgbm5OczMzNChQweFbXp6ejh16hRWrlyJK1euwNTUFAsWLEDTpk1x4sQJODg4KPRVlAepffTRR8jOzlYqL05fzZs3x7Vr17Bq1SqcPn0atra2CAkJwenTp3Hv3j2FdYC9vb2VEpcHDx7E8uXLcfHiRdSuXRurV69GTk4OvL295Wv3BgYGwsvLCzt37sTJkydhYGAAHx8f7NmzR2GpDFUGDBiA5s2bY/PmzThx4gQMDQ2xbNkyDBo0SCG2hg0bwtvbG9WqVXvnuDk4OODYsWP4888/ERYWBplMBjc3NzRq1AiXLl3CihUrcPbsWVhaWmLDhg0wNjbGmTNnFB5++KZZv4Xv08KlOKpXrw5vb2/UqVNHqe6b3r8REREq9z127FgsWbIEBw8eRK9evQAA9evXR1BQkMJYAK/WWj5w4AAWL178zvEgopLTEiVdAZ+IiIiIiIiIiKiMjRgxAn///bd8vec3+fHHHxEcHIzY2Fjo6+uXU3REHx6uoUxERERERERERBXW3LlzUatWLaWH871OJpPh3r17+OOPP5hMJipjnKFMREREREREREREREXCGcpEREREREREREREVCRMKBMRERERERERERFRkTChTERERERERERERERFwoQyERERERERERERERUJE8pEREREREREREREVCRMKBMRERERERERERFRkTChTERERERERERERERFwoQyERERERERERERERUJE8pEREREREREREREVCRMKBMRERERERERERFRkTChTERERERERERERERFwoQyERERERERERERERUJE8pEREREREREREREVCT/D6TJiSHi7vG+AAAAAElFTkSuQmCC" + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))\n", + "layer_groups = [\n", + " group['affine_residual_delta'].to_numpy()\n", + " for _, group in tanh_prime_diagnostics.groupby('hidden_layer')\n", + "]\n", + "axes[0].boxplot(layer_groups, tick_labels=[f'layer {i}' for i in range(1, 4)])\n", + "axes[0].set_ylabel(r'affine residual $\\delta$')\n", + "axes[0].set_title(r'Distribution of certified $\\tanh\\prime$ residuals')\n", + "axes[0].grid(axis='y', alpha=0.25)\n", + "\n", + "for layer_index, group in tanh_prime_diagnostics.groupby('hidden_layer'):\n", + " axes[1].scatter(\n", + " group['interval_radius'], group['affine_residual_delta'],\n", + " s=28, alpha=0.75, label=f'layer {layer_index}'\n", + " )\n", + "limit = 1.03 * tanh_prime_diagnostics['interval_radius'].max()\n", + "axes[1].plot([0, limit], [0, limit], '--', color='black', linewidth=1, label=r'$\\delta=$ interval radius')\n", + "axes[1].set(xlabel=r'interval radius of $\\tanh\\prime(I)$', ylabel=r'affine residual $\\delta$',\n", + " title='Affine residual versus constant-interval radius', xlim=(0, limit), ylim=(0, limit))\n", + "axes[1].legend()\n", + "axes[1].grid(alpha=0.25)\n", + "fig.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "worst-tanh-prime-enclosure-plots-title", + "metadata": {}, + "source": [ + "### Geometry of the particularly bad affine enclosures\n", + "\n", + "For each hidden layer, the two neurons with the largest $\\delta$ are shown. The solid curve is $\\tanh'(x)=\\operatorname{sech}^2(x)$, the dashed line is $px+q$, and the shaded region is the certified band $px+q\\pm\\delta$." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "worst-tanh-prime-enclosure-plots", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
hidden_layerneuronpreactivation_lowerpreactivation_upperaffine_residual_deltainterval_radiusdelta_over_interval_radius
0140-1.1120731.0924900.3210170.3238470.991262
117-1.0528301.0663740.3084960.3105440.993404
2241-1.2592151.2186230.3569970.3619710.986259
3212-1.2066211.2360360.3527630.3564390.989687
4332-1.5759401.6127540.4238800.4264870.993888
5343-1.5240391.5238540.4134890.4135030.999965
\n", + "
" + ], + "text/plain": [ + " hidden_layer neuron ... interval_radius delta_over_interval_radius\n", + "0 1 40 ... 0.323847 0.991262\n", + "1 1 7 ... 0.310544 0.993404\n", + "2 2 41 ... 0.361971 0.986259\n", + "3 2 12 ... 0.356439 0.989687\n", + "4 3 32 ... 0.426487 0.993888\n", + "5 3 43 ... 0.413503 0.999965\n", + "\n", + "[6 rows x 7 columns]" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABZQAAAUbCAYAAAC5tSU8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAASdAAAEnQB3mYfeAABAABJREFUeJzs3XVYFNsbB/Dv7tKtCKIoJoKIimLrtbsDGxO9dnd3d7did+c1sH8WBhYqiAoopSBdu3t+f+COLLtLLDHE+3ken3vnTL0zzO6eeefMOQLGGAMhhBBCCCGEEEIIIYQQkgYh3wEQQgghhBBCCCGEEEIIyRsooUwIIYQQQgghhBBCCCEkXSihTAghhBBCCCGEEEIIISRdKKFMCCGEEEIIIYQQQgghJF0ooUwIIYQQQgghhBBCCCEkXSihTAghhBBCCCGEEEIIISRdKKFMCCGEEEIIIYQQQgghJF0ooUwIIYQQQgghhBBCCCEkXSihTAghhBBCCCGEEEIIISRdKKFMCCGEEEIIIYQQQgghJF00+A6AFDzu7u4wMDCAra0t36FkOz8/PwQFBcHQ0BA2NjZ8h0OS+fXrF758+QIAsLe3h46OjsplExMT8fXrVzDGULp0aWhpaWV5HObm5rCyssqy7QLA169f8fPnT266ePHiKF68eIa3IxaL8e7dOyQmJqJ8+fIwMTFROyZ1tqXOOkFBQQgKCoKZmRmKFSumdrxZHZcq6lxjYWFh8Pf3h4GBAUqXLg2BQJDmOuk9LyEhIfj27Rs3bWRkhAoVKqTvYNIQGRkJX19faGhooHTp0tDW1s6S7aYmKz/DERER8PX1hZ6eHkqXLg2hMO1n8+pcj1l5fRFC8qYfP37gx48fqFChAoyMjPgOJ1slJCTg3bt3kEgksLGxgaGhId8hFUgxMTF4//59msulVXdOjzdv3iA+Ph4ODg7Q0Mi6tERCQgJev34tV1ajRg21tiX7DOrp6cHOzi5TcYWEhCAwMBAWFhYwMzNTudznz58RFhamcn61atUgEomUzpNIJPD390dkZCQsLS1RqFChTMWsCh/nJaXg4GCEhITAysoq3d8X6VnHw8MDiYmJ3LStrS0MDAzSHZcqUqkUfn5+CA8Ph4WFBczNzTO9zfRQ9/ympO61pc7+Q0NDERAQgGLFiqFw4cJqx0xyACMkByUmJjIArHbt2nyHkm2ePHnCJk2axOzs7BgABoA1atSI77BICk2bNuX+Pm/evFG6jEQiYUuXLmWFCxfmli1UqBBbvHgxk0gkWRLHvn37GAA2bNiwLNlecgMGDODiBsBmzZqV7nVjY2PZnj17WLdu3ZiRkRG3jbNnz2Y4DnW2pc46t27dYv369WPGxsZyx12+fHm2b9++DMedVXGlRp1r7OXLl6xx48ZMIBBw61haWrLdu3crXV6d87Jt2za5ZZs1a6bW8SXn6enJmjRpIhe3QCBgbdq0YR8+fMj09pXJqs/wjx8/2KxZs1jZsmXlzouxsTGbOHEii4qKUlhHnfOe1dcXISRvmzVrFgPArl69ynco2SIqKopt376dderUiRkYGHDfebdv3+Y7tALr2bNncr9Zqv59/Pgx0/sqVaoUA8ACAgKyIPK/vnz5ohBvYmJiutd/+fIlmz59OqtSpQq3ftWqVdWO5/3796xhw4Zy8TRo0IC9e/dO6fKdOnVK9dyHhYUprHP27FnWsWNHpq2tLbdsrVq1mJubm9qxJ8f3eZE5d+4cq1q1qtx6DRs2ZHfv3s2SdYoWLSq33P3799U+Rplly5axIkWKyG3XzMyMrV27NtPbVkXd85uSutdWRvcvlUrZ9u3bmZWVldw6JUqUYFu2bMmye2+StSihTHJUQUgot2vXjvsClCUxKKGcuxw5coQBYPr6+qkmlKdOncr9LUuXLi2XTJo0aVKWxJITCeVKlSoxR0dHtnXr1nSv6+XlxR2rhoYGMzQ0VDuxpc621Fmndu3a3Drm5ubMzs6O6enpcWXLli3LcOxZEVdqMnqNvXr1irtutbS0mI2NDStWrBi3zoYNGxTWUee8nD59mjk6OjIHB4csSShLpVJWvnx5BoBpa2szOzs7VqVKFa5ynRV/G2Wy6jMs+5zKvtft7e3lbgwaNWqkUNFV57xn9fVFCMnb8ntCOXnyUktLi0sqU0KZP+/fv2eOjo5K/9nY2DAAzNraOkv2ld0J5UKFCnGxi8XidK/ft29fhXs5dROn3759Y2ZmZgwA09PTY/b29lw9rkiRIuzLly8K68gSypUrV1b6d4iMjFRYR3YuNTQ0WPny5Vm5cuWYpqYmA8BEIhG7du2aWvEnx/d5YYyx5cuXczHo6+sze3t7rh7crVu3LFmnVatWzNHRkZmbm2dJQvnAgQPc/i0tLVnVqlWZtbU1E4lETFtbO1PbVkXd86uMOteWOvufPn26Ql3b1NQ0y++9SdaihDLJUQUhobxw4UK2atUq9vbtW66iTAnl3CM8PJwVK1aMde3alUv4KEsoe3p6MpFIxDQ0NNi5c+e48suXLzMtLS0mFArZ27dvMx1PTiSUVSXMU+Pv788GDx7MTp48yX7//s1VItVJbKmzLXXWmTVrFjt48CD7/fs3VxYVFcVGjhzJ3awqa9WR3ceiijrXWIsWLRgA1rFjR/bz50+u/MaNG8zAwIDp6ekp3Jhl5ryEhYVlSULZ29ubu/FIeSP07Nkzdvny5UxtX5ms/Axfv36drV+/nvn7+8uVX758mUuA3LhxQ26eOuc9K68vQkjel98Typ6enuzff/9lZ8+eZZGRkVyjDEoo505Lly5lANjixYuzZHvZnVBWlWBMy+rVq9nSpUvZy5cvuQe96iZO+/TpwwCw5s2bc/WBiIgI1qZNGwaA9ezZU2EdWUI5I0m/qVOnMjc3N7mW2CEhIax79+4MAKtfv75a8SfH93m5e/cu95bbnDlzWExMDDfv4cOHbNOmTVmyjsy4ceOyJKHs7OzMALADBw7IlUdERLA1a9ZkatuqqHN+VVHn2sro/mNjY5muri4DwNavX889AJJIJGzz5s1cvVnZwxTCL0ookxyVnoRyXFwc+/DhA/vw4QNLSEhQmB8eHs6ePXuW6o/s169f2bNnz1h4eLjS+T9+/GAeHh7M19dXZZzPnj1jnz594sq+f//OXrx4kaGEVHYklJXF5ufnx968eZPuL9m0jj86Opo9e/aMffv2Ten8t2/fshcvXqQZl6pzFhYWxt68ecO8vb1Vvr7i7+8v9zeMjY1l79+/Z58/f07XMaoybtw4pq+vz3x9fVNNKMueko4aNUph3sSJExkANmXKlEzFwljuTSinlJWJLXW2lZn9SyQSrmVqVr32lxVxZfQaS0xMZBoaGkwgEMglk2Vmz57NAKS7cpqe85JVCeVfv34xkUjETExM2J49e5TGn9Vy6jM8ePBgBoBt3749Xctn5HqkhDIhBVt6EspBQUHs9evXLDAwUOl8T09P9uzZMxYfH690fkJCAnv27Bnz9PRUOj82NpZ9+PCBvX37lkVHRytd5tu3b+zZs2dc9z8xMTHs/fv3GX61OjsSyspie/fuXbqTdek5fi8vL/bs2TOl3SqEhoayZ8+eKfx90nvOpFIp+/z5M3v9+jULDQ1Vun+pVKrwN/zx4wd7/fq1ynshdVhbWzOhUKjy/iGjcmtCObnMJE7Dw8OZlpYW09TUZN+/f5ebFxwczHR0dJimpqbcg2fG1Esoq+Lr68sAMAsLi0xvKzk+zous+wQXF5d070uddWSyKqE8YcIEBoB16dKFPXv2LEOt5dWh7vnNKFXXljr79/PzYwCYlZWV0n1ZW1szAMzb2ztTMZOsl/ZIMoTkkGPHjqFRo0bQ09ODra0tbG1tYWxsjFGjRiEmJoZbTiqVolGjRmjWrBkYY0q31bp1azRs2BBisViu/MiRI7CxsUHx4sVRtWpVWFlZoVy5cjh+/Ljccj9//kTNmjUxZswYuLu7w8HBAZaWlqhevTru3LmT5ceeEclje/PmDWrUqIGSJUuicuXKKFy4MCZNmgSpVKp03fQe//v371GzZk0sXbpU6Xa6deuGf/75R2Vcqs7Zx48f0apVK5iamqJy5cooX748zM3NsWTJEoWYN23ahJo1a+Lhw4dYsWIFLCwsYGdnh3LlyqFcuXK4f/9+hs+dh4cHNm/ejHnz5qFkyZKpLnv37l0AQI8ePRTmycqUXQvu7u5wd3eHRCLJcHwkewiFQpQrVw4AcmQAuPTK6DUWExMDsVgMPT09mJqaKqwju6bd3NzStf+cPC+FCxfG5MmT8fv3b7i4uKBIkSKoWLEiJk6ciM+fP2fLPtX9DGfU9+/fAQDW1tbpWj63Xo+EkLzDz88PEydOhJmZGYoWLYoqVarAwsICVapUwc2bN+WWPXDgAGrWrImjR48q3dbp06dRs2ZNuLq6ypUHBASgX79+KFSoEGxtbWFvbw8TExP06dNHbsBfAFi6dClq1qyJFy9eYOrUqTAzM4OdnR369OmTpcetDllsHh4emDt3LszMzFCpUiWUKVMGdnZ2eP78udL1MnL848ePR82aNRXKAeC///5DzZo1cfDgQaVxqTpnjDGsWbMGFhYWKFeuHKpUqQJTU1M0adIEb968kdtWfHw8atasiYEDB8LLywsNGjRA8eLFuXWGDx8uN8iYOu7duwcvLy80bdpUaR36x48fcHd3h5+fX6b2k588fvwYCQkJ3N8jOTMzMzRu3BiJiYl4+PCh0vUTExPh6emJjx8/IiEhQa0YMlpHyQnqnJeAgADcu3cPADBr1qx07UeddbLD2LFjYW5ujrNnz6JmzZowNjZGhw4dsH//fpX37JmR2esuvVRdW+rsv1ixYjAyMkJoaCgiIyPl1omOjsbPnz+hp6eX5v07yXlZN5wqIZk0fvx4BAUFwcDAAKVKlUJkZCT8/PywdetW+Pv74/z58wAAExMT9O7dG3v27MHNmzfRokULue3cu3cPHz58gIuLi9yooEuWLMHs2bMBAEWLFoWZmRmCg4Ph4+ODXr16ITY2FgMHDpTblr+/P5o3b47IyEiUK1cOJiYm2TZabkZ9//4dTZs2RVhYGMqXL4/o6GgEBARg7dq1KFGiBCZMmCC3vDrHrw5V58zHxwf16tVDaGgohEIhypcvj1+/fuHXr1+YPXs2vnz5gt27dytsb+XKlbhz5w6MjY1RsWJFfP36FT4+PujUqRO8vLyUJtaUYYxh5MiRsLW1xfjx49Nc/sOHDwCAKlWqKMyrXLkygKQEeUo1a9YEAISFhcHExCRdsSkTHR0NT0/PDK1jbW0NY2NjtfeZX0VERODJkycoXLgwHB0d+Q6Hk9FrzNDQEIUKFUJYWBju3r2LRo0aya1z+fJlhXVSk9PnZcCAAdi7dy8SExMhFovx4cMHfPjwAdu3b8e2bdswYMAAbtmsuP7V/QynJjw8HF5eXpBIJAgMDMTx48fx33//oVWrVmjatGm6tpFbr0dCSN5x+fJlrFu3DkKhECVLloShoSH8/Pzw5s0btGnTBg8fPkStWrUAAMOHD8fKlSuxc+dOue9ZmZ07d0IoFGLEiBFcWWBgIGrXrg0/Pz9oa2vD2toaQqEQX79+xdGjR/H69Ws8ffoUenp6ctuaMWMGHj58CGNjYzg4OMDOzi57T0QGzJo1C3fu3EGhQoVQsmRJfPnyBZ6enujQoQO8vLygr6/PLavu8atD1TmbMGECNmzYACDp3sfMzAw+Pj64c+cO6tevj8ePHyuc31+/fqFp06b48eMHypYti4SEBPj7+2PHjh0oWrQoFixYoHace/fuBQCV9wo7d+7EggULMG7cOKxfv17t/QCAl5cXwsPD0728vr4+KlasmKl9ZofU6iFAUl3k2rVr+PjxI9q2basw38HBgWtUpauri27dumHVqlWwsLBQuU9PT09ER0cjKioKr1+/xsqVK6GtrY2FCxdmwRFlDXXOy7NnzwCAexgkkUjg4+MDqVSKsmXLQlNTU2E76qyTHaysrNC7d29s2LABxYoVQ2BgIC5duoRLly5h165duHjxolx+IbPXf2avO1XSe22ps3+RSIRly5Zh1KhRaN68OSZMmICSJUvC398f69evR1hYGNauXQstLa10x0tyCM8tpEkBk1qXF1OmTGGvXr2SK/Py8mLVqlVjANjLly+58hcvXjAArHv37grbkfVTlLxLhrdv3zKhUMhsbW3Zs2fP5Ja/du0aMzIyYkWKFOFeBwwICOA6gG/SpAn78eOHWsebHV1eJI+tVatWcq+Jbd++nQFgtra2cutk9PhlcavqhsHGxobp6+urjEvZOevSpQsDwBo3biw37/Dhw9yosclfKZo2bRrX2f/u3buZVCpljCW9Ol+vXr0MvWLOGGO7d+9mAORG802tywuhUMg0NDRUbk82sEDKLjtSGzBDGVVdXjx69Ig7n+n9d/HiRbltUJcXSfr3788ApNpPmroyE5c619iIESMYAGZqaspWrVrF7ty5wy5cuMCcnJyYjo4ON5BFeqTnvGRVlxcvX75kenp6rGPHjtwrw+/fv+e6ixCJRHKv+WbF9a/uZzg1Fy9elNunpqYmW7x4sdLumVTJyPVIXV4QUrCp6vLiypUrbMeOHXJdMMTHx7ONGzcygUDAOnXqJLd8hw4dGACF7hS8vLyYQCBgHTt2lCvv0aMHA8Dmz5/PIiIiuPKfP3+y3r17MwBs7dq1XPmwYcO478SDBw+qfbzZ0eWFLDZtbW129OhRrjwwMJBVrVqVAWDHjh2TWyejxy+LW1nXDUePHmUA2KpVq5TGpeyceXh4cH2Gurq6cuVBQUGsWbNmCr/LsbGx3O/SP//8I9dl3bFjxzLd5UFERATT19dnRkZGcn3QJrdjxw7m6OiYoT5hVXV50apVqwz9/js6Osqtn1u6vJg/fz4DVPc5vWrVKgYk9e2bnKzLCwMDA2ZnZ8dMTEy4Yy1ZsqRCNwLJOTo6yp2bGjVqKNxfZ4WcPi/btm1jAFjv3r3Z9evXuWsHSBr0bezYsQpd0qizTnJZ1eVFnz59mLa2Nrt58yZjLKlLiOPHj7OSJUty8SWX2etf3esuLem9tjKz/2vXrjErKyu5/VhYWLALFy5kKFaSc6iFMsk1Vq5cCQBcy+TY2FgwxuDk5ISXL1/i/v37cHBwAABUq1YNtWvXxrlz5xAcHAxzc3MASa1CT506hbp166JatWrctmWvlIwZMwYaGhp4+fIlAIAxhiJFiqBNmzY4fvw4Xrx4gTp16nDraWtr49ChQyhWrFgOnYX0MzQ0xNGjR+WeaA4bNgyrV6/mXo2SPcVT9/jVoeycxcTE4NKlS9DT08ORI0fk5vXp0wdv3rzB8uXLcfz4cTRo0EBueyNGjICLiws3XbhwYcycORPt27fH69ev0xVTaGgopk+fjv79+6Nhw4ZpLi8WiyGVSlN9HV32VDshIQE6Ojpcubu7e7piSouBgUGGWy9mpEV0aGgofHx85Mq0tLRUPk3OqxYvXowDBw7AyckJo0eP5jscjrrX2LJly/Ds2TO4u7tjypQp3LICgQCrV6/GpEmT0vVaZE6fl1GjRkEoFGLXrl1ci66KFStiz549kEqlcHV1xd69e7F69WoAmb/+M/MZTmsfjo6OSExMhL+/P0JDQ7F06VIYGhpi7Nixaa6fW69HQkje0qZNGwBJ9ThfX1+EhoZCLBajbt26sLKyUugWbOTIkbh48SJ27twp13p0165dYIxh1KhRXFlERAROnz6NypUro1OnTvj8+TNY0rg7AIAhQ4bg6NGj+O+//xTehhs+fDicnZ2z6agzZ8qUKejVqxc3XbRoUUyZMgXOzs54/fo1evbsCSBzx68OZedM1hXdyJEj5VqVm5ub4+jRoyhfvjxu3bqFnz9/okiRItx8TU1NnDhxQq4Fa8+ePbF69Wq4u7sjJCQEZmZmGY7x+PHjiI6OxpAhQ6Crq6t0mX///Rf//vtvhretTIUKFZR2H6JKRlvCv337FnFxcXJlpUuXljuXWUHWzYiqlrDJ6yHJtWzZErNnz0aNGjW4stu3b2PIkCHw8fHBlClTcPjwYaXblJ2L0NBQ+Pv7w93dHcOHD8fJkydRokSJTB9TVlDnvMj+XgEBAWjfvj0EAgFsbW0RHh6OgIAAbNy4Ee/fv8f169chEAjUXier3bhxA0eOHMHUqVPRrFkzAICRkRF69OiBKlWqoFq1ajh+/Di2bdvGvWWX2etf3esuvftJ69pSd/9v3rzB2LFj4evrCxMTE1haWuLHjx8IDAzEuHHjuK40Se5CCWWSazx48ADTpk3Do0ePlPaNHBwcLDctq2S5urpi6tSpAJL6iYuLi1O4UX/16hUAyFWYlQkMDJSbtrW1Vej7J7eoXLmy0u43SpcuDW9vb0RFRXFdfqh7/OpQds68vb2RmJiIWrVqKU3Od+jQAcuXL8f79+8V5ilLAJcuXRoAFPpYUmX69OkQi8VYtWpVupbX0NCApqYm4uLiIJVKIRQqdjcfExMDkUiU7kRURtnb22dZclqZK1euoF+/fnJllpaW8Pf3z7Z95rRVq1Zhzpw5aNWqFQ4dOsR3OHLUvcaMjY3x8OFD7N+/H1evXkVgYCBKlCiBESNGcN8HaT1YyOnz8v37d/zvf/9DgwYNuId/yQ0aNAiurq7w8PDgyjJ7/WfXZ7hBgwZycd24cQN9+/bFuHHjUKFCBbRu3Vrlurn5eiSE5C1xcXGYM2cO9u7di9DQUKXLiMViaGgk3eq1atUK5cuXx8GDB7F8+XLo6OggMTERrq6uqFChglz3cW/fvoVEIsGbN2/kGmekpKzOmN6uf/iQ3vpkZo5fHcrOmazLp3bt2inMMzMzQ+3atXHjxg14enrKjWlibW2ttDuE0qVLw93dHZGRkWollNPq7iKrbdy4MVu337lzZ4XxG3bt2oUhQ4Zk6X5kD9CTjwWUXHR0NADIdbcCJN3jptSkSROcPn0a1apVw7lz55CYmKg0YXfgwAHu/3/+/IkZM2Zg9+7d6N69Ox49eqT2sWQldc6LbJ07d+7AyckJO3fu5Oq9169fR5cuXXDz5k1cvnwZ7du3V3udrHbixAkASQ8JUrK1tUXdunVx+/ZtvH37FvXr1weQ+etf3esuLem9ttTZf3R0NFq1aoXAwEDs3LkTLi4uEAqFYIzhwIEDcHFxQatWrfD582cYGRllKG6SvSihTHKFV69eoVmzZlyrWisrKxgaGkIoFCIyMhKfPn1SGEyiZ8+emDRpEnbv3o0pU6ZAIBBg165dKFq0KJycnOSWjYqKApDUl09q/SWl/IJKrY8qvhkaGiotF4lEACDXyb+6x68OZecsNjYWAOT6tE5O1g+ybLnklB2nsmNU5cWLF9i9ezdGjBgBX19f+Pr6cvNkP2jv3r1DXFwcbGxsuP1ZWlri69ev8PPzQ6lSpeS2GRgYiISEhGwdGCC7+1A2NTVVaAFatGjRDO0vN1u2bBlmzpyJZs2a4ezZs7ly8DN1rzEtLS0MHToUQ4cOlSvfsWMHgKSWDarwcV5kDylU9T8v+86QfR5l/5/Z6z8nPsMtWrTA6tWrMWDAAOzfv19lQjkvXI+EkLyjf//+OHnyJICkVqvm5ubc98rHjx8RFRUll1AWCAQYNmwYpkyZglOnTsHZ2Zl7y2/mzJlyrfNkdcYiRYoofHcmV758eYWyvFZvTq3OrM7xqyMr680ZuTdIrw8fPuDRo0ewtrbmEl7ZLbv7UK5cubLCw3d1Eu1psbS0BAB8+/ZN6XzZPYlsubQ4ODjA3NwcwcHBCAoKSrPFcZEiRbBz5064ubnh8ePH+PjxI2xsbDJwBNlDnfMiO1aRSIS9e/fKXestW7bE2LFjsXz5cty5c4dLDquzTlZTpw6c2es/q687ZVK7ttTZ/8WLFxEQEAAnJye5+xuBQIABAwbg9u3b2L9/P86ePat0LADCH0ook1xh165dSEhIwJQpU7Bo0SK5m+2jR48qHSVaW1sbgwcP5gZt09bWxrt37zB79myFDttlreLWrl3LvW5SkGT0+GU3IMl/3GQYYxluxSqrpHl7eyud7+XlJbdcVnr//j0YY9i6dSu2bt2qdBnZK5A3btxA8+bNASQl379+/YoHDx4o3FA8ePAAwN+BvbLDmzdvULdu3Qytc/HixXRXiNq0acO9MpvfzJs3DwsXLkTTpk1x8eJFla9n8i2rrzHZ648pB+uT4eu8yJK8ss95SrLy5K+aZsX1n1OfYdkNS0BAgNL5eeV6JITkDcHBwTh16hQsLS1x9epVhe8xGxsbfPr0SWG9wYMHY+7cudi1axecnZ2xa9cu6OvrK7Q6ldUZbW1tFbrOKAjUOf7U6s3JGzKkV/J6s7JXvLOz3pzSnj17ACBHkzhjxozBf//9l+7lHR0dM/RW09mzZ9UJK8Nk3cg9fPhQ6fyM1kXEYjEiIiIAIN1vVwkEAlhaWsLHxwcBAQG5IqGsznmpWrUqgKQ6pbIHJ7IGAsnfNlBnnayWvA6c2mc5eR04s9d/Vl93qqi6ttTZv5+fHwCobOhhZWUltxzJPSihTHKFkJAQAEmvUiVPJkulUuzbt0/lesOHD8fq1auxc+dOaGtrQyQSYdiwYQrLNW3aFOfPn8eSJUvQqFEjruKXXGRkpMon+3ldRo9f9rT06dOnCq+LHz16FNHR0Rl6TaZs2bKwsLCAp6cnrly5IjeirFQqxZo1awAA9erVU+v4UqOsJa7Mhw8fEB0djUqVKkFHR0euhXb79u1x4cIFbNy4Eb169eJadzDGuFG3O3TokOXxymR3H8r51ZQpU7B69Wo0btw41yfv1LnGEhMTIRAIFD7Dhw4dwv3796GlpYVBgwYp7IvP81KhQgUULlwYHz58UPr5l/WbnLz/9Ky4/rPyMxwVFQUDAwOl82SJ/LJlyyrMy0vXIyEkb/j58ycYY6hXr55CQuDevXsqH94VLlwYPXv2hKurK65evYqbN2/i33//VXizqXLlyjAzM8OjR49w/fp1pa9qA/m33qzO8cvqzY8ePUK5cuW4ZRISEnDw4MEMx1CvXj0cPHgQ69atg5OTk1w9/Pbt23j+/DmMjIxgb2+f4W1nhFgsxsGDByEUChW6SctO2d2Hck5xcHCApaUlvLy8cOnSJbmH3jdu3MC7d+9QrFgxufpOTEwMdHV1lfbpu2nTJsTFxaFkyZJyCcjU1vn48SOeP38OAChTpkxWHp7a1Dkvsv5zX7x4gSdPnqB27dpy27xy5QoAyH3+1Fknq9WpUwfHjx/H+vXr0b17d4XPsru7O4yMjOS+yzN7/atzflVR59pSZ/+yLjFv3LghNw4UkPQ9JPtb5cZxrQo6SiiTXEH2msagQYMwZcoUFCtWDF+/fsW2bdvw5MkTleuVKVMGrVu3xpkzZyASidCpUyelr/8MHjwYa9aswe3bt1G9enWMHDkSFSpUgFQqxdevX+Hm5oZ79+5lSf+xISEh3Csesle2IyMjuSeHQqFQ4Qmln58fgoKCULx48Wzpszmjx29hYQFbW1t8+PABvXr1gouLCwQCAdzc3LB+/XqlCem0jBkzBrNmzYKTkxOmT5+O+vXrIzQ0FNu2bcOdO3dgbGysNBGWWam1xK1Tpw6ePHmCY8eOKVTKnZ2dsWDBAjx9+hSdOnXC2LFjIRAIsGXLFjx48AAWFhbo37+/wjZlf+dq1apxCSx1ZHcfyumRfNASWR+Nnz9/5uIqVqyY3OtKMTExeP/+PXR0dBTOZ0a3pc46kyZNwtq1a1GqVCksWLBAaZ/cZcqU4V4Vlcno3yyrzos619i3b9/QunVrjBgxAg4ODpBIJLhw4QK2b98OAJg9e7bC0311z0ta0nvehEIhRo8ejYULF6r8/BcqVAiDBw/m1smK61+d86vqb9W0aVOUL18ezZo1Q+nSpSESieDr64vDhw/j+vXrACA3eCig/nlX57NCCCk4ypUrBy0tLZw9exYLFixA48aNERcXh/v372Pjxo0QiUQQi8VK1x05ciRcXV3Rp08fhcH4ZEQiEebMmYOxY8eiY8eOGDx4MFq0aAFzc3MEBATA09MThw8fxujRo7NkcFEPDw+uWzvZa94fP37kHuKVKFFCrlsIiUTCDS6dfNCyrKLO8Tdp0gQ7duzAhAkTEBUVhUqVKuHbt29Yt24dPnz4kOEYevfujTlz5uDx48do1KgRRo8eDXNzczx+/BhLliwBkPS3TK0bu6xw+fJlBAUFoVmzZlwLQVV+/PiBHz9+oGjRopnuTiq7+1BOS/KBq2X3RrGxsXL1EkdHR7kkm7e3N37//o2yZctyXZUIBAJMnToV48aNg7OzMxYvXoxq1arh9evXmDVrFoCkB8/Jk4xubm4YN24cBgwYADs7O1hYWCAgIABnzpzhBmtMORjk9evXMXXqVAwYMAC2trawsLBAaGgonj59iu3btyMmJgYtWrRQeFsro38zPs8LAMyYMQPdu3dHx44dMX36dFSvXh3h4eFwdXXF5cuXoaOjIzfwprrrpCUj561///5YuHBhqp/lsWPHyn2WM3v9q3t+lf2t1Lm21Nl/mzZtYGBggLdv36Jp06YYPXo0SpQoge/fv2PLli14/vw5dHV1lfYrT3jGCMlBiYmJDACrXbu2XHlISAgrVqwYAyD3z9DQkC1evJgBYNOmTVO6zUuXLnHLu7m5qdz327dvWdmyZRX2IftXtmxZbtmAgAAGgLVq1SrDx7hp0yaV+wDAtLW1FdYZN24cA8DmzZuX5vbTiq1Vq1YMAAsJCZErz8jxM8bYuXPnmFAoVFhu7NixzMbGhunr62corsTERNa5c2el+9bV1WVXrlyRW37atGkMALt69arCtjw9PRkA1rdv3zTPV2pq167NALA3b94onf/gwQNmaGioEK+BgQG7d++e0nVky4SFhaUrhn379jEAbNiwYeoehkoDBgxI9fjSUq5cuVSv5ZSfyWfPnjEArFy5cpneljrrlCpVKtXlAbB9+/Yp7Cejf7OsPC8ZvcZ8fX2ZhoaGwvICgYCNHz+eSaVShXXUPS+MMRYWFsYAsGbNmmXqvCUkJLAOHToo3be+vj67ceNGmttQR0bPr6q/VcOGDVP9Tt+2bZvCttQ97+p8Vggh+dOsWbOU1oVkdeOU/3r06MHq1avHALDY2Fil26xRowYDwBo2bJjqvqdOnZrqd9H+/fu5ZYcNG8YAsEePHmX4GE1NTVPdz7Jly+SWl/0uAem7jU0ttkePHjEAbNSoUQrzMnL8iYmJrE6dOgrLmJmZsfnz5zMAbNWqVemOizHG3NzclP5+AUl17bi4OG7Z2NhYBijeX8n07NmTAWBeXl7pOmcyHTt2ZADYwYMH01x23rx5DAAbN25curcv+50MCAjIUFxp+fLlCwPAunXrptb6Bw8eTPP3O+Xnq127dkrPlUQiYU5OTkq30aVLFyYWi+WWv337ttL7L9m/kSNHKtT1bty4keo6NWvWZEFBQQrHmdG/GZ/nRWbSpEkq62LHjx/PsnUY+3t/fv/+fbnyjJ63GzduMAMDA6UxtG/fniUkJKRrOxmhzvlV9rdS99pSZ/8nT55kOjo6StfR0tJihw8fztqTRLIEtVAmOUogEMDR0VFh4IQiRYrAw8MDq1evxsuXLyEWi1G1alWMHj0av379wtmzZ1UOPNCyZUtoa2ujbNmyaNKkicp9V6pUCW/fvsXhw4dx48YNfP/+Hfr6+ihdujSaNWuGTp06cctqamrC0dEx1cGtVDE3N0/1FRJlgzElJCQASN+geGnFJntNJmWrhYwcPwB06tQJt2/fxvbt2/H161eYm5ujd+/e6NmzJ5ycnLg+5tIbl4aGBs6cOYMzZ87g9OnT+PbtG3R1dVGjRg0MHz6cG2lbpkSJEnB0dFQ6yJyuri4cHR2VvmKeERUrVoRYLFb5Gnr9+vXx5s0bbNq0iXulp1q1ahgzZozK18Zkf/v0tuIuUqQIHB0dUx34hS/KBi1JLuVnUl9fH46Ojkqf1md0W+qsU6VKFblXAJVJOV/22dPS0kr3QGlZeV4yeo2VLFkS3t7e2L59O168eIGEhATY2dmhf//+Cq/yyahzXtKS0fOmqamJ8+fP4/Tp0zh9+jR8fX2hq6uLmjVrYsSIEWm2fFJXRs+vqr+Vm5sbrl27hitXrsDb2xuRkZEwMzNDrVq10LdvX4XvL0D9867OZ4UQkj9ZWloqrQvNmjUL9vb2OHbsGPz8/GBubo7OnTujX79+GDp0KOLj4xVan8l07NgR7u7uSlsnJ7dixQr06dMHBw8ehIeHB2JjY2FpaQk7Ozv06dMH1tbW3LKlSpWCo6Ojyq6BUuPg4IDfv3+rnJ/yFWfZ7096u9tILTZZ90rK6mAZOX4NDQ3cuHED69evh5ubG6RSKapXr46JEyfi/fv3cHR0VBh8L61z1qRJE7x//x7btm3D06dPER0djZIlS6Jz587o2bOn3N9XKBQqvb+SKVeuHBwdHdPd7y6Q1B90cHAw6tati65du6a5fPHixeHo6Jih33PZ72R2t7TOqNS6y5NJ+fmytraGo6OjwltHQqEQJ06cwLFjx3Dq1CkEBgaiaNGi6NatG/r06aPQlUDjxo3x7ds3HDp0CC9fvoS/vz/09PRgb2+P3r17o1atWgqxNG/eHF+/fsWRI0fw8uVL+Pn5QUtLC+XLl0fbtm3RsWNHpW+SZeT+k+/zIrN69Wq0atUKBw8ehI+PD7S1tVG9enUMGzZM5UCZ6qyTmoyet+bNm8PT01Phs9ytWzd0795d5bFmhjrnV9nfSt1rS539Ozk5oVq1ati7dy/c3d0RHh4OIyMjVK9eHS4uLnLfuST3EDDGGN9BEJIZx48fR69evbBhwwaMHTuW73DUUqNGDXz8+BFfv37N8GvnhKgycOBA7N+/H8eOHUO5cuWyrUuVvOzBgwf4559/MGLECJWDNhZEsq57oqKi0KRJEzRr1gw3b97k5tN5I4SQvEcikcDBwQHBwcHw9/fPdYm89Lh06RI6dOiAGTNmYOnSpXyHQ3Khr1+/okyZMmjatClWrFgBIHu6R8nrmjdvjvv378Pb2zvT3ZTkJ7JueNauXYujR4/i/v37cuN80Hkj5C9qoUzypLCwMHz+/Bk+Pj4YM2YM9PT04OzszHdYaomIiMCrV68wefJkSiaTbCHrG2zWrFlYvHgxz9HkLnfv3oWmpiamT5/Odyi5yunTpzFixAiV8+m8EUJI3vHu3Tv8/v0bu3btwtu3bzFt2rQ8mUwGkn5/9PX1MXHiRL5DIbmcm5sbatasCSBpUGN1xoDJrxITE/Ho0SMMGjSIkqIptGrVCkFBQUrn0XkjRB59q5I86datW+jevTs3PWfOHK7z+Lzm48ePqFGjBiZNmsR3KCSfKVOmjMIIyUTer1+/MHXq1GzrciGvStl1j42Njdx8Om+EEJJ39OzZE+/evQOQ9Np6Xq5z/v79G3PmzMlwV02k4NDW1lbomiE7uhXIyz5//gw7OzvMmDGD71ByHQcHB/z8+ZObTt69Dp03QuRRlxckT3Jzc8PUqVNhYmKCdu3aYezYsUr77yGEEEIIIaQg69WrF758+QJra2vMnDkTdnZ2fIdECCGEkDyOEsqEEEIIIYQQQgghhBBC0kX58L+EEEIIIYQQQgghhBBCSAqUUCaEEEIIIYQQQgghhBCSLpRQJoQQQgghhBBCCCGEEJIulFAmhBBCCCGEEEIIIYQQki4afAeQF/z+/Rt3795FyZIloa2tzXc4hBBCCCEkE+Lj4+Hn54dGjRrBxMSE73DyHKobE0IIIYTkH+rUjSmhnA53795F586d+Q6DEEIIIYRkoXPnzqFTp058h5HnUN2YEEIIIST/yUjdmBLK6VCyZEkASSe2fPnyPEeTfSQSCaKiomBgYACRSMR3OIQndB0QAAi/fgNxcbEo0q4dXQcFHH0nECD/XQfe3t7o3LkzV8cjGZOX6sb57dolWYuuD6IK1YWJKvS9QVKTV68PderGlFBOB9mrfOXLl0elSpV4jib7SCQShIeHw9jYOE9d+CRr0XVAACDU8wNiY2NgUakSXQcFHH0nECD/XgfUXYN68lLdOL9euyRr0PVBVKG6MFGFvjdIavL69ZGRujENykcIIYQQQgghhBBCCCEkXSihTAghhBBCCCGEEEIIISRdKKFMCCGEEEIIIYQQQgghJF0ooUwIIYQQQgghhBBCCCEkXWhQPkIIIYTkWlKpFEFBQYiPj4dUKuU7nAKLMYbExESEhYVBIBDwHY5KQqEQ2traKFq0KIRCajdBCCGEEEJIdqCEMiGEEEJyJalUCl9fX8TGxkIkEkEkEuXqZGZ+p6GRu6uNjDEkJCQgNjYW8fHxsLKyoqQyIYQQQggh2SB33xkQQgghpMAKCgpCbGwsChcuDHNzc0om84gxBolEkuuT+owxBAcHIzQ0FEFBQShWrBjfIRFCCCGEEJLvULMNQgghhORK8fHxEIlElEwm6SYQCGBubg6RSIT4+Hi+wyGEEEIIISRfohbKhBBCCMmVpFJprm8RS3IfgUAAkUiUL/vc/vDhAy5duoSoqCgMHz4cFhYWGVr/5cuXuHv3LsRiMWrUqIHGjRtnT6CEEEIIISRfy7UtlF+8eIHp06fD3t4epUuXxvv37zO0fkhICMaPHw8HBwfY29tj4MCB+Pz5czZFSwghhJDsQMlkoo78dt24ubmhSpUq6NKlC06ePIkFCxYgMDAwQ9sYN24c6tWrh7dv38LPzw/dunVD586dkZCQkE1RE0IIIYSQ/CpXJpQ3bdqEIUOGwMTEBA0aNMC3b98yVNkNDQ1F3bp18b///Q+bN2/GwYMH8fv3b9SqVQteXl7ZGDkhhBBCCCFZq1ChQjh06BA8PT3RoUOHDK9/6NAhbNy4EceOHcPu3buxYcMG3L17F1euXMGSJUuyIWJCCCGEEJKf5cqE8vDhw7kWyiVKlMjw+kuXLoWvry9Onz6NBg0aoFq1ajh69Ci0tbUxefLkbIiYEEIIIST32rNnDzZv3pxl2ztx4gRcXV25acZYqv+kf/5dvHQJmzdv5qal0r//GGNZFl9+U61aNVSpUkXt9Tdu3IgKFSqgU6dOXJm9vT3atm2LrVu3QiwWZ0WY2S5BLEVcokTuX7w46V+CWMr9S5T8/SeWSCGRMu5fymtO9o8QQgghhKRfruxDWVNTM1PrHz16FPXr10fJkiW5Ml1dXXTp0gU7d+7E79+/YWJikskoCSGEEJJdkid5lCV7VKZ/lMxQnSpiyqfSkVtKd/qJKV9eKpVizuxZaNykMVq0aJn6tlXsLCMpsMNHjuD373C4DBuhcgdMsShZGQOTSiGQMHz3/45BgwZht+sBRMWLUyyoPCpZadHiJdClc2fYVqqC2nXryi0jEgigq0V9Zme12NhYPH/+HP369VOYV69ePZw/fx7v37/PVMI6J0THi/HmezgSxBIAgAB/rhMll4tcEZNCHBcDzVAxBEJR0nyB4tLJy2T/KxQmL0sqFf5ZUJB8HYHg7zrJtsMtm2zjsv+V35/qGOTLlWxHbjmBkjL5faTcj7KyjG5HSXgq9/f3lKneh9z6Ss+XYqzy2049huRlEokEiRKGBLEUIiZIdT8qY6XvrDwpMTERQUFBCAwMRGBgICIiIhAdHY2oqCjExMSghJcXJGIxAnx9oaWlBUNDQxgbG8PIyAiFCxeGpaUlihUrluncBSGE5FW5MqGcGUFBQfjx4we6du2qMM/e3h5isRhv3rzBP//8w0N0hBDCr+joaHh5ecHb2xs/fvxAYGAgAgICEBISgqioKERHRyMmJgYN/nQzdHfJEohEIujo6MDExAQmJiYoVKgQzM3NYWVlxf0rW7Ys9PT0eD461f4mJv9MpyznpmXz5ZdHKvNTbku2Pbl1meI20xtD8v8k3yYDSzGteJypxZ2eGAAGqUSC2OgYBMcLIfyTjGHJjoebZvLbknL/TVkOufmy7Uil8qlIxgBRdAJMdDURHS/felLttoQqkrsZWDXtKFKeexXbE4vFWL1qJZhAiPqNmqVvn5loRClrjRmXIFG+gIp8CFcsuwAZw+JFC1GqdBm0adcBYonyge8UNvenwKaiHdq274C5s2fi6k03boZYKoW2KFe+OJfn+fr6QiqVwtLSUmGe7E3Ar1+/qkwoBwcHIyQkRK7M29sbQFIyTiJRcU1lsbiERETGJCA8LgGG2pp/v6+SfzCU/K9UIgVLTIBAogHBnwxxyu+vlBS+h1NsW1mR0gdfSr5zUks9snTMT0nZ8koTs1CeVFW2JeXJatX7Sf8+0p+sVvq8QFlcyv5faQI7ednfDTGpFJK4GIh04iEQCtOdrE5emt6HEcm3l/zBg6w8Wx5GKPu/XPQwQn5DqS+X8mGEfFny5eQPMCDgB954vMLHD5747O0Nby8v+Hz+jODgoFTfTuhsZAwAOHf4sMplBAIBzIsWRalSpWBjY4MKNjawtbFF5cqVUapUqaQHEamdE3oYkSfJfvty6veP5C159fpQJ958l1AODg4GABQuXFhhnqmpqdwyqtbPDZVmPuTVC59kLboO0ie1BGXKpFhqN67J56c7yZeO5KCUMXz94gOPly/h8fIF3rx+hc/eXgj48SNdx1fhTyX6Y0R4upYXCAQoWao0rG0qooJNRVSqUhUONWrBvKiFXHJSLs4Ux6VwPMkSkcmTp1JZEpcxLjmJP/OTJyeV7QPpPNdItrhC4lR+U4rHo2J+ev5uCrtPd4IbKeYrv0FKbXtMSSpDAABMCiTEAFpiQKA64ceSrwPFm0VlN1SyAmU3xABQSCqBlGlCLGUpV0mflDGkdz0l6yjcvqq6CUzvPoV/EwgawhTnPRvuL5NuagEtDfU3zqQC/A7/jVMnjmHOvPnQFKm3rb7OzujV3Qmeb1+jcpWqSdtmgj//BTKVOZfF+ufiVvVbVpB+42JiYgAA2traCvN0dHTkllFm69atWLBggdJ5UVFRCA9P3+9EZkXHJUIaHw0TDQH0M3AHw0RSiJkEGlpiLqGcX6lKjin+diT7XVO1rVQKmLLi9CbhlS2nIlYm20SKA+DKFWf9KUv9OyT57zWTSsES4yBIlEAgFCrU4VSvnYGHEaluJf0PIwDVvymMJfvtUPIEgykpA1NMwqeVrFZOWdpadX1Abn8ZehihuI+/m0sqTUxMwMd3r/H6+RN8fOsBL893CP2p+t4/sxhjCAoMRFBgIJ4+eSI3z6SQKWztq6CifVXYV3NEZQdHaP/5zk0WuML/CpX+DZQ/jJCdlOTfbEIlJ17pQ4Q0/r7pfTMCyhLmCmumfh0kzVe9nbRjkP0nAw8/lMSo7KmVsv1JpQyxMdGITZRCQ8nD8NTObXofSql6SPR3O/QwIreSSqWIjo6GQCCAMA/VO6KiojK8Tr5LKEv/ZBGU/eFkZandROSWSjMf8uqFn18oq9ynlSxKqxVi8qXTSk7K/kcikSI2NhqRcWIIRcK095HsP/LLMqTcBVNRO1Y4ToXQ/sausI804lM83j8JR1lyEsrL/56vv+Updq2w39QSgymiThGr/DZTrp38XKu6cZNKpfjm44XX7k/w+vkTvHnxDJHhv1ONIysxxuD79Qt8v37Brf+ucOVmFsVhU7kaKlWvhWq1G6BI0eJKWxD93U5SBSrlfU+6k5PcOkoqaMpqninmJY9NeUVTMQaV20gxI9UbBYV1UuwjxbZVnUPllfAUrXlUnMskSm5qpQxiEaChzSAQZibRl/F1RfEMQgGDhiDzCcaso+KDmgG/fv3C+nVrAQD37t7B3NkzAQCVKlVCz1694fHKA6dOnQSQ9PfS19OHXSV7tGzVClpaWtx2Pn78iMOHDmLI0KEwNDTC6ZMnERwchGrVHdG6TRvFvy1LSp48efIYd2/fhq6eHjp07IjSpcuojPX+vbuYO3s2tu/ajQrW1jh7+hRiY2PRoWNHMOnf1skxMTFYuXwZKtrZoWev3n/3yhi2btmM8N+/MX7iJOjp6aFFi5bQ1dXF4UOHsGx5ZdmCYFIBJFKJWon/lBhjEIvFKutt6lSa8yrZ2yPx8fEK8+Li4uSWUWbkyJHo3r27XJm3tzc6d+4MAwMDGBsbZ2G0qjHNBAhDpRCJAE3d9L9izqRJ9X5NXT2uywtCZJhUgsTYGLo+ALn6doqiFIspLqdkM0ofLijU8VNMqHqArmrbjDF8++yFR7evw/1/9/DulTvi/3yvZZRQKISOnj509fShraODQiIRGGOwMDKGRCxGbHQ0YqIjuXxDWn6H/cLj+7fx+P5tAICmljYqVnVElVr14Vi/Ccra2sv9Tis+5Ocm5KT/YcTfecp/V//+Hf8+Y2DybwooDyGVunHyZTL2QIKvNyPktpVKsjr5/zKpFNJ4CURx4r95ExUJYIV4VSXwU4k3M29GpNxeytbxyt6MUNbYI603X9L7IED+uNL3d8lTb0bgT71DMxHQ0oVAJFJ9T5bKdmRlGkKB0ocW2cHAwCDD6+S7hLKsb+TIyEiFeREREQCSRspWJbdUmjNLWetJVQlL2Y+NWCyBVqIEWrr6EIlE6r8izeR/wORa9P1ZTzFhmlaiMY0YUlkvI692y5WnEoPcD/6f1qAyUiYfV/JznxdaT0qlEkgTJBDESiAQMoVrRyEEBoW/T/J58seRYmdKtpU0V3F7KltPppNsU0q/uAUCbo7KxB4ECj9CSl/RlCtX3EbK8rSSfinjSV4eHxeLl48f4OHt63h0+0aGW2AYmxSGqbk5ChcxRyHTItA3MICunh50dPRQNTgAUrEYpiWswBgQHx+HqIhwRIaHIzLiN36GBCPwux9iU2nVBgAhgT8QEvgDD25cBgCUKlsedf5pjMat2sGhZh1oaOS7n6F8h9dkjDgGgCDftSrU0NTk6hPa2jpc3UXfwAACoRCa2lpcGWMMAQEB2DV+LExMTOB29z6MjIwAAF+++GDdmtWwtbWF6759qFmrFsJ//8bKFcvRf+AgbNqyNdlek75w1q5ZjSePH8Gukj0uXryAxQsX4L+bbqhWvbrSWHft2gl392cICwuDQCjE/Xv3YGZujjJly8ktp29ggB8/fmDzpo2oV78BrEqVAgBMmzwJu3Zsx+Fjx6H/p6KqraMDB4dquHf3zt+/LWMQCAUQCUVZ0kJbIBBAM9l5TkmdSnNeVbJkSQiFQnz//l1hnr+/PwCgdOnSKtc3NzeHubm50nkikQgiUc58L4hEoqQuCYQCrvud9JACf9YTZWg9UjDQ9ZH3MMbg4f4EN6+cx72b/+G779d0rWdS2BSly1nDqnRZWJUtj+IlSqKIuQWKmBdFEfOi0NM3kG8JfO0iJInxGNWuS7Iuv5Japob9+oXgoAAEB3xH0I/v8Pv2BV+8P+GL9yf8Dv2ldP+JCfF4/ex/eP3sfzi0ZRUsLEugUYu2aNyyLarXrpen68TK7lXTensg5ey0Eupy21aykbTeCFW67RTz00rWJ1+GCQSQCoRgyd7eU3p8Ke77FWJI577VOr5Uzl1qDyOSL5feKhljyR5GCJTHyc1WsvF0J5JVTKW8d06r5X16H3hkJJHM/a9UCml8NITh4Oq5yhoRKXtwkTIpbqitCQcrkxxpka5OfS7vfmupYGVlBQMDA3h5eSnMk5XZ2dmpXD+3VJrjxRJ8Do5CokSWWPuTiJQl/ZQkKBW/wJUnGlUlJ5lEAkl8DIThUgiEIsUv1pSJYe6/8gsyuf9PPUGaUmoJxZTbUZUITnNbsnKB/LRAkLRMyiSg0jhTLJJW60mViUV1EpQpvsBUPVFV1npSAMXDUniixwAJ04BIWwShUCN9cabcN5CiXPF4U4sz+TL0Oo+8hPh4PLh9A9fOn8LD2zcRF5t6QhcASpQqg4qVq8LathJKlS2PUmXLo0TpMtDVVd0iTVaJ/idZJTolxhgiwn/jh58vvnh/go/XB3z+6IlPnm8R+N1f6TrffLzxzccbx/fvhklhUzRp1Q7N2nZErfqNcuz7lWQMnzfbTKDwrQIgqVWlj8/nHI0lLWXLluO6D0iLiUkhTJg0BQvmzUXtOnUxaco0ufl2dvaws7OXK5s5ey4cHSpj9coVWLh46Z/SpPOyZ/cunL1wmUuelilbDgvnz8WYceNRoYINt40vPj4QCIQ4cfocAGD6zNmoVrkSFi9cgNPnLijEKZVKcffOHVS0q4S69eqBSaV4+/YNrK0rQPHXBFi4eCkunD+HBfPmYo/rAcyfMxs7tm2F68HDaNuug9yyNra2OLDfFVIpg1AohODPL6tAkDXf+7JtqPpeKUjfN3p6enB0dMSjR48U5v3vf/+DqalpqnVjQgjJDXw+fcDVc6dw7fwp/PD3TXVZfQMDONSoA7uq1VCxsgPsqjjArGixTMcgEAigp28APX0DWFqVUrpM2K+f8HzzCm89XuDdqxd48+IZwn+HKSwX+N0fx1134rjrTpiaFUXrjl3RpksP2NpXyXP3P0rfHFR6CHnruFSRSiVIFGhCU1eTHkRlUlY8jEi+TlY/jFC5bSXLy/5XypTlXv401FO1D6ni8f2KSkCpwkJIGaBmL3PZLt8llAUCAdq1a4dr164hJiZG7hW+K1euoHr16koHJcltYhMk+PE7DiFR8dDV+vslJXdrnTKxmMnWk5AyMLEEQiGDQCiVSxKm9mq3XGtLhdczUj5DUpJYVBZnypabyTaqMhmrap9pJFqJPKlUiESmAU0d+oHMLRhjePPiGS6cPIKbl88jMpV+jTU0NWHv4IgadRugeu16qFjZAUbGJtkSl0AggLFJIRibFELFylXl5oUEBeLNi2d4/eIZXjz5H96/fqnwIOl36C+cPXoAZ48egFnRYmjXrSc6du+DUmXLZ0u8JP/w8fmMWtUd+A5DztMXr2BnVynLthcaGoob/12Dj48P4mJjwcCgp6uH5+7uCst279lLriVupy5dMH/ubLg/fSqXUE5ISMDI0WO4aR0dHbRq0wanT55QGsOrly8Q+usXZs+dx5WFhISgXDnln9FixYtjwqQpWLJoAbS1tXHo4AHs2rMPXbp2U1i2sKkppFIpfv36BTMzs7RPCEk3Nzc33Lt3D4MHD4aVlRUAYPTo0RgwYADOnz+PTp06AQDevn2LK1euYNq0aXm6ZRwhJP+KjorElbMncfboAXx891rlcppaWnCsUx+1GzRGjboNUMGuMm/fa4VMi6Be4+ao17g5gKSHs16e7/Dsf/fw9OE9PH/8UKFByK+QIBzesw2H92xDmfIV0LXPAHRw6gPDPPJ2NCHqyo8PI6RSYZY8cBBLlDfCzE3yfO1xzZo12LRpE06dOoUaNWoAAObMmYMLFy5gzJgx2Lp1KzQ1NbFixQq8efMGV65cSWOLuUsxYx0YZaCfuMyQSiVIjE2Epq4WJRIJyQWiIiNw9dwpnD68D16e71QuV7S4JRq3aIsGzVqiWq26qbY8zilmRS3QtE0HNG2T1CoxLPQXnj28i0d33XDv1n8KrwOGBAXAdet6uG5dj6o1aqPngCFo2qYjNDVz5vuPkNzE7eZN9O3dA2XLlUPDRo1RqFBhiEQiaGlpIVTJq7Q2FWzlps3NiwIAAgIC5MqtSpVWaEVtbl4UYWFhiIuLU5h3280NhoaG6N3HmSsTCASpdoM0bsJEbN64Hgf2u2LLth3o2buP0uVSG/OCKPL398fu3bsBAPfu3QMAbN++HRYWFtDT08PUqVO5Zd3c3LBkyRI0b96cSyj3798fT58+Ra9eveDs7Aw9PT0cOnQIrVq1wpw5c3L+gAghJBU+nz7g5MG9uHzmGKJV9HlvZGyCRi3boGGz1qj9T2PoGxjmcJTpIxQKYVOpMmwqVYbz0FGIi4vF0wd3cef6Fdy7eQ1hv37KLf/F+xPWLJyFzSsXo01nJ3Tv5wJb+yo8RU8IIarlyoTyy5cv0aVLFwDgBlNp27YttLS0ULhwYbx48YJbNiwsDN++feMGFQGSBrW5du0aRo8eDVNTU64PvePHj6N169Y5ezCEEJJBAf5+OLR7Cy6cOIKYaOWVaMuSpdC6kxOate2ICnb2ub7lfaHCpmjZoStadugKsViMl08f4daVC3C7dgm/QoLklvVwfwIP9ycwKzoX3foORLe+A1G4CLVgJAXHuDGjYG9fGf/dui2XcD118rjS5XV05RPBsu4cxGKxXLmurmKXHKqWBQC3WzfRq09fGBoaQvYCnrm5OUJDQ1XGvmLZUq7ultoDodBfv6ChoYHChQurXIYo17BhQzRs2FDl/KZNm0JDQ4NLJsts3rwZgwcPxt27dyEWi3Hq1Ck0adIku8MlhJB0YYzh0T03HNi+Cc/+d0/pMtraOvineSu06dwd9Ro1g5a2dg5HmXk6Orpo2Lw1GjZvDYlEAvdH93Hl7Em4Xb0oV++Pj4vFuWMHce7YQVSpXhMDRoxDw+at6UEsISTXyJUJZTs7O9y5c0fpvJR93k2ePBlDhgxBsWLyfSI1bNgQr1+/RlhYGMRiMb1OSQjJ9bw832H/9o24fvEMJBKJwnxDI2O07uSENp2dUMWxVq5PIquioaGBmvX+Qc16/2DKguV4fM8NF08dxZ3rV5CYkMAtFxIUgO1rl2HflnXo3Ls/+g8bDYviJXiMnOQWZcuWw9MXr/gOQ07ZFIPUpUV2Q5iyta9YLMaXLz7o5tRd7qYxLCwMn729Ua58znQJExsbiyePH2HV2nVy5VWqVMV//11Tus7ypUuwdvVK7N1/EDu3bcX8eXPQqUtX6OvrKyzr6fkeVapWzbPfYzmtRIkSmD9/frqWbdq0KZo2bap0XvXq1VFdxQCMhBDCB6lUitvXLmHvlnX48NZD6TKVHKqjW5+BaNqmIwz/DEybH4hEItRu0Bi1GzTG9EUrcfu/Kzh9eB9ePXsst9zrF88waagzytlUxMAR49GyQxfqqogQwrtc+S2kra2d6mjTyZmYmHAjoStTqFChrAmKEEKyydtXz7Fz3Qo8vHNT6fxKDtXh5DwYLdp3zhXdWWQlkUiE+k1aoH6TFgj/HYYrZ0/g5IE9+ObjzS0THx+H4647cfrwPnRw6o2BI8ejhFVp/oImvNPR0cnS/or5IBQKYWZujpBg+Rb6GhoaKF26DJ48+XszyRjD/LmzoW9gkGPx/e/BAzjWqKlwnhs1boKjRw7D29sL5ctbc+Ub1q3FkkULsHX7TnTv0ROlS5dGk4YNsG7Nark+mIGkQRVfe3hg1JixOXIshBBCch+JRIJr509h35Z1+OL9SWG+lrY2WnXsih79h8CuSjUeIsxZunr6aNulO9p26Q4vz3c4eWgvrpw5gdiYaG6Zzx89MWf8MGxfuwwDR45DB6c+1D0cIYQ39L4EIYTwxPvje0z6tx8GdGqhkEwWiURo3ckJhy654cD5m+jYvU++SyanZGxSCL0HDcOpW4+xaf8J1P8zmImMODERZ48eQNfGNTF/8igE/vDnKVJCskbvPn1x4vgxjBr+L+bOnonjR48AAJatXIWnTx6jWeN/MGnCODT+px6KFDFD1aoOORZbojgRk6ZMVSjv1KULDAwMcO7MGa5s+7YtmD1zOtas34B+AwYCAGrWqo2uTt2xcf1afPeX/6xeuXwJ8fHx6OPcL1uPgRBCSO7DGMO9W/+hd5uGmDthhEIyuXARM4yaOgdXn7zD/NVbCkQyOSXripUwc8kaXH3yDhPnLIa5hfzb2N99v2LJ9Ano2bI+blw+l+rYBoQQkl1yZQtlQgjJz/x9v2LnuhW4cvaEQgVQW0cXnXs6o++QkbC0KsVThPwSCoXc6NjeH99j7+Z1uHHpLDeIl0QiwcWTR3H9wln0GjwMg0aMp1GwSZ60ZNkKtGjRCu/fv0V8fAL0/nQN0aFjJ7zweIu7d24jNi4WAwcNRuUqVXH61EnERP9tqWRjY4sFi5agVKnSctvV1tbGgkVLUL9BA65swKDBSIiPV4ihUePG0NBYAu0U/VC2btNWacyGhoZw7j8Ahw8dxKQpU/H792/ExsRi/6Ej6NrNSf74li5H1aoO8PPzhWWJv93VHDl0EE2aNoOtbcX0nShCCCH5gof7E2xcvkChSwcAsLAsgQHDxqJjz77Q0dHlIbrcx9DICH2HjET3fi64cvYEXLdtgN9XH27+Nx9vTB85GHZVqmHMtLmo1aARj9ESQgoaAaPHWWl69+4d7O3t8fbtW1SqlDOv2P6OSYD71zBoiAQw0s2Z11ikUgkSY6OhqasPoVCU9gokX6LrIPtER0Viz6Y1OLxnG8SJiXLzdPX00WfwcPQePAyFTIvwFOFfwmsXIUmMB2vXJVdcB998vOG6dT0unz0BSYrBw4xNCsFlzCR07+eSJwdnye34/E5gEUEw1NaAJXVxkgswMKkUAqEQwcEhcLCviA2bt6J7j54Z2sqrly/QsH5d3Hv4CA7V/vblK5ZIIRIKoK+tkSX9Kvv4JN1wly1bVul8Pup2+UleqhtTvYakhq6PnPHDzxfrFs+G27VLCvMsrUpjyNjJaNPJCZpaWjxEp1xuqwsDSY0qbl05j53rVyrtJqReo2aYPH8ZSpXNmfEWCir63iCpyarrIzA8Dia6mqhd1hQiYfaPOaJO3Y66vCCEkGwmlUpx6fQxdGlcC/u3b5RLJmtqaaGPy3BcuP8CI6fMyhXJ5NyoVNnymLd6M87dcUe7br3kEk7hv8OwdtFs9GzVAI/uufEYJSEFg7m5OfYdOKTWSPNhoWHYuXuvXDKZEEJI/hQXF4udG1bCqVkdhWRy4SJmmLZwJU7feoyO3fvkqmRybiUSidCyQ1cc++8B5q7ciKLFisvN/9/dW+jRsj42Ll+AmOgonqIkhBQUlFAmhJBs9P71Swzu2hrzJo7Er5C/g28JhUJ06umMs3fcMWnuUhQuYsZjlHlH8ZJWWLh2K45cuYu6jZrKzfP98hmj+zlh6oiBCAr4zlOEhBQMrVq3QTen7hler0mzZujVp282REQIISS3YIzh7o2r6NG8HnasXY74+Dhunp6+AYZPnIHz956jx4AhlEhWg4aGBjr1dMaZO88wftZCGJsU4uaJExOxf9sGdGtaG/9dOE39KxNCsg0llAkhJBvExkRjzcKZ6N+xOd68dJebV6PuPzhy9R7mrtyIYpYlVGyBpKaCnT02HziFrYfPwMausty8W1cuoFvTOjiwfSMSU3QtQgghhBBCsk9w4A9MHNIXE4f0xXe/b1y5QCBA5179cP7ecwwdNwV6+gY8Rpk/6Ojoot+/o3H+3gv0HjwMItHf1+uDAwMwc8xQjOzbFf6+X/kLkhCSb1FCmRBCstije27o0aI+juzZLtcqwMKyBFZs3YftR8/B2taOxwjzj9oNGuPgJTdMW7QKBkZGXHlsTDQ2LJuPAZ1a4NP7tzxGSAghhBCS/zHGcObofjg1r4t7N6/JzatUtTpcz13HnBUb6K28bGBobIzJ85bh8JW7cKxTX27e04d30bNlAxzZsw0SiYSnCAkh+REllAkhJIuE/w7DvEmjMLqfE374+3LlmlpaGDpuCk7deozm7TplyYBT5C+RSIQe/V1wxu0p2nXrJTfv47vXcO7QFDvXr0BiQgJPERJCCCGE5F9+375geO/OWDJ9AqIjI7lyk8KmmLNiA1zPXYe9gyOPERYM1rZ22HHsApZu2gWzosW48rjYGKxZOAsu3drA59MHHiMkhOQnlFAmhJAs8L+7t9C9eT1cOnVUrtyhZh0cvXoPwyfOgK6uHk/RFQymZuZYuHYrdp+8jLLWNly5RCzGjnUr0L9Tc3x4+5rHCAkhhBBC8g/GGE7s341erf6B+6P7cvPade2J025P0LlXP7UGcSXqEQgEaNWxG07deoRuzoPk5r156Y7ebRvBddsGaq1MCMk0+mYnhJBMiI2Nwcq50zCmf3e5Qff09A0wbdEq7DpxCWXKV+AxwoKnWq26OHz5DlxGT5LrS+7T+7fo36k59m5eS5VoQgghhJBM+BkchHEDe2LF3KmIi43hyosWK44N+45j4bptMClUmMcICzYDQyPMXLIGO49fRMnSZblycWIiNi1fgBF9OiPguz+PERJC8jpKKBNCiJo833jAuV0THN+/S668fpMWOHnzf+jR34VaZPBES1sbI6fMwv7zN1A+WX/VErEYW1Ytxog+XRAU8J3HCAkhhBBC8qa7N66iZ6sGeHjnply5k/NgnLjxPzRo2oKnyEhKjnXq4+i1e+g/bIzcfcnzxw/Rq3UDXD13isfoCCF5GWU6CCEkg6RSKQ5s34gBnVvg62cvrlxHVw+zlq3Fhn3HYFG8BI8REpmKlR1w6KIbho6bItda+fnjB+jduiHcrl3iMTpCCCGEkLwjNjYGS2ZMwMQhffE79BdXbm5RDNsOn8WMJathYGiUyhYIH3R19TBu5gLsOXUFllalufKoiAjMHvcvZo4ZisiICP4CJITkSZRQJoSQDAj/HYaJQ/piw7L5kIjFXHklh+o4evUuuvYZSIPu5TKaWloYPnEG9py+KleJDv8dhinD+mPpzImIj4vjL0BCCCGEkFzui/cnDOjUAmeO7Jcrb9G+M4799wC1GjTiKTKSXlUca+Ho1bvo1KOvXPl/F06jX4em+PjuDU+REULyIkooE0JIOr156Y4+bRvh/q3/uDKRSIR/x0/FnlNXYVWmHI/RkbRUrlYDR67cQbuuPeXKTx92xeBubfDd9xtPkRFCCCGE5F7Xzp9Gvw7N8PmjJ1emb2CABWu3YtnmPTA2KcRjdCQj9A0MMXfVJqzavl/u7+b31QcDu7TEmaP7wRjjMUJCSF5BCWVCCEkDYwxH9mzDkO7tEJhs8Apzi2LYcfwihk2YDk1NTR4jJOllYGiEheu2YdH6HdA3MODKP7z1QN92jXH/1nUeoyMke716+QJHDh3kOwxCCCF5RHxcHJbNmoxZY4ciNiaaK096M+8+2nfrRW/m5VFN23TA8esP4FinAVeWEB+PJdMnYO6EEYiJjuIxOkJIXkAJZUIISUVsbAxmjR2KNQtnQZyYyJXXbdQUR67cRbWadXiMjqirbZfuOHL1HmwqVeHKIiPCMX5wL2xbsxQSiYTH6AjJHv9du4bZs2bwHQYhhJA8IOC7P1yc2uLUob1y5b0HD8Oek1dgaVWKp8hIVjErWgxbD5+By5hJcuVXzp5A/47N8c3Hm6fICCF5ASWUCSFEhR9+vnDp2gb/XTjDlQmFQoycPAsbXU+gkGkRHqMjmVXCqjT2nrmKTj2d5cp3b1yNsQN7ICL8Nz+BEUIIIYTw6OXTR+jXoSk837ziyvQNDbFymysmz1sGTS0t/oIjWUpDQwMjJ8/Cpv0nYFyoMFf+xfsT+ndqjoe3b/AYHSEkN6OEMiGEKOH+6AH6dWyGj+//Dk5RuIgZth05B5cxkyAU0tdnfqCjo4u5KzdizooN0NLW5sof37uNAZ1a4OtnLx6jI/ldaGgodu/agS8+PggPD8flSxdx5NBBvHr5Qm65nz9/YveuHXj65LHCNk4cP4YDrvsgTjZIaHbw9/PDiePHcOzIYXz7+hVRUVHYvWsH3r17m637JYQQkrPOHHHF8D6dEfbrJ1dmY1cZhy/dRrO2HXmMjGSneo2b48iVO6jqWIsri4qIwLhBvbBv63rqV5kQooAyIoQQkgxjDMddd2Fk3y74HfqLK7erUg2HLrmhRt0GqaxN8qrOvfph7+lrsCz59/VN3y+fMaBzCzy658ZjZCQ/e3D/HsaPGY0N69fin7q1cfb0Kex33YcGdWtj2pS/r5+ampri4H5XOPfphZiYGK58zqwZGDp4IIyMjaGhoZEtMUokEkybMgmODlVw+tQpXLlyGfVq18CsGdMwfsxo+Pn6Zst+CSGE5KzExEQsnz0FS2ZMlOvmrXUnJ+w9ew0lS5flMTqSEyyKl8COYxfg5DyYK2OMYfOKhZg5ZghiY2NSWZsQUtBkz90HIYTkQWKxGKvnz8DJg3vkytt26YFZy9dBR0eXp8hITqhYuSoOXLyFqcMH4vnjBwCSWmaMHdADE+YsRu9Bw2jgGZKlPF69AgB4e3nh4ZNnMDQ0BABMnjgeWzZtRP+Bg1Cpkj0EAgFWrFqD5k0aYfPGDZg6fQaWLl6EDevWYvdeV3Tu0lVh29HR0Th65JBc2Yvn7oiLjcXuXTvkyqtWdUDNWrWVxjhrxjQcP3YUdx48hK2NLQRCIR79739o0bQxty4hhJC8LezXT0wbOQjPHz/kygQCAcZOn4d+w8ZQ/acA0dTSwowlq2FjXxkr5kzlHi5cv3gW33w+Y/3eIzC3KM5zlISQ3IASyoQQAiAmOgozRg/BA7frXJlQKMS4mQvQd8hIqkgXECaFCmPLodNYPX8GNwiNVCrFmgUz8cXrI6YtWpVtLUFJxh154otjz9JuITu+eQU0tTXnpt0+BGP9zU9prterphX61Lbipt//iMD0M69TXSYjXnu8goGBAfbuP8glkwHAqXsPbN+6Ba9fvUKlSvYAgDp168Gpe0+sX7sasbGxWL1yObZu34kevXor3fbvsDCMHzNa6byU5RMnT1GaUH754jm2bNqIdRs3oWJFOzCpFABQt159FLe0RGJiIooVp5tKQgjJy75+9sLYAT3w3e8bV2ZgZISlG3ehfpMWPEZG+NS19wCULW+DqSMG4ldIMADg47vXGNi5JTbsOw7ripV4jpAQwje6KyaEFHghQQEYN6g3Pr77mygyMDLCiq37UOefJjxGRvigqamJGUtWo5xNRayePx0SiQQAcObIfgQHBmDZ5t3Q0zfgOUoCAGExCfgcEp3mcpFxiQrT6VkvLCZBbjo2UaKwXsplMsLD4xU6duoMc3NzuXIdHR0AUOirfdGSpTh/7gxWrViG9Zs2o9+AgSq3rW9ggMFDhsrv79VLvH/3Dr37yg9Eqap18p5dO2FiYoL+AwYpzNPW0oadHd1MEkJIXvby6SNMHNJXbiDiUmXLY+3uwyhdzpq/wEiu4FCzDg5evIVJQ/txAzQGBfyAi1MbLN+6D/UaNeM3QEIIryihTAgp0Lw+vMe4gT0QFPCDKytWoiQ27juOshVseYyM8K1HfxeULmeNaSMGcjdaD9yuY1ivTli/9yhMzcxT3wDJdoX0tFDOTD/N5Qx1NBWm07NeIT35Uex1NUUK66VcJr1CQkLw4/t31KytmMx98fw5AMC+ShW58oMH9iMxMRECgQDVqlVPdfsmJibYuHmrXNmKZUvh5+enUK7K06dP4FCtOrS0tAD8HYwnKioKfn6+6NrNKV3bIYQQkvtcv3gGcyeORGLC3wejdRs1xbJNe2BobMxjZCQ3KVrMErtOXsLsccNw57/LAIDoqCiMH9QL0xevQtc+A/kNkBDCG0ooE0IKrEf33DBtxEBER0VxZXZVqmHdniMoYl6Ux8hIblGrfkPsPXMNYwZ0R4C/HwDg/euXGNSlFTYdOIlSZcvzHGHB1qe2et1NNLU1l+sCI73sihvhwuisGZjT49VLAEktfZNjjGHnjm2oVr06190FAKxbsxrLlizC+k2bsXL5MkybPAk379zLklhUiY2JVdrFy5HDByEWi1HFwSFb908IISTrMcZwcMcmbFg2X668S+/+mL54NXXtRRTo6uph5TZXbFgyF4f3bAOQNGjvkhkT4f/tK0ZPm6vwVhUhJP+jTz0hpEC6cvYkxg3qJZdMbtSyLXYev0DJZCKnTPkKcD17Hbb2Vbmy737fMKhra3i4P+ExMpKXyQbke/jgvlz5rp3b8fHDB6xas44r27plE+bOnon1GzdjyNBhmDtvAR4/foRTJ09ka4wVbGzw8uULxMT8HdXd0/M9Fs6bCwCoSgllQgjJU8RiMVbMmaqQTB41ZTZmLVtHyWSikkgkwsS5SzBt4Uq55PH+7Rsxa+xQuZbuhJCCgRLKhJAC57jrLswZPwwSsZgr6z14GFZt3w9dvbRfgycFTxHzoth14qJcX3HhYaEY0acL7ly/wmNkJK/y8HgFa+sKuHP7NoYOHogtmzagv3MfzJo+DTv37EOduvUAAHt378K0yZOwfOUquAz9FwDQx7kf7CtXxtxZMxEXF5dtMU6eOg2RERFo26o5Nq5fh+nTpqJPzx4oUbIkDAwMUK4ctdAnhJC8Ij4uDlNHDMTJg3u4Mg1NTSxavwODR0+kAahJuvQYMARrdh+Wu2e6fvEsxrv0RmxM2uNTEELyD0ooE0IKDMYYdq5fgZXzpsmVT5q7BJPnLYNIJOIpMpIX6OkbYO2eI+jU8++AZvHxcZg6fACunMnelqIk//F4+RJ16tbFvYePYFfJHl++fEH16o549eY9uvfoCQAI+PEDr197YMOmLRg1Zhy3rlAoxPqNm9G8ZUu8ffNa1S4UVKteHX2d+6V7+br16uPhk2do3aYdfvz4gUqVKuH+/x4DACpXqUrJB0IIySOioyIxdmBP3E32ENzAyAibD5xC2y7deYyM5EUNm7XCrhOXYGr2963Ox/duY3ifzvgdFspjZISQnETvtBBCCgSpVIrV82fg+P5dXJlIJML8NVupIk3STVNTE3NWbICFZQnsWLscQFIfcnMmDEdUVCR69HfhOUKSF0RGRsLH5zOGDhuGohYWmDBpstLlihUvjvUbNyudV6duPa4Vc3q1bNUaLVu1ztA6trYVMX3mLAAMTCqFWCLBp48fMXjI0AxthxBCCD/CQn9h7IAeeP/6JVdWtLglNu0/gXIVKvIYGcnLKlauir1nrmKUczf4f/sCAHj78jmGdm+HLYdOw9yiOM8REkKyG7VQJoTke4mJiZgzfrhcMllbWwdrdh2mZDLJMIFAgH/HTcX0xavlWmiumDMF+7au5y8wkme89vAAYwyV7KvwHUqGffz4AQkJCaha1YHvUAghhKQhKOA7hnZvJ5dMLlXOGntPX6VkMsm0ElalsefUFVSw+zuIsI/XRwzu1gbffLx5jIwQkhMooUwIydcS4uMxdfgAXDt/iivTNzTElkOn8U+zljxGRvK67v0GY+HabXJdpWxesRAbly8AY4zHyEhu5+GRdGNvX7kyz5Fk3Pu3bwEAVapWTWNJQgghfPL76gMXp7b44v2JK7O1r4o9Jy/DongJHiMj+UkR86LYeewiqtWqy5UF+PvBxaktPrxNf7dchJC8hxLKhJB8Ky4uFpOGOuPezWtcmamZOXYdvyRX6SFEXW279sCqHQegpa3Nle3ftgHLZk+GVCrlMTKSm5mZmWPUmLEwMzPjO5QMMzI2hsvQf1HRrhLfoRBCCFHB68N7uDi1RYC/H1dWvXY97Dh2HoVMi/AYGcmPDI2NsfngKTRs/rdbrbBfPzG8dye883jBY2SEkOxECWVCSL4UGxuDiS598L+7t7iyYiVKYs+pK7CplPdaBZLcq1GLNtiw95jcaNenD+3D4mnjKKlMlOreoydWrFrDdxhqad2mLTZs2gItLS2+QyGEEKLEx3dvMKxXR/wKCebK/mnWCpsOnISBoRGPkZH8TEdHFyu370e7br24ssiIcIzo2wWvnz/lMTJCSHahhDIhJN+JiY7CuIG98OTBXa7M0qo0dp24jJKly/IYGcmvajVohG1HzsLI2IQrO3/iMBZOHQOJRMJfYIQQQggpMDzfeGB4704IDwvlytp07o5VOw5AR0eXx8hIQaCpqYn5qzfDyXkwVxYdGYlR/Zzw4sn/eIyMEJIdcm1COTIyEvPnz0fjxo3RoEEDjB8/HgEBAele/9WrV3BxcUHdunXRpEkTzJs3D+Hh4dkYMSEkN4iOisSYAT3w/PEDrsyqTDnsPnkJxSypvziSfSpXq4Gdxy/CpLApV3bx5FEsnDqWksqEEEIIyVbvPF5gRJ/OiAj/zZV17tUPC9dtg6amJn+BkQJFKBRi+uJV6D14GFcWEx2FMQN64Gmyxj6EkLwvVyaUIyIiUK9ePZw6dQqTJ0/GokWL8PbtW1SvXh2+vr5prn/hwgXUqlUL4eHhWLRoEcaOHYvTp0+jVq1a+PXrVw4cASGED5ERERjl3A2vnj3mykqXs8bO4xdhblGcx8hIQWFdsRK2Hz0vl1S+dOooFkyhlsqEkMyLiIjAzZs3ce3aNQQFBWV4/S9fvuDy5cu4desWIiIisiFCQggf3rx4hhF9uyAy4m8Dqm7OgzBr2ToIhbnylp/kYwKBAJPmLkX/YWO4srjYGIwf3FuuO0JCSN6WK39dVqxYAU9PT5w/fx7t27dHkyZNcPbsWYjFYkyZMiXVdSUSCYYPHw47OzucPHkSzZs3R5cuXXDt2jX4+vpi3rx5OXQUhJCcFB0ViTH9nfDmpTtXVs6mInYevwizohY8RkYKGmtbO4VBby6fPob5k0ZRUpkQorYdO3agWLFimD17NlatWoVSpUph1qxZ6Vo3IiICXbp0QcWKFbFq1SpMnToVFhYW2LRpUzZHTQjJbh7uTzCqXzdER0ZyZT0HDMWMxaspmUx4IxAIMHbGfLiMmcSVxcfHYeKQvvjfnZs8RkYIySq58hfm0KFDqFu3LsqVK8eVGRoaomvXrjh37hwik/1YpvTu3TsEBASgQ4cOEAgEXHmJEiXg6OiIQ4cOQSwWZ2v8hJCcFRsTjXGDesklkyvY2WPH0fMwNTPnMTJSUJW3SUoqFy5ixpVdOXsC8yaOpKQyISTDbt26hREjRmDx4sV4/Pgxbt26hbNnz2LZsmXYuXNnmuu7uLjg2rVrePToEe7cuYPnz59j5cqVGDt2LM6fP58DR0AIyQ4e7k8wun93REdFcWW9Bw/DlAXL5e6FCeGDQCDAyMmzMHziDK4sMSEBk//tT91fEJIP5LqE8s+fP+Hr64sqVaoozKtSpQoSEhLw+vVrletHR0cDAAwMDBTmGRgYIDw8HJ8+fcq6gAkhvIqLi8UEl754+fQRV1bBzh7bjpyTayFKSE4rV6GiwkONq+dOYsmMCZBKpTxGRgjJa5YvX47ixYtj3LhxXFmbNm3QqFEjLFu2LNV1g4KCcOrUKfTt2xfVqlXjykeNGoWyZcti0aJF2RY3IST7eL55hTEDeyAm+m8y2XnoKEyau5SSySRXGTpuCsZM//umeHx8HMa79MHzxw95jIoQklkafAeQUmBgIACgSBHFRJCZWVJLr9T6jLO2toZQKMSrV6/kysViMd6+fcvtw87OTun6wcHBCAkJkSvz9vYGkNSdRk61LJNIJGBSKZgAkEpzJu/PpH/2KZWAUh0FV166DhLi4zF5WH88+989rqxsBVtsPnASRsbGkEqpJai6BEwKJmV54jrIzUqXt8a2I2cxvHdnhP5M+m05f/wQdHR0MGle3rjh4/U7gTHZ/+T0nkmqsv7vwf2lWdZsn/25dlTV2/LSmwKJiYm4e/cuunXrpvD6etOmTTF37lx8+vQJFSpUULr+58+fAQA2NjZy5QKBADY2Nrh69Sp+/vyptO5NCMmdvDzfYZSzfDcXA4aPxZjp8/JE3YIUPANHjIM4MRHb1iwFAMTHxWLcoF7YfOAkHGrW4Tk6Qog6cl1CWVbBF4lECvNkZYmJiSrXL1KkCJydnXH48GH07dsX7du3B2MMs2bNwvfv3+X2oczWrVuxYMECpfOioqIQHh6udF5Wi45LhDQ+GhKRAIk59GdiUinE8bEAAAH1t1Vg5ZXrQJyYiJnjR+DRXTeuzKp0WWzYdQAGerpIjI3mMbp8ICEBEnECJLExufo6yAtKWFpi457DGDWgJ8J/hwEAju/fDS1NTQyfMDXX3/jx+Z0gkogBaIBRi+4s8evXL/z+/VuuS7GMyNa/A2NgUgEkUglSfiKuX7+O06dPw8/PD4mJiShevDh69+6N1q1bp7I5BrFYrLLeFpXs9fDc7tu3b0hMTESpUqUU5snKvLy8VCaUjY2NAUDpwNSysk+fPqlMKOf1xhZ56UE5yXl58fr4+tkLI/t25eoUANBr0L8YNXU2GJP+fRZLMoUaV2S9waMnICEhHns2rQGQ1G3h2IE9sOXgaVRyqM5zdOmXF783SM7JqutDtg2JRAKw7L9fVKc+l+sSyrJKr7J+kmVlJiYmqW5j27Zt0NbWRrdu3WBubo7o6GhYW1tj1KhR2LJlC8zNVfepOnLkSHTv3l2uzNvbG507d4aBgQEXX3ZjmgkQhkohEgGaupo5s88/rTk1dfUgECom9EnBkBeuA7FYjIWTx+LB7b8DOlhalca2o+dgblGMx8jyD5GWFgBAmIuvg7zEpko1bD54EiP6dEVUZAQA4ODubdA3MpYbrCQ34vU7QRwDQKA0kf3lZzTixfxX47U1hChTRJ/vMNJlz+5d2LF9G3y++am9jWx7qMAYBEIBREIRkj9jWbp0KebMmYOOHTvC0dERjDF8/foVx48fR7t27VTHKRBAU1NTZb1NWddouZUs+a2np6cwT18/6dpLbXyRihUrwsrKCidPnsTChQuh9ef7/f3793j+/Hma6+f1xhZ55UE54Udeuz78fb9hZP8eCP319yFPp+69MWbyDIjjYniMLB+ixhXZYvDw0YiPjcah3dsBANFRURjd3wmb9h2FjZ09z9GlT1773iA5K6uuD5aQALFAAxERGhDmQAMkdRpb5LqEspWVFfT09LjX85KTtYawtbVNdRt6enrYuXMn1q9fj69fv8LIyAglSpTAv//+CwMDA1SqVEnluubm5ioTziKRSGnL6ewgEokgEAohEAogzKEbeCnwZ5+iHNsnyX1y+3XAGMOymZNw88oFrszCsgR2HD0Pi+IleIwsfxEIkr5/kEuvg7zIrkp1bNp/AiOduyE2JqkF/fa1y6Gnb4C+Q0byHJ1qfH4nMK7ypFiJihdL8e57BDRF/LXwTpQwVLI0grL4cifV5zNtyZu8ZXz9sLAw/PwZAmtr5a1oBX+2LxBArtX+qlWr0LVrV5w6dUo+GsZSbd0vm6eq3pZT9bmsIEsAKxtUWvbWno6Ojsr1hUIh9uzZg06dOuGff/7B4MGDERkZiTVr1sDa2hofPnxQmqyWyeuNLfLCg3LCn7x0fQR+98c4F2f8DP7b/WPbLj0wc/l6he5wSOZR44rsM3bmQkgZcGRPUlI5KjIS44f0w84TF1HW2iaNtfmXl743SM7LqutDkKABDR0NGBkZQyTM/nsNdRpb5LqEslAoRMuWLXH37l3Ex8dDW1ubm/fff/+hUqVKSl/5U0ZPT4/rKzkiIgJnzpzBoEGDoKGR6w6bEJIOjDGsXzIXF04e4crMLYph+9HzKFaiJI+REZI+VRxrYd2eIxg3sCfi4+MAAGsXzYaOrh669R3Ib3B5kKZIgFKm/LUO/vYrc13rJCQk4MsXHxQrVhxGRkYIDw9HRHg4LEuUkEsOxMTEwM/PF0WKmMHU1FRuG/5+foiOiUa5cuWzvX4T8OMHGADLEiUhkUjg7e0FMzNzFC5cOM11jxw6gIXz5yHo1+8M7dPKygrnzp1Dr1690Lp1a7Rv3x5FihTJ9V3FZCVLS0sAyscQkZXJllGlefPmePfuHVxdXXH37l0YGRlh//79ePjwIRYuXKiyuwwg7ze2yO0Pygm/8sr1ERIUgJHO3RDw/e8bJs3bdcK81Zvp3jabUOOK7DVxzhKIE8U4cWA3ACD8dxjG9O+OPaeuonhJK56jS11e+d4g/Miq60O2DZFIlCMJZXXqc7nyUeacOXMQGRmJadOmQfqnv75t27bB3d1dYSTqLVu2wMHBQW4QvgsXLuD27dvcdGBgIHr27AlTU1MayZqQPGz/9o04tGsLN21S2BRbD59FyVJleIyKkIypWe8frNqxHxqaf1vYLZs1CTcvn+cxKsKHh/fvw7FqZRw84IqWzZqgtmM1VK9qD6viRXHq5AluOcYY2rdpha6d2nODzQHAhfPnYF+xAlz37snWhMLFC+dRrXIl1K7piAb16sDB3g4H97vCsWpluN26mfYGMqFv374wNDTEp0+fMGTIEFhYWGD27Nly5yG/MzY2hq2tLdc9RXLu7u7Q19eHvX3arwmXLl0a8+fPx5EjR7B9+3a0bNkSbm5uqFmzJooWLZodoRNCssDvsFCM7NsVfl99uLJGLdpg8YadlEwmeZZAIMCUBcvRqaczVxYcGICRzl3xKySYx8gIIemVKxPK1atXx7lz53Du3DkUL14cpUqVwpw5c7Br1y506dJFbtmAgAB4eHjI9fdhZ2eHlStXwszMDDY2NihfvjxMTU1x//79HHstjxCStc4c3Y9Ny//24ainb4BN+0+gTHnVraoIya3qN2mBZZt2c0+CGWOYPX4Ynj68x3NkJCe9evUSALBj61YsWbYcH7x98NU/ANWqO2LEv0Pw489gwvr6+liwcDGeu7tzieYb1//DwH59MchlCJatWKWwbbFYjI8fP8j9+/XrJyQSiUJ5ygHXkjtx7Cj69+2N8RMn46vfd3zx9cfQ4cMxbswoAEBVBweFdaRSqeI+gkPAGFMoV9bqVmbEiBHYs2cP3r17hxcvXiAwMBBNmjTBkiVLcPTo0XSf5/xg8ODBeP78Odzd3bmyoKAgnDt3Dn369IGuri5X/u7dO5w6dUpuEL7gYMWb8/Pnz+Phw4fU2IKQXCw2JhrjBvWEj9dHrqxOwyZYtnkPNDVzZpwdQrKLUCjErGXr0KxtR67M76sPRvd3QmQO9c9PCFFfrn2k2bZtW3z58gVfv36FWCxGmTJllD6BHT16NJycnGBtbc2VlS9fHlevXkV4eDgCAwNRsmTJVPuGI4Tkbjcvn8eymX8HLtPU0sLa3YdhV6Uaj1ERkjlN23TA7BUbsGDyaABAYkICJg11xs7jF1GxclWeoyM5wePVKwiFQhw8chRVHZK+z/T19TFh4mR0bN8GT548Rpeu3QAAvfs6Y8f2rVg4by4KFyqMPj27o3efvli7fqPSbQcFBsKxamWl81KWT5w8BQsXL1VYLuDHD4weORyjx47DgEGDwP68NTZy1BisWLoECQkJKFeuvMJ6ERER6d73v8NHKD2G06dPY/v27Xj48CGKFy8OAChSpAg2b94MW1tbuLm5oU+fPkr3kR+NHTsW165dQ4cOHTB9+nTo6elh/fr1sLKywrJly+SWPXr0KJYsWYL79++jQYMGAIDr169j586d6NKlC4oUKYKHDx/i0KFD2LhxI1q1asXHIRFC0pCYkIApwwfg7cu/bydUr10Pq3cehHYq/aYTkpeIRCIsXr8DUZEReHL/DgDg0/u3GDe4F7YcOg1dXcrjEJJb5dqEMpD0GkSZMqm/ym5hYQELCwul84yNjalFMiF53JMHdzB7/DCu+xuhUIilm3ajZr1/eI6MkMzr2L0Pwn79xMZl8wEAMdFRGDOgO/aevgqrMuX4DY5kOw+PV2jRshWXTJYxMzcDkDRIjYxAIMCKVWvQvEkjdO7YDj179camrdtV9iWsoakJ6wryA9uEhv7C77AwlE2RBDYzM1O6jR3btkIikWDs+Ily5QKBAIULm8LMzEzpQFAikUhh379/h+FnSAjKpxiUr2hR5XW4FStWoFKlSqhXr55cuao6X36nra2N69ev4/Dhw7h79y7EYjGGDx8OFxcXhUYT9vb26NatG4oUKcKVOTs7w8bGBidOnMCLFy9gbW2Nly9fyjXIIITkHlKpFPMmjcKju25cWQU7e6zdfYQSbCTf0dLWxuodBzCyb1e8eZn0Jo6H+xNMGz4Qa3YdguafARIJIblLrk4oE0IKtrevnmPS0H5ITEjgymYvX4+mrdvzGBUhWav/sDEI/RnC9Q8e9usnRjl3w94z12CmItlG8r7o6Gh89vbCoMEuCvM+f/4MAAqJ36ioKGhoaCQlGhYsUprMlSlatChevn4rV7Zi2VLs2L5VoVyV27dvoXp1xz8J5799FovFYvz48R3NWrRQup6hoaHCPrZs2oCF8+ela9/R0dFwd3fHsGHDFOZ9+fIFAFClSpV0HUN+IhKJ0L9/f/Tv3z/V5Xr16oVevXoplNesWRM1a9bMrvAIIVmEMYbVC2bivwunubISpcpg0/6TMDQy4jEyQrKPnr4BNrgex9Ae7fH5oycA4OGdm5g7cQQWb9iZYwPAEkLSL1f2oUwIId98vDF2YE/ExkRzZeNmzJcbuIGQ/EAgEGDczAVo17UnV/bD3xdjBnSn/uPysdceryCVSuVakcocOrAfxYoXR526dbmyu3fuoHcPJ3Tp6gQdHR0sWjAv22MMCQ5BocKFFcpv3riOmJgYVK3qkC37DQ4OBmMMhZXs+8yZMxAIBOjWrVu27JsQQvi2d/NaHHfdyU2bmhXFlkOnUcScBs8k+ZuxSSFsOXgKliVLcWXXL57FqnnTC9RgvITkFZRQJoTkOqE/QzBmQA+Eh4VyZQNGjEP/4WN5jIqQ7CMUCjFn5UbUb/K3xaeX5ztMGNIHcXGxPEZGsouHxysAwMcPH+TK//fwAa5dvYK58xZwrXEe/e8henTrjM5dumLv/gMYNWYcjh45jBfP3VNuNktZWlri44cPcjdxkZGRmDVjOgDlA/IBmR+Ur3DhwhAIBHjx4oVcuZeXF9auXYvevXvD0tIy6w6UEEJyidOHXbF19RJu2sDICJsPnEQJq9L8BUVIDjIrWgxbDp+BqdnfBygnD+6B67YNPEZFCFGGEsqEkFwlNiYa4wf3xnffr1xZx+59MGbaXP6CIiQHaGpqYsW2fajqWIsre/n0EeaMGwaJRMJjZCQ7vH71CkXMzLBv3x4ccN2H1x6vsGvndvR06ophI0ai34CBAIDn7s/QrXNHtGjVGtt37YFAIMDEyVNQxMwM06ZMztYY+w8chM+fvTFl0gS8fPECly9dQttWLRAZGQFNTU1UtKukdD3ZoHzJ/61etQIxMTEK5SuWLVFY39jYGO3bt8e1a9cwZcoUuLm5YfPmzahbty5KlCiBbdu2ZetxE0IIH25duYBls/4OQq2trYP1e46igp09j1ERkvNKliqDLYdOwyBZFy+bVyzE5TPHeYyKEJIS9aFMCMk1xGIxZowegncef1ul1W3UFDOXrVM58BQh+Ymurh7W7T2Kod3b4fOnpJarbtcuYcOSuZg4VzHxVtAlShi+/YpOe8Fs3L+6PDxe4Z+GjTDYZSg2bViHLZs3okSJkti0dTs6d+kKAAj48QNTJ09Eq9ZtsXPPXq7FsqGhIRYuWoK1a1bjwf37aPBP+gYpNS1iinIp+mVOjSypfezoEYweORx2lSph/cbNmDp5IkxNTaGlYpAcZYPyqaJqUL59+/Zh+PDh2LBhA1avXg1TU1P06tULixYtghH1IUoIyWc83J9g9vhh3BshIpEIy7bsQbVaddNYk5D8ydrWDmt3Hcaoft248XQWTBmDIuZFUbtBY36DI4QAoIQyISSXYIxh5dypuH/rP67MplIVrNi6D5qamjxGRkjOMjYphE0HTmJg55YIDgwAABzesw3FS5ZCr0H/8hxd7qGtIUIlS/4Ti9oaGR8kJiEhAZ7v36Njpy5o0rQpmjRtqnS5YsWL49ad+0rn9RswkEv4pteQocMwZKjiQHep+bsfBiaVAgIB3r97h06du6hcR9mgfBllamqKkydPQiwWIyYmhpLIhJB865uPNyYM6YuE+HiubPby9WjUog2PURHCP8c69bFw7VbMGD0EACARizFlWH/sOnEZNpUq8xwdIYQSyoSQXGHf1vU4fdiVmy5WoiQ27DsGfQND/oIihCdFi1lio+sJuDi1QXRUFABg9YIZsLAsgcYt2/IcXe5Qpog+3yGozfP9OyQkJMC+ct67Gfr29SsiIyNRRUX/yVlNQ0ODksmEkHxL2bghIybNRMcefXmMipDco2WHrggJCsTaRbMBANFRURgzoAf2n7uOYiVK8hwdIQUb9aFMCOHdlTMnsGXlIm7a0MgYm1xPwEzFq9CEFATWFSthxTZXrpsDxhhmjhmKt6+e8xwZySyPV68AAJUq5b1+Md++fQMAqFq1Gs+REEJI3hYbG4PxLvLjhnTq6QyXMZNUr0RIAdR3yEj0dRnBTf8KCcLo/k4I/x3GY1SEEEooE0J49fThPSyYOoab1tTSwro9R1DGOn39bxKSn9Vt2BSzlq3jpuPjYjF+cG/4J7v5JHlPcHAwqlStilKlS/MdSoYFBwWjgo0tKlepwncohBCSZ0kkEswa+y/evZIfN2TGkjU0bgghSoyfvQgt2nfmpr9+9sIElz6Ii4vlLyhCCjhKKBNCePPF+xOmDO8PcWIiV7Zw3TYagISQZDr1dMaQsZO56bBfPzF2YE/8TvZ6LMlbJk+dhv89cc+TSYPBQ4bghccb6oaCEELUxBjDmgUzcff6Fa7Mxq4yjRtCSCqEQiEWrNmK6rXrcWUe7k+wYPJoSKVSHiMjpOCihDIhhBdhob8wfnBvREVEcGUTZi9Cy/aqB3oipKAaPnEG2nXtyU1/++yFSUOd5QbwIYQQQkjud2jXFhzfv4ubLlrcEhtcadwQQtKiraODNbsOoVwFW67s+sWz2LFuOY9REVJwUUKZEJLjEuLjMWVYf/h/+8KVOTkPRt8hI3mMipDcSyAQYM6KDahR9x+u7NWzx1g8YwIYYzxGRgghhJD0unXlAtYvmctN6xsaYqPrcZgVLcZjVITkHUbGJtjgegKmZuZc2e6Nq3Hp9DEeoyKkYKKEMiEkRzHGsGTmRLx8+ogrq9OwCaYsWJ4nX/8mJKdoamlh9Y4DKJusf/HLp4/BddsGHqMiJHeiBy2EkNzm/euXmDPh78BiGpqaWL3jIMrb2PEYFSF5TzHLEli35wi0dXS5skXTxuHFk//xGBUhBQ8llAkhOWr/9o24dOooN12mfAUs37wXGhoaPEZFSN5gaGyM9XuPwaSwKVe2ecVCuF27xGNU2UgggEQioeQgyRDGGCQSCYRCquYSQnKH4MAfmDjEGfHJBhCbs2IDatVvyGNUhORdlapWx6J127hpcWIiJv/bD35ffXiMipCChWrahJAc43b1IjYtX8BNGxcqjPX7jsHQ2JjHqAjJWyytSmH1zoPQSDZwz5zxw/Hh7Wseo8omQk0kJIoR9usnJZVJujDGEBwcDIlEAm1tbb7DIYQQxMZEY4JLX4QEBXBlg0dNRPtuvXiMipC8r1nbjhgzfR43Hf47DOMG9UL47zAeoyKk4KAmgYSQHOH55hVmjx/OTWtqaWHNrkMoYVWav6AIyaOq1ayDWcvWYcHk0QCAuNgYTBzSBwcu3EIR86I8R5eF9IwhjhIjNDQUUZEREIlEfEdUwDEA2dM1EWOAQACIhAK1uz+StUyWSCTQ1dVF0aL56LNACMmTpFIp5k4YgQ9vPbiyZm06YMTkmTxGRUj+MWD4WPj6eOP8icMAgG8+3pgybAC2HDwFTS0tnqMjJH+jhDIhJNsFB/7ABJe+cq/5zV6+HtVq1uExKkLyto7d++Cr9yfs374RABAU8AOT/nXGjmMXoJOsT7m8TCAQghmYQhwbDrEkERCL+Q6pwGKMQSoRQyjSyJb+7uMSJdAUCWGqr/7Nn0AggJaWFrS1tVG0aFHq8oIQwrttq5fKdUtla18VC9Zupe8nQrKIQCDAjCVr8N3PF+6P7gMAnj9+gKUzJ2Luqk00Rg8h2YgSyoSQbEWv+RGSfUZNnYOvn71w98ZVAMDbl8+xcMpYLNm4M99UoAUCIaBXiO8wCjwmlUASGw2hrj4EwqxvKR4eHgcTXU2ULmMKkTB/XLuEkILt8pnj2LtlLTdtVrQY1u05DF09fR6jIiT/0dTSwqod+zGwSyt8++wFALhw8gjK2VSE89BRPEdHSP5Fj0YJIdmGMYb5k0fTa36EZBORSITFG3aggp09V/bfhdPYvXE1j1ERQgghBdurZ4+xaNo4blpbRxfr9hyGuUVxHqMiJP8yMjbBhr1HYVyoMFe2Yek8/O/OTR6jIiR/o4QyISTb7NuyDjcvn+emK1Z2wIJ12+g1P0KykJ6+AdbuPoLCRcy4su1rl+HG5XP8BUUIIYQUUN99v2HysP5ITEjgyhau3YqKlR34C4qQAqBk6bJYuc0VIo2kF/GlUilmjBmCr39aLRNCshZldQgh2eLezWvYunoJN21qVhRrdx+Crq4ej1ERkj8VsyyBNTsPQktbmyubN3EUPr1/y2NUhBBCSMESHRWJCS69EfbrJ1c2cvIsNG/XiceoCCk4atRtgKkLVnDTURERmDikLyLCf/MXFCH5FCWUCSFZ7ovXR8we9y8YYwCS+rVavWM/veZHSDaq4lgLc1Zs5Kbj42IxaagzwkJ/8RgVIYQQUjBIpVLMnTgSnz994MradumBwaMn8hgVIQWPk/MgODkP5qa/+Xhj5pghENPgzoRkKUooE0KyVGR4OCYOdUZ0VBRXNnPJGlRxrMVjVIQUDG27dMeAEX/7bPzh74sZo1yoAk0IIYRks90bV+HOf5e56SrVa2L28vX5ZpBcQvKSyfOXwbFOA2760V03bFw2n7+ACMmHKKFMCMkyEokEM8cMge+Xz1xZr0H/omOPvjxGRUjBMmrKbNRt1JSbfva/e9iwdC6PERFCCCH52+3/LmPHur+v2ZsVLYaV2/dDW0eHx6gIKbg0NTWxYts+WJYsxZUd3r0VF04e4TEqQvIXSigTQrLMlpWL8L+7t7jpGnX/wfhZi3iMiJCCRyQSYemm3ShZuixXdmTPdlw6fYzHqAghhJD86fMnT8ydMIKb1tLWxuqdB2BW1ILHqAghhQqbYu2ew9DTN+DKls6ciNfPn/IYFSH5ByWUCSFZ4uq5U9i//W//rcVLWGH51r3Q1NTkMSpCCiYjYxOs2XVQrgK9ZMYEvH/9kseoCCGEkPwlIvw3Jg3th5joZF29LV0LewdHHqMihMiUt7HDovXbuenEhARMHtYfgT/8eYyKkPyBEsqEkEzzfPMKi6aO5aZ1dPWwdvdhFCpsymNUhBRs5SpUxIK1W7nphPh4TP63H36FBPMYFSGEEJI/SCQSzBjtAr+vPlxZH5fh6ODUm8eoCCEpNW7ZFiMnz+Kmf4UEY/K//REfF8djVITkfZRQJoRkSujPEEwa6oz4+L8/yAvXboV1xUo8RkUIAYCmrdtj6Lgp3HRQwA9MHTEQiQkJPEZFCCGE5H2bVyzE43u3uela9Rth3MyFPEZECFFl8OiJaNmhCzft+eYVls+eDMYYj1ERkrdlSUJZLBbj1KlTGDNmDPr06cOVX7lyBYmJiVmxC0JILiQWizFj9BAEBfzgylzGTEKzth15jIoQkty/46ehUYs23PSrZ4+xesFMHiMiJP+jujEh+dvVc6dwYMcmbtqyZCks27IHGhoaPEZFCFFFIBBg7qpNqGBnz5VdOHkEpw+78hcUIXlcphPKERER+Oeff9C9e3ccOnQIR48e5eYdP34cJ0+ezOwuCCG51JaVi+D+6D43/U+zVhg+cQaPERFCUhIKhVi4bhtKl7Pmyk4d2ouzRw/wGBUh+RfVjQnJ3zzfeCh09bZm1yGYFCrMY1SEkLTo6uph1Y4DMDI24cpWzZ9Og/QRoqZMJ5RnzpyJiIgIuLu7IywsTG6ei4sLdu3aldldEEJyoVtXLsi1zChZuiwWrdsOoZB60iEktzEwNMLa3Yehb2jIla2cNw2eb17xFxQh+RTVjQnJv8JCf2HysH5yXb0tWLOFunojJI8oYVUaSzbtgkAgAACIExMxZfhAhAQF8hwZIXlPpjM/Z86cwb59++DoqDiSrZ2dHZ4/f57ZXRBCcpkvXh8xf/Joblrnz9NeQ2NjHqMihKSmVNnyWLLhbwU6IT4eU4YPwO+wUJ4jIyR/oboxIfmTRCLBrLFDEfjdnytzGT0Jzdt14jEqQkhG1WvUTG6Qvp/BgZg2chCNMUJIBmU6oRwSEgI7OztuWnajCgAaGhpIoA8lIflKdFQkJg/rj5joKK5szooNsLa1S2UtQkhu8E+zlhgydjI3HeDvh9nj/oVEIuExKkLyF6obE5I/7Vi7HE/u3+Gm6zVqhuGTqKs3QvKiQaMmoEmr9ty0h/sTrFs8h8eICMl7Mp1QLlasGF6+fMlNJ680P378GGXKlFFru3FxcVi/fj06d+6M9u3bY/78+QqvDabGx8cHM2bMQNeuXdGxY0eMHz8er169UisWQkgSxhgWTB6Nr5+9uLLeg4ehdaduPEZFCMmIoeOmol6jZtz0o7tu2LVhJY8REZK/ZFfdmBDCn7s3rmLP5jXctGXJUli8cSd19UZIHiUQCDB/zWa5MUaO79+FS6eP8RgVIXlLpn8BO3fujJEjR8Lb21uu/Nu3b5g0aRK6dOmS4W3GxMSgUaNG2LRpE5ycnODi4oJr166hevXqCAxMu2+by5cvw9bWFk+fPkWfPn3g4uKCoKAgVKtWDVu2bMlwPISQJAd3bMKtqxe5aYeadTBu5kIeIyKEZJRIJMKiDTtQrERJrmzXhlW4f+s6j1ERkn9kR92YEMIf3y+fMWfCcG5aW1sHK7fvh7FJIR6jIoRkloGhEVbvPAh9AwOubOmMifB848FjVITkHZlOKM+bNw/x8fGoWLEiatasCcYYmjRpgooVKwIAZszI+GtAa9aswbNnz3Dx4kU4OzujS5cuuHr1Kn7//o2pU6emuf6CBQtgZGSEy5cvw8nJCZ06dcKRI0dQrVo1zJ07N8PxEEKApw/vYdOKv8ljU7OiWL5lLzQ1NXmMihCiDpNChbFq+35oaWtzZXPGD4O/71f+giIkn8iOujEhhB+xMdGYPKw/oiMjubIZS9fA1r4Kj1ERQrJKmfIVsGDtNm46Pj4OU4b3pzFGCEmHTCeUTU1N8eTJE0yYMAFRUVEwNjaGv78/hg8fjgcPHsAw2Yjy6eXq6oo6derI9T9XqFAhdO3aFSdPnkR0dHSq68fExMDc3Bw6OjpcmUAgQOnSpREbGwvGWIZjIqQgC/zhjxmjXSCVSgEAIg0NrNi2D2ZFLXiOjBCiroqVHTBt4d+uLiIjwjF12ADExcXyGBUheV921I2T+/nzJwIDA9Wuz4aGhiI4ODhTMRBSEDDGsHj6BHz+6MmVdXMehA5OvXmMihCS1Zq0ageX0ZO46QB/P8wcM4TGGCEkDVnS6VOhQoWwcuVKeHp6IiwsDF5eXli7di0KFcr4a0ChoaHw8fFB1apVFeY5ODggLi4Ob968SXUbgwcPxsePH3H9+t/Xd9+8eYObN29i4MCBcn3ZEUJSlxAfj6kjBuJ36C+ubMLsRahWsw6PURFCskLnXv3QuVc/bvrj+zdYPnsyPXglJJOysm4sc/fuXdjb26NcuXKwt7dHqVKlcPz48XStm5iYiHnz5sHMzAxly5ZFpUqVYGBggNGjRyM8PFztmAjJz4677sK186e4aftqjpg8dymPERFCssuwidNRt1FTbvrJ/TvYvmYZfwERkgdo8B1ASgEBAQAAMzMzhXnm5uZyy6gyceJEFClSBD179oSlpSW0tLTw4cMHzJ49G9OnT0913eDgYISEhMiVyfrAk0gkOfaUSiKRgEmlYAJAKs2ZwR6Y9M8+pRJIc2SPJDdKeR2sXjAD71694Oa36tQNPfq7QCqlJ7b5mYBJwaSMvg8KgMnzl+Lju9dcf3EXTx6FvUMNdO3THwD9NpAk2X0dyLYtkUgAlv0P/vNaq6N3796hTZs2cHZ2xubNm6GpqYnVq1ejd+/eMDY2RuvWrVNdf/r06Vi7di327NmDwYMHAwBu3LiBDh06wM/PD+fPn8+JwyAkz3j57DHWLp7NTRcyLYKV21zluooihOQfIpEISzbuQr8OzfD9Txdwe7esRRXHWvinWUt+gyMkl8p0QvnYsbRHwezVq1e6t5eYmAgA0NBQDE1WlpCQkOo2Hj58iClTpqB69er4999/oaWlhTNnzmDJkiWwtrZG9+7dVa67detWLFiwQOm8qKioHGvFER2XCGl8NCQiARJzKO/PpFKI45NedRbQiMUFVvLr4PrlCzh92JWbV66CLabOWQRxXAxP0ZEck5AAiTgBktgY+j7I54QAFq/djEFOHRAR/hsAsHr+DJQrXx52lavSbwMBkP11BJaQALFAAxERGhDmwJtkUVFR2bbtrK4bA8DChQuhra2NDRs2QEtLCwAwZcoUHD58GDNnzkwzoXzx4kVUqFCBSyYDQIsWLdCqVStcvnwZjDF6g4+QP0KCAjFtxCBIxGIAgFAoxNJNu1G0mCXPkRFCspOxSSGs2r4fA7u0REJ8PABg7oThOHz5DoqXtOI5OkJyn0xnKnv3TrsPqYxUmo2MjAAor+jLymTLKCORSNC7d2+Ym5vjv//+45LQXbp0QWxsLAYOHIhGjRpxrZ1TGjlypELC2dvbG507d4aBgQGMjY3TfSyZwTQTIAyVQiQCNHVzZtAz9qfFqaauHgRCUY7sk+Q+suvA/7s/Vi6YyZXrGxpi1Y79MDJVfHuA5D+iPwkLIX0fFAhW5W2xeMMOjBvUC4wxJCYmYNaEkTh08RaMTUwA0G9DQZfddQRBggY0dDRgZGQMkTD7E5sGyUZ0z2pZXTeWSqW4cuUKmjVrBl1dXbl5bdu2xbJly+Dv748SJUqo3EbJkiXx/v17SCQSiER//36/fv2CpaUlJZMJ+SMxMREzRrvgV0gQVzZ62lzUqt+Qx6gIITnFplJlTFu4EoumjQMARIT/xrSRg7Dn1BV6Q4GQFDKdUH758qXcNGMM379/x6VLlxAdHY1JkyapWFM5KysraGtr48uXLwrzfHx8AAA2NjYq1/f19YWfnx+6du2q0Mq5VatWOH36NF68eKGyJYe5ubnKZLNIJJKrhGcnkUgEgVAIgVAAYQ7dwEuBP/sU5dg+Se4jBRAXF4fpo4ciNuZvS+T5qzajVFlr/gIjOUogSPr+AX0fFBj1m7TA/9m777AorjUM4O8WerNi7713Y+8aE7vGJPaKFREELCBKExSVoth715hYEhNjildj7xqNiRFbsCE2kA67c/9ABpZeFmaB9/c897n5zu7Z+YBx9+w3Z86Zajcf630T14sLef4MrvbW8N2yh58NlO9jhKTXVigUBVJQzs/xnLbHxsHBwYiIiEDt2rXTPFanTuLn8t27dzMtKC9fvhyff/45xowZAxsbG+jr62Pv3r24efMmvvnmmxzlQ1SUrfHxwI3LF8S4R9/+GDt1loQZEVFBG/TVaNy8ehE/HNwHALj75w34eizEfM/lEmdGpFvyXFBu3rx5mrYWLVqgf//+8PPzw6+//pruczJMSKlEr1698McffyAhIUGjKPzbb7+hTp06qFmzZob9k2acpF4HOWVbXnfXJirKBEHAcveFeHT/ntg2ctI09PhsgIRZEVFBmDTLHrdvXMW5//0KADh36jfsWL8KoydYSZwZUeGh7bHx+/fvAaQ/fk1qS3pORlq2bAkfHx/Y2Njghx9+gL6+Pj58+ABXV1d8+umnmfYt7PuLcB14ykzK8+N/vxzHro2B4mPVataGi88qCIIa3Ku2+OF+IsXbXLel+Of2n7j/z18AgIO7tqBpqzboO2gYP1coU9o6PwrD/iL5ujjv2LFj0axZMzg6Ouaon5OTEzp37gx3d3e4u7sDAPbu3Yvz589j9+7dGs/dsmULdu3ahdWrV6NJkyYoW7Ys+vTpg2+//RZTpkxB165dAQD//PMPVq1ahTp16uCTTz7Rzg9IVAQdPbAbP39/SIybtGgNm/mu0iVERAVGLpfD3W8dRn7eFSHPnwEA1q/0RqPGTdC2S0+JsyMq/HIzNk6aTa1Wp/1aktSW1YxrJycnLFu2DNu3b8eYMWMAAKdPn8bAgQNx48YNHDx4MMO+hX1/Ea4DT5lJOj+ePw2Gm6O12G5gaAhPvzUwUMoRHx0pYYYkGe4nUqwp8HGPkeEDEBWZuPSq14I5qFmzFqrXrMXPFcqQtsYdhWF/kXwtKL9+/Rpv3rzJcb8OHTpg7969mDlzJvbt2wdjY2MEBQVhxYoVGDVqlMZzHz16hNOnT2sMZvfs2QN7e3v06tULderUgb6+Pu7evYs+ffpg1apV6W74R0TAvb9uY/niBWJsUaIkvNdshd7H9XSJqOgrUbIUlq3ZiknD+0GVkAC1Wo3FDjbY/eP/ULZcBanTIyrUcjM2TlqKLb1+SW0ZLdcGJH5BWL58OQYNGiQWkwGga9eumD17Njw8PHDr1i00a9Ys3f6FfX8R7hFCmRHUKsTFxWKRow0+hIeL7fM9l6N+05YSZkZS434iVLNBYyxevgrzZiRuaBsdHYWFc2Zi26Hj0DMw4ucKpUtb447CsL9Iniurr1+/TtP24cMH3L59GwsXLsz1bOCvvvoKQ4cOxZ07d5CQkICGDRvCxMQkzfMmTZqEXr16oUmTJmJbmTJlsGPHDqxZswaPHj1CfHw8atSogZIlS+YqF6Li4EN4OOZOH4+4uFixzcN/PSpUynhNRiIqmpq0bANbJzesdHcGALwOfQUXu+lYs+u7AttLgKiw0vbY2NLSEhUrVsSdO3fSPHb79m0olUo0bdo0w/4vXrxAQkICqlWrluaxpLbg4OAMC8qFfX8R7hFCmVEDWLNiKf6+fUtsG/TVaAwcPirjTlQscD8RAoBe/QZj1KQr2LNlHQDgUdC/8F7oiEXeK/m5QunS1rijMOwvkueCctmyZTN8rE6dOggMDMzw8azo6emhRYsWmT6nRo0aqFGjRrqPmZqaahSaiSh9giDAY+4sPH2SvBnmhBm26Ni9t4RZEZGURkychuuXLuB/J44BAK6c+wObVy3HVLv5EmdGpNvyY2w8YsQIBAQE4L///kPVqlUBAFFRUTh8+DD69eunMUv47du3ePv2LSpXrgxDQ0NUqVIFJiYmuHDhQprXPXv2LACgQYMGOc6JqCj49dgRfLt3hxjXadAIc92XSZgREemaWQtccefmNdy6dhkAcOL7Q2jSvAW+mjBN4syIpJXngvLy5Wl3urSwsECtWrXQtWtXzmQiKgT2bduA34//IMYt27bDFLt5EmZERFKTyWRYvHw17t29jefBTwAAmwKWo3mbdvikUzdpkyPSYfkxNnZycsKRI0cwZMgQ+Pr6wtjYGO7u7oiLi0tzPF9fXyxZsgRnzpxBp06dYGhoCFdXVzg6OmLcuHGYOnUq9PX18d1332HHjh2YMmUKatWqleufl6iwevIwCEsW2ImxsYkplq3dBkNDIwmzIiJdo6enB+81WzDy8254/zZxqakAbw80bvkJmrRoLXF2RNLJc0HZwcFBG3kQkURuX78C/yWLxLhUmbJwW861xokIMLOwwNI1WzBp2OeIj4+DIAhwtpmCfcdPcz1logzkx9i4VKlSuHjxIry8vDBnzhwkJCSgdevWuHLlCmrWrKnx3NKlS6NWrVowMkouijk4OKBdu3ZYt24d7OzsxOXgDh06hMGDB2s9XyJdFxMTjXkzJiAyxSZELsv8Ua1mbQmzIiJdVa5CJSxZtRHWY76AIAiIj4/DgpmTsOenU7AowaVVqXhixYioGHv/7i3mzZwIVUICAEAul8Nr9SaULpvx5j5EVLw0aNIMsxe4YIW7CwDg3ZvXWGA9Gev3HeWFJ6ICVKZMGfj6+mb5PDs7O9jZ2aVp79SpEzp16pQfqREVOssXz8f9v/8S4+FjJqLPgKESZkREuq5d5+6Yajcf6329AQAvngVjkd10+G3ZC7lcLnF2RAUvx98Ek9ZaywkOXol0j1qtxiK76Qh5/kxsm27vhFbtOiI+OlLCzIhI1wz5ajRuXb+GX48dAQDcuHwB6329YT3XRdrEiHQAx8ZEhcux7/bjyP5dYly/URPYOrtLmBERFRaTZtnj5tWLuPjH/wAAZ0/+gt0bAzF2mo3EmREVvBwXlDt37pzjgwiCkOM+RJS/dm5YjXP/+1WMO3brhfEzbAHw3ysRaZLJZHD29sO/d+/gycMgAMC2NX5o3rodOvXg5p1UvHFsTFR4BN27Cy8nezE2NTOHp99a6BsYSJgVERUWcrkcHn7rMLpfD4S8fA4ACPTxQPM27dC0VVuJsyMqWDkuKB8/fjw/8iCiAvTntctYu9xTjMtVrAR3//WQy+VQq1USZkZEusrENHGzonGDeiM2NgYAsMhuGvb8dBoVKlWWODsi6XBsTFQ4REVGYN70CYiNiRbbXFcEomLlKhJmRUSFTYlSpeG2YhVmjvsKKpUKKpUKC2ZNxt6fTnM9ZSpWclxQ7tu3b37kQUQFJDzsPRbMmgyVKrFwrFAo4L16M0qULCVxZkSk6+o0aIR5Hj5wn5t4W1/Y+3dYMHMiNh38EXp6ehJnRyQNjo2JdJ8gCFiyYA4eP7gvto2ZYo2ufT7jUm9ElGNNW7bGNPsFWOOTOEnr5bOncHOwxspNuyGTySTOjqhgcOVwomJEEAS4Oc7Cy2dPxbbpDs5o1voTCbMiosJk4Jej0G/Y12J8+8ZVcXMSIiIiXXR43w78fPRbMW7Wqi1mch8AIsqDsVNnoUPXnmJ8+tfj2Ld1vYQZERWsPBeUBUFAYGAgmjRpAiMjIyiVyjT/IyLd8M2OzTh14kcxbtelO8ZxAwEiygGZTIYFnstRo3ZdsW37Wn9c+OOkhFkR6Q6OjYl0y/2//8Jy1wVibFGyFLzXbOGdNUSUJ3K5HO5+61C2XAWxLcDbFX/dui5hVkQFJ88FZX9/fzg4OKB169aIiYnB9OnT0bdvXyiVSvTt2xe2trZaSJOI8urv27fgtyR5JkbpsuXg4Ze4bjIRUU4YGZvAe81WGBgYim2L7Kbj9asQCbMi0g0cGxPpjuioSMyfORFxsbFim6f/epSrUEnCrIioqChZugyWrNoofqdOiI/H/BkT8SEsTOLMiPJfnitJW7duxdq1a7Ft2zYAwOrVq3Hs2DE8fPgQUVFRGDx4cF4PQUR5FBnxAQusJyE+Lg5A4gxDz4ANKFWmrMSZEVFhVad+Q9gv9hLjt69D4WI7TVyfnai44tiYSHcsWzRXY93k8TNs0aFbLwkzIqKiplW7jphiO0+Mnz/9Dx7zbCAIgoRZEeW/PBeU79+/rzEwTvoiWbFiRaxevRpz587N6yGIKA8EQYCXkz2CHz8U2ybNskfbjl0kzIqIioKhI8ehV79BYnz53GlsX+svXUJEOoBjYyLd8NOhb/DDwX1i3LRlG0ybsyCTHkREuTPReg7aduwqxr8f/wEHd26RMCOi/JfngnJsbCxKlSoFADAxMcHLly/Fx2rUqIFbt27l9RBElAfff7NHYxOSlp90gNVsfpkloryTyWRYuDQAlapUE9vW+3rjxpWLEmZFJC2OjYmk9+RhELyc7cXYzNwCXqs3c91kIsoXCoUCngEbULqspdjm67kQf9/mZz4VXVpdPLVRo0bYv3+/GB8+fBilS5fW5iGIKAce/Ps3li1Kvv3GomQpLFm1kRsCEZHWmJmbwytwMxQf31fUajWcZ03G+3dvJc6MSHocGxMVvNiYGCyYOQnRUZFi2+IVgahQuYqEWRFRUVe6rCU8/TdAJpMBAOLj4jB/5kREfAiXODOi/KHVgvLkyZPh6OiILl26oFevXhg7dizGjBmjzUMQUTZFR0dhwcxJiI2JFtvcVq6FZfmKEmZFREVR4+atMGveIjEOefEc7o6zuHYcFXscGxMVvACvxbh397YYfzXOCt0/7SdhRkRUXLTt1BWTZiXfHfH0ySMsWWDHMTEVSXkuKN+4cUP8bysrK/j6+iIiIgJv377FokWL4OrqmtdDEFEurHRzwoN//xHj0VYz0blnHwkzIqKibNTkGejYvbcYn/71OPZt2yBhRkTS4NiYSDonfz6GAzs2iXG9hk0w28lNwoyIqLiZYjsPrdp1FONffjiMw/t2SJgRUf7Ic0G5efPmGrGtrS2uX7+O69evY/HixVynikgCJ77/Dof37RTjRs1bwnqui4QZEVFRJ5fL4ea7FmXLVRDbArwW4+/bN6VLikgCHBsTSePF02C4O84SYyNjE3iv2QIDQ0MJsyKi4kahUGDJqo0oUSp5iavlrgtw/++/JMyKSPvyXFD28fFBSEiINnIhIi0IfvwQSxbYibGpuTm8V2+Bnr6+hFkRUXFQslRpLFm1EXJ54vAiIT4e82dO4tpxVKxwbExU8OLj4+FsY4UP4WFi24IlK1CtZm0JsyKi4qpsuQrw8FsnxnGxsZg/c6LG2u5EhV2eC8rz5s1D5cqVMWTIEBw7dgwqlUobeRFRLiR+UE1CZESE2OayNACVqlaTMCsiKk5atesIq9mOYvz0ySN4Odlz7TgqNjg2Jip4G/yW4ta1y2Lc/4sR6Df0KwkzIqLirkO3Xhg/w1aMHz+4jxVuTtIlRKRleS4o37t3D/b29rh06RIGDBiAqlWrwsnJCUFBQdrIj4hyYNVSV/xz55YYfzF6Inr1GyRhRkRUHE2a5YBW7TqJ8Ynvv8PRA7slzIio4HBsTFSwLp75H7av9RfjarXqYJ77MukSIiL6aNqcBWjSorUYH9m/C78cOyxhRkTak+eCct26dbF06VIEBwfj+++/R5s2bbB8+XLUqVMH3bp1w65du7SRJxFl4dQvP2Hf1uQNsOo0aIQ5Lp4SZkRExVXi2nEbNNaO81k8Hw/+/VvCrIgKBsfGRAXn9asQuNhOE++C0TcwwLI1W2FsYipxZkREgJ6eHpas2gQTMzOxbckCOzwP/k/CrIi0I88F5SQKhQIDBgzAkSNH8PTpU/j4+CAoKAhjx47V1iGIKAMhL56l3YQkkJuQEJF0yparAHfftWIcGxONBTMnITo6SsKsiAoOx8ZE+UutVmOR3TS8fR0qts1xWYI6DRpJmBURkaZKVathobefGEeEh8Np1mTEx8dLmBVR3mmtoJwkJiYGJ0+exIkTJ/D8+XMoFAptH4KIUlCpVHCxnYaw9+/Etvmey1Gjdl0JsyIiAjp2740xU6zF+MG//8DP00XCjIgKHsfGRPlj+1p/XDp7Wox7fjYAX4yeIGFGRETp6zNgKAZ9NVqMb9+4ik3+PhJmRJR3WisoX79+HdbW1qhQoQJGjhyJx48fw9PTE8HBwdo6BBGlY/taf1y7eE6MPxs8HP2HfS1hRkREyWY6LkSj5i3F+Lvd23Dy52MSZkRUMDg2Jso/t65ewnpfbzGuWLkqXJatgkwmkzArIqKMObp6o3qtOmK8dY0vLp/7Q8KMiPImzwXl1atXo0WLFmjVqhW2bNmCfv364eTJk7h//z6cnJxQoUIFbeRJROm4ff0KNvgtFeNKVatjvudyCTMiItKkp6+fuHacafJ6lh5zbfDy+VMJsyLKPxwbE+WvD+HhcJ49BSqVCgCgUCrhFbgZZhYWEmdGRJSxpGUp9fT1AQCCIMDFdhrevX0jcWZEuZPngrKNjQ1kMhkCAwPx4sUL7N69G927d+fVYaJ89iE8HM42moPpJas2wtTMXOLMiIg0ValWA/M9V4pxeNh7uNhOE9+/iIoSjo2J8o8gCPB2noMXT5Nn+s90XIgmLVpLmBURUfbUbdgYtk7uYvz61Uu4OViLG4sSFSZ5Lihfu3YN169fx8yZM1GiRAktpEREWREEAUsXOuBZ8BOxbdqcBRxME5HO+nzIcPRLsRzP9UvnsTXQV8KMiPIHx8ZE+efHQwdw4vtDYty2Y1eNtfqJiHTdV+Ot0KVXXzE+8/sJ7N++UcKMiHInzwXlli1bZv0kItKqHw8dwM9HvxXj1u07Y9w0GwkzIiLK2jz3ZahSvaYYbwrwwa2rlyTMiEj7ODYmyh/Bjx9imctcMbYoWQrufmshl2t9n3kionwjk8mwaPlqlC2XvARWgNdi/HPnTwmzIso5fvoSFTLpDaY9/Ndx13gi0nkmpmZYsmojFEolAEClUsF59hR8CAuTODMiItJl8fHxcLaZgqjICLFtcaqCDBFRYVGyVGl4BmwQl8OKj4uD06zJGu9xRLqOBWWiQiTxg8ZK44Nmkc8qWJavKGFWRETZ16hZS1g7LhTjF0+DscTJjmvHERFRhjb4LcVft66L8fAxk9C192cSZkRElDet23fCROs5YvzkYRBWuC6QMCOinGFBmagQWbfSC3f/vCHGw8dMQrc+n0uYERFRzo2eYo1POncT41+PHcH33+yRLiEiItJZV86fwfa1/mJcs0492C50z7gDEVEhMcV2Hpq1aivGR7/ZgxPffydhRkTZx4IyUSFx6ewp7Fi/Soxr1WvAwTQRFUpyuRxuK9eiRKnSYpvP4vl4/OC+hFkREZGuef/uLRbZTRPvYtE3MIDX6s0wNDSSODMiorxTKpVYsmoTzMwtxLYlTnPw9L/H0iVFlE1aKSgnJCTg22+/xaxZszBy5Eix/aeffkJ8fLw2DkFUrL178xouttPF2MDAEF6rN3EwTUSFVtly5eG2cq0Yx0RHwcl6MuJiYyXMikg7ODYmyjtBEOA5zxavXr4Q22YvcEOdBo0kzIqISLsqVK6ChUv9xTjywwc421hxvEA6T5nXFwgPD8enn36KixcvokSJEnj//j327t0LADhw4ADev3+vMZDOLpVKhV27duHUqVNISEhA69atYWVlBRMTkyz7jho1CpGRkRk+fvDgQejp6eU4JyIpCIIAVwdrvAkNEdtsF3qgdr2GEmZFRJR3nXr0xoiJU7Fv6wYAwL27txG4zB1zFi2RODOi3MuvsTFRcXN43w7878QxMe7YvTe+Gm8lYUZERPmjV79BGDpyHA7t3QEAuHPjGtb7emPWvEUSZ0aUsTzPUHZyckJ4eDiuXr2Kd+/eaTw2adIkbNq0KcevGRsbiz59+sDZ2RmtW7dGz549sWXLFrRp0wZv3rzJsv/IkSMxfvx4jf/1798fR48eRVBQEIvJVKgc2L4JZ0/+IsZd+3yO4WMmSpgREZH22Mx3Rd2GjcV4z5Z1OPe/XyXMiChv8mNsTFTcPLp/DyvcnMW4dFlLuK4IhEwmkzArIqL8M2fREtSsU0+Md6wLwKWzp6RLiCgLeZ6hfOjQIRw5cgStWrVK81jDhg1x7dq1HL9mQEAATp48iWvXrqFly5YAgH79+qF27dpYsGABNm7cmGn/fv36pWnz8/MDAFhZ8ao2FR73//4LAd6LxbhsuQpwWRbAwTQRFRlJ62GO7t8DMdFRAIDF9jOx/+czKGNZTuLsiHIuP8bGRMVJXGwsnGysEBsTLba5rlyDUmXKSpgVEVH+MjIyhlfgFowb2AuxsTEQBAEuttNx4MQZlCxdRur0iNLI8wzl0NBQNGyYfOt9ykKXUqlEXFxcjl9z69ataN26tVhMBgBLS0sMHjwYe/bsQXR0dCa9M35NQ0NDjB07Nsd9iaQQHR2FBdaTxPVEZTIZPPzXo2SKTayIiIqCGrXrwmGxlxi/e/Mai+1nQK1WS5gVUe7kx9iYqDgJXOaOf+/eEeNRk2egQ9eeEmZERFQw6tRvCDsXTzF+ExoCN8dZ4sakRLokzwXlChUq4MaNG2KcctB88eJF1KhRI0evFxYWhnv37mkUk5O0atUKUVFRuH37do5e8/Lly7hz5w6++OILlCxZMkd9iaTi57EQj4L+FePxM2zRpkNnCTMiIso/g78eg56fDxTji3/8D7s3rZEwI6Lc0fbYmKg4OX/qN+zZsk6M6zVsAuu5LhJmRERUsL4YPQFd+3wuxmd+P4GDu7ZKmBFR+vK85MXgwYMxY8YMHD58GLVr1xbbnzx5Ant7ewwZMiRHr/fs2TMAQLlyaW9zLV++PADg+fPnOXrNLVu2AACmTJmS5XNfvXqF0NBQjbagoCAAiRsFqlSqHB07t1QqFQS1GoIMUKvzXPfPFkH98ZhqFTgnTFonfz6G7/ZsF+PGzVthymxHqNX5f/7xPCAAkAlqCGqB5wEV6HuCk9dK/HXzOl4+fwoAWLPcEy0/6YCGTZvn85EpK/l9HiS9tkqlAoT8X9YpP8dz2h4bExUXb0JfYbH9TDE2MDTCktWboG9gIGFWREQFSyaTwWVZAO7euoHQkBcAAH9PF7Rq1wG16jaQODuiZHkuKC9evBg///wzGjRogObNm0MQBHTv3h2XLl1CjRo1sGDBghy9XtJtgEpl2tSSNtOL/bgEQHZERUVh//79qF+/Pjp3znp259q1a+Hm5pbuYxEREQgLC8v2sfMiMiYe6thIqBQyxOf9z5QtglqNhNjE5URk8oIpYlNaIS+ew3OerRgbm5hi8TI/CAlxiE/I/9tkeR4QACAuDqqEOKiio3geFHMF+Z5gpK8HVx9/zBj7JdRqNRLi4+E8ywrbvjsGExPTfD02ZS6/zwMhLg4JMiXCw5WQF8A+AREREfn22toeGyd58uQJnJ2dcerUKSQkJKB169bw9PRE8+bNs+xbuXJlvH//Pt3H5HI53rx5w02rSVKCIMDNwRpvXydP7HFw9UKN2nUlzIqISBolS5WGu+9azBg9FIIgIDY2Bk6zrLDz6G8wMDSUOj0iAFooKJcuXRqXLl2Ct7c3fvjhB1hYWODp06eYNm0aXFxcYGZmlqPXMzVN/MIYFRWV5rGkwX9OXvPbb79FeHg4XF1ds/X8GTNmYPjw4RptQUFBGDx4MExNTWFhYZHtY+eFoBcH+Vs1FApAz6hgBvjCx9mvekbGkMkVBXJM0qRSqeDh5IAP4ckXLuZ7Lkf1ArwSyfOAAEChrw8AkPM8KPYK+j2hVceumGzjgI3+PgCAp/89hv9SD7iuCMz3Y1PG8vs8kMUpoTRUwtzcAgp5/heUk8ab+UHbY2MAePHiBdq3b48GDRrgt99+g4mJCVxcXNCpUyecP38eTZs2zbT/vXv30qy/+M8//6BNmzbo06cPi8kkuf3bN+Lcqd/EuEff/hjyNfe+IaLiq22nrhg7dRZ2rF8FAAj65y5WLXWFo+tSiTMjSqSVqa8lS5aEj48PfHx88vxa1apVg76+Ph4/fpzmsaS2OnXqZPv1tmzZAgMDg2xvxmdpaQlLS8t0H1MoFFAoCqawolAoIJPLIZPLIC+gYo4a+HhMRYEdkzRtDfTD9Uvnxbjf0K/Qb+hXBZoDzwMCAJks8f0HPA+KPSneEybNcsCV82dw4/IFAMCP3x1A+y498dngLwrk+JRWfp8HSa+tUCgKpKCc3+M5bY6NAcDT0xNv377Fvn37xHHqpk2bcOrUKTg4OOCXX37JtL+JiUmatgMHDgAArKystJIjUW7d//svBHgtFmPL8hWwcFmAxvrjRETF0XR7J1w+9wf+vn0TALB/20a079ITnXr0ljYxImhhUz4fHx+EhIRoIxcAictadO3aFWfOnEmzvt3JkydRvXr1bBeUg4KCcObMGQwdOhSlS5fWWo5E+eHPa5ex0X+ZGFeuVgPzPLTzRZSIqDBRKpXw8N8AM/Pku4K8nefg6X+PpUuKKJu0PTYGgIMHD6JLly4akx709PQwePBg/P7773jz5k2OXi8+Ph67du1CuXLlMGjQIK3mSpQT0dFRWGA9CfEflz2UyWTw8N8AixLcSJ2ISE9fH0tWbYShkbHY5uZojTehryTMiihRngvK8+bNQ+XKlTFkyBAcO3ZMK5uczJs3Dy9evICfn5/YduzYMZw+fTrNunN79+7F4MGD8ffff6d5na1bt0IQhGxtxkckpQ/h4XC2mSL++1EolfBavQkmpjm/LZaIqCioUKkyXJYFiHFkRAScbawQHx8vYVZEWdP22Pj58+cIDQ1FgwZpl79q2LAh1Go1bt++naPXPHbsGEJCQjB+/Hgud0GS8vN0waOgf8V4wgw7tG7fScKMiIh0S7WateHo6i3Gb1+HwtV+JtRqbp1O0srzkhf37t3D1q1bsXPnThw5cgQVK1bEuHHjMHHiRI2drXOiZ8+e2LBhA+zs7HDgwAEYGxvj0qVLcHZ2TlMcvnv3Lo4ePQoHBweNdpVKhZ07d6Ju3bro1q1bbn88onwnCAK8nefg+dP/xLYZDs5o1KylhFkREUmv5+cDMXTkOBzauwMAcOfGNWz0W4aZcxdKnBnllCAIUAuAWhAgpPp/tSAgOl6FEgW0Z0R+0/bY+PXr1wASl9FILakt6TnZtXXrVshksmwtd/Hq1SuEhoZqtAUFBQFIHG9rYzJJdqhUKghqNQQZoFZnf06MoP7YT60Cv3rrllO//ITvdm8T40bNWsJqtgPU6oI5pwCeH5QxmaCGoBZ4blAaUrxvDBg+AudP/Ybfj/8AADh/+nfs27YeIyZMLaAMKLu0dX4kvYZKpQKE/F8CKjfjuTwXlOvWrYulS5diyZIl+Omnn7BlyxYsX74c3t7e6Nq1KyZNmoQxY8bk+HWtrKzw9ddf4+rVq0hISEDz5s1RtmzZNM8bNWoUWrdujYYNG2q0R0dHIzAwEDVq1Mj1z0ZUEH74dh9OfH9IjNt2TFx8n4iIgDmLluDG5QviDLZta/3QpmNXtGrfSSxSCoIANQADpRzyFGtuRsYmIDZBDbVagBrpFTIT+1qaG8JIL3lN3dcRsQj9ECs+VxAAlSBoHk8Aqpc2hqV58k7br8JjcPdFeJrXTx3XsjRFgwrmYr+3kXH47e8QqNUCBCQeL3XOagGoUcYEPeonL3kQHa/CljOP0hRnU/ZRCwIqWBhiQkfN8ZDnsbuIiVen2ycpZyN9BbyGNtHot+THf/Df22gI0Dxe6uNvGd8axvrJw8zlJ+7h4sO3Gf6dG1U0h/PnBbcBbX7Kr7FxeuvJJrWl3nAvMy9evMDx48fRo0cP1KpVK8vnr127Fm5ubuk+FhERgbCwsHQf07bImHj8ducpZHLAUKmATAbIkPg7SPnfhko52tdIXi5HUKtx7fEbxMvCIZfLkvsAkMs0+zepYAp5ijW8g9/FIDJO9fFx2cfnJ/dPaq9goQ99RXKROzwmAVFxKshkH/sgVZ+P7cZ6cihT9EtQC1CpBY2fR56iT1ESGvISHnNni7GxsQkWL/ODkBCH+IS4AstDUKuREBsNIHEtdyJRXBxUCXFQRUfx3CANUr1vOC7yxO0bV/Hq5QsAwGpvdzRv0Qq16xWN8VNRoa3zQ4iLQ4JMifBwpcb3m/wSERGR4z5a2ZQPSNzcZMCAARgwYABCQkKwc+dOBAQEYOzYsbkaNAOAmZkZunfvnulzGjRokO4tgKamphg8eHCujktUUJ48DILPonlibFGyFNz91kLOQQtRgUoqxqT8wh6boEKCKuNiW1Jc2lQfeikKAm8j4/A+Ki5Nn5SFT7VaQLXSxihhrC/2CwmPwaPQyDRFxNRFzNqWpqhRJnmDrVcfYnDhwZtM+6gFAbUtTdGuZvJ+AmHR8fjmSnCmhU+1AFQpaYiBDUuI/dSCAK8f/04sKH4sgKZ33JLG+pj3WX2N37Pr93/hTURchoVP4WPf7RPbin2MjIxR2yoAqtAYQCYHZDIsvSUHbl1I83dcN7olyqUo8O44/xi//Z31GnOuAxuiaeXkn/H0vVDsufRfxh0+mt6tFno3TD7evyERWPV7UJb9hrSolKagvP9ycJb9OtQqrVFQTlCpcfKfrH++WmXTbsj2z8sPiIrLfCaCqUHaYeL76Hi8icy62JP6LsysimE5qIcWGtoaGyfNQk6vcPv+/XsAQKlSpbL9ejt27IBKpcr2knAzZszA8OHDNdqCgoIwePBgmJqawsLCIoOe2iXoxeHbP0OhUmd+spQy0UOXhhWT+6lV2HsrCC/Csz5v91m10Xg/33cyGDeCsy6YB3zVFJVMjcT48NXH+PmvrNfRdvqsHlpWS17e7OdbL7DjQsbvPUkF6Qkdq+GzxuXF9mtP3iHg9wdpCuzypP/+WPju1aAshreuLPZ79j4ay37+N7l4LdMstuNjQbthRXOMbV9V7BeboIbXT/+IBW/IPh4r5fFlgKWpASZ2qq7xM2z84xGi4hJw5dwf0Os4AaWhBgQBLdu2x4mXhpCFvIK+Qo4JHatp9Pvp9ku8jojTLLank2//puU1LhBee/IOz97HpPl9JH05l0GAEB+HtrXNUdIo+f388ZsoPH0bleaCRdL1hqTXql7aBKVNkz/L30XG4dn7GI3cUl5ESHq9ksZ6KJliDBAbr8LryDjIIfv4+0z8m8lk0HgtfaUcJinem9WCgJh4lfjcNBcxUvw35YxCP/HvIzcyhowbVFMKwse7KPQK+NwobWQCd791mD5yCARBQHx8HBbPtcXO73+FoaFR1i9ABUJb54csTgmloRLm5hYFsmG1qalpjvtoraCcJCYmBidPnsSJEyfw/PnzfN9Fm6iwio+Lg7PNFERHRYptrisCUbZcBQmzIm1RqQWNAlnqW73VQmIhztxQT+MD4n1UHKLiVB9nJ2Z8m3jlkkYaXyhCwmPw/H10toqKFUskDzhehsXgZvD7NMep+l8Y1Kp4PLz+DIAcdcubahTcXkfE4qfbL6BWAwLSFgST4lplTdE3xZfe6HgV1pwMyrKIWd7CENO7ac6cW3TkDiLjVOn+XIKQmIehngJ+XzXX6Lf46F94+Doi06IwAOyc2Bamhsm/01W/B+HCg6w3ulr+RVPUskz+AD7253McufE8y34OfeqiQ+0yYnwz+D02nH6YZb/R7apqFJSfv4/BjvNPsuzXu2E5jYJydJwKx++8zLJfk0rmGgVlGYDr/73Psp+lmUGatpDwGISEx2bZVxAEjS+/hsamkCmyvg0rdVEyu1+gU9ensvu9W52qY3avBaaeTZrt46Xql9lshaQihlwm05hxmcTCSA96Cnnyc1IUIpIKJSlnGCepWsoYSrlcfI5cnlhQSjnzUy5L+7uoX94MiqQ+qQorcpmsyCx3kZ68jo0rV66MEiVK4N69e2keS2pr1KhRtl9v69atKFu2bLYnXlhaWmpsBpiSQqEosLG+QqHI1kzsxHM+OSc1gOxer1AqlBr/XgRk7x+nQqHQOGZ2jyeXyzX6ZfVmkHQHg1ym2U8lyLK8QAQAUfFqjX4Jahmev4/Jsl8pUwPNPAH89fxDlv2qlTZO0+/io7cIj04AyjWGabnGYvu/McC/fycurWKolGNS55oa/c4EvcH9kKxnT33aqDxMDJKPeTboLc7cz3pJmBoVSqK0efJn68WH7/DttadZ9pvdqw661k2+i/bm03Cs+d+DLPt93bYKvmxdRYwfvo6Ay9G/suzXvX5ZzOqRvEH92w+xmLLrWpb9GlU0h8fgxhptY7ZcgkotiJ8lqS9AyGRAaRN9LB/eTKOf2/d/4dWH2OTnQ/P9PKnNe2gTjRn4O84/xv1XERqfT+kV6qd0rYkypsljiN//DsHN4PfpXrRILsDL8HmT8qhWOvlvePtZGK48epvqDoO0FwhaViuJuuWSL+w8fx+NS4/eokbwB0CdgCe3X0Ehl6e5iFG9jAnqlU/uFxGTgKtP3qa4EJB+niWN9VAnxfFUagF3noVl+PtM/L0ABnoKVC2VvDEbADx7Fy2OmVL+7VJ/xpYy0dfoFxWXAEHQ/Fun/pvwYkTG1EiceSqTK9K8x+W3Nh26YMIMO2xd4wsAeHT/HlZ5uWG+5/ICzYMypq3zI+k1FApFgRSUczOe01pB+fr169i6dSv27NmD9+/fo1atWvD09MSECRO0dQiiImXNck/8ffumGH81zgpdevVN97np3c4sCIChnlzjQz4iJgGxCar0i22C8PGWbaCChSEMUszeePUhJnHWoFqASq1CXEwM5PrxgEyePANRnXibdsoB3vP30bj38sPHLzgZr49Zv7yZxsApJDwGp++FZlr4FASgbnlTdK6TPEgPi47HzvOP0y/OChDzrFbaGKPaac5sWXjkTrq/k5RxSWM9LB6o+aXc6dBtvPoQm3ER8+OM031Wn2j8LTyO3cWfT7Oe1bR2VEuUt0ieEbPvcjB+vZv1rKZFAxqieZUSYnwu6DV2X8x6RuXULjU1CsoPQyOw8Y+0RcxeTxILqb8JiV+mBjevqFFQDouOz1bRNDI2QaOgrFYLOJ+NIm1EbEKatsdvotJtT8lYP+0HYVRcAiJjs1GMTFUCyO7HdpoiXzZ7pi5iZvdWptT9FNnul7oYma1u6RZpDZTy5C9j8o9fXuSJP7tM/MKkn+a1KpYwgoFSkW7xUiaTiQVHtQAoUuRXt5wZ9BVy3LlxBW9CQxKTEtRo2KQ5qtWoKeZhqKdZxWxSyQL6qYqmKQuhSTPWyqeY1QwATSuXwMSO8nRzTBmn/AIKAPXLm2P+Z/XTfJFL/fOWTvXFrnJJI/h+2QwyGT7+DtL/vRooNc9vY30F9lh98vGWeM3jZPUFcM2o3K3TP6tHrVwNjgc0q5jp4y/Dsi5qFTbaGhvLZDIMGjQI3377LcLDw2Funji7Xa1W44cffkCHDh1Qvnz5LF4l0ZkzZ3D//n04ODhAXz/tv1FdN/fT+pDLE5djSRoLJV2gTbpQmHKGcZJxbSogXqYHyOTiRVAhxdgh6bM99T+bTxuVQ8uqJZKXokl1LEEA1BBgZqj5lap5lRIw1ldCSLXUTtLz8bGtTKoLb1VKGqN7vbLJ+QEaY5ykPMules+yMNJDs8oW4vMhQLwonfK4lmaa/fQUMlQtZSweSyPXFL8ni3Qu+JgZKsXnpf4Zk9rS+wKcEJ/55ziQ/vtXNq4lAEj7WZr6sy/DYyJ3/VL/iNnOM/XxstctTb+83NwRHadKM6ZITZnO3/DVh1i8yMZ7duq/45M3Ubj7PDzLfuMTNH8bQa8icC4o67HjJzVKaRSUH7yKwLE/X2TZz8JIT+Pz/L+3Udh14UnyWFid/vh6QLMKGgXl0IjYbN2l1LZGKcxPcQdXXIIabj/czbJftdLGaSZNOB++jfCYzP9NGerJsdeqnUab2/d3cf9V1hdoto5vrXE3XcBv9xMnW8ggjj1SF75lMhkW9mugMdni6M1n+PnOS43idVLhG7Lkcc7odtXQslpJsd/Vx2/x3fVnaWb8p74Y0a2eJTrVSZ6k8eRNJA5cCU5zsST1RZMGFczQs0E5sV9ETAL2XflPzCdNgf7j8S3N9NGluubdX4euJ35nyihPmQww0lOge33NC7QXH7xBdLxKHGcmjU1TX4xoW6OU+PNMsZuH87f/xeP/ngEQ8MMfV1H56C9o0fqTNBcDqpcx1hg/vomIRURsAtLcYZLq4oKFkZ5Gv9h4FWIT1KkuHKW9GCGXywpkeQbSDXkuKK9evRpbt27FzZs3YWhoiGHDhmHSpEno1q0br2bpqKRBY+orjjHxKkTHqqCUxUOAKs06jkkzEcuYGmgM1l9/iEVYTLzm2o8pinxJRcOaZUxgnmJA+vx9NJ68icp49ubH/25QwRxVUlyNff4+GpcfvU2+3frjIDll4VMQgAYVzdEqxQfS6w+xOHTjWapbtNOu/Vjb0hSDmlfS+L2sOHEv00Jk0hqVtr3ravyu53/3JyJiEzRncaoFxMbGIkzvE1S2bgchPgZGp30x28lN7Lfg0G0EvYoQj5ORHRPbwMww+Xe6/vSDbBXrlg1rolHg/eVOCA7deJZlvzm966JTneQvP389D8e6U1nPwhj5SVWN470Kj8X+K1nf3h2TYKlRUI6NV+F/90Iz6ZHUL23x8O/n4VkOuqPi0n65fh8Vj7fZub07VQEs+8XB3BX5cjJTUaMfctkvl8XPtMXIj/+PFLMnUw3y5DJZuoXhsmYGMDFQpH0+EmdgymSAoV7afrUtTWGsr0xT1EueUZkYp/7i26SyBYz1FWmKiCn/Xy6TaQy0AaBZ1RIw1E+naAqIecohQy1LzYFo40rmsO1VJ9PCp1wm07gAAQA1y5rAe2iTTAuf6f1OS5saYOv41pkXTQEAaqhiojT67pui+cUku1z6N8z6SekY+UnirdZvPymDr/t2xpvQxGUeLp0whPWx31Grbvprx3WqU0bjC0Z21bY0RW3LnN/2VcpEH21rZH/pgSQGSgWql0m7LEVWZDKZxq3dpBvyY2zs4uKCI0eOYOLEidi0aROMjIzg4uKChw8fYvPmzRrP9fDwwLJly/Drr7+iffv2Go9t2bIFALK1GZ8uqm1pCqVCpjGezI7mlc2gZ2SS44shn6S4qyOn/XLTt2W1khrFlOxqUME8zQXx7Khc0hj+XzfPcT8jfQV2pFiaKLsiIz4gYq8tgp88BmQyKJVKBO4+hMbNW2sUstMrys7/rD4SVGpxPJL6+UmxSaqleka0rYr+TSuK46Y0Y3+1GvGx0ahYQvOztWvdsqhtaaox7k8ayyddEBAA1LHUvLBYr7wZrDrX0LgokN7FiIYVzTX6lTU1wBetKqe4IJD8cyXmndg39YVMA6UcPetbprhQkf7FiCol094O36JqyTQTQVLnnN7FhEoljaCvlGv8/jVzTf9vaKyvgLmRMs2Fh9QXI1K/TWa3SJ/b0kPq9+XcXrzI7lr2qcf62b94kfYHzOqCAJD2YklejpmgViNOlfXlj9Sv/yEmIVt3qKW+0+JdVDzuvcz6boj6FTT/PYVHJ2S6b0MSmQwaBeWo+AQcv5313XuNKpqlKSjvvfRfln+PMqb6aQrKuy4+ydYFmoPT2ouzXfT09FBtgA2iXyd/Rz30DDj0LO1dDoEjW2hMJvr22lOcyMaSTAv7N0DLqsmfR8fvvMTOC0+y7GfVpQY+a5x8x/Wlh2+w4pd/xe8yGV2M+LxJeXzdNnlppeC3UfA4lnihJc0FgRSF7GaVS2Bipxpiv5h4FRYf/SvDAn1S3woWhpjaVfNu2IDf7icuIaRxkUTz2AZKeZp+h64/1VySCYCgiodCTw8KmVzMY1jryhrj9osP3yD449JK6V000VPI0a1u2n3kdEmeC8o2NjZo0aIFAgMDMWrUKJQoUUILaREAeP54V5wtkdGmNyWN9eE9THPDHMeDt/AiLCZt4RPJ610CiW9KKQtgy0/cx61szKhM/ab03fXsvSm59G+AFinelC4/epvtN6WUBeX/3kRlq99AtaBRUP4Qm4Cfs3F7d4JKwKDmybFaELJ1e3d6H64vw2IyuGosg8I0MTd1XDS8V2+GgWHyQFb9cVOWrOS6yJcqTu926PSPl7viZ25v705bjMy4Y8o34fRmbFoY6wEf/z2lV2yTy2QoaZJ20FyjrAnMjZQZFj4zmgnYqKI5zAyVaYqIGkVMedoiX4uqJWFqoMy0iCmTAZVKaH4xaF6lxMd+aYu0KWdLVi+tOfipX8EMrgMbpv1d/PEcQkIc+vRsDIVCkeYLReWSRlg7qmW6hU+FXCZ+WCsVmr8bIz0FvpvePldFlZVfNstxHwBpPvSz69NG5YGcfz9Hk0oWaFLJIsf9KlgYoYJFztc/MzFQasyQyS6FPG0xPD1qtYCc7/mbP0qVKQs337WwHvMFACA2NgZOs6yw8+hvGu+hRFLKj7FxrVq1cOrUKcyZMwflyiV++W3SpAl+/PFHdOrUSeO5sbGxiIyMTLNb94cPH/Dtt9+iW7duqFtX8wI4UUFY7jofwY+SZ3BOnu2ANm0/yVbf1LfsZ1fFEpl/rqrVKsRHK6GXqhBdpZSxxveP7Mptv/IWhuLF05wwN9LDzB61c9wPAJz75W4jL6dcbqDq8Gm9XPWb2rUmrLokLoGS+g7BlLPwDVLdpfRp4/LoXKeMOPkoo0J2CWPNMW7jSuZYNqwJLE4/hyo+Dt26N4RMJk8z+z/lXZtA4p0D8z+rn+7dDykvRqRccxtIXBd7ateaGhcSNArtH9vSu5A2uEVFxCaokwv66fysynTW4/qkRmnUKGOSJs+Uv2O1IKS546NGGRPExKvTv6shRVvqi92lTPRRq6yJ5rJz6fxNUt9ppq+Uo6SxXqq7OzQvZAiCkGYmfeq7DjOS+ttI9i9e5K64n/5FgdxdiDAwNAKQ9aSntBc+snW4XN8NkebuCwEp6hoZv0psqjsT4lVqvI7I+udL/Z1YpRayNfv+Qzo1mmtP3mXrbtjU3y0vPHiDB6GRGfRINrB5RY1/G+eCXmd690XDCuZFv6B87do1tGyZu9smKXPBb6OhyuJffHpFx+h4VbbWUUss8iX/g89tkU+qGY5Z9kszEzO7/TTJZR9v785kNqVchnRv765cyhiRsQkpngv89zAI4WHvkThtWY0aNWqgZl3NjasaVjRHCWO9NEW+1AXC1B+eLauVQAljPchkybdNp+4jl6UdlLeqVhIWRon9IAgQEuKg1DeAQi7XmB2ZumjVtHIJ8fbujHKVyRJnXaRUy9IUfl81y7TwmfR7T6m0qT72Wn2S5jiJBdrM/8Bbx7fJ9PGMOOZy8DusVeWsn5SOtjVK5WqGY/UyJrma4VjCWD/doqLc3ACqeKBs2fRndOkp5GlmymYH71yhvGrfpQfGTLHGro2BAICgf+4iwGsx5rovkzgzokT5NTZu3rw5Tp48CbU68ct8RmvdLVq0CPPnz4exsWZRy8TEBC9fviyUS11Q4Xfi++/ww8F9YtyibXtMtJ4jYUZUWCQuiZX434psL0iWWNTMzV08ZoZ6MDPUE8fC5cqbZevuBhMDZa7G8HoKeeIkhlwY2jJ33ze+aJ27frk93udNKuDzJjnfJ6hr3bIaa5RnV9PKJXBoRod0C+UpL0aknoRUxtQAW8a3Ti6Qp1PkBwCFTACgWXhcMbypWFBPuVRiyjZlOksyzeheG7HxqnTzTFm0T/0dakCzimhfqzQO79+Fu3/eBGQyyGRydO3zOVp36CIW6lMvydS6ekmUMtHXvOCR4mdNKtqXTbUkU43SJvi0Ubl0f66UuVdIdbdHSWM9tK1RKt2fK+XdFKm/V+or5ahjaZri7o70/ybpXWwsY6ovXixJfYd5Up6pL14A+FjTyN7FgdxI/TU4q+J+YfjaLBOye29GMfbXX3+hcePGuHPnTo42PcmL91Fx6Ot/BmpBgOLjOjTprcloYaSHhaluIV598j7eRMRlWPhMimf3rKPxpvbDrWd48io8cWq+XJ5iRmXy2kFymQz9m1XQKEDd+O8dHoZGZjkzsmXVkhpvTE/fReHx6yiNdXdSFz5lMhkqlzDSWGPuQ0w8XoTFpHsccWMgGWCir9S4kqtSC/gQE59lETPp//PDvm0bsMJ1gRh3+7QfVmzYqVNFtsSZGpG5ujWUig75zz9AFR8Lod8QngfFnC6+J8THxWHC0L4a69D7bdmb4Tr0lHf5fR68DItBCSM9fFKzdIFsPCLF2K4okWpsfPXxuxwveaGL72HFzfPg/zDi8y6ICE9cP9fU3Bz7fz6LCpVyV5zSJp4flBGOhSkjuvS+8SEsDCM+74IXTxOXk1Tq6WH74V/QoEnu7uykRKkL+kBycTr1EosRMQlIUKvF56tUKsTHREJhYJy4d8PH1yhnbqgxxn31IUbc5ye9wndUXAIalDfX6bFxjmconz17FgDE2+uS4sykvhWPssdzcONcrROXcuffnOjXpDzio3O+xlyLqiU1lrLIrsoljVG5ZM5vCUu6apxT2b29O7/8e/cOArwWi7Fl+QpwWRagU8VkIqLCQE9fH0tWbcSoft0RHZV4i5mrgzUOnDiDsuVyPgOGKC84NibKWEJCAhbaThWLyQDg7O2nE8VkIqLCzszCAp7+G2D1ZX+o1WokxMfD2cYKe378H4yMc373KiVKWkoTsqzvjDBNNQtcrVYhXhEPPSODTOtqlmaGQCarFhaGDatzXFDu3LkzgOQ1UZPizHASNBV30dFRWGA9CfFxiesAyWQyePhvQImSOb8tioiIgGo1a2Ou21K4Oc4CAIS9e4vFc2YgcNd3kKezXiBRfuHYmChjWwNX4tbVS2I86MtR6NN/iIQZEREVLc3btMNkGwds9PcBADx5GISV7s5YuNRf2sSoyMtxQfn48eOZxkSUlp/HQjx+cF+MJ86cg9btOTuJiCgvBgwfiQt/nMQvPxwGAFw6exq7NgZi3DQbiTOj4oRjY6L03bxyEZsClotx1Rq14ODqLWFGRERF06RZDrh05hRuXbsMADi8byfad+mBnp8PlDYxKtJyXFDu27dvpjERaTp5/Ad8t2e7GDdp0RpWtnOlS4iIqIiQyWRwWuKL2zeuimvHrVnuidbtO6FRM24YTAWDY2OitD6EhWGh7VSo1WoAiet6eq3eBGMTU4kzIyIqepRKJTz8N2DE510Q+eEDAMBzvi0at2iFchUqSZwdFVV5vid0+/bteXqcqCgLefEMHvNmi7GJqSk8V22Enl7O14AmIqK0zCwssCRgIxSKxDXKVAkJcJplhciIDxJnRsUVx8ZU3AmCAC/nOeKFPgCY6eCMBk2aS5cUEVERV6lqNSzwXCnG4WHv4WI7HSqVSsKsqCjLc0F5woQJeXqcqKhSqVRwsZ2G8LD3YtuCJb6oXLW6ZDkRERVFzVp/AqvZyXd+PH3yCMsWzZMwIyrOODam4u7Yd/vFpYgAoG3Hrhg9xVrCjIiIiofPBn+BfkO/EuNrF89i54bVEmZERVm+7loTExMjzhgiKm52rAvAtYvnxLjf0K/w2eAvJMyIiKjommg9By3athfjH7/bj+NHvpUwI6K0ODamoi748UMsc0m+wGdRshTc/dZys1QiogIy130ZKqWYxLZupRf+unVduoSoyMrxGsoAcPHixUxjAIiNjcXx48dRpUqV3GVGVIjdvn4F632TNx2pVLU65nn4SJgREVHRplAoEteO69sZH8LDAADeC+3RtGUbVKpaTeLsqKjj2JgIiI+Lg9MsK0RHRYptrisCUbZcBQmzIiIqXkzNzLEkYAMmffE5VCoVVAkJcLaZgj0//g8mpmZSp0dFSK4Kyu3bt880TqJQKBAQEJCbQxAVWhEfwuFsM0Vcq0ihVMJr9Sa+eRMR5bMKlSrDZVkA5k4fDwCI/PABzjZW2HTwR65dT/mKY2MiYL3fUtz984YYfzl2Mrr04iaVREQFrUnLNphqNx9rVywBkHj3yHLX+XBdsUbizKgoyVVB+fDh5DWxhgwZohEnMTExQYMGDVC5cuXcZ0dUCC1d6IhnwU/EeNqcBWjcvJWEGRERFR89Px+IISPG4vC+nQCA2zeuYpO/D2Y4OkucGRVlHBtTcXf53B/YsS75YkmtuvUx29lNwoyIiIq38TNscfHM/3D90nkAwA8H96FD157oM2CoxJlRUZGrgvLgwYPF//bz89OIiYqznw59g+NHDopx6/adMW6ajYQZEREVP/aLluDG5Qt4/OA+AGDrGl+07dQVrdt3kjgzKqo4Nqbi7P27t1hkNx2CIAAA9A0MsGT1ZhgaGkmcGRFR8aVQKODut15jObglTnPQpEUbVKjM5bco7/K8O4Ktra0W0iAq/IKfPIL3QgcxtihREh7+67j5DhFRATMyNoF34Bbo6esDAARBgIvtVLx/91bizKg44NiYihNBEOAxbzZCQ16IbbZO7qhTv6GEWREREZC4HJzzUj8xjggPx0LbqUhISJAwKyoq8lxQFgQBgYGBaNKkCYyMjKBUKtP8j6ioi4+Ph7ONFaIiI8Q2F59VsCxfUcKsiIiKr7oNG8NmgasYv3r5Ah7zZosz6IjyC8fGVJwc2rsDp078KMadevTBl+MmS5gRERGl1LvfYAz6cpQY37xyEVvX+EqYERUVeS4o+/v7w8HBAa1bt0ZMTAymT5+Ovn37QqlUom/fvpylQcXCBr+l+OvmdTEeNnoCun/aT8KMiIhoxISp6NitlxifOvEjDu3dIWFGVBxwbEzFxcN//8FK9+T16UuXLQfXFYGQyWQSZkVERKk5uHqjao1aYrzJ3wc3r1yUMCMqCvJcUN66dSvWrl2Lbdu2AQBWr16NY8eO4eHDh4iKiuIaclTkXTl/BtvX+otxzTr1YLfQQ7qEiIgIACCTyeC6cg1Kl7UU21a6O+Phv/9ImBUVdRwbU3EQGxMD59lTEBsTLba5+a5BydJlJMyKiIjSY2xiiiWrNkKppwcAUKvVWGg7FR/CwyXOjAqzPBeU79+/rzEwVqlUAICKFSti9erVmDt3bl4PQaSzEjchmaaxCYnX6s0wMjKWODMiIgKAUmXKws13rRjHxkRjwazJiI2JkTArKso4NqbiINDHHf/evSPGo61mon2XHhJmREREmWnYtAVmOiTfVfLiaTC8nedwOTjKtTwXlGNjY1GqVCkAgImJCV6+fCk+VqNGDdy6dSuvhyDSSYIgwH2uDV69TN6EZPYCN9Rp0EjCrIiIKLX2XXpgtNVMMQ765y4CvF2lS4iKNI6Nqag7979fsXfLejGu16gpZjoulDAjIiLKjtFTrNG2Y1cxPvH9Ifx46ICEGVFhlueCckqNGjXC/v37xfjw4cMoXbq0Ng9BpDO+27Mdp3/5SYw79eiDr8ZbSZgRERFlxHquC+o3bibGB7ZvxB+/n5AwIyoOODamouZN6Cu4OliLsaGRMbxWb4K+gYGEWRERUXbI5XK4+62FRclSYtsyl7kIfvxQwqyosNJqQXny5MlwdHREly5d0KtXL4wdOxZjxozR5iGIdMKDf/+GLzchISIqNPT09eG1ehOMjE3ENjcHa4SGvMykF1HecGxMRYlarYabgzXevg4V2xwWe6F6rToSZkVERDlRtlwFLF6+WoyjIiPgNMsK8XFxEmZFhVGeC8o3btwQ/9vKygq+vr6IiIjA27dvsWjRIri6uub1EEQ6JTYmBk6zrBAbm7z+JjchISLSfdVq1sZct6Vi/P7tGyyeMx1qtVrCrKio4diYiqr92zfi3KnfxLjnZwMw+GteICEiKmy69v4Mw8dMEuO7f97Aer+lmfQgSivPBeXmzZtrxLa2trh+/TquX7+OxYsXQ+/jLpJERUWAtyuC/rkrxmOmWHMTEiKiQmLA8JHo3X+wGF86exq7NwZKlxAVORwbU1F076/bWJVi7flyFSrCeak/784jIiqkbBe6o1bd+mK8Y10ALp/7Q8KMqLDJc0HZx8cHISEh2siFSOed+f0XHNi+UYzrN27GTUiIiAoRmUwGZy8/VKhcRWwLXO6Ju3/eyKQXUfZxbExFTXR0FJxmTRZvh5bJZPDw3wCLEiUlzoyIiHLL0NAIS1ZvFtfAFwQBi+ym4/27txJnRoVFngvK8+bNQ+XKlTFkyBAcO3YMKpVKG3kR6ZzQkJdwdZgpxkmbkOjp60uYFRER5ZSZhQU8/TdALk8cBqkSEuA0ywqRER8kzoyKAo6NqajxdXfG4wf3xXii9Ry0atdRwoyIiEgb6tRvCFsndzEODXkB97k2EARBwqyosMhzQfnevXuwt7fHpUuXMGDAAFStWhVOTk4ICgrSRn5EOkGtVmOx/Qy8f/tGbJvrthTVataWMCsiIsqt5m3awcp2rhgHP34In8XzJcyIigqOjakoOXn8Bxzau0OMm7RoDavZczPpQUREhcmX4yajU48+Ynz6l5/w3Z7t0iVEhUaeC8p169bF0qVLERwcjO+//x5t2rTB8uXLUadOHXTr1g27du3SRp5Ektq9aQ0unTklxr37D8bAL0dJlg8REeXdJGt7tGjbXoyPfbsPPx/9TsKMqCjg2JiKipfPn8Jj3mwxNjE1xZJVm7gOOBFRESKTyeC6IhCly5YT23zdnfHw338kzIoKgzwXlJMoFAoMGDAAR44cwdOnT+Hj44OgoCCMHTtWW4cgksTft29izXJPMS5fqTKcvfy4CQkRUSGnUCjg4b8BZuYWYpuX8xw8+++JhFlRUcGxMRVmKpUKi+ymIzzsvdi2YIkvKlWtJl1SRESUL0qWLgN3v7ViHBsbAycbK8TGxEiYFek6rRWUk8TExODkyZM4ceIEnj9/DoVCoe1DEBWYqMgIOM2yQkJ8PABALpfDM2AjzCwssuhJRESFQYVKlbFwqb8YR374AGcbKyQkJEiXFBUpHBtTYbR9rT+uXTwnxv2GfY3PBn8hYUZERJSf2nXujjFTrMX4/t9/YfUyNwkzIl2ntYLy9evXYW1tjQoVKmDkyJF4/PgxPD09ERwcnKvXEwQBR44cga2tLaytrbF9+3bEfyzqZde7d++wceNGzJo1C05OTjh37lzWnYhSWL54Pv579ECMJ9s4oEWbdhJmRERE2tar3yAM/nqMGN++cRUb/ZdJmBEVBdoeGxMVlNvXr2CD31IxrlytBua58z2RiKiom+m4EPUbNxPjfVs34OzJXyXMiHRZngvKq1evRosWLdCqVSts2bIF/fr1w8mTJ3H//n04OTmhQoUKOX7NhIQEDBkyBJMnT0aZMmVQq1YteHh4oEOHDggPD8/Wa/zvf/9D3bp1cfjwYdSpUwfly5eHs7MzPD09s+5MBODno9/h+4N7xbhZ608waZaDhBkREVF+cVjsheq16ojx1kBfjdl5RNmVH2NjooLyITwczjZToFKpAAAKpRJLVm2EiamZxJkREVF+09PXx5JVG2FoZCy2uTrMxOtXIRJmRboqzwVlGxsbyGQyBAYG4sWLF9i9eze6d++ep/Vl161bh6NHj+L777/HwoULYWdnh9OnT+Pu3btYuHBhlv0fPXqEgQMHYtq0aTh+/DhsbGxgY2ODU6dOYcyYMVn2Jwp+8gheTnZibGpuDk//DVAqlRJmRURE+cXI2AReqzdDT18fQOKdUi62UxH2/p3EmVFhkx9jY6KCsszFEc+Ck9eRn27vhMbNW0mYERERFaTqterA0dVbjN+9eY3F9jOgVqslzIp0UZ4LyteuXcP169cxc+ZMlChRQgspARs3bkTz5s3RoUMHsa1y5coYNGgQtm/fjtjY2Ez7e3p6wsjICC4uLmkeq1aNG0lQ5uLj4uA8ywqRERFim8vSAFSsUlXCrIiIKL/Va9QENvMXi3HIi+fwmDsbgiBImBUVNvkxNiYqCD8eOoDjRw6KcZsOXTBumo2EGRERkRQGfTUavfoNEuOLf/wP+7aulzAj0kV5Lii3bNlSG3mIPnz4gDt37qB169ZpHmvTpo34eGa+//57dOvWDffu3YOLiwumTJkCDw8P3L9/X6u5UtG0Zrkn/rp1XYyHjRqv8WZKRERF14iJ09CxWy8x/t+JYzi0d4eEGVFho+2xMVFBCH7yCEsXOoqxRclScPdbC7lc63u4ExGRjpPJZHD29kO5ipXEtlVL3fDPnT8lzIp0TY7v3z979iwAoFOnThpxZpKemx1Pnz4FAJQvXz7NY0lrzgUHB6NVq/RvvQoNDcXr169x48YNfPLJJ5g+fToaNWqEQ4cOwdPTEzt37sRXX32V4fFfvXqF0NBQjbagoCAAgEqlEtcTy28qlQqCWg1BBqjVBTOQE9Qfj6lWobjezHD+9O/YtTFQjGvWrQ/bhe5Qqwvm764LeB4QAMgENQS1wPOAiuV7wqLlqzDis654+zpxPLDS3RnNWrdFzTr1JM5MOvl9HiS9tkqlAoT8XxpCm+O5/B4bE+W3+Ph4ONtYISoy+e68RT6rYFm+ooRZERGRlMwtSmBJwEZM+WoA1Go1Ej5+Vuw+dhJGxiZSp0c6IMcF5c6dOwOAePtnUpyZnNwqGhMTAwDQ09NL85j+x3UNk56TnsjISACJReBvvvkGw4cPBwBYW1ujS5cumDx5Mnr37o1SpUql23/t2rVwc3NL97GIiAiEhYVl+2fJi8iYeKhjI6FSyBCf8z9TrghqNRJiowEAsmI4G+F16CssnjNTjA0MDeG+fBUUghrx0ZESZlawivt5QB/FxUGVEAdVdBTPg2KuOL4nmJkYY6HXCsyZMg4AEBsTDedZk7Fp/xEYGBhKnJ008vs8EOLikCBTIjxcCXkBrDUckWJZq7zK77ExUX7b4LcUf91Mvjvvi9ET0a3P5xJmREREuqBF2/aYaD0Hm1etAAA8fnAfK92dsXCpv7SJkU7IcaXy+PHjmcZ5ZWpqCgCIjo5O81hUVBQAwMws412Gk/qbm5uLxWQAUCgUmDhxIiZPnoz//e9/GDZsWLr9Z8yYodEPSCxODx48GKamprCwsMjZD5RLgl4c5G/VUCgAPaO0xfV8OebHWbh6RsaQyRUFckxdoVar4enkiPdv34htc1yWoF7TFhJmJY3ifB5QMsXHC3hyngfFXnF9T+jc+3OMnDQde7esAwAE3fsH6wNWwmGxl8SZSSO/zwNZnBJKQyXMzS2gkOd/QTlpvKgN+T02JspPl8/9ge1r/cW4Zp16sHPxkC4hIiLSKVaz5+Ly2dP48/oVAMDhfTvRvksP9Px8oMSZkdRyXFDu27evRty8eXPI5XJYWlpqJaFq1apBqVTiv//+S/PYkyeJOw7XqlUrw/5lypRBiRIlULp06TSPJeX45s2bNI+lfE5GP4tCoYBCUTBfphUKBWRyOWRyGeQF9AVeDXw8pqLAjqkrdqxfjcvnTotxr36DMGzU+GK5I3txPg8omUyW+P4DngfFXnF+T5g1bxGuXTyHe38lrhd3YPsmtO/SE5179pE4s4KX3+dB0msrFIoCKShrczyX32PjJFFRUVi1ahVOnTqFhIQEtG7dGvb29ihbtmy2X+Pvv//Gpk2bcPv2bZiZmaFfv34YP358gY1vSbe8f/cWi+ymizPm9Q0M4BW4BYaGRhJnRkREukKpVMIzYCNGfN4FkR8+AAA859uiUfOWKF+xssTZkZTyfM9ixYoVceHCBW3kAiBxWYuOHTvi7NmzaW4HPHXqFCpVqoR69TJfw7B37954+vRpmqUxkjblq1GjhtbypcLvz2uXsW7FEjGuULkKFnr7F8tiMhERJdM3MIDX6k0wNDIW21wdZiI05KWEWZGu0/bYGEhcoqNTp07Yvn07Jk+eDHt7e5w/fx4tW7bEs2fPsvUa27ZtQ8uWLSEIAhwdHTFt2jRcuXIFc+fO1WquVDgIggD3uTYIDXkhttk6uaNO/YYSZkVERLqoUtVqcFriK8bhYe+xyG56ge0xRropzwXlSpUqoWPHjtrIReTg4ID//vsPGzduFNtOnTqF33//HQ4ODhqFvkOHDmH8+PFisRgAHB0dER8fjyVLkouEISEhWLVqFerVq4fu3btrNV8qvD6EhcHJxkp8I1QoFPBatQlmBbS0CRER6bbqtepgrttSMX7/9g0Wz5kOtbq4bFFIOZUfY+Ply5fj1q1b+P777/HFF1/gs88+ww8//IDo6OhsFYSvXbuGKVOmIDAwEH5+fujTpw/69OmD9evXw8ODyxsUR9/t2Y7Tv/wkxp17foovx02WMCMiItJlfQcNQ79hX4vxtYvnsGNdgIQZkdTyXFCePHkydu3apY1cRP3794ePjw9mz56N3r17Y9CgQejbty9mzJiB2bNnazz3+vXr2LFjB0JCQsS2Nm3aYOfOnfD390fr1q0xdOhQNG7cGObm5jh27BiUyoLZ5I50myAI8FxgixdPg8W26Q7OaNqqrYRZERGRrhn45Sj07j9YjC+dPY0d61dJlxDptPwYG+/cuRPt27dH3bp1xTYLCwsMGTIE3333XZabDHp5eaFixYqYOHFimseMjY3T6UFF2YN//4avu7MYly5bDouXr+bdeURElKl57stQqWp1MV7v643bH9dWpuInz5XVAQMGwNvbG7dv38bAgQNRqVIl6OlpbiLXvHnzHL+uo6Mjxo4di/PnzyMhIQH+/v7pLlUxbNgw1K5dW2OADQCjRo3CgAEDcObMGYSFhWHu3Llo27Yt5MVkh3rK2uF9O/Dbj0fF+JNOXTFumo2EGRERkS6SyWRw9vLDnZvXxIuQ61YsQYs27dC8TTuJsyNdo+2x8Zs3b/D48WN89tlnaR5r3rw5Nm/ejNu3b6N9+/bp9ler1fj1118xcOBAHDp0CPv370dISAiqVKmCsWPH4tNPP83Rz0eFW2xMDJxmWSE2NnlpQHe/tShZuoyEWRERUWFgYmoGr9WbMHHYZ1AlJEClUsHZZgr2Hj8NUzNzqdOjApbngnKrVq3E/962bVu6z0m9FnJ2lStXDkOGDMn0OS1atECLFi3Sfczc3Bz9+vXL1bGpaHvw799Y4eokxqXKlIW733pecCAionSZWVjAe/VmTBreTxxAO9lYYd/xP2BRoqTU6ZEO0fbY+OXLxDW709t8r0yZxCLgixcv0jyW5NWrV/jw4QN+/vln/Prrr/D09ETVqlXxzTffoG/fvvD29sb8+fMz7R8aGqrRFhQUBABQqVQFtn6iSqWCoFZDkAFqdfbHa4L6Yz+1ClyoBgjwWoSgf+6K8Zgp1mjbsQvU6uK5DibPD8qITFBDUAs8NyiN4v6+0bBpc0ybMx9rfDwBAM+Cn2DpQke4+62VODPdoK3zI+k1VCoVIOT/HUS5Gc/luaC8b9++vL4EUYGKjo7CgpmTNGZmuPmuRRnLchJmRUREuq5JyzaY6bgQq7xdAQAhz5/BzcEaKzft5q3iJNL22Dg+Ph5A4j4PqSUt45b0nPTExsYCSJzp/Mcff6Bz584AgE8//RRv3ryBi4sLRo8ejcqV09+pfe3atXBzc0v3sYiICISFhWX/h8mDyJh4qGMjoVLIEJ+DrzCCWo2E2GgAgKyYTxw4d/okDuzYLMb1GzXB5Bk2iI+OlDArafH8oAzFxUGVEAdVdBTPDdLA9w3g6zETcOH077h+KXET4uNHDqJNuw7oOzDzCaHFgbbODyEuDgkyJcLDlZAXwPeMrJZPS0+eC8pff/111k8i0iF+Hgvx4N9/xHjMFGt06NpTwoyIiKiwGDPFGlfPn8H5078DAE7/ehz7t2/EiAlTJc6MdIW2x8YWHzcKTm+g/+HDB43nZNa/dOnSYjE5ybBhw3D06FH88ccfGDlyZLr9Z8yYgeHDh2u0BQUFYfDgwTA1Nc302Nok6MVB/lYNhQLQM9LLukNSv48zb/WMjCGTpy3KFxehIS+xxNlRjI2MjeG5ahOMLYr3HRY8PygjCn19AICc5walwvcNQA+Au996jPq8G8LevwMArPR0QYt2HVG5WtqlaosTbZ0fsjgllIZKmJtbQCHP/4KyqalpjvtoZXe6hIQEHDlyBKdPn8abN2+wd+9eAMBPP/2E3r17p1k3jkgqv/14FN/t2S7GjZq1xEzHhdIlREREhYpcLoe73zqM+KwrQkMSlxnwX7IIzVt/ggZNmkubHOkMbY6Nq1atCiMjIzx8+DDNY0lt9erVy7B/iRIlUK5cuXQ33zM3T1zvMCoqKsP+lpaWsLS0TPcxhUKR7szp/KBQKCCTyyGTyyDPwRc0NfCxnyJH/YoSlUqFxXNm4v3bN2LbXLdlqFG7bia9igeeH5QRmSzx/QY8NygVvm8kqlCpChYuC4Dj1LEAgMiICDjbTMXW745D7+MFmeJIW+dH0msoFIoCKSjnZjyX5/n54eHh6Ny5M4YPH47du3dr3OZ34MABHDx4MK+HINKK58H/wWP+bDE2MTWFV+DmYv1mR0REOVeydBl4BmwQ191PiI/H/JmTEPEhXOLMSBdoe2ysUCjQu3dvnD59GnFxcRqP/fLLL6hfv366G1en1L9/fwQHB+Pdu3ca7bdu3QIANGzYMEc5UeGyfa0/rpz/Q4z7DBiCAcPTn5FORESUXT369scXoyeK8d0/byDQx0PCjKgg5bmg7OTkhPDwcFy9ejXNIHXSpEnYtGlTXg9BlGfx8fFwtrFCRHjyl31nbz9UrlpduqSIiKjQat2+E6xmJ98+/vTJI3g5zcn1RsRUdOTH2NjZ2Rnv37+Hi4uLeI5t27YNly5dSrO+8YYNG9CuXTv8+eefYtuCBQtgYGCA2bNni+st//nnnwgICEDnzp3RoUOHHOdEhcONKxexwW+pGFeqUg1OXn5c952IiLTCzsUDtesnX5jevWkNzp78VcKMqKDkuaB86NAhbNu2TWNH6yQNGzbEtWvX8noIojxbt9ILf16/IsaDvhyFTwcOkzAjIiIq7CbNckCrdp3E+MT3h3Bk/y4JMyJdkB9j47Zt2+Kbb77Brl27UK1aNdSrVw+2trZYvXo1vvzyS43nBgcH49KlSwhPcRG9Vq1a+Pnnn3H58mWUK1cODRs2ROvWrdG9e3ccOXIkx/lQ4RD2/h2cbazEndsVSiW8AjfD7ONSJ0RERHllaGiEpYFbYGBoJLYttp8hLg1HRVeeC8qhoaEat8mlvNqtVCrT3JpHVNDO/e9X7FgXIMbVa9WBo9vSTHoQERFlTaFQYMmqDShRqrTYtnzxfATduythViS1/BobDxkyBMHBwThx4gS+/fZbvHr1CtbW1mmeN23aNFy4cAHNmjXTaO/UqRP++ecfXLlyBbt378bLly/x7bffolSpUrnKh3SbIAjwmDsbIc+fiW3Wc13QuHnaCx1ERER5UaNOPcxNUWN5//YNFs6eJl7QpKIpzwXlChUq4MaNG2KcctB88eLFLNd0I8pPIS+ewcVuuhgbGBhi6dptMDI2kTArIiIqKsqWqwAPv3ViHBsbgwUzJyE6KlLCrEhK+Tk2VigUaNCgAZo0aQIDA4N0n1O5cmW0a9cOZmZm6T5eq1YttGzZkoXkIu7grq3434ljYtyha0+MtpopYUZERFSUDfpqND4dOFSMr144g21r/CTMiPJbngvKgwcPxowZMxAUFKTR/uTJE9jb22PIkCF5PQRRriQkJMBplhXC3r0V2xzdlqJOfW48Q0RE2tOhWy+Mm2Yjxg/v38PyxfMlzIikxLExSe3fu3fg57lQjEuXLQc337XiRqJERETaJpPJsGCJLyql2Kdqo/8y3LhyUbqkKF/leVSxePFixMbGokGDBmjTpg0EQUD37t3RoEEDAImbgBBJYYPvUtxM8ebVd9AXGPz1GAkzIiKiomq6gzOatGgtxke/2YOfDh+UMCOSCsfGJKXoqEjMt56EuNhYAIlf8D3916NUmbISZ0ZEREWdmbk5vAM3Q6FUAgBUKhWcbawQ9v5dFj2pMMpzQbl06dK4dOkS7OzsEBERAQsLCzx9+hTTpk3D2bNnM7zdjig/nT/9O7atTb69olrN2nDyWskdrYmIKF/o6enBO3ALzMwtxDZvZ3v89+iBhFmRFDg2Jin5LJqHJw/ui/HEmXPQtlNXCTMiIqLipFGzlrCe6yLGIc+fwX2uDQRBkDAryg9aue+pZMmS8PHxwd9//413797h/v378PX1RcmSJbXx8kQ5EhryAovspotvWPoGBli6ZitMTPkFjoiI8k+FylWweEWgGEdFRmD+jImIjYmRMCuSAsfGJIWfDh/E9wf3inGzVm0xxW6ehBkREVFxNNpqJjp07SnGp078iIM7t0iYEeUHLqRFRUpCQgKcbabg3ZvXYpvDYm/UbdhYwqyIiKi46P5pP3w1fooY37t7GwFeiyXMiIiKg+DHD+HtbC/GZuYWWLJqE5QfbzsmIiIqKHK5HG6+a1G6bDmxzddzIe79dVvCrEjbcjzC2L17d44PMnr06Bz3IcqNTQE+uHbxnBj3GTAEQ0eOkzAjIiIqbmyd3HDzykXc++tPAMCBHZvQukNn9OjbX+LMKD9wbExSi4+LwwLryYiKjBDbFvmsQoXKVSTMioiIirNSZcrC0389ZoweCkEQEj+rZk3GnmMnYWRsInV6pAU5LiiPGZPzTc04aKaCcOnsKWxZvVKMq1SvCWdvP66bTEREBSpxqaUtGNWvu1jgcXechfqNmqJilaoSZ0faxrExSW31Mnf8ffumGA8fMwk9PhsgXUJEREQA2nbqiokz52BLYGKd5smD+/BZNE9jiTgqvHJcUL59W3OKukqlgqenJ0qWLIlBgwahXLlyCAkJwZEjR/D+/Xu4uLhk8EpE2hMa8hILZ08V103W09fH0jVbYWpmLnFmRERUHFWtUQvO3n5wtrECAHwID8P8mROx5dufoKevL3F2pE0cG5OUzvz+C/ZsXivGdRo0gt1CDwkzIiIiSjbFbh6uXjyLW1cvAQC+P7gXbTp2xedDhkucGeVVjgvKjRtrrkW7YMECdO3aFdbW1hrt/fr1w6pVq3Dw4EE0bdo0b1kSZUKlUmHh7Kl4+zpUbJvjsgT1G/O8IyIi6fQdNAyXz53G0QOJSyL8des6/L0WwdF1qcSZkTZxbExSefXyOVwdZoqxoZExvFdvhoGhoYRZERERJVMqlVgSsBEjPuuCD+FhAABvZ3s0adEKVarXlDg7yos8b8q3detWjBo1Kt3HRo8ejU2bNuX1EESZ2rxqOa5eOCPGvfoNwvAxEyXMiIiIKJGj21LUrt9QjPdv24jffjwqYUaU3zg2poKQOKFiGt6/fSO2zXNfhhp16kmYFRERUVoVKlfBIp9VYhwVGYH5MychLjZWwqwor/JcUA4PD0dISEi6j4WEhCA8PDyvhyDK0OVzf2BTwHIxrlS1OhYuDeC6yUREpBOMjIzhs247jE1MxTb3ubPw36MHEmZF+YljYyoIWwN9ce3iWTHuO+gLDBg+UsKMiIiIMtbjswEYPmaSGP9z5xYCl7lLmBHlVZ4Lyh06dMD06dPx+vVrjfbQ0FBMmzYNHTt2zOshiNL1JvRVmnWTl63dCjNzrptMRES6o1rN2nBZ5i/GkRERmDdjAmJioqVLivINx8aU325cvoCN/svEuHK1GliwZAUnVBARkU6zW+iBOg0aifGeLevwx+8nJMyI8iLHayin5uvriy5duqBq1apo166duPHIxYsXoa+vjzNnzmT9IkQ5lLRu8pvQ5BlAds4eaNCkuXRJERERZaDPgKG4cfkivtm5GQDw7907WOG6AAuX+kubGGkdx8aUn969eQ2nWZOhVqsBAEo9PXgHbuZG1EREpPMMDA3hvXozRg/oiZjoKACA65wZ2PPTaVSoVFni7Cin8jxDuVmzZrhx4wZGjRqFJ0+e4Pjx43jy5AlGjx6NGzduoEmTJtrIk0jDRv9luHzutBj36NsfX46bLGFGREREmbNb6IGGTVuI8eF9O/HjoQMSZkT5gWNjyi9qtRqL7Kbj1csXYtuseYs03leIiIh0WY069TDPPfkum7D37+BkPQnx8fESZkW5kecZygBQs2ZNbjBCBebc/37F5lUrxLhS1epY5LOat/kREZFO0zcwwNI1WzGqXzdxl2svJ3vUb9wUteo2kDg70iaOjSk/7Fi/CudP/y7GXXr1xajJMyTMiIiIKOcGDB+JqxfP4cfv9gMA/rx+BWuXe2K2k5vEmVFO5HmGMlFBevHsKVxsp4mxvoEBfNZth5mFhYRZERERZU+lqtXg5rtWjGOiozBv+gRERUZImBUR6bobly9g3YolYly+UmW4rlzDCRVERFToyGQyLPBcjhq164ptOzes5nrKhQwLylRoxMfFYd6M8Qh7/05sm+u2DPUbN5UwKyIiopzp2vszjJliLcaPgv6Fl5O9uMksEVFKSesmq1QqAIBCqcTSwC2wKFFS4syIiIhyx8jYBEvXboWBoZHY5jpnBl48eyphVpQTLChToeG3ZBH+unldjPt/MQKDvx4jYUZERES5M3OuC5q1/kSMjx85iMP7dkiYERHpIrVaDRe7aRrrJtvMX4wmLdtImBUREVHe1a7XEPM9fMSY6ykXLiwoU6Hwyw+HcGD7RjGuXb8h5nsu521+RERUKOnp6cE7cDNKlCotti13XYB/7vwpYVZEpGu2rwvAhdMnxZjrJhMRUVEyYPhI9Bv2tRgnradMuo8FZdJ5j4L+hcc8WzE2NjHFsrXbYGRkLF1SREREeVSuQiV4BmwQL47GxcZi3owJ+BAeLnFmRKQLUq+bXKFyFa6bTERERQrXUy68WFAmnRYdFYl508drbFbk4hOA6rXqSJgVERGRdrTv0gOTZtmL8dMnj+DuaM31lImKuXdvXmOB9WSo1WoAiesme3PdZCIiKoK4nnLhxIIy6SxBEODt7IAH//4jtn09YQr69B8iYVZERETaNcV2Htp06CLGJ38+hn1b10uYERFJKWnd5NCQ5HWTZy9wRZMWrSXMioiIKP9wPeXChwVl0lmH9+/Ej4cOiHHjFq1g6+QuYUZERETap1AosGTVRpQuW05s8/dajJtXLkqYFRFJJfW6yV17f4aRk6ZLmBEREVH+43rKhQsLyqST/r59C8sXzxdji5KlsHTNNujp60uYFRERUf4oXdYS3oGbIZcnDs1UCQmYP3MSXr8KkTgzIipI6a2bvHhFINdNJiKiIo/rKRcuLCiTzgkPe49508cjLjYWQOKbiqf/elSoVFnizIiIiPJPq3YdYT1vkRiHhrzAAt7qR1RscN1kIiIq7riecuGhlDqBzJw6dQqnTp1CQkICWrdujYEDB4ozdzLz888/4+eff073MQ8PD5iZmWk7VdISQRDgaj8Tz4KfiG2TbRzQoVsvCbMiIiIqGGOnzsLt61fxvxPHAADXL51H4DJ32C30kDgzIspPXDeZiIgoUdJ6ym6OswAkr6e88Ztj0NPTkzg7SqKTM5TVajXGjRuHgQMHIiIiAgqFAjNnzkSvXr0QHR2dZf+LFy8iICAAFStWRPXq1TX+p1AoCuAnoNzavi4Ap389LsafdOoKq9lzJcyIiIio4MhkMriuDES1mrXFtt2b1uC3H49KmBUR5Teum0xERJQsvfWU1/hwgoUu0ckZylu2bMHOnTvx66+/olevxJmpY8aMQePGjeHq6oply5Zl63VmzJgBU1PT/EyVtOjS2VMaC65blq+AJas28SIAEREVK6Zm5li+YQfGDuyNmOgoAICb4yzUqlsfNerUkzg7ItK2qxfOct1kIiKiFJLWU7576zoeBf0LANi1MRDN27RDtz6fS5wdATo6Q3ndunVo1KiRWEwGgNq1a2PAgAHYtGkT1xIsgl48DdZYM06pp4ela7aiZOkyEmdGRERU8GrVbYBFPgFiHBUZAYepYxEZ8UHCrIhI20JDXsJpFtdNJiIiSs3I2ATL1m3TWE95sf0MBD95JGFWlETnCsqRkZG4efMm2rZtm+axdu3a4d27d7hz5062XsvX1xeOjo5YtmwZLl26pO1USUtiY2LgOG0cwt69FdvsF3mhWetPJMyKiIhIWp8OHIYRE6eK8eMH9+E+1waCIEiYFRFpS3x8PBZYT8Kb0Fdim62zO9dNJiIi+qhW3QZw9vYV44jwcMybNh4xMVkvh0v5S+eWvAgODoYgCKhYsWKax5La/vvvP7Ro0SLT16lXrx5ev36NsmXL4uzZs5g/fz6GDRuGvXv3Ql9fP8N+r169QmhoqEZbUFAQAEClUkGlUuX0R8oVlUoFQa2GIAPU6oKp+wvqj8dUq6AukCMmbsK31MUBf9++Kbb1G/olho0aB7W6YH7XpEmK84B0j0xQQ1ALPA+I7wkSs5m/GHf/vIFbVy8DAH778Sh2N1+DUZMLdm3V/D4Pkl5bpVIBQv7f5l9Q4zmizKzx8cCNyxfEuHf/wRgxYWomPYiIiIqffkO/wq2rl/Ddnu0AgHt3b8Nn0Tws8lklbWLFnM4VlJM23TMwMEjzWFJbVhvzjRkzBosWLYJcnlyI3bZtGyZOnAhXV1d4eXll2Hft2rVwc3NL97GIiAiEhYVl+TNoQ2RMPNSxkVApZIgvoD+ToFYjITbxdyuTF0wR+8g3e/H9N3vFuE79hnBY6I6EmKgCOT6lJcV5QDooLg6qhDiooqN4HhRzfE+QnsfKQIwf+jnevnkNAFi91A116tVDiwK8kye/zwMhLg4JMiXCw5WQF8C6sREREfl+DKLMnPz5GHZtDBTjarXqwGVZANdNJiIiSof9Ii/8desG/rlzCwBw9MBuNG/9CQZ+OUrizIovnSsom5iYAEi/aJzUlvScjNSqVStN24QJE+Dj44M9e/ZkWlCeMWMGhg8frtEWFBSEwYMHw9TUFBYWFln+DNog6MVB/lYNhQLQM9IrmGN+nBGsZ2QMmTz/N8L76+Z1+Hm5irG5RQks37gTpiVL5/uxKWMFfR6QblJ8vJNDzvOg2ON7gvQqVK0B7zVbMGPUUPFuKZc51th97CTKlitfIDnk93kgi1NCaaiEubkFFPL8L6hx02aS0n+PHsDVYaYYGxoZY/n67TAxNZMwKyIiIt1lYGgIn3XbMbp/d4SHvQcALF3oiHqNmqJeoybSJldM6VxBuWrVqlAoFHj69Gmax4KDgwEANWvWzNVrV69eHSdPnsz0OZaWlrC0tEz3MYVCAYWiYL5MKxQKyORyyOQyyAvoC7wa+HhMRb4f8+3rUMybMQHxcXEAEnfwXLJqI6pUy93flrSnIM8D0l0yWeL7D3geFHt8T9ANrdt3xmwnN/h6LASQ+Dm6YOYkbNj/PfQyWcpLW/L7PEh6bYVCUSAF5YIazxGlFh0dhbnTxyPyQ/IGm87efqhVt4GEWREREem+SlWrwd1vPWwnfg0AiI2Nwdzp47H7h5MwK6DJn5RM5+5dNTQ0xCeffIJz586leezMmTMoV64cGjTI+YBLEATcu3cPVapU0UaalAcJCQlYYD0ZIS+ei21T7eajQ7deEmZFRESk20ZOmo7e/QeL8a1rl+HvtUi6hIgoRwRBwDIXR9z/+y+x7YvRE/H5kOGZ9CIiIqIknXv2wcSZc8T46ZNHcHW05qbVEtC5gjIA2Nra4uHDh9izZ4/YdvnyZZw4cQI2NjYaayP/9NNPsLW1xaNHjwAAarUaR48eTXMyeXl54dGjR5gyZUrB/BCUoTU+Hrh64YwYd+nVF5Nm2UuYERERke6TyWRwWRaAGrXrim37t23ET4cPSpgVFaSff/4Z8+fPh4ODA/bv34+EhIRs9Tt27BimTZuW7v/Cw8PzOWtKcvTAbvxwcJ8YN2rWEvaLlkiYERERUeEzzX4B2nToIsanTvyosS8BFQydLCgPHz4czs7OmDhxIoYPH45x48ahZ8+e+PrrrzFv3jyN554/fx4BAQF49uwZgMQvW7t370aDBg0wfPhwTJ48Gc2bN4erqyscHBzg6OgoxY9EH/364xHs3LBajKtUrwl333UaFwmIiIgofSamZli+YSeMTZLXAPacNxv/3PlTwqwov6lUKnz11VcYMWIElEolSpcujXnz5qFr166IjIzMsv/Vq1exYcMGNGjQAM2bN9f4n55ewezVUdz9c+dPLFs0V4zNLUpg2dpt0E9nI3IiIiLKmEKhwJJVG1HGMnkvkcBl7rh+6byEWRU/OreGchJPT0+MHz8eZ86cQUJCAmxtbdGiRYs0z+vXrx/KlCkjrqssk8lw8OBBBAcH48qVK3j58iX69u2Lzp07o1y5cgX9Y1AKD//9B24Os8TY0MgYyzfs5Fo3REREOVCjdl24rgjE3OnjASSuH+cwdQx2/XASJUtxY9uiaOPGjfjmm29w6tQpdO3aFQAwYsQINGjQAIsWLcLKlSuz9TqTJk3ihoQS+BAWhrnTxyMuNlZs8/DfgAqVuRQfERFRbpQua4mla7di6lcDxE2r58+chL0/nUIZS9b+CoLOFpQBoHbt2qhdu3amz2nfvj3at2+fpr1KlSpcL1mHfAgLg8PUsYiOSp5F47IsAHXqN5QwKyIiosKp5+cDMXHmHGxd4wsAePE0GPNnTMSa3d9BqdTp4R3lwvr169G0aVOxmAwkbjY9cOBAbNmyBd7e3tAvgM0ZKefUajUW28/As/8ei22TZtmjU4/e0iVFRERUBLRo0w42C1zh5+kCAHgTGgKnWZOxds9hjocLANcZoHynUqngPHsKnjwMEttGTJyKvoOGSZgVERFR4TbNfgE6ptjQ9uqFMwjgJn1FTkREBG7fvo02bdqkeeyTTz5BWFgY7ty5k63X8vLywqxZs+Dm5obTp09rO1VKx84Nq3H61+Ni/EmnrphqN1/CjIiIiIqOUZNnoOdnA8T42sVzWLuC+xMUBJbsKd+tX+mNc//7VYxbftIBs53cJcyIiIio8EtcP24TxgzsieDHDwEAe7esR/3GzdBv6FcSZ0fa8vTpUwiCgPLly6d5rEKFCgCA4OBgtGzZMtPXadKkCRISElC1alVcvnwZbm5u6NevHw4ePAhDQ8MM+7169QqhoaEabUFBiZMEkm4xLQgqlQqCWg1BBqjV2Z8TI6g/9lOroM7H/NJz7eI5rPHxEGPL8hXg4b8eMhmgVhfM740yJ+X5QbpNJqghqAWeG5QG3zd0z8JlAbj/91/47+N4eMe6ADRp3miAiB8AAQAASURBVApd+3xW4Llo6/xIeg2VSgUIMq3ll5HcjOdYUKZ89euPR8TbcQGgXMVKWLZ2GzeAISIi0gIzCwus3LQL4wd/iqjICADAkvl2qFmnHho0aS5tcqQVMTExAJDukhYGHzd0i46OzvQ1JkyYgMWLF0MmS/5Csm/fPowcORKLFi2Cj49Phn3Xrl0LNze3dB+LiIhAWFhYlj+DNkTGxEMdGwmVQob4HHyFEdRqJMQm/n5kBbgJ9KuXLzB/5iSo1YlfJxVKJTx8A2FqbIT46Kw3UqSCIdX5QYVAXBxUCXFQRUfx3CANfN/QPQZKOZb4r8Xkrwcj9uO4adGcGdjyzVFUq1GrQHPR1vkhxMUhQaZEeLgScln+F5QjIiJy3IcFZco3/969A1d7azE2MDTCyo27UapMWQmzIiIiKlpq1W0AN9+1cJw6FkDiJn32U8Zg9w8n+ZlbBJiYmABIv2gcFRUFAFlutFetWrU0bSNGjICnpyf27duXaUF5xowZGD58uEZbUFAQBg8eDFNTU1gU0ObKgl4c5G/VUCgAPaPsT0wQPs4E1jMyhkyuyK/0NMTFxsLZbibev30jts12ckXL9l0K5PiUfVKcH1Q4KD5exJPz3KBU+L6hm+o3a4UFnivg6pBYg4qKjICT7QxsO/QzTApwQ2JtnR+yOCWUhkqYm1tAIc//gnJuNm1mQZnyxbu3b2BvNRox0VFi2yKfVWjQpJmEWRERERVNPfr2x2QbB2xetQIAEPL8GebPnIg1uw/xrqBCrmrVqlAoFAgODk7zWFJbzZo1c/3aJ0+ezPQ5lpaWsLS0TPcxhUIBhaJgvkwrFArI5HLI5DLIc/AFTQ187KfIUb+8WOnujL9uXRfjTwcOw8iJ0zVmiJNukOL8oMJBJkt8vwHPDUqF7xu6a8Dwkbj75018s3MzAODR/XvwmDcby9ZuK7DPYG2dH0mvoVAoCqSgnJvxHOfnk9YlJCRgwcxJeP70P7Ft7NRZ3ISPiIgoH021m4/OPT8V42sXz4m7XlPhZWBggPbt2+PcuXMQBEHjsdOnT6NChQqoX79+jl9XrVbj77//Tnf2MuXe4X07cWjvDjGu06ARXJb5s5hMRERUAOa4eKJZ60/E+PefvsfODaslzKjoYkGZtC7AaxGunP9DjNt37QHredx1noiIKD/J5XJ4+G9AtZq1xbYD2zfi+4N7JcyKtMHe3h6PHz/G9u3bxbZz587h119/hZ2dHeQp1uj7/vvvMW3aNDx48ABAYuH4wIED4lq+ACAIAlxdXfHkyRNMnz69wH6Oou7OzWtYtmiuGJuZW2D5hp0wMjaRMCsiIqLiQ09fH8vWbkPpsuXEtsBl7rh09pR0SRVRLCiTVh37bj/2blkvxlWq14TX6s0FdjskERFRcWZmbo6VG3dprBXn7Wyvcfs9FT6DBw+Gh4cHpk2bhgEDBuCrr75C7969MWHCBNjb22s89/Lly9iwYQNevHgBAJDJZDh27Bjq1KmDAQMGYPTo0WjYsCF8fHywcOFC2NraSvATFT1vX4di7rRxiI+LA5D4e1+yaiOqVKshcWZERETFS9ly5eGzbhsUysRVftVqNRZYT8aLp2mXD6PcY0GZtObOzWtYssBOjI1NTLFy0y6YW5SQLikiIqJipkadevDw3yDGcbGxcJgyBqEhLyXMivJq4cKFCAoKwsiRI9G/f39cu3YNmzdv1pidDACDBg3CunXrULt24kx1mUyGXbt24fz585g+fTp69OiB5cuX48WLF/Dw8OBSDFqQkJCA+TMnIeTFc7Ftqt18dOzeW8KsiIiIiq/mbdrBYbG3GIe9ewuHqWMRE5N2k2PKHW7KR1rx+lUIHKeORVxsrNjm4b8eteo2kDArIiKi4qlr788wxXYuNvr7AABevXwBh6ljsHH/DzAwNJQ4O8qtKlWqYMSIEZk+p02bNmjTpk2a9nLlyuHzzz/Pr9SKtdVL3XDt4lkx7tKrLybNss+kBxEREeW34WMm4q9b13Hs230AgH/u3IK3swNcVwTygroWcIYy5VlcbCzmThuHVy9fiG1T7eahWx9+aSEiIpKK1ey56PZpPzG+c+MaPOfbptnYjYhy75cfDmH3pjViXLVGLbj7rU8zc5yIiIgKlkwmw4IlK1C/cTOx7di3+/Dt7m0SZlV0cKRDeSIIApYudMCta5fFtm6f9sNkG0cJsyIiIiK5XA4Pv3WoXb+h2PbT4W+wY/0qCbMiKjqC7t2Fm6ONGBsZm2D5hh0wMzeXMCsiIiJKYmhohBUbdsKiZCmxbYXbAty6eknCrIoGFpQpT/ZsXouj3+wR45p16sHddy1nZRAREekAYxNT+G3eixKlSottgcvc8cdvP0uYFVHh9yEsDA5TxiImOkpsW7R8FWrXa5hJLyIiIipoFSpXgdfqTWKdKiE+HnOnT+D+InnEqh/l2pnff0GA12IxtihREr6b98DE1EzCrIiIiCililWqYsWGnVDq6QFIvLvI2WYKgu7dlTgzosJJrVZjoe1UBD9+KLaNmWKNPv2HSJgVERERZaRd5+6YOddFjF+/eol5MyYgPi5OwqwKNxaUKVce/Ps3nG0mQ61WAwAUSiV81u9Aleo1Jc6MiIiIUmvRtj0WeK4Q46jICNhNHIl3b15LmBVR4bTe1xtnT/4ixq3bd4b1vEUSZkRERERZGTfNBj0/GyDGt65egp+nSyY9KDMsKFOOvXv7BnYTRyIyIkJsm++5HK3bd5IwKyIiIsrM4K/HYNSk6WL8/Ol/mDt9PGdmEOXArz8ewZbVK8W4XMVKWLpmC5RKpYRZERERUVZkMhkWrwhEjdp1xbYDOzbh6IHdEmZVeLGgTDkSHxcHx6lj8Sz4idg2YuJUDB0xTsKsiIiIKDtsnNzQoWtPMb5+6TyWujhCEAQJsyIqHP69eweu9tZibGBgiJUbd6Fk6TISZkVERETZZWJqhhUbd8HELHmpVi9ne27SlwssKFO2CYIAb2d73Lh8QWxr37UHbJ09JMyKiIiIskupVMJr9WZUq1VHbDuyfxf2b98oYVZEuu/d2zeYYzVKYxM+F59VaNCkuXRJERERUY5Vr1UHngEbIZPJACRu0uc4bRxCXjyTOLPChQVlyrY9m9fi6Dd7xLhG7brwDtzKW/yIiIgKETMLC/hv2QtzixJim6+7My78cVK6pIh0WHx8POZNn4AXT4PFtnHTbPDZ4C8kzIqIiIhyq0vPTzHDcaEYvwl9BYcpYxETEy1hVoULC8qULWd+/wUBXovF2KJESfht3Qczc3MJsyIiIqLcqFqjFpau3QqFQgEAUKvVmD9zIh4/uC9xZkS6x89jIa5dPCvGHbr21NgpnoiIiAqfCTNs0WfAEDG+++cNeM6z5VJw2cSCMmXpwb9/w9lmMtRqNQBAoVTCZ/0OVKlWQ+LMiIiIKLc+6dQNDou9xTgiPBy2E0fg3ds3EmZFpFuO7N+FAzs2iXHVGrXgtXqzeDGGiIiICieZTIZFy1ejXqOmYtvxIwexa2OghFkVHiwoU6bevX0Du4kjERkRIbbN91yO1u07SZgVERERacPwsZMwbPQEMQ5+/BAOU8YgLjZWwqyIdMOtq5fgvdBBjE3MzOC7eQ/MLCwkzIqIiIi0xcjIGCs37dbYYHf1UjecP/WbhFkVDiwoU4biYmPhOHUsngU/EdtGTpqGoSPGSZgVERERaYtMJoOj61K07dhVbLt55SI85s3m7X5UrIW8eAbHaeOQEB8PIPHfimfARtSoXVfizIiIiEibKlSqjOXrd0DxcX8wtVqNBbMm48nDIIkz020sKFO6BEGAx7zZuHH5gtjWoWtPzHZylzArIiIi0jY9PT34rNuuUSj76fA32LRquYRZEUknJiYaDlPG4k3oK7FthuNCdOn5qYRZERERUX5p0bY95rn7iHFEeDjmTB6FD+HhEmal21hQpnRt9F+Gnw5/I8Y1ateFV+AWKD9esSEiIqKiw8zCAgHbDmjc7rfBdymOH/lWwqyICp4gCFgy3w53/7whtvUZMAQTZthKlxQRERHlu2GjxuOL0RPF+PGD+1g4ewpUKpWEWekuFpQpjR+/O4CN/slXZkqVKYuAbQdgZm4uYVZERESUnypVrQbfTbuhb2Agtrk5WuPmlYsSZkVUsHZvWqMxqaJewyZY5LMKMplMwqyIiIioIDgs9kLLTzqI8dmTv2DdSi8JM9JdLCiThuuXL8BzgZ0YGxgYwnfzHlSqWk3CrIiIiKggNG3VFm4r14hxfFwc7KeMwdMnjyTMiqhg/PH7CQR4LRbjEqVKY+Wm3TAyNpEwKyIiIiooevr68Fm3HeUrVRbbtq3xw89Hv5MwK93EgjKJHj+4jwU2UzU2H/HwX48mLVpLnBkREREVlD4DhmKGg7MYv3/7BrYTRyI8LEzCrIjy1/1/7sJ5lpW4GaVCqYTPuu2oULmKxJkRERFRQSpZugx8N+2BgaGR2ObmaI3bN65KmJXuYUGZAADv3rzG7AkjNBYct1ngip6fD5QwKyIiIpLCROs56P/FCDF+8jAITrbTEB8XJ2FWRPnj7etQ2E0agajICLFtgecKtGrXUcKsiIiISCr1GjWBm2/yXXtxsbGwtxqNF8+eSpiVbmFBmRATEw27SSPxPPiJ2DZs1HiMmWItYVZEREQkFZlMhoXefhoFteuXLsB7oYM4g5OoKIiNiYHDlDF48TRYbBs1aTqGjBgrYVZEREQktd79BmPqnPli/Cb0FewmjUBkxAcJs9IdLCgXc2q1GovnzNCYut++aw/Mdffh5iNERETFmJ6+PpZv2ImqNWqJbT8c3Ictq1dKmBWR9giCgCUL7HDr2mWxrWP33pjt7C5hVkRERKQrrGwc8enAYWJ8/++/sHD2VKhUKgmz0g0sKBdzgT4e+O3Ho2Jcq259eK3eDKVSKWFWREREpAssSpREwLb9sChRUmxbt9IL3x/cK2FWRNqxba0/fjx0QIxr1WsAr9WboFAoJMyKiIiIdIVMJsOi5avQuEUrse2P337G6qVuEmalG1hQLsYO7d2OHesCxLiMZTmsWLcVpmZmEmZFREREuqRqjVpYsXEn9PX1xTbP+ba48MdJCbMiypuTx3/AGh8PMS5Zugz8t+yDqZm5hFkRERGRrjE0NMLKjbtRrmIlsW3XxkAc2b9Lwqykp9MF5du3byMwMBD+/v44e/Zsrl/n9OnT8PT0xM6dO7WYXeH2x+8n4O3sIMZGxibw27IH5SpUlDArIiIi0kXN27TDYp8AcTksVUIC5k4bj3t/3ZY4M6Kc++fOn3Cxmy7Gevr6WLFhJypWqSphVkRERKSryliWQ8DW/TAyNhHbvJztce3iOQmzkpbOFpTt7e3Rpk0bXL16Fffv38eAAQPwxRdfID4+Pkev8+LFCwwbNgwuLi7YuHFjPmVbuNy+cRXzZ0yEWq0GAMjlcnit3oT6jZtJnBkRERHpqu59PoOdS/KMzqjICNiM/0pjMzMiXRca8hJ2k0YgJjpKbFu41B/N27STMCsiIiLSdXUaNILX6k0aEywcp45F8OOHEmcmDZ0sKO/duxe+vr7Yu3cvtm/fjjVr1uD06dM4evQovLy8cvRakyZNQpMmTWBpaZlP2RYuTx4GwXbiCMTGRItt8z1XoEuvvhJmRURERIXBiAlTMWryDDF+/eolZo0bjvCw99IlRZRNMTHRsLcajVcvX4ht42fYov+wryXMioiIiAqLLr36wmaBqxiHvX8H24kj8CEsTLqkJKKTBeVVq1ahTp06GDp0qNjWtGlTfP7551izZk22d1Ncv349Tp06hc2bN4tXEIqzN6GvMGvscLx/+0ZsmzTLHsNGjZcuKSIiIipUbJ3d0bv/YDF+FPQv7K1GIy42VrqkiLIgCAJc7Wfir1vXxbZun/bDTMeFEmZFREREhc2YKdYY9OUoMX784D7mzZyAhIQECbMqeDpXUI6OjsaVK1fQvn37NI917NgRoaGhuHv3bpav8+DBAzg4OMDd3R21atXKj1QLlciID7AZ/xWeBT8R2wYMH4Hp9k4SZkVERESFjVwuh9vKtWjRNnmsdv3SeSy2nyEup0Wka9av9Mavx46Icb2GTeDhtw5yuc59HSIiIiIdJpPJsGDJSrRq11Fsu3TmFFa4LoAgCNIlVsCUUieQ2n///Qe1Wo3KlSuneSyp7dGjR2jSpEmGr6FSqTB27Fg0aNAAdnZ2OTr+q1evEBoaqtEWFBQkvm52Z0fnlUqlgqBWQ5ABanXeBroJ8fGYO308/rlzS2xr37UHnJashCCokXS+C+qPx1SrwK+DxRfPAwIAmaCGoBZ4HhDfEwhA2vNAT18PyzfsgNXw/ngU9C8A4JcfDsOyfEXMdnLNxesnvrZKpQKE/L+rrKDGc6Qbjn33Dbau8RPj0mXLwXfLHhibmEqYFRERERVWevr68Fm/A+MG9cbTJ48AAAd3bUHl6tUxfMQYibMrGDpXUI6KStwgw8DAIM1jhoaGGs/JyLJly3DlyhVcvXoVCoUiR8dfu3Yt3Nzc0n0sIiICYQW0LkpkTDzUsZFQKWSIz8OfSRAELHF2wMU//ie21W/UBB4rVkNIiEN8Qlzyc9VqJMQmrq0s42yNYovnAQEA4uKgSoiDKjqK50Exx/cEAtI/D4wN9LFi3VZMGTEEb14nXozfvWkNypQpgy/HTMjZ68fFIUGmRHi4EvICWKYsIiIi349BuuHS2dNY5pZ8R56BoRH8tuxB+YppJ68QERERZVeJkqXgv3Ufxg/pg4jwcACAv+ciWJYti14DhkmcXf7TuYKykZERACA2nXX4YmJiAADGxsYZ9r958yZcXV0xb948NG3aNMfHnzFjBoYPH67RFhQUhMGDB8PU1BQWFhY5fs3cEPTiIH+rhkIB6Bnp5fp11q3wwk9HvhPjSlWrw3/bfliUSbtJoaBOnK2jZ2QMmTxnhXgqOngeEAAo9PUBAHKeB8Ue3xMIyPg8qFq7HgK278eUrwYiKjISABCw1B1lK1RCnwFDsv36sjgllIZKmJtbQCHP/4KyqSlnphYHQffuYt6MCVB9XNNQJpPBa/UmNGrWUuLMiIiIqCioUbsulq/bAetxw6FKSEjcs2GuLcpXro6mrdpKnV6+0rmCctWqVSGXy/H8+fM0jz179gwAUK1atQz7HzlyRFy/z9PTU2yPjIxEcHAwPD090bFjR3Tv3j3d/paWlrC0TFtsBQCFQpHjGc+5pVAoIJPLIZPLIM/lF/hvd2/TuL2vRKnSCNx5EGXLVUj3+Wrg4zEVuT4mFX48DwgAZLLE9x/wPCj2+J5AQObnQYMmLeCzfgdmT/haHEgvtp8Ji5Kl0L5Lj2y9ftJrKxSKAikoF9R4jqQTGvISsyd8jcgPH8S2OS6e6NbncwmzIiIioqKmbaeuWLjUH24O1gCA2JgYzLEaje1HfkHlqtWlTS4f6dy9q8bGxmjZsiUuXLiQ5rHz58+jdOnSaNSoUYb9u3Tpgvnz50MQBMTExIj/EwRBbIuPj8/PH0EnnDz+A5a5OIqxoZExArbtR9Ua3KCQiIiItKt9lx5wXR4oxgnx8XCcOg53bl6TMCsqrqKjImE3aSRePnsqtn01bjJGTJwmYVZERERUVA0cPhJWs5NrcO/evIbN+K8Q9v6dhFnlL52boQwAM2fOxIQJE3Ds2DH0798fAHD37l38+OOPcHR0hFKZnPbp06dx5swZjBs3DlWqVEGPHj3Qo0fa2TCbN29G1apVNWYtF1WXz/0BJxsrcaa2QqHA0jVb0bh5K4kzIyIioqLq86Ff4t3b1/D1WAggsahnM/4rbPn2J9SoXVfi7Ki4UKlUcJplhb9v3xTbOnXvBTsXT8gKYH1uIiIiKp6m2s3Hs/+e4KfD3wAAnjy4D8epYxG481vop7NPXGGnczOUAWDcuHGYOnUqvvzyS0yfPh329vbo2rUrevbsiUWLFmk899dff4WLiwuePHkiUba65e6fN2BvNRrxccmb7Tl5+aJzzz4SZkVERETFwajJMzB+hq0Yh717i5ljhiHkxTPpkqJiZaW7M/747Wcxrt+4KVx9ArjMCREREeUrmUyGhUv90LJtO7Ht2sVzcJ9rA0EQJMwsf+jkDGWZTIb169dj4sSJOH36NBISErB371706tUrzcyCbt26AUhcezkzdnZ2KFWqVH6lrBMeBf2LWeO+RFRk8s7lNgtcMfjrMRJmRURERMWJ9VwXvHsdiqPf7AEAhDx/BusxX2Dztz/BokRJibMr3J48eYIzZ84gISEBrVu3RuPGjXP1OlevXsXFixdRsWJFDB06VMtZSmfv1vU4sH2jGJerWAl+W/bA2MREwqyIiIiouNDT14dXwAZMG/0FHj+4DwA4fuQgKlWthun2ThJnp106WVBO0rZtW7Rtm/muiL169UKvXr2yfK158+ZpKy2d9PL5U8wcMwzv374R28ZOnYVx02wkzIqIiIiKG5lMBidvP7x//w6nf/kJAPDw/j3YTvgaa/ccgpExi3u54eHhAU9PT3z22WcwMTHBzJkz8fXXX2PTpk2Qy7N/0+GbN28wYMAAvHz5Eh07diwyBeVTv/wEX3dnMTYxM8Oq7QdQxrI84qMjJcyMiIiIihNzCwv4b9uHiUM/w9vXoQCAzatWoFKVahj45SiJs9MenVzygnLm3ds3mDl6GEKeJ99OOujLUbBZ4CpdUkRERFRsKZVKeK3ehBZt24ttf16/gnnTJxSLzZG17fDhw1i0aBE2bdqEI0eOYM+ePfj999+xY8cOrFy5MkevNW3aNFSvXh1ly5bNp2wL3p/XLsNplpV4O6lCqYTPuu2oXa+hxJkRERFRcVSpSjX4bdkLA0Mjsc1zgR0unT0lXVJaxoJyIRcZ8QGzx38lTqUHgG6f9oOTtx83HiEiIiLJGBoawW/zXtRp0EhsO3fqN7g7zhI3Dqbs8fX1RY0aNTBmTPIyZu3atUPv3r3h5+eX7d/nrl278P3332PLli05mtWsyx4/uA/bSSMRGxMttjktWYl2nbtLmBUREREVd42bt8KSVRvF2pwqIQGOU8fi3l+3Jc5MO4rGSLKYiouNhcOUsfjr1nWxrVW7TvBatQlKpU6vZkJERETFgJmFBVbvOIhKVaqJbT8d/gYr3Z2K5OYk+SE2NhYXLlxAx44d00wW6NKlC168eIF//vkny9cJDg7GrFmz4OTkhIYNi8bM3dCQl7Ae+wXC3r0V2ybbOHD/ECIiItIJ3T/thzkunmIcGRGBWeO+xLP/nkiYlXaw6lhIqVQqLJw9BZfPnRbb6jduBt/Nu2FgaChhZkRERETJypYrjzW7v8PEYf9n767DmzrbMIDfSeotbZECxaF4cbdhxZ0BQ4e7Dxm6MYZtQzaGuwyH4s5gFPnQ4Tq8xaWl1CV5vz9KTpMmaVM9abl/18XWvOe85zxHkjx5jsXeR27LmuVwyuSc4R5OkhqePXsGtVqNvHnzGgzTtj1+/DjeIrEQAr169ULevHkxcWLi1vnbt2/x7t07vbaHDx8CiMlH1Wp1oqaXVGq1GkKjgVAAGo0SwUFBGN7zG7x67ieN0/qbrug/8ntoNLExCc3nfho1eF48xcX9g0xRCA2ERnDfIAP83KD4GNs/OvXqj1cv/LBp1VIAwId3bzC0e3us2nEArlmymphOzDTUajUgUv/uA0nJ51hQToeEEJg5cRSOH9onteUvVBgL1m2DUyZnGSMjIiIiMpS3QCEsWLcd/Tu1REhQEICYh5M4ZXLGt/2HyhydZQsNDQUA2NvbGwxzcHAAAAQHB8c7jT///BP//PMPzp07B2tr60TNf/HixZg6darRYcHBwQgMDEzU9JIqJDwKmogQqFUKhEZqMHZgL/x355Y0vHrtehg9eSqiw0P1+gmNBtERMbfDUGSQ23xQyuH+QSZFRkIdHQl1WCj3DdLDzw2Kj6n9Y/B33+Ptqxf4+2BMHc/3ySOM7NUJf67eBPvP+ZzedCIjEa2wwqdPVlCmwe1sE8oljWFBOZ0RQmDuzxOxe8tfUlv2nO5Y9Jc3smTLOA9XISIiooyleKkymL9mK4Z0ayfd7/aPGT/CMVMm1GjWUeboLJfd5yvPIiIiDIZp24wVm7Xu3buHCRMmYMSIEahSpUqi5z948GB06NBBr+3hw4do06YNnJyc4OLikuhpJoWwjoTSXwOFQoOZk0bg8vmz0jDPshXw65I1sHdwNOz3+Wxla3sHKJSqNImV0g/uH2SKysYGAKDkvkFx8HOD4hPf/jF13hJ8/PgRl/93GgBw+8Y1TPl+JGYvW2dw21pFpBWs7Kzg7OwClTL1C8pOTk6J7sOCcjqzePYMbF69THrt4poZi/7yhnsew8sgiYiIiCxJ+crVMGf5enzXpwuio6IAADMnjMJYhS1af90hgd5fpjx58kChUODVq1cGw7Rt+fLlM9l/69atiIyMRM6cObFw4UKpPTQ0FK9evcLChQtRsWJFVK9e3Wj/7NmzI3v27EaHqVQqqFRp82NapVJBoVRixbwZOLLHW2rPV9AD89dsgaOT8av0NIg5Q0ihVEHJH/4UB/cPMkWhUEKhVADcNygOfm5QfOLbP+zsHTB3+V/o900L6SqrMyeO4pfJY/HDr/P1npWhnYZKpUqTgnJS8jmen5+OrF44D6sXzZNeO2bKhEUbvFGoaHEZoyIiIiIyX406Xpj55wooP18GKITA3EnD8b9/jsocmWVycnJCmTJlcPHiRYNh58+fh4uLCzw9PU32r1y5MgYOHAhfX1/cu3dP+qdWqxEWFoZ79+4Z3CPZUu38awW2rlokvc6SzQ0L1m9H5qzZZIyKiIiIyDxOmZzx59pteieF7tm6AUvnzZIxqqThGcrpxKbVS7FoduyTIe3sHfDnmq0oUbqcfEERERERJYFXs1b4cfYC/DR6CABAHR2NScP6oLyHO7zq15M5OsszYMAADB48GCdOnED9+vUBxDyIb9++fRg8eDBsPl+aDQAXLlzApUuX0K5dO7i7u6NZs2Zo1qyZwTR37NiBQoUK6Z21bMl2ee/A0l9/lF47ODrhz7VbkSdfAfmCIiIiIkoktxw5sXD9DvRu1xSBAf4AYp4t4pbDHe279ZI5OvOxoJwO7Nq8HnOnxj6R28bWFr+v3IhylavJGBUREZHlE0LE/F96bTg8TpM0jjRE/3+x48cZz3DaOtMwmLZ+fHEmaTRO4/M2jFXo9dOdtuk4405Df9oANBqIyDAgVAFIZxbrj6d7MZ6I8zruSEIApeq0Qp/v/bHqtx8AAJER4fh+7BhcunRJOnuZYvTv3x8+Pj5o27YtBg8eDAcHByxfvhwVK1Y0eGDevn37MGPGDJQrVw7u7u4yRZyyTp48iYF9e0nvA5WVFX5bupYnVhAREVG6VMCjCOav3owBndtIzxb59YexyOqWHfUaN5c5OvOwoGzhju3bgVkTvpNeq6ys8NuSdahSq46MURERfZnMKU7qDkec8UwW/XQ66RYg9YqBRobrtxsWJ03FmqjipEYDTWQkFFFWgEK/yGdWIdVUMTZOvPEVJ3XHkW4tZqRAqftaoVPR1PZRfG4w9qBkhe5/FbpthuMrFLHTMjW9mHaFwTSkljjzQJzhyrjDlQq913oxxI1PZ/7Gl8P4cmpruAoo9J4mrVQAGo0aUeECtvaO0j3WlDFBQAFF7LRMrGtTMZQbMxIuqijMm/UzSpQshQMHDrCYbIRKpcKWLVtw9OhR+Pj4IDg4GPPmzcPXX39tcM+7atWqYciQIciVK1e80+zduzfc3Cz/gc5+fn5o06YNIiMjpbZhP81BvtLV8SIgLJ733+c/NAIiMhKKSCvpaeu6feK+B2P+VMRtijNcOw/DmRudtt5ww2kb/0yK5/PKSKy6E9WbtpFgEv4s1I/V9LxNj0dERETxK12hMn5ZtAqj+3WDRqOBRqPBpGH9sGiDN9yLlZc7vASxoGzBzvx9EL+MHy794FYqlZjx53J85dVI5siIKKXEPTvSnAJlSp49qTcdnTbbKDU0URqEh0ZBoVTHDjc3Tt0ip270qXX2pE4Mhv1imHP2pEDMj2WjxUnd1zpFs/h/kBsW1YwVJ7XTgpHhseMZLw7qFhWNFScV2pe6w40UJ+NOS0to1FALFaxsVVCqrIz2Q5z1oIB+cTJmlnEKpXFea4dLy5nAutaPQb+4m1BB1dS6VihM94tbQDZVCDYsoJq3rvX3Ef09KG7xyNx9KL4YElsAUqvVCAxUwsXFJcUfwjZnxk/wyJMDnTp1QpYsWVJ02hlNo0aN0KhR/HlgixYt0KJFiwSnNXPmzJQKK1XlyZMHo0aNwpQpUwAAQ77/AV26doNGEzuO8e+VmD80ajWiFTFPSofOw3H0DmjFnamR7ybdacf9rjOIwUhf7Z8aYbxT3O9AUwffjInv+9z4eIZtuq3G5pbg97V22gr911qmDgbqzUOnWQghfU4Z62K80G6kWA/jL6TPYKEBIiOACJXBAQeFkSBNHjCI+zlsdJhhrGZNO04fcw9GmBxubPkSeTDC2LR5MIKIKPFqN2iCiTPnYfr4kQCAiIhwjOzTGb+s3onyZcvKG1wCWFC2UMeOHMaM0QOgUccWcqbMWYiGzdvIFxRZvJQ4e1Jo1IiO1kATpYZCabo4GXf65pw9qRtDnMkm+exJU9PQHS8pZ08mpjhplM6IepNS6BQnta8TKnLF+cucsye104svT4/v7Elrzedit4DeLzxzzp40OHPKxPIk5+zJmFkq9OanUCig+wBcpUKhX5iLZ12bG0N8hcbEFCfji0GvGBznx1dChUZjxcnExGDwNxTQaNT49MkaLs7OUKlUpqfFH4WURAqFAoMHD5Y7DLJQCoUCP/74I3Lnzo1bt25h3i9T4/28kXKIz9+1arUagZ+s4Ozsonf2e7wHamGYOyQ0nqkCsNEcJZHTSehArLHCdkIFYGP5TnzLnKjpGJ2e6b667UbXie4wI4V5jTSekTYIo9PWaGImJTRqRIVHwdrOBlBoDzgkfvvqMXP7Gtl8cf6Ok6vqnoSQUF5uJFajocbZvqZGNz5tvTEM4o/b3+T7S3faRg5IGByMAKQkRHvlkk6qqj9fwy7xHozQHc85JApQR+GTfxgUqs8HG8ws1scdN+7wBIv1ep3ixm/8889g2nrDjE/b8DdH0g9G6MVodFjSD0bozzv+8YjSo7adu+P92zfSg/mCP33CDwO7YNnWA0ChrDJHZxoLyhbo5MmT6N6lI6Kjo6S2CTPmokW7Tmkei24ia6o4qR0e265ti79gFzcRkv3ekwbTjjMNcwupJpbXWIEybpKjezZGnJnHjqQ7f+0Xa9zXZhUo4xTatLPSaKCIikmgFEq1xZ89qT9ebD/9ONPm7Enta8NEzfi6Ts7Zk3HnkdJnT4ZkdUR4uBKZ82f+XEQ0nuzprusE405MkZZJosVQqwWslApYqZRQqXgrAiKSR58+fcwaL+73tBAxt3BRKRX8DCMDMVdfWMHFJfaAg7kFc91xeTACBizpYITufMw9GKF0tEZ0hAZOrnaxPxwA6eoI4+tW/oMRxq6MMHYwQnfclDgYEbfN+HiGbdL2TWA+Rt9jutM2cuAhvoMRCYn3YIRGA0TFPFdCoXPVlsJIn7i/xfSmozu+mQcjEpy2XicjccVz9UByDkboxmH28hlbJiRh3ZmIN26fhMaTW9/hYxDw4T22rlsBAAh4/xbf9eqAmufPIZd7TpmjM44FZQvj6+uLFi1aIDw8XGrrPepHVG3WEX7+ofEWJwGDuqNpOiPGfiDHfDAqQhSQ7pOpMLc4GftXYs6eNDY8drwUuvekTkEvbqExtc6e1C3w6RYoY//WjpdwcTIxMZhTaIz7d9zlFBo1QoNVcHRyhvJzIVF3eHwFSlNnT8Z3abexGBJz9qTZ07LALw1LprFRAWolXOytU/zydiIiIiJLEzcvjzM0TWMh+flncURYmAI582dJMBeOe2UEYP6BguQcjNCfXsoejDA1nS/5YIT2LH6NRo3wUMDG3hEKneeLaHT6aHT6SG0WfjBCrz8PRhi0mX0wQnvAIURh9NkNsfMwLNZ3HP4DXr5+i9NH9gAAXvg+RZs2rXH+3DmLfMYIC8oWJl++fJg4cSImTZoEABjw3QT0HTYiVe49Gbc4KTRqhIeqYO/g9LmQaH6RT3+epouaiSlO6i2PsRjiFBn152lYQObZk+ZRq9UIRCRcXOxYSCQiIiIiIopH/AcjAB6QyFhirmxQpNhzJeIekDBWSDZehE5MQTbheZicTyIPaiQUq/404x8vvivUk3Mwwli8xorVxmLQbzMcT6NRIzxEAVsHJ+mAg9ErI3Q66R6M+PXPpRjV9xPOn/4H9g6OmPbzzxZZTAZYULZIEydOhKOjI169eoVZs2akWfEy5oNRDRcXJxYSiYiIiIiIiIhSEa+OyFhi6moCLi7OSa6rHTu4F506dcIPP/yAqlWrpnCEKYcFZQs1YsQIuUMgIiIiIiIiIiKiNOLk5IT9+/fLHUaCLPO8aSIiIiIiIiIiIiKyOCwoExEREREREREREZFZWFAmIiIiIiIiIiIiIrOwoExEREREREREREREZmFBmYiIiIiIiIiIiIjMwoIyEREREREREREREZmFBWUiIiIiIiIiIiIiMgsLykRERERERERERERkFhaUiYiIiIiIiIiIiMgsLCgTERERERERERERkVlYUCYiIiIiIiIiIiIis1jJHUB6EBERAQB4+PChzJGkLrVajeDgYDg5OUGlUskdDsmE+wEBQKCfH8LDw/Dh9m3uB184fiYQkPH2A21Op83xKHHSU26c0fZdSlncP8gU5sJkCj83KD7pdf9ISm7MgrIZ/Pz8AABt2rSRNxAiorQ2caLcERARpRo/Pz9UqFBB7jDSHebGRPTFYC5MRF+QxOTGCiGESOV40r2PHz/Cx8cHefPmha2trdzhpJqHDx+iTZs22L17NwoXLix3OCQT7gcEcD+gWNwXCMh4+0FERAT8/PxQp04duLq6yh1OupOecuOMtu9SyuL+QaZw3yBTuG9QfNLr/pGU3JhnKJvB1dUVrVu3ljuMNFO4cGF4enrKHQbJjPsBAdwPKBb3BQIy1n7AM5OTLj3mxhlp36WUx/2DTOG+QaZw36D4pMf9I7G5MR/KR0RERERERERERERmYUGZiIiIiIiIiIiIiMzCgjIRERERERERERERmYUFZZK4ublhypQpcHNzkzsUkhH3AwK4H1As7gsEcD+g9Iv7LsWH+weZwn2DTOG+QfH5kvYPhRBCyB0EEREREREREREREVk+nqFMRERERERERERERGZhQZmIiIiIiIiIiIiIzMKCMhERERERERERERGZhQVlIiIiIiIiIiIiIjKLldwBkLzCwsJw6NAh7Nu3D0FBQfjtt99QqFAhs/trNBps3rwZ//zzD6Kjo1GxYkX06dMHDg4OqRg1pYa9e/fi4MGDCAkJQenSpdG3b19kyZIlwX4nT57EwoULjQ6bNWsWihQpktKhUjL5+PjA29sb/v7+KFq0KHr37o08efKkWX+yDP/++y82bdqE169fI3/+/OjZsyeKFi2aYL/Xr19j6NChRof16NEDLVu2TOlQKRXdvn0bW7duxZ07d1CrVi2MHDkyUf3v3buHdevWwdfXF+7u7ujatSvKly+fOsESmSk6OhonTpzAzp078f79e3z33XeoWbNmoqZx4MAB7N+/H8HBwfD09ETfvn2RLVu2VIqY0tKDBw+wdu1aPH36FDly5ECXLl1QqVIls/q2b9/eaHujRo3Qv3//lAyTUklYWBjWrFmDixcvQqVSoW7duujSpQtUKlWa9CfLdvz4cezatQsfP35E8eLF0adPH7i7uyfY79q1a5g+fbrRYePGjUPlypVTOlRKY48fP8a2bdvw77//onjx4pg2bVqi+vv6+mL16tV4+PAhsmTJgg4dOuCrr75KpWjTBs9Q/oJ5e3sjf/782LhxIwICAqQCkbmioqLQvHlzjB49Gp6enqhZsyaWLVuGypUr48OHD6kYOaW0nj17olu3bsifPz+8vLywe/dulClTBk+ePEmw79OnT+Ht7Y2aNWuiU6dOev+yZs2aBtFTYkyZMgUNGzaEk5MTGjdujAsXLsDT0xOXL19Ok/5kGZYtW4aqVasiMjISTZs2ha+vL8qUKYNDhw4l2Dc4OBje3t7ImjWrwXu+WLFiaRA9pZTSpUujY8eOUCqV8Pb2xvnz5xPVf8+ePShbtiyeP3+Opk2bIjw8HFWqVMHatWtTJ2AiM1y8eBF58+bF3LlzERERAW9vb/j5+SVqGgMHDkTHjh2RO3duNGjQAAcPHkSpUqXw4MGDVIqa0srhw4dRunRpPH78GE2bNoVGo0H16tWxbNkys/p7e3sjMjLS4PuPxaL0ISAgAFWqVMGff/6JGjVqoGzZshg/fjyaNGmCyMjIVO9Plm3cuHFo3rw5smTJgsaNG+P06dPw9PTE9evXE+z7+vVreHt7w9PT0+DzIXfu3GkQPaWmpk2bolGjRvj48SOOHTuGf/75J1H9z507B09PT/z7779o3LgxHB0d4eXlhZkzZ6ZSxGlE0BfLz89PBAUFCSGEmDVrlgAgLl26ZHb/P/74QwAQ58+fl9revn0rnJ2dRZ8+fVI8XkodO3bsEADEjh07pLaQkBCRL18+0bhx4wT7r1mzRgAQN2/eTM0wKQVcunRJKBQKMWfOHKlNrVaLKlWqiBIlSgi1Wp2q/ckyPH36VNja2oqRI0fqtbdp00bkyJFDBAcHx9v/wYMHAoBYsmRJaoZJaeDu3btCCCGioqIEANGxY0ez+wYGBoqsWbOK9u3b67UPHTpUODg4iBcvXqRorETmevv2rXj//r0QQojt27cLAGLz5s1m99+/f78AIDZs2CC1hYeHi8KFC4s6deqkdLiUhoKDg0WOHDlEixYt9NrHjBkjbG1txdOnTxOcBgAxevTo1AqRUtnAgQOFo6Oj3nfU1atXhUKhELNnz071/mS5Tp8+LQCIhQsXSm3R0dGiXLlyoly5cgn2P3TokAAgjh07lpphkky0ObMQQuTIkUPUrFnT7L7R0dGicOHCokaNGkKj0Ujtv/zyi1AqleL69espGmta4hnKX7A8efLAyckpyf1XrlyJsmXLomrVqlKbm5sbvv76a2zatAmhoaEpESalspUrV8LNzQ1t27aV2hwcHNC9e3ccOXIEvr6+MkZHKWnVqlVQKBTo27ev1KZUKtG/f3/cvXsXZ8+eTdX+ZBk2bNiAiIgIDBgwQK994MCBePPmDfbu3StTZJTWihcvnuS+u3btwocPH4zuR6Ghodi4cWNywyNKEjc3t2RdIbVy5Uq4uLigU6dOUputrS169uwJHx8fnqWcju3fvx9v3rwx+rkVERGB9evXyxQZpYWIiAhs2LABLVq0QK5cuaT2cuXKoWrVqlixYkWq9ifLtmrVKtjY2KBXr15Sm0qlQt++fXHt2jVcunRJxuhIbsnJmU+ePImHDx+iX79+UCgUUrv2NkmrVq1KdnxyYUGZkiQ4OBi3b99GxYoVDYZVqlQJYWFhZl0aQvK7cOECypcvD6VS/+NAey+5CxcumDWdWbNmoXPnzhg4cCDWr1+P8PDwFI+VkufChQsoXLgwXFxc9Nq12zqhy92T258sw4ULF+Dg4GCQGCV2O3p7e6Nr167o1asXfv/9d7x79y7FYyXLpf1uiJsHeHp6wt7enp8HlG5duHABZcuWNbgfKr/r0j9Tn1seHh5wdXU1e9ueOnUK3bt3R/fu3TFjxgw8ffo0pUOlVHD79m0EBweb/P3633//ISAgINX6k2W7cOECihcvbvAsqMR+9i9evBhdunRBv379sHz5cgQHB6d4rJS+mPruyZw5MwoVKpSu8wo+lC8D2LJlC3bs2GHWuEuWLIGbm1uy5/ny5UsIIZAjRw6DYdq2Fy9eJHs+ZL5jx46Zff+36dOno3jx4ggNDUVAQECyt2OFChVQokQJ5M2bF7du3cLgwYMxa9Ys/P3337xnlAV5/vy50aOr5m7r5PYny/D8+XOj7/msWbNCpVKZtR1z586NihUrokSJEnj+/Dl+//13/Pzzz9i9ezfq1KmTGmGThXn+/Dmsra2ROXNmg2HZs2fn5wElW3wP/Y1rypQpKF26dLLnqdFo8OrVK9SqVctgGL/rLEdUVBQ6d+5s1rgNGjTAwIEDAcR8bgExn1Fx5ciRw6xtmzlzZlSvXh3lypXDhw8fsH79ekybNg2rV69Gly5dErEUlNa02z+h3z3GvtdSoj9ZtufPn6NatWoG7Yn57C9evDjKlCmDggUL4sGDBxg/fjymT5+Oo0ePJusMV0rfEvrsSM8HJVlQzgBu3boFb29vs8adM2dOihSUtQ8dsLIy3IWsra0BgGeoprFHjx6ZvR+MHDkSQMpsxxYtWqBHjx56l29069YNVapUwYABA7B//36zYqLUFxkZmaxtndz+ZBlMbUcgZlsmtB1z5cqF+/fvw9HRUWobNmwYypUrh06dOuHx48ewt7dP0ZjJ8iR3PyJKiPahv+YYOHBgihSUmd+mD2q12ux9w9XVVfo7MjISCoXC4OxzwPzPrfv37+v9lho5ciTq1q2L3r17o3bt2siTJ49ZcVHaS+77m58PGVtyf+dUrVoVN2/e1JtGv379UKZMGXTr1o0PMP+CJfTZkZ4/N1hQzgA6d+6McuXKmTWusSPySaG997Kx+ySHhIQAADJlypQi8yLzNGrUCNu3bzdr3BIlSgAAHB0doVAokrUds2XLZtBWtmxZNG3aFPv370doaKjBpUMkDycnp2Rt6+T2J8vg5OQkHSnXpVarERERkeB2NPZ+dnZ2xuDBgzF27Fj873//g5eXV4rFS5bJyckJERER0Gg0BrdMCgkJQc6cOWWKjDKKevXqmZ3XlClTJkXmaWdnBysrK37XWTgbGxuz942CBQtKfzs5OUEIgfDwcNjZ2emNFxISYta9t+OemGNlZYUxY8agbdu2OHDggMH9mclyJPf3K3//ZmzJ/Z1j7Mz0/Pnzo3Pnzli2bBmePXuG/Pnzp0ywlK7ofnZkyZJFb1hISEi6/txgQTkD8PT0hKenZ5rOM2/evLC1tcWzZ88MhmnbihQpkqYxfekKFSqEQoUKJaqPtbU18ufPnyrbMWfOnFCr1QgMDGRB2UIUKVIE9+7dM2g3d1sntz9ZhiJFiuDy5cuIiIiAra2t1O7r6wshRLLe8wDg7++fInGSZStSpAg0Gg38/Pz0fiCFh4fjzZs3aNKkiYzRUUaQP39+WX58Fy5cmPmthVMqlWjfvn2i+2m33bNnz1CsWDGpPTo6Gi9fvjR6ubs5+P2XPuhu/7iePXsGKysrvQMQKd2fLFuRIkVS7TcxEPP5wILyl0n3syPuVSzPnj1D2bJl5QgrRfChfJQkKpUKdevWxZkzZ6BWq/WGnThxArlz55bOgiXL5uXlhatXr+LTp0967SdOnICDgwNq1qyZpOleuXIFzs7OKXKLFUoZXl5eeP36Ne7fv6/XfuLECQBAw4YNU7U/WQYvLy+o1WqcOXNGrz252/HKlSsAYooxlPFpz0I/efKkXruPjw80Gg0/Dyjd8vLywq1bt/Dhwwe99hMnTsDGxob3iU/HTH1unT17FhEREfz+y+A8PDxQoEABg+0vhMDJkydRu3Zt2NjYpFp/smxeXl54+vSpQVH5xIkTUCgUSb767sqVK1CpVChQoEAKREnpkanvntu3b+Pt27fpO2cWREKIWbNmCQDi0qVLRodv3bpVtGvXTty6dUtqO378uFAoFOK3336T2o4cOSIUCoVYsGBBqsdMKeP27dvC2tpajBgxQmq7evWqsLOzE+PGjdMb9++//xbt2rUTp06dktqWLl0qPn78KL3WaDTS/jRp0qRUj5/M9/btW+Hq6iq+/vprER0dLYQQwtfXV7i5uYmOHTvqjXvjxg3Rrl07sWPHjiT1J8sVFhYmChQoIGrWrCnCwsKEEEL4+/uLIkWKiJo1a+qN++LFC9GuXTuxbNkyqW3btm3iyZMneuMdPXpU2Nraijp16qR2+JQKoqKiBACT7+OwsDDRrl07MWvWLKlNo9GISpUqiWLFiomAgAAhhBChoaGievXqokiRIiIiIiItQieK1/bt2wUAsXnzZqPDT506Jdq1ayf+/vtvqe3BgwfC1tZWDBgwQGq7deuWcHBw0MuVKH2qWbOmKFSokHj//r0QQojw8HBRp04dkT9/fuk7UQgh1Gq1aNeunZg6darUdvjwYXH9+nW96V29elW4ubkJDw8Pvf5kmRYvXiwAiL1790ptCxYsEADE0aNH9cb9/fffRbt27aTvuMT2p/TlxYsXIlOmTKJz585CrVYLIYR4/PixyJw5s+jRo4feuJcvXxbt2rXT2w9Wr14t3r59qzfeqlWrBADRp0+fVI+f0k6OHDkMfjPp6t69uxgzZoxeW6tWrUSOHDmEn5+fEEKI6Oho0bp1a5EtWzbh7++fqvGmJhaUv2CPHj0S7dq1E+3atROlSpUSAET9+vVFu3btRPfu3fXGnTJligAg/vnnH732ZcuWCQcHB1G1alXh5eUlbGxsDN48ZPl27twpMmfOLMqUKSMaN24s7O3tRffu3UVkZKTeeCtWrBAAxF9//SW1zZo1S+TPn19UqlRJNG3aVBQoUEA4ODiICRMmSF/GZDlOnz4tcufOLYoUKSKaNWsmnJ2dRZMmTURgYKDeeMeOHRMAxLRp05LUnyzbrVu3RLFixUTevHlF8+bNhZubm6hatap4+fKl3nh3794VAPQKK7t27RKlSpUSnp6eolmzZqJMmTJCqVSKDh06iA8fPqT1olAy/Pjjj6Jdu3bi66+/FgBEnjx5pLxAt8AWFBQkAIjmzZvr9ff19RWVKlUS2bNnFy1atBB58uQRJUqUEPfu3UvrRSGShISESPtxtWrVBABRtWpVqe3Tp0/SuH/99ZcAIFasWKE3jX379omsWbOKUqVKiSZNmggHBwfRqVMnHijJAF6+fCmqV68usmbNKlq0aCHy588vihYtqnfSjBCxB9q8vLyktrNnz4pq1aqJIkWKiCZNmojKlSsLlUol6tWrZ3CglSzX+PHjhY2NjahXr56oXr26sLe3F4sWLTIYr2PHjgKAePXqVZL6U/pz4sQJ4e7uLooVKyaaNWsmnJycRMuWLUVwcLDeePv27RMAxOzZs6W2pUuXCg8PD1GuXDnRrFkzUaRIEWFjYyMGDRokwsPD03pRKIXNnz9fyiNsbW1F1qxZpddbt27VG9fFxUWULVtWry0gIEA0aNBAuLi4iGbNmonChQuLfPnyifPnz6fhUqQ8hRBCpOkp0WQxAgICcPz4caPDbGxs0KpVK+n1nTt3cOfOHdSpU8fgFgafPn3ChQsXEB0djXLlysHd3T1V46bUERYWhvPnzyMkJASlS5c2eo+np0+f4vLly6hSpQry5csntUdERODOnTvw8/ND5syZUaFCBTg6OqZl+JQIUVFRuHDhAgICAlCkSBEUL17cYJy3b9/i1KlTKFWqlMFwc/qT5dNoNLh8+TJev36N/PnzG71/V1BQEI4cOQIPDw+UL19eahdC4L///sPDhw9ha2uLMmXKpNhDXynt+Pj44N27d0aHVapUSbo8U61WY9euXXB3dzd6G6Rr167Bz88POXPmRMWKFQ0e0keUlqKiorBnzx6Tw1u1aiVdlu7r64uLFy/q7e9a4eHhOH/+PIKDg1GyZMlEP6eCLNuNGzfw7NkzZM+eHZUrVzb43BJCwNvbGzly5MBXX32lN+zJkyfS7b9KliyplxNT+vDmzRvpVgRVqlSBq6urwTgXLlyAn58fWrZsqffMCXP7U/oUGRmJCxcuIDAwEEWLFkXRokUNxnn9+jXOnDmDMmXK6A2Pjo7GnTt38OzZM2TKlAnly5eHi4tLWoZPqeTSpUtG77ENxHwPlCxZUnq9d+9eODg4oEGDBgbj3r17Fw8fPkSWLFlQtWpVWFml78fasaBMRERERERERERERGbhKSREREREREREREREZBYWlImIiIiIiIiIiIjILCwoExEREREREREREZFZWFAmIiIiIiIiIiIiIrOwoExEREREREREREREZmFBmYiIiIiIiIiIiIjMwoIyEREREREREREREZmFBWUiIiIiIiIiIiIiMgsLykRERERERERERERkFhaUiYi+IJcvX4ZCocDu3bu/yPkTEREREWnJnZvKPX8ioqRiQZmIKIM5c+YMFAoF9u/f/0XOn4iIiIhIS+7cVO75ExGlBiu5AyAiorRTqVIlCCG+2PkTEREREWnJnZvKPX8ioqTiGcpEREREREREREREZBYWlImITMiZMyd69uyJ8+fPo1q1arC3t0ehQoUwd+5ck+NeunQJtWvXhr29PcaPHw8ACA4Oxrhx4+Dh4QFbW1vkyJED/fr1w4cPH6T+z58/h0KhkP7Z2tqiRIkSmDFjBqKjo/XmFRwcjAkTJqBo0aKws7NDoUKFMGrUKHz8+BEbNmzAV199BQBo2bKlNL05c+YAMLxP29OnT6FUKvHTTz8ZLNPbt29hY2ODcePGmR1jYuev9eLFC/Tq1Qs5c+aEra0tPDw8MHnyZEREREjj/P3331AoFPj777+xZMkSFCxYEHZ2dqhevTouX76c4Pbs3r077O3tceXKFb32b775Bo6Ojrh161aC0yAiIiL6UjE3Zm5MRCQRRERkVI4cOYSXl5do3ry5uH37tvjw4YOYP3++UKlU4scffzQ57s2bN8WLFy/Ezp07RUhIiKhQoYIoUKCAOHTokPj06ZO4fv26qFSpkihVqpQIDQ01Ou8PHz6ITZs2CScnJzFlyhSpPTg4WJQrV07kzp1b7Nq1S/j7+4unT5+K33//XSxatEgIIcTp06cFALFv3z6D6V66dEkAELt27ZLaGjRoIPLlyyfUarXeuLNnzxYAxL179xIVY2Ln/+7dO5E3b15RvHhxce7cOREYGCh27NghXFxcROPGjYVGoxFCCHHs2DEBQHTu3FlMnz5dvHnzRjx69EhUrVpV5MqVS4SHhxuNUyskJESUKlVKFCxYUPj7+wshhPj9998FALF+/fp4+xIRERF96ZgbMzcmItJiQZmIyIQcOXIIe3t78fbtW732vn37CltbW/Hu3Tu9cW1tbcWbN2/0xv31118FAHHp0iW99kePHgmVSiWWLFkSbwxjxowROXPmlF5Pnz5dABBnz5412SexSeuWLVsEAHHkyBG9cUuUKCFq1aoVb3zGYkzs/CdOnCgUCoW4du2a3rhLlizRm442af7mm2/0xvPx8REAhLe3d4Kx3r9/X2TKlEm0aNFCnDlzRlhbW4sBAwYk2I+IiIjoS8fcmLkxEZEWb3lBRBSPatWqwc3NTa+tTZs2iIiIwNmzZw3GzZ49u17bvn37UKBAAVSqVEmvvVChQihYsCB8fHykti1btuCrr76Ci4sLlEqldDnc69evERISAgA4ePAgChQogBo1aqTYMrZp0wZZs2bF6tWrpbZz587h7t276NOnj9645sSYWMePH4eHhwfKli2r196+fXtpuK7mzZvrvS5VqhQA4PHjxwnOq2jRoli1ahX279+PBg0aoEyZMpg/f36S4iYiIiL60jA3Zm5MRATwHspERPHKkSOHybb379/rtefOndtg3NevX+Pp06ewsrKClZUVVCqVlGw+fPhQulfchg0b0LlzZ9SrVw83btxAREQEhBDS/duioqIAxNy7zdh8ksPW1hbdunXD7t274e/vDwBYtWoVMmXKhA4dOkjjmRtjYn348AE5c+Y0aM+WLRusrKwM1rO7u7vea2dnZwDAx48fzZpfs2bN4O7ujvDwcHz33XewtbVNUtxEREREXxrmxsyNiYgAFpSJiOL15s0bk21Zs2bVa7e2tjYYN1u2bChVqhSio6MRHR0NtVoNjUYDEXPLIRw9ehQAsH79epQoUQI///wz8ufPL03ryZMnetNzc3PDixcvUmTZdPXp0wcRERHYuHEjQkJCsG3bNnTq1AmOjo7SOObGmFhZsmQxup4/fPiA6OhoZMuWTa9doVAka34DBgxAQEAAypYti1GjRuHly5fJmh4RERHRl4K5MXNjIiKABWUionidP3/e4CyAPXv2wMbGBrVq1Uqwf8uWLXHnzh2znpIc92yAT58+Yc+ePXptLVq0wNOnT/G///3P5HS0ia7uU6ATUrp0aVSpUgWrVq3Ctm3bEBQUZHBJn7kxJnb+Xl5eePjwocE68vb2loanlCVLlmDjxo1YsGABDhw4ACEEOnXqZPC0cCIiIiIyxNw48TEyNyaijIgFZSKieFSvXh29e/fG3bt3ERAQgEWLFmHNmjUYO3aswdkBxowYMQIVKlRA69atsXv3bnz48AEfP37E+fPnMWjQIGzcuBEA0KpVK1y7dg1Lly5FcHAwbt26hbZt26Ju3boG0ytbtiy++eYb7N69Gx8/foSvry/mz5+PxYsXAwA8PDxgb2+PI0eOIDQ01Oxl7dOnD65fv44pU6bA09MTVatW1RtuboyJnf93332HXLlyoWPHjrh48SI+ffqEXbt2Yfz48WjQoIHBfeGS6vLly/juu+/Qs2dP9O3bF7lz58amTZtw9uxZTJw4MUXmQURERJSRMTeOxdyYiL5ocj0NkIjI0uXIkUP06NFDnD17VlSqVEnY2tqK/Pnzi19//VVoNBqj4xoTGhoqfvrpJ+Hp6Sns7OxE1qxZRc2aNcWyZctEWFiYEEIItVotpk2bJvLnzy/s7OxEhQoVxKFDh8SsWbMEABEQECBN79OnT2Ls2LGiUKFCwsbGRhQqVEiMGjVK+Pv7S+OsX79eFC5cWFhZWQkAYvbs2UII40+S1p2uo6OjACDmzZtnMDwxMSZ2/n5+fqJ79+4ie/bswtraWhQoUEBMmDBBWj9CxD7J+tixY3p9o6KiBAAxadIko+tfCCH8/f1FgQIFRJkyZURoaKjesKlTpwqFQiF2795tsj8RERHRl465sT7mxkT0JVMIIURaF7GJiNKDnDlzokmTJli7dq3coRARERERyYq5MRERafGWF0RERERERERERERkFhaUiYiIiIiIiIiIiMgsLCgTERERERERERERkVl4D2UiIiIiIiIiIiIiMgvPUCYiIiIiIiIiIiIis7CgTERERERERERERERmYUGZiIiIiIiIiIiIiMzCgjIRERERERERERERmYUFZSIiIiIiIiIiIiIyCwvKRERERERERERERGQWFpSJiIiIiIiIiIiIyCwsKBMRERERERERERGRWVhQJiIiIiIiIiIiIiKzsKBMRERERERERERERGZhQZmIiIiIiIiIiIiIzMKCMhERERERERERERGZhQVlIiIiIiIiIiIiIjILC8pEREREREREREREZBYWlImIiIiIiIiIiIjILCwoExEREREREREREZFZWFAmIiIiIiIiIiIiIrNYyR0AfXkuX74MJycnFC9eXO5QUlVYWBiePn0KpVKJQoUKwdraWu6QSMeHDx/w5MkTAECpUqVgZ2dnctzo6Gjcvn0bUVFRKFy4MFxdXVM8juzZsyNfvnwpNl0AePr0Kd6/fy+9zpUrF3LlypXo6aTk8r958wZv3ryBm5sb3N3dTY4XFhaG27dvmxyeI0cO5M2b1+TwgIAAPH/+HE5OTihQoAAUCkWSYzZFjvWS3Pmbu17evXuHZ8+eSa+dnZ1RtGhRs+JKSFBQEHx9fWFlZYUCBQrA1tY2RaYbn6ioKDx9+hRCCBQoUAA2NjZJntanT5/g6+sLBwcHFChQAEql8WPzSdmHHz58iI8fP8Y7/yxZsqBQoUKJjpuILBdzY5JTUvKJpOQt5mJunDBz85q0zivkXi/m5mhxRUZGwtfXF0ql0mQ/5sampWZunJT5UBoSRGkoKipKABBVq1aVO5RU8ebNGzFt2jRRqlQpAUD6Z29vL3r27Cnevn0rd4j0Wf369aXtc/PmTYPhYWFhYtWqVaJdu3bC2dlZGnfXrl0pGseaNWsEADFgwIAUna4QQvTo0UNvP5w0aZLZfVNy+Y8fPy6+/fZb4eLiohdP4cKFxZo1a4z2uXr1qt64cf+NGDHCZL+6desKhUIhjZs7d26xcuXKRMdtjNzrJanzT+x6WbJkiV5MXl5eiV6+uO7evSvq1aunF4NCoRBNmzYV9+7dS/b0jVGr1WLmzJkiS5Ys0jwzZ84spk+fLtRqtdnTefnypZg0aZIoVKiQ3npxcXERo0aNEsHBwQZ9krIPN2/ePN4+AETnzp2Ts0qIyMIwN2ZuLIek5BNJyVuSgrnxGpP9EpvXpEVeIfd6SUqOpvX+/XvRv39/4eTkJPVzdXUVEydOFCEhIXrjMjfWl1a5cXK2L6U+FpQpTWX0pHnz5s16iXLx4sVFzpw5pbYiRYqIwMBAucP84m3atEkAEI6OjiYLyg8ePJC2m5WVlciUKVO6LSh7enqKihUrisWLF5vdNyWXv2rVqtK0smfPLkqWLCkcHByktlmzZhn00SYcWbNmFRUrVjT4N3fuXIM+165dk7apjY2NKFasmHB3d5fmM3/+/ETHHpfc6yUp80/KevH29hYVK1YU5cqVS5GkWaPRiMKFCwsAwtbWVpQsWVKUKVNGZMuWzeSypoTvv/9eWs4CBQroJaOjR482ezra9ykAkSVLFlGqVCkpdgCiTp06Bkl4UvbhkSNHGh23YsWK0rbesGFDstcLEVkO5sbMjeWQlHwiKXlLUjA3Nr0uE5vXpEVeIfd6SUqOJoQQz58/Fx4eHtJ4+fPnF56ensLGxsbo70PmxvrSKjdO6valtMGCMqWpjJ40nzhxQkyePFncunXLoD1z5swCgJgxY4ZM0ZEQQgQGBgp3d3fx9ddfS0mLsYLy8+fPRe/evcX27dvFx48fRdeuXdNtQdnY8iUkJZd/0qRJ4q+//hIfP36U2oKDg8XgwYOlImdAQIBeH23C0aNHD7Pn07BhQwFAtGrVSrx//15qP3bsmHBychIODg7i1atXiY5fl9zrJSnzT856CQgISJGk+eHDhwKAKFu2rAgKCtIbdunSJXHgwIFkTd+Yu3fvCpVKJaysrMTu3bul9gMHDggbGxuhVCoNPqtNOXr0qPjjjz/E8+fP9doPHDggndVy7NgxvWFJ2YdNCQ0NFa6ursLZ2VmEhoYme3pEZDmYGzM3lkNS8omk5C1JwdzY+LpMybwmJfMKuddLUnI0IYTw8vISAESxYsXEv//+K7UHBgaKWbNmicePHxuNkblxjLTKjZO6fSltsKBMacqcpDk8PFzcu3dP3Lt3T0RGRhoMDwwMFJcuXRJPnjwxOY2nT5+KS5cumTzj4eXLl+L69evC19fXZJyXLl0S//33n9T24sULceXKlSQnSn/88YdU1EkOY7H5+fmJmzdvGnwZmZLQ8oeEhIhLly6JZ8+eGR1+69YtceXKlQTjMrXOAgICxM2bN8XDhw9NHlF8/vy53jYMCwsTd+7cEY8ePTJrGU0ZMWKEcHR0FL6+vvEWlOP60grKcaXG8qvVaumo+IkTJ/SGJTbhiIqKElZWVkKhUOgVTbUmT54sABg98p0cab1eEjv/5K6XlEqaP3z4IFQqlXB1dRWrVq0yGktKGz9+vAAghgwZYjBs1KhRAoAYO3ZssufTu3dvAUAsXbpUrz0lC8p//fWXACD69u2b7GkRkWVhbszcWAh5c2MhkpfPJCZvMQdzY+PrMiXzmtTMK+TOjXWZytFOnDghnRn88OHDRMXC3DhhaZEbxzcfSju8izVZjC1btqBOnTpwcHBA8eLFUbx4cbi4uGDIkCEIDQ2VxtNoNKhTpw68vLwghDA6rSZNmqB27dqIjo7Wa9+0aROKFSuGXLlyoWzZssiXLx88PDywdetWvfHev3+PypUrY9iwYbh8+TLKlSuH3Llzo0KFCjh58mSSlq9IkSIAkOyb7OvGdvPmTVSqVAl58+ZF6dKlkSVLFowePRoajcZoX3OX/86dO6hcuTJmzpxpdDrt2rXDV199ZTIuU+vs/v37aNy4MbJmzYrSpUujcOHCyJ49O2bMmGEQ84IFC1C5cmWcPXsWv/76K3LmzImSJUvCw8MDHh4eOH36dKLX3fXr17Fw4UJMmTIl3hv+J8fly5dx+fJlqNXqVJl+RqJUKuHh4QEg/vfFmzdvcOPGDbx+/drkOKGhoYiOjoaDgwOyZs1qMFy7vU+cOJHMqFOfuevFHJayXrJkyYIxY8bg48eP6NOnD7Jly4YSJUpg1KhRePToUarM08fHBwDwzTffGAzTtiX181zXixcvAMR+xhtjzj4cn9WrVwMAevbsmaT+RJQ+MTc2D3PjpOfGKSGhvIW5sfniW5cpmdekt7wiqbmxqRxty5YtAIBOnTpJ001rzI2TlxubOx9KXVZyB0CkNXLkSLx58wZOTk7Inz8/goKC4Ofnh8WLF+P58+fYs2cPAMDV1RWdO3fGqlWr8Pfff6Nhw4Z60zl16hTu3buHPn36IEuWLFL7jBkzMHnyZAAxTxB1c3PD27dv8fjxY3Tq1AlhYWEGX6rPnz9HgwYNEBQUBA8PD7i6uiJz5sxJWr6jR48CABo1apSk/nG9ePEC9evXR0BAAAoXLoyQkBC8evUK8+bNQ548efDdd9/pjZ+U5U8KU+vs8ePHqFGjBvz9/aFUKlG4cGF8+PABHz58wOTJk/HkyROsXLnSYHq//fYbTp48CRcXF5QoUQJPnz7F48eP0bp1azx48MBokcwYIQQGDx6M4sWLY+TIkcleTlMqV64MAAgICEjWk41DQkJw9+7dRPUpUqQIXFxckjzPtPbp0ydcuHABWbJkQcWKFY2Os23bNqxbt056Xbx4cfzwww/o0qWL3niZMmVC5syZERAQAB8fH9SpU0dv+IEDBwDE/HCzdOasF3NZ0nrp0aMHVq9ejaioKERHR+PevXu4d+8eli5diiVLlqBHjx7SuCmx/9+7dw8AUKZMGYNxS5cuDSDxyx0YGIgHDx5ArVbj9evX2Lp1K44cOYLGjRujfv36RvuYuw+b8uTJE5w8eRJFihRBzZo1ExUvEaVvzI0Th7lx4nLjlJJQ3sLc2HzxrcuUymvSY15hTm6cmBzt0qVLAIAWLVoAAPz9/fHixQtkz54dOXLkSN2F0cHcOIY5uXFS5kNpQOYzpOkLE99lfWPHjhXXrl3Ta3vw4IEoX768ACCuXr0qtV+5ckUAEB06dDCYTrdu3QQAvcvObt26JZRKpShevLi4dOmS3viHDx8Wzs7OIlu2bCIiIkIIIcSrV6+kG73Xq1dPvHz5MjmLLS5cuCCsra1FiRIlpHkklW5sjRs31rv36dKlSwUAUbx4cb0+iV3+S5cuxXupWbFixYSjo6PJuIyts7Zt2woAom7dunrDNm7cKGxtbQUAcfr0aal93LhxAoBQqVRi5cqVQqPRCCFiLg+qUaNGoi9vWblypQAgfHx8pLbUuOWF9qEC5l5iaeqyvnPnzknr09x/+/bt05uGpV/W1717dwFALFiwwGCY9pIohUIh8uXLJwoVKiSsrKykZf35558N+gwaNEjg84MeZs+eLU6ePCn27t0r2rdvL+zs7KSHOaSktF4vSZl/ctZLSl3Wd/XqVeHg4CBatWolPTX7zp070qVqKpVK3L59Wxo/JfZ/pVIprKysTMakfVBhYh7ksW/fPr15Wltbi+nTpxu9BD0p+7AxP/zwgwAgpk+fbnacRJR+MDdmbixXbqwrOflMQnkLc2PzxbcuUyqvSe28Qq7cODE5Wo4cOaTP0J49ewqVSiX1K1euXLy31WBurC+tcuPEzIfSDs9QJovx22+/AYB09kVYWBiEEGjfvj2uXr2K06dPo1y5cgCA8uXLo2rVqti9ezfevn2L7NmzA4g58r1jxw5Ur14d5cuXl6a9bt06aDQaDBs2DFZWVrh69SoAQAiBbNmyoWnTpti6dSuuXLmCatWqSf1sbW2xYcMGuLu7J3m5Hj16hFatWsHBwQHbt2+HjY1NkqelK1OmTNi8ebPeWSEDBgzAnDlzcP/+fURGRkrzSuryJ4WxdRYaGor9+/fDwcEBmzZt0hvWpUsX3Lx5E7/88gu2bt2KWrVq6U1v0KBB6NOnj/Q6S5YsmDhxIlq0aIEbN26YFZO/vz/Gjx+P7t27o3bt2slavoRcvnw5Rabj5OSU6LNTE3PWh7+/Px4/fqzXZmNjY/SIdWqYPn061q9fj/bt22Po0KEGwzNlyoT58+ejZ8+ecHZ2BgB8/PgRU6dOxR9//IGpU6eiQ4cOKF68uNRn1qxZuHTpEi5fvoyxY8dK7QqFAnPmzMHo0aMRGRmZ+guXDAmtl6SwhPUyZMgQKJVKrFixAg4ODgCAEiVKYNWqVdBoNFi7di1Wr16NOXPmAEj+/h8dHQ2NRhPvZZHW1tYAgMjISNjZ2Zk9j4oVKyIqKgrPnz+Hv78/Zs6ciUyZMmH48OF64yZlH45Lo9Fg3bp1UCqV6N69u1kxElHGwdw4cZgbm58bpxRz8hbmxuaJb12mVF6THvMKc3PjxORo4eHhAIBx48bh6NGjyJkzJ1xdXfHkyRNcu3YNjRo1wtGjR1GvXr1UWy7mxonLjRMzH0pDclaz6csT31kYp0+fFjVq1BAKhcLoEbbJkyfrjb9u3ToBQPz6669Sm/bhHhs3btQbt2HDhmYdxdMeSdWeUVC2bNlkLe+jR49E3rx5haOjozh79myypqWlja1GjRpGhzdo0EAAEB8+fJDaErv8yTkLw9g6u379ugAgatasaXR6Z8+eFQBE/fr1pTbtWRjbtm0zGP/WrVsCgPj222+NTi+ufv36CVdXV/HmzRu99i/5oXzah3Ho/sudO7fJaaXk8v/222/SWUTh4eGJ7t+6dWuTZ1ZERESI5cuXi7Zt24rq1auLDh06iBMnTkhHxPPkyZPs+HXJvV7MnX9S10tKnIXx/PlzAUDUqlXL6HAfHx8BQDRo0CDJ8zDG2tpaKBQKk2dZ2NjYCJVKlax5HD16VLi5uQkA4tChQ2b3i28f1nXkyJFUWTdEZDmYGycfc+PE58ZxJSWfSW4+ZwpzY+PrMiXymrTIK+TOjbXiy9Hc3d0FAOHg4CAOHz4stQcEBIj27dsLAKJixYpGp8vcOH6pnRsndz6UsniGMlmEa9euwcvLSzpzIF++fMiUKROUSiWCgoLw33//ISoqSq9Px44dMXr0aKxcuRJjx46FQqHAihUrkCNHDrRv315v3ODgYAAx9wvSHnkzRnukTCtnzpxJXqYHDx6gfv368Pf3x8GDB1GjRo0kT8uYTJkyGW1XqVQAoPcgj6Quf1IYW2dhYWEAoHffPl3ae71px9NlbDmNLaMpV65cwcqVKzFo0CD4+vrC19dXGhYSEgIAuH37NsLDw1GsWDGT6zWtpfZ94rJmzWpwlDst7hk2a9YsTJw4EV5eXti1a1eSHsTTuHFj7NmzB0+ePDEYZmNjg379+qFfv3567cuWLQMAFC1aNGmBp7KUWC/xkXO9PH/+HABM3mNT+5mhfT9q/07u/p87d248ffoUfn5+yJ8/v964r1+/RmRkZLIfztmwYUPMmTMHPXr0wLp169CkSROz+sW3D+tatWoVgPTz0BwiSjnMjROPubF5uXFKSO28xZgvPTdOibwmPeUVyd3H4svR8uTJg1evXmHIkCFo3Lix1O7q6oo1a9Zg9+7d+PfffxEUFJQqvw2ZGxsyNzdO7nwoZbGgTBZhxYoViIyMxNixYzFt2jS9L4zNmzcbvUG7ra0tevfuLT2YwtbWFrdv38bkyZMNLp3TXvY3b948eHl5pe7CIKZA2aBBAwQGBuLAgQMGD8JKa4ldfiurmI8G3S8xLSGE9CVoLjc3NwDAw4cPjQ5/8OCB3ngp6c6dOxBCYPHixVi8eLHRcTp16gQAOHbsGBo0aJDiMSTFzZs3Ub169UT12bdvn/RwiYQ0bdoUTZs2TUpoSTZlyhT8/PPPqF+/Pvbt2wd7e/skTefDhw8AYPalWACwceNGAJD9vWhMSq2XpEiL9aJNZLXv87i07dmyZZPaUmL/L1OmDJ4+fYozZ84YJM1nzpwBEPsAkuTIkycPAODVq1dm9zFnH/b398eePXvg7OyMr7/+OnlBElG6w9w4dX3JuXFyyZW3fOm5cXLzmvSUV6TUPmYqRytbtiwuXbpktHjq5OSEzJkz4927dwgODk6VgjJzY0NJ+X2XlPlQymJBmSzCu3fvAMQcLdVNmDUaDdasWWOy38CBAzFnzhwsX74ctra2UKlUGDBggMF49evXx549ezBjxgzUqVNHSgp1pdQRyGvXrqFhw4YICQnB/v37U/XeS+ZK7PJrj4pevHgRGo0GSqVSGm/z5s0ICQmBo6Oj2fMvVKgQcubMibt37+LgwYNo1qyZNEyj0WDu3LkAkOJnqgDGzzbQunfvHkJCQuDp6Qk7O7sUOQslpaT2feLS2tixYzFnzhzUrVvXrMQwODgYTk5OBu0BAQHS2RUVKlTQGxYVFQWFQmGwf2/YsAGnT5+GjY0NevXqlcwlSVmJXS9JIfd6KVq0KLJkyYJ79+4Zff9r7w2ne4/IlNj/W7Rogb179+LPP/9Ep06dpLO3hBCYP38+AKBly5ZmTdvU/gjEFuULFSpkVp/49uG4042IiEC3bt3S9CADEVkG5sap60vOjZMjLfIWU7703Di5eU16yStS6jcDYDpHa9WqFVauXImDBw9i2LBhesMuXbqEd+/eIVOmTKl2QIe5sb74cuOkzIfSDgvKZBFKlCgBAOjVqxfGjh0Ld3d3PH36FEuWLMGFCxdM9itYsCCaNGmCnTt3QqVSoXXr1tKRKl29e/fG3Llz8c8//6BChQoYPHgwihYtCo1Gg6dPn+LEiRM4depUos8uiOvGjRuoX78+AgICMGvWLDg7Oxs8iMLBwQElS5bUa/Pz88ObN2+QK1cu5MqVK1kxGJPY5c+ZMyeKFy+Oe/fuoVOnTujTpw8UCgVOnDiBP/74w2jSnZBhw4Zh0qRJaN++PcaPH4+aNWvC398fS5YswcmTJ+Hi4pIqRa34zjaoVq0aLly4gC1btqBUqVIGw2/duiU9tMHf3x9AzINktNvU3d0duXPn1uujHVa+fHnpSzopSpUqlWIPMUmqxC5/aGgo7ty5Azs7O731OXr0aMybNw/58+fH1KlTcefOHYN5FSxYULq8EwAaNWqEPHnyoFGjRihYsCCEELhx4wb++OMP+Pn5IVeuXOjYsaPeNJ49e4YmTZpg0KBBKFeuHNRqNfbu3YulS5cCACZPnmz0TITEbjM510tS5p/U9ZIQc9ebUqnE0KFD8fPPP5t8/2fOnBm9e/eW+qTE/t+tWzdMnToVFy9eROvWrTF8+HAoFAosWrQIZ86cQc6cOQ0eSGNqW9WvXx+FCxeGl5cXChQoAJVKBV9fX2zcuBFHjx4FAL0HJAFJ24d1rV69GkD6uCyViFIec2PmxqmVG2slNp9Iat7C3DhlcsCk5DW6EpNXpKfcOCk5WvPmzVGmTBkcPnwYbdu2Rc+ePZE5c2ZcuXIFs2bNAhCzvhP7vmZunPK5cVLmQ2lItrs30xfJ1INH3r17J90cX/dfpkyZxPTp0wUAMW7cOKPT3L9/vzT+iRMnTM771q1bolChQiYfulGoUCFpXO1DNBo3bpyo5fv9998TfLiHsQdzjBgxQgAQU6ZMSXAeCcXWuHFjAUC8e/cuycsvhBC7d+8WSqXSYLzhw4fH++ARU3FFRUWJNm3aGJ23vb29OHjwoN742gePGLvJ/t27dwUA0bVr1wTXV3wSeiifh4dHvNvS2D6pHRYQEGBWDHI9eMQciV1+7QNrPDw89Nrz58+f4PtizZo1en3q1atnctzcuXOLf//91yBeX19fYWVlZTC+QqEQI0eOFBqNxuhyJnabyblekjL/pK4XIeJ/8Ehi1ltkZKRo2bKl0XgdHR3FsWPHEpxGUpw5c0ZkypTJYJ5OTk7i1KlTBuOb2la1a9c2ub5tbW3FkiVLDKaVlH1YS/uwxMKFCyd/JRCRRWNuzNzY2LzTIjdObD6R1LwlsXkWc2PT6zKxeY1WYvOKxG4zOddLUnI0IYS4f/++0c9YAOKrr74SQUFBRvsxN46RVrlxUrcvpQ2eoUxpSqFQoGLFitJZF1rZsmXD9evXMWfOHFy9ehXR0dEoW7Yshg4dig8fPmDXrl1Gz64AYo502draolChQvFeQufp6Ylbt25h48aNOHbsGF68eAFHR0cUKFAAXl5eaN26tTSutbU1KlasmOgHVeXIkSPBS1GKFStm0BYZGQnAvAd/JBRb0aJF8f79e4MHjCRm+QGgdevW+Oeff7B06VI8ffoU2bNnR+fOndGxY0e0b99euvecuXFZWVlh586d2LlzJ7y9vfHs2TPY29ujUqVKGDhwIAoUKKA3fp48eVCxYkWjD9Kwt7dHxYoVk315S4kSJRAdHW3yUqrSpUvHe6mcsX1Su/3NPaKdLVs2VKxY0eA+VpYgscvv6OiIihUrGpztWqZMGb17gBkTd/ixY8dw+PBhHDhwAA8fPkRoaChy5cqFevXqoVu3bkYvwc2bNy8ePnyIpUuX4sqVK4iMjETJkiXRvXt3VK1a1eh8te89Gxsbsx/2Ied6Scr8k7JeEpLY9WZtbY09e/bA29sb3t7e8PX1hb29PSpXroxBgwYhX758SYojITVr1sTNmzexYMEC/PvvvwBizhoZNmwYChYsaDC+qW114sQJHD58GAcPHsTDhw8RFBQENzc3VKlSBV27djX4/AKStg9rnT59GhUrVtQ7M4WIMibmxsyN5cqNE5tPJDVvYW6ccjlgYvMarcTkFektN05KjgbEfC7cunULixcvxunTpxEUFIQ8efKgZcuW6Ny5c6LPTmZunDq5cVK3L6UNhRBCyB0EUXJs3boVnTp1wvz58zF8+HC5w0mSSpUq4f79+3j69KnBZWJESdWzZ0+sW7cOW7ZsgYeHR6pdNpqenTlzBl999RUGDRpk8qGNX6J3797h2bNnCA4ORr169eDl5YW///5bGs71RkRkuZgbExnH3DhhzPGMY25MZEiZ8ChElicgIACXL1/Gtm3bMGzYMDg4OKBbt25yh5Uknz59wrVr1zBkyBAmzJQqOnXqhMqVKzO5McLHxwfW1tYYP3683KFYFG9vb1SuXNnkmW1cb0REloW5MZH5mBubxhzPOObGRIZ4ywtKl44fP44OHTpIr3/44QdkyZJFxoiS7v79+6hUqRJGjx4tdyiUwRQsWFDvMtO4DxAk4MOHD/j+++9T7bKy9Cp79ux6+07cy5G53oiILAtzY6KEMTdOGHM845gbExniLS8oXTpx4gS+//57uLq6onnz5hg+fHiynhpMRERERJReMTcmIiKitMSCMhERERERERERERGZhfdQJiIiIiIiIiIiIiKzsKBMRERERERERERERGZhQZmIiIiIiIiIiIiIzMKCMhERERERERERERGZxUruANKDjx8/wsfHB3nz5oWtra3c4RARERFRMkRERMDPzw916tSBq6ur3OGkO8yNiYiIiDKOpOTGLCibwcfHB23atJE7DCIiIiJKQbt370br1q3lDiPdYW5MRERElPEkJjdmQdkMefPmBRCzYgsXLixzNBmTWq1GcHAwnJycoFKp5A6HkonbM2MJPHoM4eFhyNa8ObdnOsf3ZsbC7Zl0Dx8+RJs2baQcjxKHuXHS8D2b8XCbZjzMezMmvlczHm7TlJWU3JgFZTNoL+UrXLgwPD09ZY4mY1Kr1QgMDISLiws/DDIAbs+Mxf/uPYSFhSKnpye3ZzrH92bGwu2ZfLxdQ9IwN04avmczHm7TjId5b8bE92rGw22aOhKTG/OhfERERERERERERERkFhaUiYiIiIiIiIiIiMgsLCgTERERERERERERkVlYUCYiIiIiIiIiIiIis7CgTERERERERERERERmYUGZiIiIiIiIiIiIiMzCgjIRERERERERERERmYUFZSIiIiIiIiIiIiIyCwvKRERERERERERERGQWK7kDICIiIiKihL158waHDh1CcHAwOnXqhGzZsiWq/7Nnz3D69GlER0ejUqVKKFWqVCpFSkREREQZmUUXlP38/HDgwAEEBwejR48ecHNzS1T///77D6dOnZKS5kqVKqVSpEREREREqePcuXOYNGkS7t+/jyxZsuDWrVuoVatWogrK06ZNw/Tp09G0aVM4OjpiyJAh6NSpE1asWAGlkhctEhEREZH5LDJ7PH36NGrUqIFatWph2bJlGDt2LF68eJGoaUyaNAmlS5fG33//jUuXLqF+/fro2rUr1Gp1KkVNRERERLqEENBoYv6pNQLRag2iPv+LjNYgIlqN8KiYf9FqjdzhWiwhBCZPngxfX1907Ngx0f137dqFH3/8EStWrMDu3buxceNGHD9+HOvWrcPcuXNTIeLUERmtkfaXiOiYf5HRsftU9Od/6s/7m0YjIETMPyIiIiJKORZ5hrKtrS3mzJmD6tWrY8aMGbh27Vqi+m/btg0zZ87E5s2b0alTJwDA4MGDUbVqVXh6emLixImpEDURERF9yWIKV4D4/Deg/RsQiBkWO25sm9Dt/3mYtnN840SroxEaEokIRTiUKpXBfLRxSK+ldtNxwqB/zPhSzOJzkfjzSBrpNaARQhpXo9GfZuwy689Tdzkz2VqjXD5XKBSKxK/8DK5GjRrJ6j9v3jwULFgQ3377rdRWrVo1NGzYEL///jtGjx5t8Wcph0RE4+aLQERGq6HA531EZ1fR/qlQKHT+jh2i0OmiXVQFFFAoYv6vVMT2BwClAlAqFHptCoXudBR609SdB4wOU0Co1QgLDUdgtBVUKqs4fXSmbbAMkJY5Jt7YhVYYWT4pXp1pxF1nutPRHd/Y8unGZ2x+xmLm+5iIiChjs8iCcpUqVZLV/48//oCHh4dUTAaAihUrokmTJvjzzz8xbtw4qFSq5IZJRET0RUqocApAr4iZ3MKpdpoQ+vMxVTiNO1+9cXTnq1dYNb9wKhDzf0BIhdPYZTa+bnSXMza+2GWSYvzcoFOHNVoABgCh0UATEQqFjRpQKuKs8zgxxa4a/XWj7aOI+Vuh0AlYodtHSBUk3aITEFtI0qnd6RfbTBXEdNr9QyKRP4sSGgGoWIdKURERETh37hw6d+5sUOSrXbs2Dh8+jHv37qFkyZIyRWieKLUGweHR+BgWiUx21gBi3y8xf3/+v+7nQJzPI92/jfaF/ntMO77uW0IIIa1HYeI9I0Ts3wqdYQqhAaJCobBWQ6HSL+ArdP4yXag2XtTVm048xWiYKkbrxmBk3gkV8LWv4yuoxxTxY6akLdTrFvG1fysVCpMFfP1l0l1Gw2K7qfVgtEgeTwE+bgE/7nQ0ajVCwqKgtoqESqUyKMonpoCvG6upuONdPhbxEyUkJAR+fn7w9/dHYGAgPn36hKCgIGS9fgNRUZEICA2FlZUVnJ2d4erqChcXF2TJkgV58uSBnZ2d3OETEcnOIgvKyREeHo6LFy+iS5cuBsNq1aqFAwcO4O7du3wICRGRjuDgYLx+/RqvXr3C+/fvERYWhtDQUISFhSH7rdvQaDQICguDnZ0dnJyc4ObmhuzZs8PNzQ2ZM2fOsD9idIuUGp3iYHyF09i/YwuY0nhmFFd1C4qmzihNauFUrVEjPDgUtuFKKJTK2MIphDRd3cKpdrlFKhZOpX4m1o3ucuquC/22lCuc6iyCycIpYOKHfnyFj88FCv1iDvQKJ7rjKvXmpS0+xCmGCDXUsIKVvRUUSpVeUTfuuLrz0s4vbpuc1BqR8EiUJM+ePYNarUbevHkNhmnbHj9+bLKg/PbtW7x7906v7eHDhwAAtVqdZreTU6vVEBoNsjpYIZN9OvwJIwQ0GjWiwqJhZWcNhVKlM8jwswvQ+USL8/mJOMNNFsx1pq/tpn8gL06bsXh0P2eNxZNAAV/7WrdvnEkbfNbH89FslO7s4vZJqABvvBit3z9mLMNitDZgERUC5fsoKJRKgwK8YTE6TgE/7jAjxWjd0UwV8PVjjXsWfvwFfG2bse8yg0J2nPkZK5LHDounSB73jHZT28jIMFPrQW+96xT8Hz96hLt3buPO7Vu4c+cOnj15Ar/nfgjw94cxbZxdAAC7ly0zOhwAcuTIgXz58sHDwwOlSpVC6dKlUbp0aeTNm9divldJn/b7irdAzTi4TVNWUtZjOszG4ufr6wu1Wo08efIYDNMmzU+ePDFZULaUpPlLww+DjIXb0zIJIfDkyRNcuXIFd+/exf3793H//n08ePAAwcHBJvtJifXKFUaHOzg4oGAhD3h4eKCghwdKepZCuXIVUdCjEJTaoiUMi6t6BUUThVO9YfEUbPUKp4DBfLSFU+160GjPNNUpnAozCqex4+muWMNCqN4PdcQGFLdwqm1LSuFUL4Q404w7hvTT5vPZcfgYDSiU0njxFU61EzAoUJr4kZdQ4TS2b+zfSu14Bj9idX+06v9A151X3BiMzisD/sATGgWiVIC1StqcukON9dD701ghSS5Co4HQfP7eEKm/rb6k76fQ0FAAgL29vcEwBwcHAIj3O2Dx4sWYOnWq0WHBwcEIDAxMgSgTFhIeBU1ECNQqBaLS6U8YodFAHRke8xlrxi1GFCb+ThZjE8p4H49GzzSPO0yviA/DF7r5RNwifmz+ooZaHQWFKgyKz48nEnH768xce6d4Y/HEDcFoUd9gVOMHFYSR7wBTBXzdYSZ3BZ0RRJyRFULn74QK8HolacP8Q9sY/1n4cd8bsQlAeFgY7t+8hlvXLuP29X9x98Y1hIWGmFqqJHvz5g3evHmDS5cu6bW7uGZGmQqVULZiFZSvWBnFSnjCysYGQOzDq5Q6C62XD8VTgIeR4j50i/I67Qr9lfO5TWEwnrEDA/Gte/1YdSduZHicAwza5TM2TVPLZ7Cdk5nHaTQahISExBxQsfBbPJF5uE1TVny5oCnpMxuLR0hIzBeGsaRZ26YdxxhLSZq/NPwwyFgseXvqJdFAnEJhTKuppDxuQTH2rEedM39gIqk3Nh9p/Lg/TqA/PZ1G3YKi0bNUdfqro6Nx++YNXDh3BreuXcGdm9cR4P/BvBWVCKGhobh96yZu37qp1+7g6IQiJUrBs3xFlKtcA8VKl4WVlY3eOHELy7Ftuk3CcHzErn/tGacAYs86NZJz6jYbO0NGy1jhVC/9lgqUcRPr2GkajKvToIDOWUJxfljFFFWNF051RtH/f9wzXONJuIVGIDoCsLIVUCjjqyTKVGU0+eve/K4WUh9NE0KjQXREGACYVZyyZCIyEtEKK3z6ZCWdMZeakpI0p1faS7MjIiIMhmnbjOXNWoMHD0aHDh302h4+fIg2bdrAyckJLi4uKRitacI6Ekp/DVQqwNreOk3mmdKEJuZAhrW9g94ZypR+CY0aUWGh3KYAEPeqLOlvIwV83QJ9nKK2/oF44wVz3WGvX/jh3Mm/cd7nOK5d/B8iI8KTvAj2Do6w/fyZ6agRUKujER4Wanb/wI8BOH3iGE6fOAYAcHByRrnqtVGxZn1UqFkXLpmz6uWyis/LoYBOMqvXpn0dSz/n1LaZLsCbKtbqTiOh2+ho+xo9G1wnMIMc1dgw3WkYy29NnIUfdxkScxa+QqGARgNERigQpAJUKoVOX2MxGyn4m1gPporkZp2Fb2L7KeKZn+HBG8MCfoJXCBjdJ9LfbXTUajWEEHB2duYtbVOAk5NTovtkuIKyNiE2ljSHh4frjWOMpSTNXxp+GKSOuPc51U3yEv2AqDiFPFPThAA0UCNSYYNwYQMFlIaX4sczX6MPiAL02uI94/TzRLVnn5r7gCi9IpSRdSGdzacTi5QD6yS2cZfD2HLqt+kmyfEUTqU1YXjEX9enj/646HMUV8/54ObFMwgJ+mRizPgpVSrY2trB1s4ejvZ20KjVcNBoEBUZiahIw89XY0JDgnH98nlcv3wem1Ysgp29PcpUrIoa9Rrhq4ZN4JbDXUqMpMLp54UzlejE/F+n3cITHUvDYkbGkpG2pyLSClZ2VnB2doFKmfrv66QkzelVnjx5oFAo8OrVK4Nh2rZ8+fKZ7J89e3Zkz57d6DCVSpVmeZtKpYq5pYBSAWU63d81wOdlUKXbZSB93KbyeP3yOY7u3YnDe7xx/87NhDsAyJErNwoXK4kChQojZ568yJkrD3Lmyo3MWd3glMkZDo6OUKlUUB7eB3VUBMY0bwulUoXo6GiEBAch+NMn+H94h1cv/PD6xXO8eu6HZ48f4L+7txHw4b3ReYYGf8L/ju3H/47th0KhQPkq1dG0TXt4NWsNF9fMyVoHRm9lE6ddf3ydX1Uizm8aqc2wgG/QFzD4bac3romYpJca3fGMxGMwPRF/PEaWxehtdDQaKKLCgCBFbDU6buAAoNCJXQG9++ADCRSTdf9rpIieUP/YcYwXo+MW8LXT0b40HGa6gB873LAYrbsM2le6BfyE7oOvbZPij7eQbXx+5txrXqNWIyw0CmGKCKisrIwetDA8sz+RBXydznEPMMQ3v7jt8c0vdj3J+7s2Kflchisoa+9b9PLlS4Nh2rb8+fOb7G8pSfOXSLt+k7KOE3pAVKIKp587xzdO3KPpceeT5MKpkWlKfUVs4RTQv89p7L1dRYKF09jW2OXUHcewcCoSXTjVaNQQEaGAjSb2Pq2661cvJYmz/rTzUMT8rT3jVEAYfMjqfr/HPZIqfaka+9Iw8uGu810W82WgU+jUm26c4qf0MCrdLw/dL684X+LxFU5jl8X0l44xAf4f8M/hffj7wB5cPnfGrEu5c+bOgwIeRVDAowjyFyyMHLlyI1v2HMjqlh1ZsrrBxtZWGlebWI/6nFhHRkQgwP89Aj58gP/7t3jh5wu/J4/g9+wJnj56AN8nj4zOMzwsDBfPnMTFMyfxx7SJKF2+Euo3aYEmbdoje85cCcZMyccfvhlLRtqe2uVQqVRpUlD+kvI5JycnlClTBhcvXjQYdv78ebi4uMDT01OGyIiIzBcWFopj+3Zh7/ZNuHrxXLzjOru4omylqihXuRrKVKiMIsU9kSmJJ4ZZWVnBxTUzXFwzI3e+/ChdvpLBOB/evcX9Ozdx49+LuHrxPG5evYyI8DC9cYQQuHLhf7hy4X/49cdxqFHHCy07dEHtBk1gZZX4soxeQUrva5MnWxgTcw97AWt7hyTnTAbPRtFpB2D0963uAL3Ct94JQ3GnF/fXss5vdej//pZOntLtr9GJx2DewrBNt29C8ejWK4zFE2facfdGofP7XjuCiDOy3sNs4yvAazRAVBhgrYHy84Nu4/4uB/SL8qbOwjdWHI5TT9avK+iGbKRIHrcYrT/ccH6IU/zWFuIz2VqjWM5MshebTclwBWVHR0eUK1cOFy5cMBh27tw5ZM6cOV0nzQkVTgGdN7GRYmFiC6cxo8QWFE1O00ThNDY+I/c51ZmmWhONiJAQ2IQpoFCoTBZOY2NN5H1O4ymcSjHqFk6NrBtT92JL6D6niS2cAjD45DVWOAV0ipk6fRJ6QJRBMVP6O3aauuNqhyv1CqcKabrG5iWEEmqlNazsrGO+sBMonOrNz0I/LC2NWq3GhdP/YPeWv+Dz92FER0WZHNfB0QnlKlVF6QqVUKpcRXiWq5issyJsbG2Rwz03crjnNjo86NMn3L99A3duXMWNfy/h8rnTCPoUaDDezauXcfPqZSz49WdU+6oeWn7TBXUaNJUuNSQiosS7cOECLl26hHbt2sHd3R0AMGDAAAwePBgnTpxA/fr1AcQ8iG/fvn0YPHgwbGxs4pskEZFsHv93D96b1mK/9xYEfzJ+5Z1SqUTpCpVRq34j1KrXEIWLl0zT2+5ldcuOGnW8UKOOFwAgKjISt69fwdl//sbpE0fw4O5tvfGjo6Jw6u/DOPX3YWTP6Y62nbujbefucMvhnmYxU+Lp3SqPRfxUl1ABX/ugW2t7G0ARc5DA5JnrCRTw9foibl1Hr6IDgXgK+AbzNhKPGQV8AAiOiEb+LA7QCEBlobtYui8onz17FufOnUPnzp2RO3dMcWPQoEHo378/jhw5gsaNGwMAHjx4gAMHDmDEiBGwtrb8+66FR6lx/3UQItXqVCucal/rFk6lcYwUT5NcONXOQ6dwKkSco6mazw+KshYQSmWKFE6hM358D4iKO65S74iRYeFU2yfuUSu9eX3hhVONBoiKVsLaOv2fNWdpAj68x44Na7Bry3q8efnC5HjFS5VFzboNUK12PZQuXwnWaVgsyOTsjErVa6FS9VoAYorf92/fwIUzPjhz4iiuX76g9xmj0WjwP5/j+J/Pcbi4Zkabzt3xTfc+yJnL8OGqRERfojdv3mD79u0AIJ00sXXrVpw5cwb29vbo06ePNO6+ffswY8YMlCtXTioo9+/fHz4+Pmjbti0GDx4MBwcHLF++HBUrVjT57BAiIrkIIXDpf6exbul8nD/1j9FxlEolKtesjSat2qF2w6ZwzZwljaM0zdrGBuUqV0O5ytUw5PvJePXiOf45sh+H9+zA7WtX9MZ9+/oVlv3+K1b+OQcNW7RFz0EjUKRE+j0BjiilJFTA1ygEhEoJK5XS4p7blBJeByb9fvBpxSILyq9evcLGjRsBxBSMAWD9+vX4+++/4eDggMGDB0vjHjp0CDNmzEC1atWkgnKfPn1w9uxZtGvXDv369YODgwNWr16NmjVr4qeffkrz5UmK8Cg13gVF4F1wBJxsrYwXM5G4wmls38/D9O4Tk0DhVHd60gzjFHVNzMucwmlKXIJClJE9eXAfm1YvxQHvrYgw8bARz3IV0KBZazRo1hq58pq+H2ZaU6lUKFmmPEqWKY9eg0fi/ds38Dl2CMcP7sHFs6f0isuBHwOwbsl8bFi+EPWatEDXPoNQpmIVGaMnIpJfWFgY7t27ByDm1m1DhgxBUFAQ7t27Z3A/6GrVqmHIkCHIlSv2VkIqlQpbtmzB0aNH4ePjg+DgYMybNw9ff/31F3X7DyKybBqNBscP7cX6pQtw58ZVo+MULVkKrb/phoYt2iCrm/FbVVoa99x50KX3QHTpPRB+Tx/j0J4d2LttI14995PGUavVOLxnBw7v2YEadbzQc/BIVKhaI0OfhERE6ZtFFpSjoqLw+vVrAICnp6d0i4rXr1/D0dFRb9xatWph9OjRyJMn9kw2pVKJtWvXonfv3vDx8UF0dDSWL1+O5s2bp7sjF+4udnBOp0+yJqLku3frBpbP/w0+Rw8aHe6Wwx0t23dGq45dkTd/wTSOLmmyZc+Bdl17ol3Xnnjz6gUO7NyGfds36d17Wa1W4+8De/D3gT2oVP0r9BsxFhWr1WRSTURfpAIFCmDhwoVmjduiRQu0aNHC6LBGjRqhUaNGKRkaEVGyCSFw8uhBLJk7E4/u3zUYbmtrh4Yt26J9154oVb5Sus4H8xYohP4jvkefoaPxv5PHsWPDapz955jeCRbaK/cqVquFIWMnoWylqjJGTERknEUWlPPly4c5c+aYNW6TJk3QpEkTo8Nq166N2rVrp2RoRERp4v7tm1g+/zecPHLA6PAadbzQoXsf1KjbIEkP8rAUOdxzo/eQ79Br8EhcvXgOW9Yswz9HDkCj0UjjXD53GpfPnUb5KtXRf+Q4VKnJz3UiIiKijOB/PsexZM5Mo2cku2bJio49+6HDt32QOUtWGaJLPSqVCl95NcJXXo3wwvcZNq5agt1b/tJ7mN+/58+gd7umqFW/EQaPmYRinqVljJiISF/6rUIQEWVAL/18sfC3n3Fk706DYTa2tmjW9ht06TMQHkVLyBBd6lEoFKhQtQYqVK2BV8/9sG39SuzcvE7v4StXL57DoC5tUL1OfQwf/xOKliwlY8RERERElFRPHv6HeT9Pwv98jhsMc8+TF937D0PLb7rA3t5BhujSVu58+fH91F/Qb/gYbF23AtvWrUTgxwBp+JkTR3H2n2No0b4Thoz9AW45csoYLRFRjPR1/wciogwqOOgT/vxlKtp5VTUoJtvZO6D7gGHYf/Y6fvh1foYrJsflnicvRkyciv1nrmPQ6Ilwcc2sN/yczwl0aVYHP44ajNcvn8sUJRERERElVlBgIOb+PBEdG9cyKCa75XDHhBlzseufS/imR98vopisK3PWbBg4agL2nrmKAd+Ng6POPfKFENi3fTO+rlcFqxf9johwy39gFxFlbCwoExHJSK1Ww3vjWrSpUwnrlsxHZESENExbSN535ipGTJyabh48klIyubig7/Ax2Hf2GoaNnwJnF1dpmBACB7y3oF39aliz+A9ERUbKFygRERERxUuj0WDn5nVoU7cSNq1aCnV0tDTMJXMWjPphOnafuoz23XrB2sZGxkjl55TJGf1HjsPeM9fQY9AI2NrZS8NCQ4Kx6Ldp6NCgOs6dOiFjlET0pWNBmYhIJvdu3UDPNo0wc+IoBHx4L7UrFAq06fQtdvtcxoiJU5Elm5uMUcrP0SkTeg4agT2nr6DHwOGwsbWVhoWHhWLhrz+jU5OvcPHsKRmjJCIiIiJjnj56gAGdWmHG+O/w0f+D1K6yskKXPgOx2+cyuvYdDDudwikBrpmzYPj4Kdj5zwU0btVOb9gLv2cY+m17TB4xAP7v38kUIRF9yVhQJiJKY2GhIfh9+g/o3srL4AEkVWrWwaaDPvjh1/m8P1oczi6uGD7hJ+z85xKat+ukN+zpowcY1KUNJg7ri3dvXssUIRERERFpRUVFYfXCeejctDauXPif3rDqdepj6+HTGP3jTL2r0MhQzlx5MHPBCqz2PoSSZcrrDTu0ezvae1XD3u2bIISQKUIi+hKxoExElIbOnDiGDg1rYMOKRVCr1VJ7voIe+H31ZizeuJMPm0uAe+48+HneYqzacRBFSnjqDTuydye+aVgDB3dtZ1JNREREJJO7N6+heysvLJo9Xe+Wbu558uL31ZuxYN12FCxSTMYI05+ylapi3Z5jmDBjLhwzZZLaAz8GYOqYoRjS7Wu8efVCxgiJ6EvCgjIRURoIDvqEn8cOw4heHfHquZ/UbmVtjb7Dx2DL4dOo7dUYCoVCxijTl3KVq2HD/n8wespMvYeWfAr8iB9GDsCYAd3x4d1bGSMkIiIi+rKo1WqsXjgPPdo0wn93bkntCoUCnXsPwLajZ5nzJoNSqUT7br3gffw8vJq21Bt24YwPOjaqhSN7vWWKjoi+JCwoExGlsqsXz6Fzk9rYs22jXnu5ytWw+aAPBo2eCFs7O5miS9+srKzQpfdAeJ+4iMatvtYbdvLIAXRoWANH9+3k2cpEREREqezVcz8M6NQKi2ZP13vonkfR4liz8zDGTJkFB0eneKZA5nLL4Y7flq7D3BUbkD2nu9Qe9CkQE4f1w6Th/fEp8KN8ARJRhseCMhFRKomMiMCfv0xFv29a4OVzX6ndMVMmTPrld6zYth+FihaXMcKMwy1HTsxcsBKzl65D5qzZpPbAAH9MGNoXP4wciJDgIBkjJCIiIsq4Du7ajo5NauHqxXNSm0qlQv+R32PjgZMoXaGyjNFlXHUbNcO2Y+cMni9yeM8OdGpcC5fPnZEpMiLK6FhQJiJKBc8eP0TPNo2wbsl8vbNjK1arha2Hz+Drzj2gVPIjOKXVb9oS24/9Dw1btNFrP7R7O7o2r4e7N6/JEhcRERFRRhQWGoIfRw3GDyMHICQo9uB9nvwFscr7EAZ8Nx7WNjYyRpjxZXJ2xs/zFuPXxWvg4ppZan/z6iUGdWmDVQvmQqPRyBghEWVErGYQEaWwo/t34duW9XH/zk2pzdrGBt9Nnoalm3fDPU9eGaPL+DJnzYZfFq3GL4tXwyVzFqnd7+lj9GzbGBtXLuYtMIiIiIiS6emjB+jRphEOeG/Ra2/dsRs2H/JB6fKVZIrsy9SgeWtsPXoG1evUl9o0Gg0Wz5mB4T2/QYD/BxmjI6KMhgVlIqIUEhkRgV8mj8WEIX0QEhwstRctWQob9p9At35DeFZyGmrYvA22HD6FitVqSm3RUVGYN20yRvbujMCPATJGR0RERJR+HTuwG9+29MKj+3elNhfXzJi9dB1+/O1P3itZJm453LFg3XaM+WkWrKytpfZzPifQpWltXLt0XsboiCgjYWWDiCgFPPd9it7tmmL7X6v02r/p3hfrdh9D4WIlZYrsy5Y9Zy4s2bQbA74bp1fMP3PiKL5tUV/v6eNEREREFL+oqCjMmToB4wf3RmhI7AkUnuUqYONBH9Rv2lLG6AgAFAoFOvcagFU7DupdGfn29Sv079gSm9cs49V6RJRsLCgTESXT+dP/4NsW9fXuz+vo5IRZi1Zh3LTfYGNrK19w9PmBMOOwbMtevadgv/B7hp5tG+PwHm8ZoyMiIiJKHwL8P2BIt6+xefUyvfaOPftj1faDcM+dR6bIyJhS5Spi44GT+MqrsdSmVqsx56cJmPb9cERGRMgYHRGldywoExElkRACG1cuxrDuHfAp8KPUHnOLi3/QqEVb+YIjAxWq1sCmQ6dQ9au6UltEeBgmDe+HedMmIzo6Wr7giIiIiCzYw/t30KNVA/x7/qzU5uDohFkLV+L7qb/wwXsWysU1M+at3IjhE36CSqWS2vds24iBnVvj/ds3MkZHROkZC8pEREkQGRGBqWOHYt60yXpPTW79TVes2XUE+Qp6yBgdmZI5S1YsWLcdPQaN0GvfuHIxhnRrx/sqExEREcVx6u/D6NW2CV74PZPa8nsUwfo9x9Co5dcyRkbmUCqV6DFwOBau3wFnF1ep/fq/F9G9lZfeVZZEROZiQZmIKJHevXmN/h1bYt/2zVKbSqXCuJ9/ww+//Qk7O3sZo6OEqFQqDB8/Bb8sXg17B0ep/fK50+jVtjH8nj6WMToiIiIiyyCEwNol8zGqb1e9+yVXr1Mf63YdRcEixWSMjhKrSq06WL/vODyKFpfa3rx6iT7tmuH4wb0yRkZE6RELykREifDw/h30aNMQN69eltpcMmfBog078U2PvlAoFDJGR4nRsHkbrN19BHkLFJLanj1+iB6tG+LqxXMyRkZEREQkr+joaEwfPxILfpmq9wC3Ln0G4o/VW5DJxUXG6Cip8uYviDW7jqBOw6ZSW0REOMYN7oVNq5bIGBkRpTcsKBMRmeni2VPo3a4p3rx8IbUVLl4Sf+09jso1vpIxMkqqwsVKYv3ev1Gpeuz2C/wYgIFd2uDAzq0yRkZEREQkj9CQYIzq2xW7t/wltVlZW+PH3/7E6B9nwsrKSsboKLkcnTJhzvK/0GfoaKlNCIG5P0/CnKkToFarZYyOiNILFpSJiMxwYOdWDOvRASFBQVJb7QZNsGbnYeTOl1/GyCi5nF1csXD9drT+pqvUFh0VhR+/G4Sl82bpnZVDRERElJF9ePcW/Tu2wtl/jkltLq6ZsXTTbrTu2E3GyCglKZVKDB47CT/NXQSVzgGCzauXYdygXggLC5UxOiJKD1hQJiKKhxACK+bPxo/fDUJ0VJTU3rFHP8xZ/hccHJ1kjI5SirWNDX747U8MGz9Fr33F/NmYMeE7REdHyxQZERERUdp4+ugBerVtrPeQtlx58mGV9yGUr1JdvsAo1bRs3xkL1m2DY6ZMUts/R/ZjUOc2+BjgL2NkRGTpWFAmIjJBrVZj1qTRWDpvll77d5OnYezUX6BSqWSKjFKDQqFAz0Ej8NuStbC1tZPad21ej3GDeyEiPFzG6IiIiIhSz50bV9G7XVO88HsmtRUvVRZrdh1BwcJFZYyMUlvVWnWxaschZM/pLrXdvHoZ/To0x9vXL2WMjIgsGQvKRERGREVGYvKI/vDeuFZqs7G1xa+L16BbvyF8+F4G5tWsFZZt2QMX18xS28kjBzC0e3sEBQbKGBkRERFRyrty4X8Y2Lk1AnXOSK1Rxwsrtu1Dtuw5ZIyM0kqR4iWxbs8xFC1ZSmp7/OA++rRrBr+nj2WMjIgsFQvKRERxhIWFYnS/bji6b5fU5uziiiUbd6FB89YyRkZppXSFyli14yBy5MottV258D/0/aY53r15JWNkRERERCnnzIljGPpte4QEB0ttrTp0wbxVm3hrty9M9py5sHzrfr3bm7x87os+7Zvhvzu3ZIyMiCwRC8pERDqCAgMx9Nv2OHvyb6ktW/acWLF9P8pVriZjZJTWChYphtXeh1CoSDGp7eG9O+jboQVePfeTMTIiIiKi5Dt2YDdG9euKiIjY23p17TsYP85eAGtraxkjI7lkcnbGwr924CuvxlLbh3dv0a9jC1y7dF7GyIjI0rCgTET02Yd3bzGgUyu9ZCl33vxY5X0QhYuVlDEykkvOXHmwcsdBlK1YRWp7/uwJ+n7TnJf/ERERUbq1Z+sGTBzaF2qdBw8PGDUe302exlu7feHs7Owxe9l6NG3TQWoL/vQJQ75tj8vnzsgYGRFZEhaUiYgAvHn1Av2+aYH7d25KbR7FSmCV90HkyVdAvsBIdi6umbFo407UqOMltb1+8Rx9O7TAk4f/yRgZERERUeJtXrMMP38/HBqNRmob/eMM9B/xPYvJBACwtrbGz78vQcee/aW28LBQDO/ZERfP+MgYGRFZChaUieiL9/rlc/Tv2ArPHj+U2kqVr4gV2/bDLYd7PD3pS2Fv74C5KzagTqNmUtv7t6/Rv2NLPLh3R8bIiIiIiMy3adUSzPlpgvRaqVTix9/+RJc+g2SMiiyRUqnE2J9moc/Q0VJbRHgYRvbujHOnTsgYGRFZAiu5A4jP+fPncfLkSURHR6NSpUpo3Lix2UdMIyIisHfvXty5cwd2dnaoXbs2qlevnnBHIvqivHrxHAM6tcIL36dSW5WadTB3xV98EAnpsbG1xa+L1+CHkQNwbP9uAID/+3cY0LElFm3wRonS5WSNj4gytsjISCxfvlwvNx42bBhcXFzM6v/48WMsW7ZMLzfu378/bG1tUzlyIrIUm1YtwdyfJ0mvVVZWmPHncjRs3ka+oMiiKRQKDB47CVbWVlj2+68AgIiIcIzq2xVzlq1HzXoNZY6QiORikWcoCyEwYMAAeHl54fnz5wgKCkKPHj3QrFkzREREJNj/0aNHKFWqFEaPHo2goCA8fvwY9evXR8+ePfUu6yGiL9ur534Y0LGlXjG5ep36+GP1ZhaTyShra2tMn78czb/uKLUFfgzAwC5tcPPqZRkjI6KMLCwsDHXr1sXcuXPRokULdO3aFXv27EGFChXw5s2bBPv/888/KFOmDC5duoSuXbuiUaNGmDt3LmrWrImgoKA0WAIiktum1Uv1islW1tb4bclaFpPJLP1HjsOg0ROl15ERERjd/1ucOn5ExqiISE4WeYby2rVrsXz5chw4cADNmsVcXtyrVy+ULVsWP//8M2bMmBFv/y5duiAwMBB37txBtmzZAAAdO3aEl5cXypcvjxEjRqT6MhCRZXvp54sBnVrh5XNfqa1GHS/MWf4XbO3sZIyMLJ2VlRV+mrsINra22LV5PYCYB5UM7d4eSzftQYnSZWWOkIgymnnz5uH8+fO4ceMGSpUqBQDw8vKCh4cHxo0bh7Vr15rsq9Fo0KtXL+TJkwdHjx6FlVVM+l+vXj2ULFkS06ZNw2+//ZYWi0FEMtm0einmTo0tBlpZW+PXxWtQV+dWXkQJ6Tt8DFRWVlj4688AgKjISIwd0B2/LVmLOg2byhwdEaU1izxDedGiRShevLhUTAaA4sWLo3nz5li6dCmidZ5EG9fjx49x8eJFdOvWTSomA0D9+vVRtmxZzJ8/P1VjJyLL99LPF/07tdQrJtes15DFZDKbUqnEpFm/6z2oJPjTJwzp9jX+u3NLxsiIKCNas2YNqlSpIhWTASBLlixo27Yttm3bhtDQUJN979y5g2fPnqFdu3ZSMRkAChcujEqVKmH16tW8go8oA9u8ZpleMVllZcViMiVZr8EjMWLiVOl1dFQUxg3uhf/5HJcxKiKSg8UVlENDQ3HlyhVUq1bNYFiNGjXg7++P27dvm+z/6tUrAECuXLkMhuXOnRtPnjyBn59fygVMROnK65fP0b9TS7x6Hvs5UKt+I8xZtp7FZEoUhUKBsT/NQsce/aS2wI8BGNS1LR79d1fGyIgoIwkICMCjR49Qvnx5g2Hly5dHWFgYbty4YbL/p0+fAACurq4Gw1xdXfHhwwc8fPjQYBgRpX/b1q3UewAfi8mUEroPGIbRP8ZeNR4VGYkx/b7F5XNnZIyKiNKaxd3ywtfXF0II5M6d22CYtkj87NkzlC1r/JJid3d3aTrGpg0AT548Qd68eY32f/v2Ld69e6fXpk2y1Wo11Gq1mUuSPGq1GkKjgVAAGo3F1f1TnNB8Xl6NGjxHJv2z1O354d1bDOrS1qCY/Ovi1bCytoJGkzbv7/RGITQQGmFx29NSjPpxOiIjI6TbX3z0/4CBndtg2Za9KOBRWObo9Fnqe5OSJiNtT+1yqNVqQJj3AObkSKt8LiW8fPkSAODm5mYwLHv27ABiT6gwplChQgCAW7f0r57QaDS4c+cOAODFixcoWrSo0f7MjVNORnrPUgxL3qb7vbfg1x+/l16rrKzwy6JVqNOwCXPeeDDvNU+nXv0RHR2N+TOnAIh5UN/I3p2w8K8dKFOhsszRGbLk9yolTUbfpukhN7a4gnJYWBgAGH3itN3nswfju6yvUKFCqFSpEjZu3Ijx48dLRegDBw5IibR2HsYsXrwYU6dONTosODgYgYGB5i1IMoWER0ETEQK1SoEoy9tMKU5oNIiOiNkuCmX6/JFAsSxxe376+BFDenaC75NHUlvNul6YPm8BFJpoRIWZvpXOFy8yEuroSKjDQi1me1qa0ZN+QkRYKA7u3gEA8H//DoO6tMHidVuRJ38BeYPTYYnvTUq6jLQ9RWQkohVW+PTJCkpF6ifNwcHBqT6PlBIVFQUAerer0LK2tgYAREZGmuyfM2dOfP3119iyZQv69++PmjVrAgBmz56NZ8+e6c3DGObGKScjvWcphqVu03+OHsK072OfG6RSqTB93kLU/KoOosJCZIwsHWDea7aO3Xog5NNHrFz4OwAgLDQUI3p2xII1m1GsZKkEeqctS32vUtJl9G2aHnJji8vGHBwcAADh4eEGw7SFYO04pmzatAnNmjVD+fLl0apVKwQFBeHIkSNo2LAhjh07BhcXF5N9Bw8ejA4dOui1PXz4EG3atIGTk1O8fVOSsI6E0l8DlQqwtrdOk3nKSXw+Sm5t7wCFUiVzNJRclrY9g4OCMGpgLzz6757UVrlmbfy6dC1sbXmbi4SobGwAAEoL2Z6W6sc5C6ERwOE9MUXl92/fYFifrlixdR/c8xi/KiatWdp7k5InI21PRaQVrOys4OzsApUy9ZNmJyenVJ9HSnF2dgYAhIQYFoGCgoL0xjFl1apV6N27N+rWrYtSpUohKCgISqUSvXr1wpo1a/SeOxIXc+OUk5HesxTDErfp/3yOY8rY4dK90RUKBX6auwgNWraTObL0gXlv4vQfNQGR0dFYv3QBgJjfXSP7dceyLXvgUbS4zNHFssT3KiVPRt+m6SE3triCcr58+aBUKvH8+XODYdq2ggULxjuNIkWK4Pbt2zh69Cj+++8/ODs7Y/bs2ViwYAGOHz8OT09Pk32zZ88uXT4Yl0qlgkqVNjuqSqWCQqmEQqmAMgO+OeLSAJ+XV/VFLG9GZ0nbMywsFKP7dcOdG1eltrIVq2Deig2wt3eUMbL0Q6GI+SyCBWxPS6ZUqjB13mKo1dE4tn83AODNyxcY1uMbrNx+AFmyGV6untYs6b1JyZeRtqd2OVQqVZokzWmVz6WEfPnywdbWFk+ePDEYpm0zdbsKLVdXV+zcuROvXr3CgwcP4OzsjNKlS2P48OGwt7dnbpxGMtJ7lmJY2ja9evEcvh/YC9E6Vx1MnDkXzdp+I2NU6Qvz3sQbPv4nRIRHYOva5QCAwAB/DOnWDiu3H0C+gh4yRxfD0t6rlHwZfZumh9zY4grK9vb2qFKlCv73v/8ZDDtz5gzc3NxQsmTJBKdjY2ODFi1aSK+FENi/fz+aNm2KTJkypWjMRGSZIiMiMHZAd1y5EPt5UsyzDOav2QoHx/RzdhqlH1ZWVpj2xzJERUXh5JEDAIBnjx9iWI9vsGzLHjhliv8sQiKiuKysrFC/fn2cOnUK0dHRere++Pvvv1G4cGF4eJj3g93d3V163khYWBh27tyJLl26GL3VHBGlL3duXMWIXh0RER57e8fvJk/D1116yhcUfREUCgXGTJmJ8LBQ7Nm6AUDMs2uGdGuH1TsPwS2Hu8wRElFqsMgbjQwbNgwPHz7E9u3bpbarV6/i8OHDGDp0qF7l/OjRoxg/frx0DzgAuHnzpvREa60ZM2bgyZMn+PXXX1N/AYhIdmq1GpNH9Mc5nxNSW6EixbBogzcypdHlufRlsra2xqwFK1GlZh2p7d6t6xjVtyvCw03fw5+IyJQJEybg3bt3mDFjhtS2bds2nD17Fj/88IPeuGvXrkWDBg30HsJ37Ngx3LhxQ3odFBSEXr16wcrKCrNmzUr9BSCiVPXkwX0M7d4BITr3wOw/8nt06zdExqjoS6JUKjFp1u9o0rq91PbyuS+Gdu+AoDS61z4RpS2LLCh36dIFo0ePxrfffotvv/0WAwYMQL169dCmTRtMmDBBb9xTp07h119/hZ+fn9T2+vVrVKlSBV27dsWIESNQqVIlLFu2DIcOHYr3kj4iyhiEEPjtx3E4fmif1JYnf0Es3rgTmbNklTEy+lLY2NpizvL18CxbQWr79/xZTBjSJ96HXxERGfPVV19h/fr1mD9/PkqUKIGKFSuiZ8+emDVrFrp376437sOHD3H8+HF8/PhRasuZMyf69euHYsWKoWbNmsidOzc+fvwoXf1HROnX29cvMaR7ewQG+EttXfsMQv+R42SMir5EKpUKP81dhFr1G0ltD+/dwcg+nXlSBVEGZHG3vNCaM2cOevXqJV3e17NnT1SvXt1gvMaNG8PJyQkFChSQ2ho2bIhz587h6NGjeP36NRo3boz69evDzo4P3yL6Eqz8cw52bFgtvc6e0x1LNu7i5VaUphydMuHPddvQt30zPHn4HwDg1N+HMe374fhp7iIoM+DTiIko9XTt2hXt27fHjRs3EB0djVKlShm9jVuvXr1Qt25dlC5dWmorXbo0Lly4gEePHuH169fw8PBAzpw50zJ8IkoFQYGBGNq9A968fCG1te7YDd/9MB0KRerfc5MoLmtra/yyeDWGdP0a1/+9CAC4duk8Jgzpg9nL1uvdtomI0jeLfjd7enomeEbxV199ha+++sqgPXPmzOjYsWNqhUZEFmrn5nVYOi/28t1Mzi5Y+NcO5MqbT8ao6EvlmjkLFm3wRu92TfH6RcyDZQ/s3ApnV1eM/nEmf+wRUaLY2tqicuXK8Y7j4eFh8p7K8Q0jovQlIjwco/t3w6P7d6W2Og2bYuLMecwvSFb29g74ffVm9OvQHI/+uwcg5qSKGeNH4sfZC7h/EmUQPD2KiDKMk0cPYtbE0dJrW1s7/L56MzyKlpAxKvrS5XDPjcUbdiJz1mxS2+bVy7BhxSIZoyIiIqL0Sq1W44fvBuLf82eltrIVq2DGghU8A5QsgotrZiz8awdy5s4jte3dvgkLfpkqY1RElJJYUCaiDOHapfOYOLQvNBoNgJgHQ8xcuBLlK1eTOTIiIH+hwliwbjscdS5P/2PGjzi6b6eMUREREVF6I4TA3KkTcfzgXqmtYOGi+H31ZtjbO8gYGZG+7DlzYdFf3nDVeYbNuqV/YuvaFTJGRUQphQVlIkr3Hv13FyN7d0ZERLjUNmHmXNRt1EzGqIj0lShdFnOW/QUra2up7cdRg3Hlwv9kjIqIiIjSk7VL5mPrutiCnFsOdyxYvwMurplljIrIuAIeRfDn2q2wd3CU2uZMnQCfY4dkjIqIUgILykSUrr159QLDundA0KdAqW3AqPH4unMPGaMiMq5KzdqYMnuh9DoqMhKj+nbF48/3lyMiIiIyZd+OzVj468/SaydnZyxYvx3uOrcVILI0nmUr4Lela6FSqQAAGo0GE4b2xe3rV2SOjIiSgwVlIkq3goM+YXjPjnjz6qXU1q5bL/QbPlbGqIji16xtBwwd96P0OuhTIIb1/Abv3rySMSoiIiKyZBfP+GDauBHSa2sbG8xdvgFFipeUMSoi89So44UJM+ZKryPCwzCyd2e88H0mY1RElBwsKBNRuhQdHY0JQ/rg4b07Ulv9Ji0w7uff+ORgsng9B41A+269pdevXzzHiJ6dEBIcJGNUREREZIke/3cPYwf1gDo6GgCgUCgw7Y+lqFS9lsyREZmvbefu6D1klPTa//07DO/5DT4FfpQvKCJKMhaUiSjdEULgtynj8D+f41Jb6fKVMG3+MulSKiJLplAoMHbqL6jdoInUdv/OTXw/qCeioqJkjIyIiIgsif/7dxjRqxOCP32S2r6bPA0Nm7eRLyiiJBo8dhKatG4vvX766AHG9P8WkRERMkZFREnBgjIRpTsbVy6G94Y10uvcefNj3sqNsLOzlzEqosSxsrLCzAUr4FmugtR2/tQ/mDVxFIQQMkZGREREliA8PAyj+nbFy+e+UluHb/ugS59BMkZFlHQKhQJTZi9AxWo1pbZ/z5/F1LHDoNFoZIyMiBKLBWUiSldOHN6PP2bE3n82k7ML5q/ZgizZ3GSMiihp7B0c8ceqzciTv6DUtmfbRvy1fGE8vYiIiCij02g0mDpmKG5evSy11azbAGN+msXbu1G6ZmNriznL/0IBjyJS2+E9O7BkzkwZoyKixGJBmYjSjdvXr2DyiAHS2ZsqKyvMXrYeBYsUkzkyoqTLks0NC9Ztg4trZqntz1k/4eTRgzJGRURERHJaOncWju7bJb0uXLwkZi5cCSsrKxmjIkoZzi6u+HPtNr2TglYvmocDO7fKGBURJQYLykSULrz088V3fbogIjxMavvh1/moXOMrGaMiShn5Cnpg9rL1sLK2BhBzn/BJw/vj3q0bMkdGREREaW3v9k1YtXCu9DqrWw78sXoLnDI5yxgVUcrKnS8//li9GbY6ty2cNm4Erl++IGNURGQuFpSJyOIFB33CyN6d8OHdW6mt7/AxaNm+s4xREaWsitVqYtLMedLr8LBQfNenM969eSVjVERERJSWLp87gxkTvpNe29rZ44/Vm+CeO4+MURGlDs+yFTDt9yXS66jISIwZ0B2vnvvJGBURmYMFZSKyaGq1GpOG98ej/+5JbU1at8fAURNkjIoodbT6pit6DBwuvX77+hVG9e2GsLBQGaMiIiKitPDc9ym+H9gD0VFRAGIeYDbjz+UoWaa8zJERpR6vZq0waPRE6bX/+3f4rm8XhIYEyxgVESWEBWUismiLfpuGMyeOSq/LVqqKH3/7kw8joQxr6LgfUbdxc+n1nRtXMWXUYD75moiIKAMLCQ7CqD5dEPgxQGobMXEq6unkBEQZVZ9ho9G4VTvp9YO7tzF5xADmv0QWjAVlIrJYB3duw7qlf0qvc+bOgznL1sPWzk7GqIhSl1KpxPQ/lqKYZxmp7fjBvVg6b5aMUREREVFq0Wg0+OG7QXpX5LX+piu69RsiY1REaUehUODH2X/Cs1wFqc3n2CEs/G2ajFERUXxYUCYii3Tz6mVMGz9Cem1n74DfV27SexIwUUZl7+CI31dtRLbsOaW2VQvm4tDuHTJGRURERKlh6bxZ8Dl6UHpdtmIVjJ8+h1fk0RfFzs4ec5dvQA73XFLbuiXzsd97i4xREZEpLCgTkcV5+/olxvTvjsiICKlt2u9LULRkKRmjIkpbOdxz4/dVG/WffP39cNy9eV3GqIiIiCglHd23E6sWzJVe53DPhdnL1sPG1lbGqIjk4ZYjJ+at3AQ7ewepbfr4kbh++YKMURGRMSwoE5FFCQ8Pw+h+3+L929dS24BR41G/aUsZoyKSR8ky5TF13iLpdUREOEb37wb/9+9kjIqIiIhSwt2b1/HTmGHSa1s7e8xdsRFZ3bLLGBWRvIqXKoNpvy+RXkdFRmLswB54+/qljFERUVwsKBORxRBCYNr3I3DnxlWpzatZK/QdNkbGqIjk1bB5G/QeMkp6/eblC4wb3AtRn58AT0REROnPh3dvMbp/N0SEh0ltP81ZgBKly8oYFZFlqN+0JQaPmSS9/vDuLcYO7KF3BSsRyYsFZSKyGGuXzMfhPbH3iC1WsjSmzl0EpZIfVfRlGzRmIr7yaiy9vnLhf5g7daKMEREREVFSRUZEYOzAHnjz8oXU1nvIKDRq+bWMURFZlt5DR6FB89bS61tX/8UvP4yFEELGqIhIi1UaIrIIp/4+jEU6T/HNks0Nc1duhL2Do4xREVkGpVKJaX8sQ36PIlLb9r9WYefmdTJGRURERIklhMAvP4zVuyds7QZNMGgMDxQT6VIoFJgyewEKFy8pte3ZugE7NqyRMSoi0mJBmYhk9/TRA0we0V862mxlbY3ZS9fBPXcemSMjshyZnJ0xb8UGOGbKJLX9+sP3uHbpvIxRERERUWJsX78Ke7ZukF4XKlIM0/5YyivyiIxwcHTC3BUb4OziKrXN/mk8rl48J19QRASABWUikllIcBDG9P8WIcHBUtvEGXNRrnI1GaMiskwFPIpg5p8roVAoAADRUVH4flBPvHn1IoGeREREJLdrl85jzs+xZyI7u7hi3qpNcMrkLGNURJYtT74CmLlwpXTQRR0dzfyXyAKwoExEshFC4KfRQ/Dk4X9S2zfd+6J1x24yRkVk2WrVb4jBYydLrz+8e4uxA3ogIjxcxqiIiIgoPu/evMb3g3pBHR0NIOZ2Vr8sXo28+QvKHBmR5ateuz6GjftReu3//h3zXyKZsaBMRLJZu2Q+ThzeL70uW6kqRv0wXcaIiNKHXoNHomGLNtLr29evYMbEUXxICRERkQWKiozEuMG98OHdG6lt6LgfUbVWXfmCIkpnvh0wDI1bxT648vb1K/hl8hjmv0QyYUGZiGRx7tQJLJ4dWzzOlj0nfl28BtY2NjJGRZQ+aB9SUrRkKantgPcWbP9rtYxRERERkTG/z/hR7yF8Xk1bovuAYTJGRJT+KBQK/PDrfBQp4Sm17d2+CdvXr5IxKqIvFwvKRJTmXvr5YtKwftBoNABiHsL325I1cMuRU+bIiNIPewdHzF2+AS6Zs0htc3+eiJtXLskYFREREek6sHMrtq5dLr0uWLgopsxZKD0PgYjMJ+W/rpmltjk/T8RVPqSaKM2xoExEaSo8PAxjBnRH4McAqW30jzNRtlJVGaMiSp9y5c2HWToPKdE+pM///TuZIyMiIqJ7t25gxvjvpNeOTk6YvWw9HJ0yyRgVUfqWO19+zFq0Su8hfeMH98aHd29ljozoy2KxBWUhBHbv3o2RI0di6NChWLt2LaKioszu/+nTJyxbtgyjRo3C8OHD8ccff+D169epGDERJUQIgZkTRuH+7RtSW8sOndHh294yRkWUvlWtVReDxkySXr99/QoThvZF9OeH/hBRxpPc+0XyfpNEqS/wYwDGDuiOiIjYh4b9NHcxChYuKmNURBlD1Vp1MWz8FOn1+7evmf8SpTGLLChHR0ejbdu26Nu3L7JlywYPDw9MmzYNNWrUwKdPnxLsf/78eRQoUAB//vkncuXKBQ8PD3h7e6NgwYLw9vZOgyUgImO2rVuJAzu3Sq+LlyqL8dPn8JI/omTqOWgE6jRsKr2+fO603j3KiSj9u337Npo0aQJ7e3vY2dmhWrVq8PHxMauvEAIrVqxA+fLl4ejoCAcHBxQvXhy//PILIiMjUzlyoi+PWq3GpOH98PK5r9TWe8go1G/SQsaoiDKWb/sP1XtP/Xv+DBbPmSFjRERfFossKC9ZsgR79uzB3r17MXnyZHz33Xfw8fHBnTt3MHny5AT7jxs3Dmq1GmfOnMGYMWMwYsQInDhxAvnz58fw4cPTYAmIKK6rl85j7rTYsyhdMmfBnGXrYWdnL2NURBmDUqnE1LmLkbdAIalt3dI/cfzgXhmjIqKU8vTpU9SuXRv29vbw8/ODv78/qlevjkaNGuH8+YTvGzljxgz0798fbdu2xevXrxEQEIDx48dj0qRJ6N+/fxosAdGXZdnvv+CczwnpdbXa9TBw9AQZIyLKeBQKBabMWYh8BT2ktnVL5uOfIwdkjIroy2GRBeXly5ejXLlyqFGjhtSWJ08etG7dGmvXrkVERES8/d++fYt8+fIhc+bYG7VbW1ujVKlSeP/+PS/zI0pj/u/fYcKQPlB/vgRJqVRi1sKVcM+TV+bI6P/s3XdYU9cbB/BvEqbs4cC99957dGnrqNbaWvceiBtFhooCgqg4wIUVt9VWrXu0atVqq/6ctWrdCwcuZM8kvz+QCyFsAjcJ38/z+LTnJOfeN1xCTt57BukPCysrLFq7GcbpbtLMm+GExw/uiRgVEWnC/PnzkZCQgA0bNsDe3h5mZmZYsmQJKlWqhJkzZ+bYftu2bahSpQrmzJkDCwsLmJiYYNiwYfjiiy/w008/sW9MpEFnThzD+sAlQtmhfAX4rFgHmUwmYlRE+sncwhL+azap9H/nTnfEs8cPRYyKqHjQSEI5OTkZu3btwsSJEzFgwACh/vDhw3la9xgAoqKi8O+//6J58+Zqj7Vo0UJ4PDu9e/fGnTt38M8/aeu0vnr1CmfOnEGvXr04vZ6oCKVM+RuDN2EvhboJM2ejVfvO4gVFpKdq1K4LD79lQjkmOhozxg5BbEy0eEERFUOa7BsDwN69e9GxY0dYW1sLdVKpFL169cLZs2cRFhaWbXtbW1vI5fJM47S2tmbfmEhDXjx7irlTxwtlY2MTLFqzGdY2tiJGRaTfatSuCw/fpUI5JioKM8YNRVxcrIhREek/g4IeIDIyEl27dsX58+dhbW2NDx8+YPv27QCAnTt34sOHDyod6ZyEhoYCAMqUKaP2mIODAwDg2bNnaNasWZbH8Pb2ho2NDb744gs0b94cRkZGOHv2LAYPHgwvL69sz//69Wu8efNGpe7+/fsAUhJjmXXGC4NcLodSoYBSAigUWjmQXKOUio+vVyGHQuxgqMDSX8+1yxbh4rm0NR47ff4lBo+ZAIWiaN5LVHASpQJKhZLvTx3R7etvcOPKRfy8eT0A4OG9O5g/cxJ8VgQDSgX/1uoRffrsTH0dcrkcUBZ+crMw+3OF0TcODw9HrVq11B6rXbs2lEolbt68idKlS2d5DG9vb/Ts2RPTp0/H1KlTYWRkhO3bt+PUqVNYv3593l8kEalJTEjArAkjEBnxQahz8fJHnQaNxAuKqJj46pvvcP3yRezaGgIAuHf7JnzdnTFvyUreNCUqJAVOKLu5uSEyMhKXLl1Cs2bNVN6sI0eOxNy5c/PUaY6PT9kF19DQUO0xIyMjledk5d69e9i+fTtsbW3RoUMHGBkZ4e3bt9i1axe+/vprdOzYMcu2q1atwrx58zJ9LDo6GhEREbl9KQUSE58ERUIM5DIJkgp+mbSeUqFAckIcAEAi1f8Eur5LvZ7nz55WmfJXtnwFuHn5ITmed4t1SmIi5MmJkMfF8v2pIyZMc8Gtf67i32tXAAC/H9yLuvUb4LtBw/m3Vo/o02enMjERyRIDREYaQFoEX/yiowtv1L6m+8bv378HAJXRyalS6969e5ftMbp06YKNGzdi1KhRCAgIgEQigYGBAfz8/DBkyJBs23Kwhebo000gSpH+mi719sDN61eEx3p82x89+/3AQRQ6hgMpdNdUj/m49c9V3PrnKgDg0O4daNi0Bb4ZMIR/f/WQvl9TXRhsUeBM5Z49e7B3795MRwzXrVsXly9fztPxzM3NAQBxcXFqj8XGpiShLCwssmyvVCrRt29fxMXF4caNGzAzMwMATJkyBZ999hn69OmDhw8fwsrKKtP2jo6O6Nevn0rd/fv30bt3b5ibm2fZTtOUhomQvldAJgMMTdWT6/pG+bGjZWhaAhIp1xfTdUqFHK9ePsd81+lCnZGRMRau3gDb0mVFjIzyQ/bxZp6U70+dYWhqhoWrN2Jwj0/x/l1KIihosS8aNm+N2rXr8G+tntCnz05JogEMTAxgaWkFmbTwO82p/c3CoOm+carM1jlOrctp9NXy5csxdepU+Pj4wNHREYaGhvj1118xatQoPH78GIGBgVm25WALzdGnm0CUIvWanjh6SJgZBADVatbGNNe5SIqLETE6yhcOpNBZEgBeSwIx/NsewkyBxZ6uqF6jBmrXrc+/v3pG3z9TdWGwRYF7Y2/evEHdunWFcvoOrYGBARITE/N0vEqVKsHAwABPnz5Ve+zJkycAgGrVqqk9lurZs2e4desWJkyYICSTU+Pq168fTp48iYsXL+Lzzz/PtH2pUqVQqlSpTB+TyWRFtpmCTCaDRCqFRCqBVMe/JOaGAvj4emXF4vXqu4TERMyZPhER4e+FuhnzfFG3YVMRo6L8kkhS/haB70+dUqZsefiuXI/xA3pDoVBAnpwMt0ljsHHXQdibWfBa6gF9+uxMfR0ymaxIEsqF2Z/TdN/Y3t4eABAeHq72WGqdnZ1dlu3j4uIwa9YsdO3aFa6urkL9wIEDcfXqVSxZsgROTk6ZLqkBcLCFJunTTSBKoVTI8fTxQyz0THtvlTAzg9+qDbCwtRcxMsovDqTQbRWr14LX8jWYMvwHKJVKJCUlwmPqBGzadwxmJqb8+6tH9P0zVRcGWxQ4oezg4ICrV6+iQ4cOAFQ7zefPn0eVKlXydDwjIyO0a9cOZ8+ehVKpVDneqVOnUK5cuSw7vEBKRx1IG82cXkxMyh3izJbTICLNWe7riZv/XBPK3b/5Hn1+GCpeQETFVPM27THe2R0r/VP2Dwh78RzzZ03Dsg07dD4BSaStNN03Llu2LOzt7fHff/+pPXbr1i1IJBI0aNAgy/ahoaGIj49HnTp11B5LTXzfu3cvy/41B1tojj7dBKIUsXGx8JjmhJh0I7tm+69A1RpZf18l7caBFLqvfZcvMGbKTKxduhAA8PL5M3g6T4R/0Dr+/dUj+v6ZqguDLQo8Lrx3795wdHQU1lJL9eTJE0yfPh19+vTJ8zGdnZ3x9OlTBAcHC3WnTp3CiRMn4OzsrNIx37NnD4YNG4Z79+4BSOl0t2jRArt27cKdO3eE5719+xarV69GmTJl0Lp16zzHRES58/uhvdi5cZ1QrlazNlx9FnMzBCKRDBs/Ge06fyaU/z7zB7YEB4kYEZF+K4y+cd++fXHmzBm8fftWqEtOTsa+ffvQpUsXYRRzan18fLywHEbZsmVhZGSE69evqx336tWUdSbzmuQmohT+c2fhwd20mz3fDx2NL3rk/T1ORJo1atIMtO30qVD+69QJbAtZK2JERPqnwAnluXPnIiEhAXXq1EGLFi2gVCrRpUsXYRRE+ql1udWjRw/4+/tj8uTJ+Pzzz/H111+jW7ducHR0xOTJk1Wee+XKFWzatAlhYWFC3Y4dO9C0aVM0btwYX331FXr37o2aNWuiRIkS2L9/P0xMTAr2ookoU08e3ofXzElCuYRZyjqupiXMsmlFRIVJKpVi3tLVKO2Qtn756sULcPXi3yJGRaS/CqNvPHv2bFhaWmLAgAF48OABXr58ibFjxyIsLAyLFi1Sea6npydMTU1x7tw5AICZmRkmT56MkydPws3NDU+fPsWrV6+wcuVKrF27Fr1790a9evUK/sKJipn9P2/DgV9+Esr1GjXFFPf5IkZERKmkUim8lq9F6bLlhLrg5Ytx7X/nRYyKSL8UOKFsZ2eHCxcuYOrUqYiOjoaVlRVCQ0Mxbtw4nD17NtsN9LIzY8YMPHnyBI6Ojhg0aBBu376NoKAgtVGOffv2xYYNG1CzZk2hrmrVqjh16hSuXLmC8ePHY8iQIThx4gSuX7+OFi1aFOj1ElHm4uJiMXP8MJUpf+6+S1Gles1sWhFRUbCxtYNv0HrIPi4LJZfL4eo0Cu/fvhE5MiL9Uxh943LlyuGvv/6CtbU12rZti/r16yM0NBSnT59G06aq+xMYGhrC2NgY0nQb1CxcuBDbt2/HuXPn0KZNGzRs2BCbN29GQEAAdu7cWeDXTFTc3Lt9E34eM4SypZU1/FaGwMjYWMSoiCg9axtb+GXo/7pPGoPw9+9EjoxIP0iUmW0ZTSpu3ryJ+vXr499//y2yERwfYhNx6XE4DGQSWOroxiN5oVDIkRQXA0NTM71c/0bfKZVKeDo74eCutFEa3w4YChefxbyeekB69ADkSQlQdu/D66njNq8NxPIFc4Vyqw6dEbjplyJbA5U0S58+O19FxMPa1BCtqtoVyTpxYvTt9An7xvmjT+/Z4iw6KhJDen2GJw/TlrUJ+HEbOn3+pYhRkaaw36t/MvZ/23X+7ON+IgUeX0ki0vfPVF3oG/MdREQFtm/nVpVkct2GTeA0003EiIgoMwNHjUeHTz4Xyhf+PIWQoADxAiIiItIhSqUS3rOmqCSTB40ahw6ffiFiVESUnUGjJ6D9J2nv0XOnjmPz2kARIyLSDwYFPcCOHTtyfE7//v0Lehoi0lL3bt+E/xwXoZwy5W89jIw45Y9I20gkErj7LMaIfj3xIvQpAGDtUj80at4KLdt1FDk6Iv3AvjGR/vpl83r8fnCvUG7Ssg3GTHIWLyAiypFUKsXcxYEY9FUXhL16AQBYtcgbjZu3QuMWrUWOjkh3FTih/MMPP+T4HHaaifRTbEw0Zk0YgYSEeKHOa9laOJSvgKS4GBEjI6KsWFpZYUHQjxjVrzuSk5KgVCrhMXkMth06hZKly4gdHpHOY9+YSD/dvnEdAd4eQtnWviQWBAbDwKDAX6mJqJBZ29hi/pJAOA75DnK5HHK5HG4TR2Hb4dOwsbUTOzwinVTgT7+rV6+qlJVKJZ4/f46DBw8iJiYG06dPL+gpiEhLLZzjgscP7gnlYY5T0P6Tz6FQyEWMiohyUq9RE0zz8Ib/3JTZBe/evIb7pNFYte1XfjEmKiD2jYn0T0x0FFydRiIpMRFAyowfnxXrYF+qDAdREOmIBk2aYcIMD6zwmwcACHv5AnOnOWJZyE9cT5koHwr8rmncuLHKvyZNmqBHjx5Ys2YNmjZtit9//10TcRKRljm4e4fKusmNmrXEuGmuIkZERHnx3dBR+Kz710L58vlzCF66UMSIiPQD+8ZE+kWpVMLX3RnPHj8U6kZNcuZSUUQ6aOBoR9X1lP/4HVuCg0SMiEh3FeptmCFDhmD58uWFeQoiEsGj+3fh5zFDKFtaWcMn8EcYGurmrutExZFEIoGH33JUqFxVqFsftAR/nzkpYlRE+o19YyLdc+CX7Tiy9xeh3LRVW4yaNCObFkSkraRSKeYFrEJph7JC3Up/L1y/dEHEqIh0U6EmlN++fYt3794V5imIqIjFx8fB1Wkk4mLTpvd5LlkJh3LlRYyKiPLDwtISC1dtgJFx2iaac6aOx9vXYSJGRaS/2Dcm0i0P7/4Hv9kzhbKVjS18VnDdZCJdZm1jC9+g9ZDJZAAAuVwOV6eR+BD+XuTIiHRLgRPKb9++Vfv36NEj7N+/H/369UOrVq00EScRaYkALw/cu31TKP8wYiw6ff6liBERUUHUqtcAznN9hfL7t28wZ+o4KBQKEaMi0l3sGxPph/j4OMxyGomE+Dihbt6SVShVpmw2rYhIFzRq3goTZs4WyqnrKbP/S5R7Bb61WrJkySwfq1GjBoKCuB4Nkb74/dBe7N66QSjXadAYk2Z5ihcQEWnENwOG4sLZUzhxeD8A4MLZ09i4ejlGTJgqcmREuod9YyL9sGSeGx7cuS2UB45yRIdPv8imBRHpksFjnHD5/Dmc+yNlb4OzJ3/D1nUrMWTsRJEjI9INBU4oL1q0SK3OysoK1apVQ6dOnYRpBESk20KfPoaXy2ShbGZuDt+V61WmyhORbpJIJJjttxy3/7mGF6FPAQBrlixAs1Zt0ag5R1MS5QX7xkS677eDv2LP9k1CuV6jppjoMkfEiIhI01LXUx74VSeEvXwBAAjy90Kz1u1Qr1FTkaMj0n4FTig7OztrIg4i0mJJiYlwnTASMVFRQp2771JUqFRFxKiISJMsrKywIOhHjPz2K8iTkyGXy+E2aTS2Hz4NK2sbscMj0hnsGxPptmdPHsF71hShbGZhgQVBP8LQyEi8oIioUNjY2sEn8EeM+a4HFAoF5MnJcHUahe2HT8HcwlLs8Ii0WqFuykdE+iFw4Xzc+ueqUP5mwFB07dVXxIiIqDA0aNIcTjM8hPKr56HwmjkZSqVSxKiIiIiKRlJiItycRqkMopi9cDnKV6wsXlBEVKiatGiNMVNchPLzp4/h6+7M/i9RDvI8Qvns2bN5Pkn79u3z3IaItMOZ40ex7cdVQrlarTqYPneBiBERUWEaNMYJF/86g79PnwQA/HHsIH7ZvB7fDR0lcmRE2ol9YyL9Eeg3T2UQRd+Bw/B5997iBURERWKE0zT8768/cfl8ymf60X270KpDZ/TqN0DkyIi0V54Tyh06dMjzSXhnh0g3vXoRCs/pE4SyiWkJ+AWth4mJqYhREVFhkkqlmB+wGv27dcS7N2EAgABvDzRq3gq16jUQOToi7cO+MZF+OHPiGLatXy2Uq9eui2lzfESMiIiKikwmg/fyNejfrSMiwt8DABbOnomGTVugcrUaIkdHpJ3ynFA+cuRIYcRBRFomOTkZ7pPGIOJDuFDn4uWPqjVrixgVERUFW/uS8F62Bo6DvoFSqUxZR33iKGw9cAIlzMzFDo9Iq7BvTKT7wl4+h+c0R6HMQRRExU+pMmXhuTgIU0emjEqOj4uFm9MobNz7GzeiJ8pEnhPK3bp1K4w4iEjLBC9diGv/Oy+Uv+rzHXp++4OIERFRUWrZvhNGTJiG9UFLAABPHtzDwjkumLdkpciREWkX9o2JdFumgyjmL0SVGrVEjIqIxNDxs27oP3wMdmwIBgDcuXUDy33nYoann8iREWkfbspHRGounD2FkJUBQrlS1eqY5b0IEolExKiIqKiNmeqCRs1bCeWDu37CoT07RYyIiIhIs0KCluDqxb+F8pe9+6En100lKrYmu85DrXoNhfKODcE4c/yoiBERaacCJ5SVSiWCgoLQoEEDmJqawsDAQO0fEemO8HdvMXvKeGF9RyNjY/gGrYeZuYXIkRFRUTMwMIDPinWwtLIW6nzdnfHk4X3xgiLScuwbE+mOa/87j3XLFwnlilWqwdVnMQdREBVjKd9/f4RpCTOhztPZCa9fvRAxKiLtU+CE8rJly+Ds7IzmzZsjPj4e48ePR7du3WBgYIBu3bphypQpGgiTiIqCUqmEp7OTsBEXAEz18OZGXETFmEO58pizKFAox8XGwHXCSCQmJIgYFZH2Yt+YSDdERUTAY8pYKBQKAICBoSEWBK7jIAoiQqWq1eHi5S+UI8Lfw2PyWMjlchGjItIuBU4oh4SEYNWqVdiwYQMAIDAwEAcPHsTDhw8RGxuL3r17F/QURFREdmwMxtmTvwnlzl27o9/gESJGRETaoEvX7vh+2BihnLqeHBGpY9+YSPsplUoscJ+Gl6HPhLoJMzxQp0Fj8YIiIq3So29/fNm7n1C+fP4cQoICsmlBVLwUOKF87949lY5x6h2bsmXLIjAwEDNnzizoKYioCNy99S+WL0hLEJUq44DZC5dzyh8RAQAmu3qiVt202Qo7NgTj1G+HRYyISDuxb0yk/Q7u3oHfDvwqlFu174RBoyeIGBERaRuJRAJXn8UoX6mKUBe8bCGuptu4nqg4K3BCOSEhAba2tgAAMzMzvHr1SnisSpUquH79ekFPQUSFLC4uFq5OI5GUmAgg5cNz/tI1sLaxFTkyItIWxiYm8F25XmU9ufkzJiLs5XMRoyLSPuwbE2m3Z48fYuHstBs71rZ2mBewGlIp96snIlVm5hbwDfoRBoaGAACFQgH3SaMR8SFc5MiIxKfRT8169ephx44dQvnXX3+FnZ2dJk9BRIUgYL47Hj+4J5SHO05Fi7YdRIyIiLRRparV4eqzWChHfAjH7CnjuZ4cURbYNybSLkmJiXCbOBpxsTFC3dxFgShZuoyIURGRNqvbsAmcZs4WymEvnsNr5mRhE3ui4kqjCeVRo0ZhxowZ6NixIz777DMMGTIEgwcP1uQpiEjDTh45gD3bNwnl+k2aYcxUFxEjIiJt1v2b7/FVn++E8uXzZ7Fp9XIRIyLSXuwbE2mXNUv9cOufq0L5uyGj0PGzbiJGRES6YOAoR7Tt9KlQ/uPYQezaukHEiIjEV+CE8tWraR/Io0ePRkBAAKKjo/H+/XvMmTMHnp6eBT0FERWSVy9C4eUyWSibmZvDZ8U6GH6c0kNElBkXL3+Uq1hZKK8J8MWNq5fEC4hIi7BvTKSdLp47o3IDtFrN2pjsPk/EiIhIV0ilUswLWAW7kqWFugAvd9y/c0vEqIjEVeCEcuPGjVXKU6ZMwZUrV3DlyhXMnTuXiSkiLSWXyzF7yjhERnwQ6lx9AlA+XZKIiCgz5haW8FkRDJmBAYCUvyfuE0cjOipS5MiIxMe+MZH2+RD+HnOmjhemqBsZG8Mn8EeYmJiKHBkR6Qpb+5LwWrZa2LQ+MSEBbhNHIz4+TuTIiMRR4ISyv78/wsLCNBELERWhDSuX4sqFv4Ry97798WXvb0WMiIh0SYMmzTFumqtQfv7sCfw8ZogYEZF2YN+YSLsolUp4uUzGm7CXQt1k13moUbuuiFERkS5q1b4zhoybJJQf3LmNFb6c6UDFU4ETyi4uLihfvjz69OmDgwcPcmMeIh3wz+WLCF62UCiXr1QFLvMXZtOCiEjd0HGT0LxN2gaeR/b+gsN7fhYxIiLxsW9MpF32bN+EU8cOCeV2XT7H98NGixgREemy8dNcUadBY6G8c2Mw/jzxm3gBEYnEoKAHuHPnDkJCQrB582bs3bsXZcuWxdChQzFixAhUr14938dNSkrC+vXrcerUKSQnJ6N58+ZwdHSEpaVljm2//fZbREdHZ/n4gQMHON2Qiq2oyEi4TxojfMGVGRhgQeA6mJlbiBwZEekamUwGr2Wr0b9rB0R8CAcA+Ho4o0GzFqhQqYrI0RGJo7D6xkSUd4/u3cGS+e5C2a5kKXguDhKmrBMR5ZWhkRF8VgRjYPcuiIuNAQDMm+GEHUf/hH2p0jm0JtIfBR6hXLNmTfj5+eHZs2fYv38/WrRogUWLFqFGjRro3LkztmzZkudjxsfH45NPPoGvry8+++wz9O3bF7/88guaN2+ON2/e5Nh+/PjxmDJlisq/AQMG4NixY3j58iWTyVRsKZVK+LpPw4vQp0Kdo7M76jVqKmJURKTLSpUpizmLAoVybEw03CeNRlJSkohREYmnMPrGqSIjI3H8+HEcPXo0T8tq7N+/H7t27cryn0KhyHdMRNoqMSEBbpNGIyHd+qaeS1bC1r6kiFERkT6oVLU6Znj6CuXwd2/h6TyBn6dUrBR4hHIqmUyGnj17omfPnggLC8PmzZuxfPlyDBkyBIMHD87TsZYuXYpz587h2rVraNiwIQDg888/R/Xq1eHi4oKQkJBs23/66adqdYsXLwYAjBkzJk+xEOmTA7t+wrH9e4Ryy3adMGTsRBEjIiJ90PmLr/DtoBHYtTXl8/nmtStYE+CLiS5zRI6MSDya7BsDwNq1azFt2jQ0aNAAZmZmOHfuHKZPnw4fH58c2+7evRsxMTEqdcnJydi3bx+sra3Rp0+fPMdDpO2CFs7H3Vv/CuWBoxzRtpP690Qiovzo9d1A/HX6BI4f2gcA+Pv0SezYsBYDRo4XOTKiolHgEcoZxcfH4+TJkzh27BhevHgBmUyW52Ns2LABLVq0EJLJAGBvb48+ffpgx44diI2NzfMxQ0JCYGpqioEDB+a5LZE+ePLwPvznuAhlKxtbzF+6ClKpxv8MEFExNMVjPqrWqCWUN61ejovnzogYEZF20ETf+MSJExg/fjy8vb1x/vx5nDhxAr/++it8fX0RHBycY/tNmzapjUoeMWIEAGDQoEH5iolIm/116ji2rV8tlGvVbQCnmbNFjIiI9I1EIoG771KULltOqFvhNw93bt4QMSqioqOxTNKVK1fg5OQEBwcHDBgwAI8fP4a3tzeePXuWp+OEh4fj3r17aNKkidpjTZo0QVxcHG7cyNsb9O+//8bt27fx3XffwdraOk9tifRBUmIi3CeNEdZ4AgDPxUEoWdpBxKiISJ+YmpbAgqD1MDI2BpCyxM6cqeMR/v6dyJERiUNTfWMA8PPzQ9myZTF58mSh7ssvv0SnTp3g6+ubTcuspc744+w90jfv3rzG3OkThLKxiSl8AtcJn09ERJpiaWUN72VrhUFaKd+7RyMuLu+DIIl0TYGXvAgMDERISAiuXbsGExMT9O3bFyNHjkTnzp3ztdnBixcvAAClSpVSe6x06dIqz8mt9evXA8hdh/n169dq6zTfv38fACCXy4tsp265XA6lQgGlBFAo9H8EqVLx8fUq5OCqQ5oX5O+F2zeuCeXvhoxE+08+h0JROL/PvJ76RaJUQKlQ8nrqgcJ+b1arWQuTXT2xyNMVAPAm7CXmz5iIxcGbuQFSIdCnv7Wpr0MulwPKwv9dKcz+nKb7xklJSTh9+jT69u2rNqvok08+wZw5c3D37l3UrFkz18cMCwvDoUOH0KpVKzRo0CDPMRFpK6VSiXnOTnj/Nu37nLPnAlSpnvv3BxFRXjRt1RbDJ0zF+sAlAIBH9+9iqfdsuPksETkyosJV4ITypEmT0KRJEwQFBWHgwIEFHgGcuomPgYF6aKmb6SUmJub6eDExMdi5cyfq1auHtm3b5vj8VatWYd68eZk+Fh0djYiIiFyfuyBi4pOgSIiBXCZBkuaWutZaSoUCyQkpG2ZIuASDRl04dwZb160UytVq1sb4qTOQFBeTTauC4fXUM4mJkCcnQh4Xy+up44rivdm7X3/8deo4zp06AQA4c/wodm5Yi74/5H3NWMqePv2tVSYmIlligMhIA0iL4OZDdHR0oR1b033jJ0+eICkpCZUqVVJ7LLXu3r17eUoob968GcnJyRxsUcT06SaQttqxIRjnTh0Xyl26dcfX3w3kIArKNQ6k0E+F/V4dNXE6Lp49jRtXLwEAdm/dgNYdOqPzF18VwtkI0P+/v7ow2KLAmcrLly+jadOmBT2MwMLCAgDUNg4B0jr/lpaWuT7ezz//jOjo6FxP53N0dES/fv1U6u7fv4/evXvD3NwcVlZWuT53QSgNEyF9r4BMBhiaGhbJOcWk/NjJMzQtAYmU6/hpyvu3b+DlOl0oGxubYEHQjzC3tivU8/J66heZkREAQMrrqfOK6r05d3EQBnzVGW9fhwEAgvx90LxdJ1SvVafQzlkc6dPfWkmiAQxMDGBpaQWZtPA7zebm5oV2bE33jVP7vyVKlFB7zMzMDAAQFRWVp2OGhITA0tIS33//fY7P5WALzdGnm0Da6P6d21jhm/a7WrJ0GbjM8UFyfOFNPec11UMcSKGXiuK9OscvAEO/+QqxMSmf214zJ6NmzVooWbpMoZyvuNP3v7+6MNiiwL0xTXaYgZSRFkZGRnj8+LHaY48ePQKAPI3AWL9+PUxMTHK9m3apUqUyXW4DSNmtu6g2LZHJZJBIpZBIJZDq+JfE3FAAH1+vrFi83qKgVCoxf+ZklSl/0+b4oHqtuoV+bl5P/SKRpPwtAq+nziuq96ZdydKYv3Q1HAd+AwBISIiHx+Sx2Lz/OExMTAvtvMWNPv2tTX0dMpmsSBLKhdmf03Tf2OjjTb3k5GS1x1Jn9pmYmOT6eOfOncN///2H8ePHCwnp7HCwhebo000gbRMfF4u5M6cgKSllJqtEIoHXsjWwcyiXQ8uC4TXVPxxIoZ+K4r1auWYduHj5Y+40RwBAZMQHeLnPwMotu9SWrKKC0/e/v7ow2ELrbu8bGBigS5cuOHPmDORyuUqH//jx46hWrRqqVauWq2PdvXsX586dw6BBg2BjY1NYIRNppZ82rMW5P34Xyp27dkffgcPEC4iIipVW7TtjyNiJ2Lw2EADw4M5tLPeZCxcvf5EjI9It5cqlJMTCwsLUHkutS31ObuR1Mz4OttAcfboJpG2WL5iHR/fuCOXhjlPRom3HQj8vr6n+4UAK/VRU79Ueffvj79MncXTfLgDApb/+xLYfV2PouEmFds7iSt///urCYAutvE3i6uqKsLAw+Pn5CXW7d+/G2bNn4e7urvLczZs3o1u3brh586bacfKyGR+RPrlz8wZW+HoK5VJlHDB74XJuikVERcrR2R11GzYRyj9v/hFnjh8VMSIi3WNlZYXatWvj8uXLao9dunQJZmZmqF+/fq6OFR0djZ9//hktWrRA48aNNRwpkThO/XYYu7aGCOV6jZtizFQXESMiouJslvdilC1fUSivXOSNW/9cFTEiosKhlQnlTp06YePGjfD390eDBg3QqlUrDBo0CF5eXhg+fLjKc+/evYtjx44hPDxcpT45ORlbtmxBnTp10KFDh6IMn0hUcbExcJs4CkmJ6af8rYW1ja3IkRFRcWNoZASfFcEwLZE2rd7T2Qlvwl6KGBWR7hkxYgQuX76MS5cuCXVhYWHYu3cvBgwYAFPTtKVkbt68iV27duHdu3dqx0ndW2T06NFFEjdRYXv96gXmz5golEuYmcNnxTphM3cioqJmYWkJnxXBwohPeXIy3CeNEdZWJtIXWplQBoAhQ4bg1atXCA4OxuLFi/Hq1St4eHioPW/o0KE4cuSI2siMhIQEhISEYNeuXUUVMpFWWDLfHY8f3BPKIyZMQ/M27UWMiIiKs4pVqqkscxER/h5zpjpCodDH/ZiJCsekSZPwySefoGfPnli+fDnWrVuHTz75BBUrVoSvr6/Kc3/66Sf069cPt2/fVjvO+vXrYWFhgR9++KGoQicqNAqFAnOmOiLiQ9rAolnei1ChUhURoyIiAho2a4lRk2cI5aePHmCxp6uIERFpnkYSysnJydi1axcmTpyIAQMGCPWHDx8WNgvJD1NTU7Rp0wYdOnTIcsOPGjVqoFu3brC2tlapNzMzQ7du3VC3buFvQEakLU4c3o9ff9oslBs0aY7RU2aKGBERUcp6cl17fSOUL547LaytTKSPNN03NjY2xm+//YaFCxfin3/+wdmzZzFu3Dj873//g52dncpz69evj759+8Le3l6lPiIiAmXLloWnp2e+Nl4h0jZbgoPwv7/OCOVuX3+Lr/p8J2JERERpRkyYhsYtWgvlfT9vw++H9ooXEJGGFXhTvsjISHTt2hXnz5+HtbU1Pnz4gO3btwMAdu7ciQ8fPqh0pImocLx6EQrvWVOEspmFBaf8EZFWkEgkcPUJwI0rl/Ai9CkAYNViH7Ro2wH1GjUVOToizSqsvrFMJsOQIUMwZMiQbJ/Xv39/9O/fX63eysoKv/zyS57PS6SNbv1zFSsXeQvlsuUrYpb3Yu4XQkRaw8DAAN7L1qL/lx0QHRkJAPCZNRX1GzeHQ7nyIkdHVHAFHqHs5uaGyMhIXLp0SW0d45EjR2LdunUFPQUR5UAul2P2lHGIjPgg1Ln5BKBcxUriBUVElE5m68m5TRyNmOgokSMj0iz2jYkKV2xMNNwmjoY8ORlAys0WnxXBsLC0FDkyIiJVDuUrwH1BgFCOiozA7CljIZfLRYyKSDMKnFDes2cPNmzYgGbNmqk9Vrdu3Ux3pCYizQoJCsCVC38J5R7f/oBuX/cVMSIiInUNm7XE6Mlpy/CEPnmEhXNcRIyISPPYNyYqXIvmzsKzxw+F8ujJM9GwWUsRIyIiytoXPb9Bz35pexdcvfg3Nq5aJl5ARBpS4ITymzdvVNYpTj/NyMDAAImJiQU9BRFl4/qlC1i3PG3DqwqVq2LmPD8RIyIiytoIp2lo2qqtUD60eweO7OUGuqQ/2DcmKjy/HfwV+3/ZLpSbtGyDEU7TRIyIiChnMzz9UD7dhqFrl/rhxpX/iRgRUcEVOKHs4OCAq1evCuX0nebz58+jShXusktUWKIiI+ExOW3KjMzAAAsC18HM3ELkyIiIMieTyeC1bA0sLNM22/X1mI7nT5+IGBWR5rBvTFQ4XoY+g4/rVKFsbmkJr2VrhaWUiIi0lZm5RcrSbwYp25jJ5XK4TxqD6KhIkSMjyr8CJ5R79+4NR0dH3L9/X6X+yZMnmD59Ovr06VPQUxBRJpRKJXzdpwkbXAHAhBkeqNuwiYhRERHlrEzZ8pi9cLlQjomKgvuk0UhKShIxKiLNYN+YSPPkcjk8powVNrYCAHffpdzYioh0Rv3GzTB+uptQfv7sCfy59BvpsAInlOfOnYuEhATUqVMHLVq0gFKpRJcuXVCnTh0AgKura4GDJCJ1h/bsxLH9e4Ryy3adMHiMk4gRERHl3qdf9UKfH4YI5RtXL6ks30Okq9g3JtK8kKAAXPvfeaH89XcD8UUP3pwhIt0yZOxENGvdXigf2rOTS7+RzipwQtnOzg4XLlzA1KlTER0dDSsrK4SGhmLcuHE4e/YsLCw49Z5I0549foiFs9M2trKyscX8pasglRb4LU1EVGSmz/FB5Wo1hHJIUAAunz8nYkREBce+MZFmZdwvpGKVanD29BUxIiKi/ElZ+m01LK2shTou/Ua6SiPZJxsbG/j7++P27dsIDw/HvXv3EBAQABsbG00cnojSSUpMhNvE0YiNiRbqPBcHoWRpBxGjIiLKO9MSZlgQ+CMMjYwApCzlM3vKWER8CBc5MqKCYd+YSDMy7hdiYGiIBYHrUMLMXOTIiIjyp7RDObWl3zwmj0FycrKIURHlXYETyv7+/ggLC9NELESUC2uW+uHWP2mb/Xw3ZBQ6ftZNxIiIiPKvVr0GmDRrrlAOe/kC3i6ToVQqRYyKKP/YNybSjJT9Qqar7hfi7I46DRqLFxQRkQZ88mVPlaXf/rnyP/y4YpGIERHlnUFBD+Di4gJ3d3f06NEDI0eOxJdffsmddokKycVzZ7BpddrdzGo1a2Oy+zwRIyIiKrgfRozD+TN/4Nyp4wCAk0cP4tefNuGbAcPEDUyPJckVkCuUkCuUUChT/wsoFErIP5alEgnKWJmotHvwJhrxSXIolEhp8/H5af9Nqe9Q0x5SiURod/HRe4RFxmd5TjNjA3xZv0xR/xgKBfvGRJpx+NefcWz/bqHcsl0nDOJ+IUSkJ6bP8cGVi3/jyYN7AID1gUvQqn1nNGnZRuTIiHKnwAnlO3fuICQkBJs3b8bevXtRtmxZDB06FCNGjED16tU1ESMRAQh//w5zpo4XRu0ZGRvDJ/BHmJiYihwZEVHBSCQSeC5Zif7dOuDdm9cAgMXz3NG4eWtUrVk738dVTVymJTBTE6EKZUoS1NbMCMYGaQm/t9EJeBedmJL8TNde8TEZmpo4rVnaHKUs0xKuoe9jcT00QuWcCgXUjtGwgjUaV7AW2j0Pj8OeK6Fpx84Y48f6xhWs8U3T8kK78NhEzNt/DwpIoPzYRp7udSkUgEKpRM3SFpjTs67Kz2bAuguQK7IfBW5nZoR1Q5ur1C39/S5efIjP8WfftrodpLK0hPJvN1/hytMPWT6/XllLvUkos29cePZcDYUEgKmRAaSSlL8dUgkglUgglUogAVDCSIYv6qn+Lp1/8A6xSXKhjexjO6G9VAKpRILGFawhk6b93j57H4uYhGTh8fRtZKnnlAD25sYwlKVN/IxLkiNZrvjYJuU5UokEUCqgUCg5AyMXnj15BD+PGUKZ+4UQkb4xLWGGBSvWYWjvz5GclASFQgH3yWOw4+ifKmssE2mrAieUa9asCT8/P/j4+ODw4cNYv349Fi1aBF9fX3Tq1AkjR47E4MGDNRErUbGlVCrh5TIZb8JeCnVT3b1Qo3bdbFoRUXGhyGSEaGpSMTUxaWIog7lx2hfxhGQ5nr6LTRkhmpq0zGTkqFUJQ9RxsBTayRVK/H4rTK1NauI0td7SxBC9GpdViXPDuUeIjk8WEqaqI1uVqD9xHW7/cxWvd3kiIT4ObpNGY9Pe37H23DM8fBOTdi6Vc6e93sABTWBTwkg4X+CJezhz722OP78FfeqjdrrXePxWGH6+FJpju0mfVldJKN97HY31Zx/l2M7IQKqSUI6MT8Ifd97k2M7OzFilrFQCoR8ScmwXlyRXq5NKAPVaVfJMkl6ydKOOs6NQKIF0g3Kl0uzbKfQowca+ceE59m9Ypr+X6dmaGakllLddeIrnH+JyPP7Osa0hQ9rv6qa/Hmd7IyTViv6NUd62hFDeev4Jjtx4lW0bqQRw/aoOmlVKW1f76L+v8NPFpypJaKkEkAkJ7ZT6vs3Ko1PNkkK7my8isPXvp5BKITwvNVEuQVrCvEVlG3xap7TQ7m10AnZcfJZpcl1I1EskqGBris61SgntkuUK7L/+Qv1cEtXzW5gYomUVW5XXfeVJOJIVSpXjp8YtJN4VcvhOUt0vZOqCICQaWePFhziVNhljLWEsU5kdoVAqIUHK6yMi0ja16zfERJc5WOo9GwAQ9uI5FrhNg2/Qev7dIq1X4IRyKplMhp49e6Jnz54ICwvD5s2bsXz5cgwZMoSdZqIC2rV1A07/dlgod/i0K/oNGSliRETiUKYbrSlMz0+XkFQiZaRYeqHvYxH3cYp++sRrxtGqLarYqoxMu/o0HK8jE1SSnhmXBVAolOje0AHW6ZKYZ++9xfXQD2lJ1ixGyX7fogJqlbEQ2v157w32Xn2R/ahapRIDWlZE13QjOf/3+D18D/+X48/uqwZlMKJdJaH8OjIBLrtv5NiuWSUbuHdXTSgHn3mYY7ty1qZqCeVz99/hfUxiNq1kKFG1iVC6d/smAhfOQ3jDAXj6PjbHc2YccSvLIYkptMuQoJLmsgOf8Xw5JU1TZRwYnNn5UpNIqaMgpRIJjA1UR+YZyiQoa2kEmUyWknBK93zZx4SLTCJBhXSJrlQda5SEQqkURnamJKzSnVMigamR+jINfZqWQ0yCXOX50nTnTK03kKnGOrh1JXzTpJwQp1Sies7IOP3biIZ9Y81TIOcbD5m9DXN7wyLjezGHQfxp7TKcVJGLhgolkDHU+CQ5ouJzfi/EJqg+Jyo+GXfConJsV9pS9fMxKj4ZJ/97nWO7llVsVRLKSXIltp5/mk2LFJVsS6gllFecvJfj+12mTMbDa1eEcr/BI3E2riw2/HQ1m1YpQoY1V/lMXn78Hv689zZdYj0t+SwkwKWAR/e6qF4qbaO/A9df4Oi/r9KS15CotZdKgW+blkNd+7Sv1NeefcDhGy9VEt2SjMlzSNCqqi2aV0772byKiE85XxaJcsnHpH0luxJoWjHtJkRckhxn771VH7GfYfS+pYmByo1TALj1IlLlxkXGEf9SpPwtz7j0UWRcEpSA+rkyjOQnotwZMHI8/j5zEufP/AEA+P3gXrTt9Cl6fTdQ5MiIsqexhHKq+Ph4nDx5EseOHcOLFy+4ZhxRAT24extLvTyEsn2pMpi7KJAdNT2nTDftPTVxmX5UpoWJIYzSJZc+xCYiPDYpw4jRdCNWP47qrGxnhpIWaV8oX0XE48bzCJXp/Aql6hT9Ko/CUdpMhvLp4nsVEY99156rTP/PmASVK5SoV84KfZqUE9pFxSdh/oFbqknZDKNNFQolqpQ0w+weGafon0d8kiLbn5uVqSE2DG+hUrf4t7u5SkZuG90KptK0z6xjN8Nw8dH7HNu1r2Gv8uX13usonLid8xf0rvVKq5Sj4pPx6G1Mju3ik1XHleZ6xGiG/Eauk635TNJmlsDJrGn6L68yiQQGUhlq1WuIOzf/AQD8FLIWXX2+RDlrC+ELa2oSM30CUyqVwCDDCWqWsUCyQpkhMZuWBE2tzzjyt1EFa5gayYQv/6kJVtWkLVC9pLlqu/JW8OpdL11cmSdrzYxVu1/VS5lj++hW6UYh5u7LuIWJIZb0qQlDUzNIpXnrb034JH9LL6RPKuVFZknt9CTIeRkNXcW+seYs6N0AUilQwshAmK2Q/mahQpn53yjHLtUQl5hyY1GZvk26z6+UGyyq7b6oVxqNK1hnea7UY5lneE/XcbAUnqvMeDMzOQmQyqBQSmBdwlClnb25MeqXtRQ+hzO2Tz2nhYlqO0OZFDYlDDM9Z/rXmvFnk/tEu2o51+0yuRaK7D/GAQBJiWkzL6rWqIUpHvMx+8Dd3J0zw9/O1M8w5cf/lwslVRmXIYmIS8LLiJz/LkXFJyH9V+o3UQm49Dg8x3alLU1UEspvohKw//qLHNt9WruUSkI5Kj4Jq089yLFdvbKW8OpdX6Vuzr5/c7xpYm9uhOAhqksfue65kePPRoKUEf/pby76H/0P/z5XTWJL0302pyanPXrUQel0s39+vfocFx6+U0l0Z7YMzXctKqBaus/lK0/Cce7+W/WlcdKP/pdI0KqqncoN/vDYJNx/FYXHV19AJpVClmGkf+o5K9uVQI3Sae1iEpLxT2hEjiP+LU0NUN4m7TNRrlAiNDxWbcR+xpkGhjKpWv8hWa4QzsXvhrpLKpVi3pJV6N+tA8Lfpcys8587C42at0Klqlwqi7SXxhLKV65cQUhICLZt24YPHz6gWrVq8Pb2xvDhwzV1CqJiJyE+Hm4TRyMhIa3TNn/pKtjY2YsYlWal/2KXMRlpZCBFCaO0P1NJcgVefIhT2zgq40hQcxMD1EzXwVMolTh1543KqNL0Iz5Tk6JmRgbo3tBBJb6fLj5FZFxSWnI3w+jR1C93rl/VUWkXfOYh7r+OVlurNeNo00XfNoRduhG1q/64j5P/vc6xgz//63qoX85KKP9+Kww/XXyW48/bsXM1fFY3LZF573VUjl9EPnsSjqYVLFQSypHxSTh2MyzH86W/fkBKUvPBm5yTppmN0JKojeVSl9masLlOgGYcbZrLfrnaKNUsOvSSj7GkfnHK2PE3MzZAGUuTtARmJqM4M0t+2pgZoW01O9Ukaboka2oytHa6L0sAYGliiCFtKqVLlCJDkjblvzZmqokLqQSY16te2vOzSNYaydTXuVzWv7Gwfqksi58DADzuvA4Du3dBfFzKjYCLAaPw05EzKFk6b2vsdq1XBl3r5X1d3lplLFS+XOaWdQkjlZsLuZVyzZhk1DfsG2teSQtjGMgksDQ1zPnJ6dQra5XzkzLRuqpdvtp1rFkSHdMtSZFKoZAjKS4my5tA7WvYo32NvPfxmlWywfphLXJ+YgaV7cywbVSrLBPmqfXp15gHABNDGfy/baianE93Qzq13sRQ/XNgXKeqSEhWqCa+P/alYmJjsXHNCkR/SEnIGhkbY8HH/UI+q1MaTSomqvSp0rdPnaVklGEmR60yFlACQjuVNuled8YZGdamhqhkWwIKqN50yHjOjOdLSfxJVGLKTMaloHObpM/4kZmbBD2Qed8kNyPwM2+Xc0Ml1G8oxCbKEZ2Q8wj8jMd/FRGPu2HRWTw7zZf1VfvwT97H5mo5qVKWJiqf+e+iE3DhSSSOI/t+9deNy6oklMMi47Ho2J0cz9e6qi1mdkvbHyI+SY6pO6/n2K6KvRmWfNdIpW7ExkvCzzSrEfFmRgZYM7iZSju/I//h2ftYtWVqMibCPbrXUUli77kSilsvIjNNlAvtpUC/ZhVURrZfevweN55HqMeXIdaWlW1Vlg8KfR+Lmy8jMx+xn27Ef0W7EihrnbavUHR8Mh69i1FtAwXkifEwMgEMZCmDBsxNDFSWSpMrlIiIS1L/WWacnVAICXz7UqUxd1EQpozoDwCIi42B+6Qx2LDnKAyN8t63JCoKBU4oBwYGIiQkBNeuXYOJiQn69u2LkSNHonPnzrxLRpROxk536qhOJdRHmryOjEd8sgLrVy7D0/AEGDnUhEQqRdevv0OJKk1w7dmHlI2dylup3Pm/+TwCr6MSMt2UKX0y9LO6pVU+PC8+eo9/Qj+oTefPOOW+d+OyKtPlLjx8h73XXqgkalPbyeUKKCGBXKFE32bl0S3dFP3rzz7A59BtoUOela71SmNsp2pC+U1UQq46XI3KW2Fur3oqdUEn7+fYroyliVpC+dSdN3gTlfMapRmFhsfi/uucO7/JmfwActPB19So0dxO7c/YLqdRsalJxoxxGUglsDc3Spf0VE9GyqQSlLNW32yyZRVbJMkV6UZ+Qm0kqKmh+hf0rxqUQWR8srAJU8Yp96mjTTN+KezXvAK61S+TLqmbybIAUonaEhvft6iAb5uVT3tN6ZLD2elUs6TKmpi5VcXeDM5da+XquQpF2uhmcxMD9E43ejy3JBIJGpTPX3Im4w2GrFSuVgMzPH3h5TIZABD+7i08nScgcNMv3JCJtB77xqQrZNLMl7bJTbv0y0PkRdvqmSfMlUolZo4bipcnDwp1k13noUadlP5c13xu2tmzUVn0bJTz8zLq0agsejQqm+PzUm8SpPqsbmmVG/dqSXek9LMNM9x0reNgiTWDm2abvFZkMjrdpoQhZveoo/7cDIlwa1PVhJRSqcTI9lXUz6VQTbSbGav/fnSqWRIRHwdbpB/pn3H0f8Z+T1V7s3RxZj3iP+PPxsxYBlszI9VzQT1WtSR9bjrUUB9AkNsV/fO9RI5au/zdTMjYNvXnmSL7Y76NTsjVCPyMsT14E5OrdeUzJvdvvYzEgesvs3h2mtKWJioJ5VsvI7H2dM7LrA1sVRF9m6UNfXnwJhrzDtzKsd1ndUrBsUvaCOC30QkYv/VKNi1S1C9niflfq474H/TjBQCqie70iWiJJOWmaMZ2fkf+Q3hM4sfnOKD+1I148/I5lEoF3imVGL/6CKpWr4lZX9VW+d3ZcyUUz97HqY6+B9TO2bdpeZWbsJcev8ejtzEqbTLOkJNKUm5Uph/0FBoei6fvYtMS7RlvKnw8r4OVKWzN0v7exCQk43VUgtp3oow3MEwMZSrf41JzEFxGR7sVOKE8adIkNGnSBEFBQRg4cCCsra01EBbpOiEJmW40prGhVKWDEB2fjIi4JMiVSiQnJyMxPg4SQyUgkapM8y9nY6qSsHkTlYBbLyNVlgDIONpUoVSiRmkLNEg3gvNtVAIO/PNCbWp9+hGqCiVQo5S5SpIlNjEZCw7/p7qMgEI9WVvexhQeGaboj9p0CVHxScL5MmNubIDNI1uq1C08eidl6rtdZzgM6yzUXwdwfd9Nobx5REuYp/uZHv73Ff5+8C7H69O0oo1KQvm/V5E4nMPmMQDQIcOomYi4JNx5lfOafbGJ6qMRMkukZpT/pKlqOdfroWbSoctN4lQqgVrH2drUSEicZpaQTL/eaHpVS5qjVXyySsJVbcq9RAI7c9UvBnUdLDGgVUW1ZGfGZG36kdsAULesJTx61BFiSd+hSC1bnXoJwwzbd1WyK4GNw1uoj27NIXFqZmygNnUytyZ/ViNf7dJvQJQXVezN8tXOJJOkNuXd198Pwt9nTuL4oX0AgPNn/sC2H1dh8BgnkSMjyh77xkR59+uOzTh5NC2Z3K7L5/h+2GgRI9KMlFk5SNnsMZvugZGBFKUsTLJ+QhaMDWVokm4JjLzElXEQRW71b1kxX+2GtK2cv3ZtKmNIm7y3/bpxWXRv6KCSMBe+A6ZLpFuYqKZEKtiWwLeNS+GzLvWghFR9mZyP/186w/UqbWmMqZ/XUEt0Z7ypkHFNaiOZFN+3qKCWXJena6dUpiw/klGbqnYf9wlRH32fWs6sX1rO2jRlBH0mI/bl6WLIbGCIkYFU+A6d1dc5tSR9LpPtaiPwc5ukz3DC3J4vv8n9zNrFJua01XHmHr2NUR28ZGQPk0pp37nfA3j/JFztnNefReDG84gcj/9lfQeVhPLFR+9xPBdL83n2qquSUL7w8D22Xch57fxxnaqqbIx77dkHLPkt5yWL+jQph8FtKgnlh29iMHPXPyrPyThKXCqVoEUFC0z8PG1gTUxCMqb9fD3z0fPpEuEVbEwx8VPV75b+R/9DQrJCvU2645gayVQGvAHA3qvP8S4mMcdz9mzkoDLj58rTcIRFxquP2P/4HV6hVKJluqWRtFGBE8qXL19G06ZNNRELZfC/xylrdxoZSIU/2BmXBDA1kqlN591zJTTlrrECGZYEUF0f1fmLmip3ejb/9Rj3UqfoK5VZjlad93U9lQ7P+rOPcOJWmHDczKacA8CcHnXRuKK1UP79dhi2/P0kx5/D2I5VVUYl3HsdheXH7+XYrnfjsioJ5Yj4pFzdGc24hppSmbJpRU4yrmkFpCzRkCTP/oMps5+XUp67zYnUR5vmqpla4jQ3a7BKJeofzqZGKSMG0k9zT0kqAhKlEjKDlI2iMk4BtzI1RPPKNipJT7Up+lL1Kfrmxgb4vkUFlfNlNs0//V3RVG5f1VYflZphtGpmU/R9+zYQfkYqMeaQOJ32Rc0cf6aZ6Va/jMpo7tyq7WCpttlKbtiUMIJNxeynUUnNjCBPSlAZ62Agk8LSlCNFqfBIJBK4+y7Fv9cu49XzUABAkL8XmrdpjzoNGosbHFE22DcmyptH9+5gsaebULYrWQqei4M4Io0KxEAmhUE+7vGbGEhR0twQ9qXM87Q/gYWJITrUyPtMM2NDGb5vUSHP7YD874cw9fP8fU/JrF1m67ZnnPX3bbPy+KqBQ5Yj9lNzHRk3Dm1e2QYOVnXVE+UK1eR+xkEg5WxMMbpDFdVEuVyBpKRESGWGUCIlWVctw0wLU0MZvmpQJscR/5VsVc+nVEJ9vf1Mltmxy+Q7qq2ZUdrP5eO5kpPkiI6JBiABpFJIIEH4+3ewsU1bhin3a9lnvF65aqb297cgyfbcnU+1nFm7tFH4aY8lZsi1yBXKfM0uBlKS9HFJ2d8YMDc2UEson7n7Bo/f5bxfz5f1yyB9yuj4rTCcf5j1fj11HCz0P6HMDnPhCf7zYY531+zNjdQSyiduv87lFBZAlu6N+/hdLG7mInGalKz+po1PznkBr7xOmc+qXX5HmxZk4ypLUwO1dUVVRo5mMUW/cQVrxCXKVTdlyjBF3zjDB65CocD7Swfw4eVbQCGHUiHH51/1QoMmTdWm3BtnWJvu68bl0LFmyQyjU6E2WtUhw53x3k3K4csGDplO589umkmHGiUz7TjltEZgZXszuGVYczg3zD4mlPOjeT7/GFvlcZ1GItIcSytreC8PxpjvekChUCA5KQluE0dj26E/UMIsf9OtiQob+8ZEuZeYkAC3SaOREB8n1HkuWQlb+7wn5oio6EklEiCHUfhmxgaZDr7Kib25sdrScrlR0sIYXzZQHYGf0/dTIGUvjFEdqub5fDKpBHN61s35iZnw/aZBpvU/b16PhbNnCGWvG19iybqtwnfyOT3rZpu8Tk1uZ9x7pX+LCikj9zMk5TMufVPZTnUz5dZV7eBgZZrpcjPp22fcg6SibQn0b1lBbWmbjO0ztrMwMUDHGvaZPldY51+hQAUb1dcnkQDVSpqlWzs/w9I8SBkAmXFTXAAwNpSq/ywyznzOZDxV/kfE5+352ijP7+qzZ88CANq3b69Szk7qcylvpBJJplPw08vslzDXCVeF6jQWM2MZLE0NhDVNVZKn6TdfyjAMtoKtKZpUtM50U6b0icmMU/RrlDZPWWdU8vGOlDwJRkZGwiL5qYnQjKMua5a2wMxutdKNMs185GjGPxLlbEwRPLhZ5jFmkzg1MZRh43DVJSlyKz93f7f9uAo3dq0Qyp/36A3Xod1zNUojv+vZpXzI56spEVGhatKiNUZNckbwMn8AwNNHD7DY0xVzFgWKHBlRCvaNifIvcOE83L31r1AeOMoRbTt9KmJERETi6zd4BP4+fQJnjh8FAJz+/Qh2bd2AfoNHAFBfazy37C2MYW+R9y/+FWxLoIJtiZyfmEElOzNUssv7MoIOVqaYkkMuJeMa9kDKTIFF/fKxcD6AkEw2t01NgKf/b0aeX9dDslyhnpxPt6yqUqm+iesPLSugewOHj89RH+mf1cx/bZLnhHKHDh0ApP0gU8vZyeyHTjkb06EqDGQSmBkbqGwclX7kaMZfSgCY26sulEqojDZNvxZLan3GBOX0L3K3qVNGX9Z3UFt4PzfqOFiizsdkcW7uGKayNTPK167bhjJpvv54FqXbN64jyN9LKJcpVx7uC5Zyyh8RFWsjJzrjwtnTuH4pZcOTfT9vQ+tOn+CLHn1EjoyIfWOi/Dr3x+/Yvn6NUK5VryGcZs4WMSIiIu0gkUgwZ1Eg+nftgLevU/Y6Wurlgaat2qBazbzP9KX8SV0HH8g6H5PfGc05Jdpf5WLVAbHlOaF85MiRbMukOc0q2cBAJlFZRD038jMthMQXGxMN90mjkZyUBACQSqXwXh4MCyurHFoSEek3AwMD+CwPRv8vOyA6MmVpJh/XqWjQuDkcyudvGRwiTWHfmCjv3r15DU/ntE1WTUxLYEHgOhgZ83sMEREA2NjaYf7SVXAc+A0AICEhHm4TR2PzvuMwNsn7Jp5EmpbnhHK3bt2yLRNR/iye54YnD+8L5VGTnNGkRWsRIyIi0h4O5SvA3XcpXCeMBABER0bCffIYBO88AAODAm8JQZRv7BsT5Y1CoYDn9Al4//aNUDfD0xeVq9UQMSoiIu3Tqn1nDBk7EZvXpiz1dv+/W1jh54kZnn4iR0YE5G/hlXQ2btxYoMeJCPj90F7s27lVKDdq3gojJzqLGBERkfb5okcffP3dQKF8/dIFrA9cLGJEROrYNybK3o4Na/HX6RNC+dOveuHr7weJGBERkfZydHZH7fpp6wLv2BCMP0/8JmJERCkKnFAePnx4gR4nKu5ePg+Fz6ypQtnc0hI+y4M54o6IKBPOnr6oVLW6UP5xxWJc/d95ESMiUsW+MVHW/vv3H6zwmyeUS5ctBw+/ZdwvhIgoC4ZGRlgQuA4mpmmb4s2b4YS3r8NEjIpIAwnl7MTHx0Mmy36DNaLiTC6Xw2PyGERFRgh17gsCuCYoEVEWSpiZw2fFOhgYpuwvoFAo4DF5DCIjPogbGFEusG9MxVlcbAzcJ41GUmIigJT9QnyWB8PSylrcwIiItFylqtUxc17aMhfh795i7nRHKBQKEaOi4i5fQyDPnz+fbRkAEhIScOTIEVSowMQYUVZCggJwLd3Iul79BuCLnt+IGBERkfar06ARnGbOxjKfOQCAV89D4eM6FX4rQzjKjUTBvjFRzpbMd8fjB/eE8ginaWjSso2IERER6Y5e3w3EX6dP4PihfQCA82f+wE8hazBwlKPIkVFxla+Ecps2bbItp5LJZFi+fHl+ToGoqCgsWbIEp06dQnJyMpo3bw4XFxc4ODjk+hjXrl1DcHAwbty4AQsLC3Tv3h1jxoyB4cdRTURiun7pAtYt9xfKFatUw4x5XFyfiCg3Bo5yxPk//8D5M38AAI4f2of9nbZxHU4SRVH0jYl02YnD+/HrT5uFcsOmLTB68kwRIyIi0i0SiQTuvktx4+olhL14DgBY4TcPzVq3R+36DUWOjoqjfCWUf/31V+H/+/Tpo1JOZWZmhjp16qB8+fJ5Pn5kZCTatWsHpVIJPz8/mJmZwcfHB02bNsWFCxdQsWLFHI+xdu1aTJkyBVOmTIGPjw8AYM+ePXBxcUFAQECeYyLSpKiICLhPHgO5XA4AMDA0hM+KYJQwMxc5MiIi3SCVSjFvySr079YB4e/eAgD8585Co+atULlaDZGjo+KmsPvGRLrs1YtQeM+aIpTNLCzgzf1CiIjyzNLKGj7LgzHm+55QKBRITkqC+6TR2HrwJExLmIkdHhUz+foU7927t/D/S5cuVSlrwsKFC3H79m3cuXMH1apVAwA0b94cVatWxYwZM7Bz585s21+4cAETJkxASEgIhgwZItR37NgRCQkJGo2VKK+USiW8Zk3Gy9BnQt0EZ3fUbdhExKiIiHSPfanS8Fy8EpOHfw8AiI+LhdvE0dj46zEYGRuLHB0VJ4XdNwZS+g+7d+9Wmb03ePBgGOfhd/3du3fYuXOnyuy9Tp06aTxWolRyuRyzp4xTWefezScA5SpWEi8oIiId1qRlG4xwmoYfVywGADx+cA8BXh5w910qcmRU3BR4U74pU6ZoIAxVW7duRZs2bYRkMgBYWFjgm2++wd69exEVFZVte19fX1SsWBGDBw9WeywvnW6iwvDrT5tw4vB+odyqfScMGuMkYkRERLqr/Sef44cRY4XynZv/YOUibxEjouKuMPrGycnJ6NWrFxwdHVG+fHnUq1cP/v7+aNOmDSIiInI+AIDjx4+jZs2aOHLkCBo0aIDKlSvDy8sL8+fP13i8RKk2rlqGKxf+Eso9vv0B3b7uK2JERES6b/TkmWjYtIVQ3rN9E04eOSBiRFQcFTihrFQqERQUhAYNGsDU1BQGBgZq//Li7du3ePr0KRo2VF8DpmHDhkhMTMQ///yTZXu5XI7jx4+jXbt22L59O3r27ImWLVuib9++2LdvX55fH5EmPbh7G4s93YSyrX1JzF+6BlJpgd+KRETF1kSXuahZt75Q3rpuJf46fULEiKg403TfGABWrlyJgwcPYv/+/Zg1axYmTpyIP/74A3fv3oW7u3uO7R88eIDevXtj4sSJOHDgABwdHeHo6Ijjx49jxIgR+XmZRDn65/JFrF2atj9I+UpVMJP7hRARFZiBgQG8lwfDzDxtyUwvl8kIe/lcxKiouCnwwlXLli2Dq6srfvjhB/z7779wcnLCo0ePcPz4cXz22WeoXbt2no736tUrAIC9vb3aYyVLlgQAhIWFZdn+zZs3iImJwcGDB3Hy5En4+fmhYsWK+Pnnn9G7d294enpi7ty5WbZ//fo13rx5o1J3//59ACnJ6tQ1bwubXC6HUqGAUgIoFPqfbFQqPr5ehRwKsYMpJPFxsZg1YSQSEuKFOs/FQbC1t4dCUTS/V0WlOFzP4kSiVECpUPJ66gF9fW8aGhnCe/laDO75GRLi4wAAc6c54qcjp2FrX1Lk6AqPPl3P1Nchl8sBpaTQz1eY/TlN940BIDg4GE2aNEHr1q2FunLlyuHrr7/G5s2bsWTJkmxn4Xl7e8PMzCzT5DPXdKbCEBUZCfdJafuFyAwMsCBwHczMLUSOjIhIP5SrWAmuPgHwmDwGABAZ8QGzp4zH6u2/QiaTiRwdFQcFTiiHhIRg1apVGDFiBDZu3IjAwEAAwIsXLzBo0KA8ryEndDoyeQOk1iUlJWXZPnWN5A8fPuDw4cPCLtudOnVCWFgYvLy8MHz48Cw39lu1ahXmzZuX6WPR0dG5nlZYUDHxSVAkxEAukyCp4JdJ6ykVCiQnpCQBJHo6WneJpxse3v1PKA8cMRbNW7ZCUlyMiFEVjuJwPYuVxETIkxMhj4vl9dRx+vzeLF+uHCbPmg3/j7NA3r99A89pjli0OkRvZ4Ho0/VUJiYiWWKAyEgDSCWFn1COjo4utGNrum8cGRmJW7duYdSoUWqPtWjRAtu3b8eNGzfQvHnzLI9x4MABfPLJJ7h58yZ27tyJsLAwVKhQAT/88EO+EtxE2VEqlfDzcMaL0KdCnaOzO+o1aipiVERE+ufL3t/i79MncGhPyj5jl8+fxaY1KzBiwlSRI6PioMCZynv37ql0jOVyOWQyGcqWLYvAwECMHj0af/31V9YHyMDKygoAMl0nObXO2to6y/apj9nb2wvJ5FS9e/fG7t278eeff2LgwIGZtnd0dES/fv1U6u7fv4/evXvD3NxciK+wKQ0TIX2vgEwGGJoaFsk5xaT8OELX0LQEJFL9u5t24vB+7P15u1Cu27AJJrjMgaGRkYhRFR59v57Fjezj76mU11Pn6ft789sho/C/8+fwx9FDAIC//zyF3Tu3YcCIceIGVkj06XpKEg1gYGIAS0sryKSFn1A2TzdFVNM03TcODQ0FAJQpU0btsdS60NDQLBPKb968wbt373D58mW0a9cOTk5OaNasGXbv3g1fX19s3LgRAwYMyPL8nL2nOfo0qyA7h/f8jKP7dgnl5m07YNBoR72bkQcUn2tanHBmnn7S5/eqs6cvrl26gOdPHwMA1ixZgOat26F+k2biBlbI9PmaAroxe6/ACeWEhATY2toCAMzMzPDq1SuUK1cOAFClShVcv349T8erWLEiSpQogQcPHqg9ltp5zW4khZWVFRwcHDKd9pf65SEuLi7L9qVKlUKpUqUyfUwmkxXZ1AGZTAaJVAqJVAKpjn9JzA0F8PH1yvTu9b549hTerml3CM0sLOC7cj2MTUxFjKpw6fP1LI4kkpS/ReD11HnF4b05e+EK3Lp+FWEvXwAAghZ6oVmr9qjToJHIkWmePl3P1Nchk8mKJKFcmP05TfeNU2ffGRqqDzAw+njDLz4+Xu2xVDExKTOhHj58iN27d+Obb74BAIwfPx6dOnXCmDFj0LVrV9jZ2WXanrP3NEefZhVkJfTJYyycPVMoW1nbYLbPIsgT4qB/6eTicU2LHc7M00v6/F41NpDCc+FSjBv0rXCj133yGGzacxhmZoV3A11s+nxNAd2YvafR3li9evWwY8cOTJ8+HQDw66+/Ztk5zYpUKsUXX3yB06dPIyEhQSUxfOzYMdSrVw+VKlXK9hg9e/ZESEgI3r17p3L+q1evAgDq16+fVVMijUpKSoL7pNGIjowU6twXBKB8xcriBUVEpMesrG3gtWwtxvbvBaVSiaTERLg6jcS2Q39w7U4qcproG2c3ICI1WWxhkfXvdmp7KysrIZkMpPS5hw8fjrNnz+KPP/7At99+m2l7zt7THH2aVZCZ5KQkeLpMRWxs2nJusxcuR9nK1USMqnDp+zUtjjgzTz/p+3u1cev2GDvVBasWLwCQMqhtme98eC5ZKXJkhUffr6kuzN7TaEJ51KhRGDt2LPbt2wcjIyP88ccfmDVrVp6PM3v2bLRu3RouLi4ICAiAVCrF6tWrcenSJezZs0fluStXrsS6deuwceNGNG7cGADg6uqKHTt2YMKECdi0aROMjY1x+fJlrFixAp988onKhiZEhSl46UL8c+V/Qvnr7weha6++IkZERKT/mrVuh1GTnLFu+SIAwLPHD+Hr7gyvZWsgKYI7/ESpNNE3rlSpEgwMDPDkyRO1x54+TVmjtlq1rBN29vb2sLGxEUZNp5e64XV4eHiW7Tl7T3P0aVZBZtYEeOHWP1eF8reDRqBLtx4iRlT49P2aFkecmaefisN7dZjjVFw4exqXz58DABza8zNad/gEX33znciRFQ59v6a6MHuvwOPCU0f9AsDo0aMREBCA6OhovH//HnPmzIGnp2eej9m0aVPs3bsXe/fuRdmyZVGpUiXMnj0b69atQ58+fVSe+/LlS1y/fl1leHblypXx+++/4/bt27Czs0PVqlXRpk0bdO/eXS0hTVRYLp49jQ2rlgrlytVqYIanr4gREREVH6MmzUDTVm2F8pG9v+DAL9uzaUGkGZruGxsZGaFDhw44e/YslEqlymN//PEHypcvj1q1amV7jM8//xyhoaFqo5zv3r0LIGUpDqKC+Ov0CWxas0IoV61RC1Nne4kYERFR8SKTyTB/6WpYWlkLdb4eznj6SH05WSJNKHBCOXVUcKopU6bgypUruHLlCubOnZvpem+58dVXX+HRo0f4+++/cfz4cbx69SrT3a2dnJxw9epVNGm2Mv/wAAEAAElEQVTSRKW+ZcuWuH79Ou7cuYODBw/i/fv32Lp1a5FNy6Pi7f3bN/CYMk744mdkbAy/lSEwLWEmcmRERMWDgYEBfFYEw8ombVTmwjkueHTvjohRUXFQGH1jZ2dnPHv2DKtXrxbqTpw4gZMnT2LmzJkqI+937dqFQYMGCcliAJg5cyaSk5Ph5ZWW4Hv16hVWrFiBOnXqoHPnznmOiSjVm7BXmDN1vFA2NjaB78r1MNHj/UKIiLRRmbLlMWdRoFCOjYmGq9MoJH7cj4FIkwqcUPb390dYWJgmYlEjkUhQpUoV1KhRAwYGma/OUaZMGTRu3BhmZpkn6sqVK4e6desW6m7eROkpFArMneaId2/S3hdTPbxRo049EaMiIip+SpUpi3lLVgnl+LhYuEwYgfj4rDfnJSqowugbf/XVVwgICMC0adPQpUsXdO/eHd27d8ekSZPg5OSk8txr165h27ZteP36tVDXrFkzbN26FStXrkTjxo3Rq1cv1KtXD3Z2djh48GCW/WyinCgUCsyZOh7h794Kdc6eC1C9Vl0RoyIiKr66dO2O74akDcb879/rCFyY+ea6RAVR4ISyi4sLypcvjz59+uDgwYOQy/Vx/16i3Nv24yr8dfqEUO7StQf6DR4hYkRERMVXh0+/wMBRjkL5wZ3bCJjvLmJEpO8Kq288depUhIaGYtq0aRg1ahTu3r2LZcuWqa0L3q9fP2zZskVtGYz+/fvj+fPnWLhwIQYNGoRjx47h8uXLqFq1qkbio+Jp4+rluHjutFD+rPvX6PPDUBEjIiKiKe7zUbNufaG8ff0anDl+VMSISB8VOKF8584dTJ8+HRcuXEDPnj1RsWJFuLm54f79+5qIj0in3Lx+BUH+adNJS5cth9n+y7kJFBGRiCa6zEG9Rk2F8u5tG/H7ob3iBUR6rTD7xvb29ujZsyf69OmDihUrZvqcRo0aYdCgQcKGe+mZm5uja9eu+O6779C8eXP2T6hArl+6gDVLFgjlsuUrwsNX/SYHEREVLWMTE/gGrVdZctNz+gSEvXwuYlSkbwqcUK5Zsyb8/Pzw7Nkz7N+/Hy1atMCiRYtQo0YNdO7cGVu2bNFEnERaLzoqEq5Oo5CclAQAkEql8FmxDlbWNiJHRkRUvBkaGWFB0I8ws7AQ6rxcJiP06WPxgiK9xb4xFQeRER/gNmm0MAJfZmCABYHrYMH9aoiItELlajXg4uUvlCM+hMN90hgkJyeLGBXpkwInlFPJZDL07NkTe/fuRWhoKPz9/XH//n0MGTJEU6cg0lpKpRK+7s54ni45MWaKC5q0aC1eUEREJChfsTI8fJcK5ZioKLg5jUJSYqKIUZE+Y9+Y9JVSqYTXzEl49TxUqJswwwMNmrYQMSoiIsqoR9/++KrPd0L56sW/sT5wsYgRkT7RWEI5VXx8PE6ePIljx47hxYsXkMlkmj4FkdbZ//M2HN23Syg3a90eI5ymiRgRERFl9EXPb9Dnh7RkXsZliogKA/vGpG92bd2Ak0cPCuXWHbtg8BinbFoQEZEYJBIJZnkvQoXKafsl/LhiMS79fVbEqEhfaCyhfOXKFTg5OcHBwQEDBgzA48eP4e3tjWfPnmnqFERa6f6dW1g4x0UoW9nYwnv5Gn5hJCLSQtPnLkC1mrWF8tZ1K/Hnid9EjIj0FfvGpI/u3voXAV5pG5valSyF+QGrIZVqfJwSERFpgJm5BXyD1sPA0BAAoFAo4DF5LMLfvxM5MtJ1Bf7kDwwMRJMmTdCsWTOsX78e3bt3x8mTJ3Hv3j24ubnBwcFBE3ESaaW42BjMchyBhPg4oW7eklUoVaasiFEREVFWTE1LwG9lCIxNTIW6udMd8frVCxGjIn3CvjHpq7jYGMxyGonEhAQAKSPfvJatgV3JUiJHRkRE2anToBEmu80Tym/CXsJz+gQolUoRoyJdV+CE8qRJkyCRSBAUFISXL19i69at6NKlC3f3pWLBb/ZMPLp/VygPGTsRHT79QsSIiIgoJ1Vr1sbMeX5COSL8PTwmjxU2lyIqCPaNSV/5z3HBkwf3hPIwxylo1b6zeAEREVGu/TB8LDp82lUonz35G34KWSNiRKTrCpxQvnz5Mq5cuYIJEybA2tpaAyER6Yb9v2zHwV0/CeWGTVvAcYaHiBEREVFuff39IHTt1VcoXz5/Dj+uWCRiRKQv2DcmfXT411+w/5ftQrlh0xYYO3WWiBEREVFeSCQSzF0chFJl0mZKLff1xK1/rooYFemyAieUmzZtqok4iHTKg7u34ecxQyhbWlljQdB6GH5cl4iIiLSbRCKB24IlKF+pilDHTUpIE9g3Jn3z7PFD+LpPF8oWllbwWbGO/V4iIh1jY2sHnxXrhHXvk5OS4Oo0CtFRkSJHRrrIIK8Nzp5N+aLVvn17lXJ2Up9LpA8yXTc5YBUcypUXMSoiIsorcwtL+K1cj2F9uiI5KQkKhQLuk8Zgx9EzsLGzFzs80hHsG5M+S0xIwKwJIxEbEy3UzfFfgbIVKooYFRER5VfTVm0xespMrA1IWf4t9MkjLHCbDp8VwVyei/IkzwnlDh06AICweHdqOTtc6Jv0if8cFzy8d0coDxo9AR0/6yZiRERElF91GjTGZLd5WDLPDQDw9vUreEwZi8BNvwijN4iyw74x6bOghfPx37/XhXK/wSPxyZc9RYyIiIgKaqTTdFz6609cPn8OAHBs/2607tAZvb4bKHJkpEvynFA+cuRItmUifXZw9w6V9eMaNGkOJ5c5IkZEREQF9cPwsbj01584/XtKn+b8mT8QEhSAUZOcRY6MdAH7xqSvzpw4hm3rVwvlGnXqYaqHl4gRERGRJshkMngvX4v+3ToiIvw9AGDhHBc0aNIcVWrUEjk60hV5Tih366Y6ErNx48aQSqUoVaqUxoIi0kaP7t2Br3tacsHSyhq+XDeZiEjnpW5SMrB7Z7wMfQYAWLvUD42at0KLtjmPNqXijX1j0kcvn4fCc5qjUDYxLQHfwB9hbGIiYlRERKQppcqUxbwlqzBlRH8AQHxcLFwmjMCmfb/D1LSEyNGRLijwXM6yZcvi77//1kQsRForLi4WLo7DER8XK9R5LlkJh/IVRIyKiIg0xcraBn4rQ2Dw8SZhynrKo/H2dZjIkZGuYd+YdF1SUhLcnEYi4kO4UDdznh9HrRER6ZkOn36BgSPHC+UHd25j0RwXESMiXVLghHK5cuXQrl07TcRCpLUWzZ2FB3f/E8oDRzmi0+dfihgRERFpWv3GzTDFfb5QfvfmNdwmjoZcLhcxKtI17BuTrlvp74V/rvxPKHf/5nuuq0lEpKcmzpqLeo2aCuV9P2/Dwd07RIyIdEWBE8qjRo3Cli1bNBELkVY6vOdn7Nu5VSjXa9wUE7luMhGRXuo/bAw+Tbfh1OXzZ7F2qZ+IEZGuYd+YdNmZ40exJThIKFepXhOuPoshkUhEjIqIiAqLoZER/FaGwMLSSqjzdXfGw3QD6ogyk+c1lDPq2bMnfH19cePGDfTq1QvlypVTW1O2cePGBT0NkSge3b+LBe7ThbKFpRX8gkJgaGQkYlRERFRYJBIJZvsH4s6tfxH65BEAICQoAI1btEbbTp+KHB3pAvaNSVe9DH2GuenWTTY2MYXfqhCYljATMSoiIipsZStUhOeSlZg+ehCAj+spOw7H5v3H+RlAWSpwQrlZs2bC/2/YsCHT5yiVyoKehqjIpa6bHBcbI9TNXRyEshUqihgVEREVNgtLSyxctQHDv+mKxIQEKJVKzJ4yDtsPn0Jph3Jih0dajn1j0kVJiYmY5TQCkREfhLpZ3otQvVZd8YIiIqIi0/mLrzBwlCO2/bgKAPDw3h34zZ6JeUtWihwZaasCJ5R/+uknTcRBpFWUSiV83Z3x4M5toW7AyHHo0rW7iFEREVFRqV2/IZzn+mKB2zQAwIf37+DqNAprd+xXG21KlB77xqSLAv3m4d+rl4Vyz34/oFe/ASJGRERERW3irLn45/JF3Lh6CQBwcNdPaNaqLdfRp0wVOKHcv39/TcRBpFV+3bEZh9ItRF+vcVNMmuUpXkBERFTkvhkwFFcu/IWj+3YBAK5fuoBVi7wx2W2eyJGRNmPfmHTNH8cOYdv61UK5ao1acJnvL2JEREQkBkNDQ/gGrceArzoJM1b8Zs9E3UZNOGOF1BR4Uz4ASE5Oxq5duzBx4kQMGJB2J/vw4cNISkrSxCmIisztG9exaO4soWxlbYOFqzZy3WQiomJGIpHA3TcAlavVEOo2rw3EmeNHRYyKdAH7xqQrnj99gnnOTkLZxLQEFq7awDUziYiKKYfyFTBvadpNxoT4OLiMH47YmGgRoyJtVOCEcmRkJDp06IB+/fph69atKtP8du7ciV9++aWgpyAqMpERH+AyfhgSExIApCQTvJevhUO58iJHRkREYihhZo6FqzfA2MRUqJs7zREvnj0VMSrSZuwbk65ISkyEq9NIREVGCHVuC5agas3aIkZFRERi6/hpVwwZO1EoP35wD77uztwDglQUOKHs5uaGyMhIXLp0CeHh4SqPjRw5EuvWrSvoKYiKhEKhwNxpjnj+7IlQN3LidLTt/JmIURERkdiq16qLWd6LhHJkxAfMmjACSYmJIkZF2op9Y9IVyxbMwc3rV4Ty198PQvdvvhcxIiIi0haOMzzQqFlLoXz415+xd8cWESMibVPghPKePXuwYcMGlR2tU9WtWxeXL1/OpBWR9sk4jblV+04YM8VFxIiIiEhb9Oo3QGWDqpvXr2CpzxwRIyJtxb4x6YKTRw5gx4ZgoVytVh3MmOcnYkRERKRNDA0NsSDoR1jZ2Ap1i+bOwr3bN0WMirRJgRPKb968Qd26aYtzSyQS4f8NDAyQyNE7pAMu/X0WK/29hHKpMg7wWbEOMplMxKiIiEibzPTyR7VadYTyzo3BOLZ/t4gRkTZi35i0XejTx5g3M20qs2kJs5R1k01LiBgVERFpmzJly8Mr/XrKCfFwcRyOmOgoEaMibVHghLKDgwOuXr0qlNN3ms+fP48qVaoU9BREhepN2Cu4TRwFhUIBAJAZGMBvZQhs7OxFjoyIiLSJqWkJ+K/eiBJm5kKdl8sUPLh7W8SoSNuwb0zaLDEhAbMcRyA6MlKoc1sQgCrVa4oYFRERaat2XT7HMMcpQvnJw/vwcZ3G9ZSp4Anl3r17w9HREffv31epf/LkCaZPn44+ffoU9BREhSY5ORluE0fh3ZvXQt0Ut3lo1LyViFEREZG2qlytBuYsWiGU42JjMGPsUERHRWbTiooT9o1Jmy2e74bbN64J5T4/DMFXffqJFxAREWm98dPd0KRlG6F8bP9u7Nq6QcSISBsUOKE8d+5cJCQkoE6dOmjRogWUSiW6dOmCOnVSpoS6urrm67gvX77EuHHjULt2bVSvXh39+/fHf//9l6u2ixcvRpkyZTL99+bNm3zFQ/pp1WIfXLnwl1D+9Mue+GHEOBEjIiIibfd5994YOHK8UH7y8D7mOTtxpAYBKLy+MVFBHdy9A7vTJQBq1KkHZ09fESMiIiJdYGBggAWB61RmcS+e54p/r3FfiOKswAllOzs7XLhwAVOnTkV0dDSsrKwQGhqKcePG4ezZs7CwsMjzMd+8eYM2bdrgxo0b2Lp1Kw4cOACFQoHWrVvj9u2cp5VGR0cjLCwMf/31F65du6byz87OLj8vk/TQ6d+PYNPq5UK5YpVqmLMoUGVqKhERUWYmunqqjNQ4efQgtgQHiRgRaYvC6BsTFdS92zexwG26UDazsMCiNZtgYmIqYlRERKQrSpUpC58VwUK+JDkpCTPHD0P4+3ciR0ZiKXBCGQBsbGzg7++P27dvIzw8HPfu3UNAQABsbGzydTwfHx+8fPkSv/zyC5o3b446depgy5YtMDc3x4wZM3J9nFKlSqmNUJZKNfKSSceFPn2MOdPSRpcZm5jCf80mmFtYihgVERHpCkNDQ/gGrYddydJCXdDC+bh8/pyIUZG20HTfmKggoiIjMWPcUCTExwl18wNWo0LlqiJGRUREuqZV+84YP91NKIe9eA6PyWMgl8tFjIrEopXZ1Z07d6J9+/YoW7asUGdsbIw+ffrg6NGjCA8PFzE60nVxcbFwHjNYZTMSV5/FqFG7bjatiIiIVJUsXQYLV4VAJpMBAORyOWZNGIHXr16IHBnpo2fPnmHYsGGoXLkyypcvj969e+PGjRu5auvn5wdra+tM/71+/TrnA5DOUiqVmOfshGePHwp1Q8dPRucvvhIxKiIi0lXDJ0xF+0++EMrnz/yBdcv9RYyIxGKQ1wZbt27N80kGDRqU6+e+evUKr169Qr9+6ptD1KtXD3K5HDdu3EDHjh1zPFbbtm0RHh4Oe3t7dO7cGTNmzFBJUlPxo1Qq4e0yBfdu3xTqevcfjJ7f/iBiVEREpKuatGyDyW7zEODlAQB4//YNZjmOwNod+2FoZCRydFQUCrtvDABhYWFo06YNqlevjoMHD8LMzAyzZ89G27Zt8ffff6N+/frZto+Pj0dERARCQ0NhZmam8piVlVWe4yfdsSU4CH8cOyiUm7VuD0dndxEjIiIiXSaVSuG1bA0GftUZL0KfAgDWLV+E+o2bo/0nn4scHRWlPCeUBw8enOeT5KXTnLppnq2trdpjqXU5baxnZmYGDw8PfPnllyhZsiQuXrwINzc3bNu2DX/99ReqV6+eZdvXr1+rHT91l265XF5kQ/nlcjmUCgWUEkCh0MqB5BqlVHx8vQo5FIV4nh0bgnF03y6hXKdhYzh7LoBCwSkamlRU15OKhkSpgFKh5PXUA3xvFo7+w8fgnyv/w/FD+wAA1y9fxFKf2XCeu6BQz6tP1zP1dcjlckBZ+HsZaLI/V9h9YwDw9vbG27dvcfnyZZQunbLMSkhICGrUqAFnZ2ccPXo0V8exsrKCubl5nuMl3XT5/DkELZwvlO1LlYFv0I8wMMjzV0AiIiKBpZU1/Ndswoi+3ZCYkAAAmD1lLLYdOoWyFSqKHB0VlTz3JjJOrZPL5fD29oaNjQ2+/vprlC5dGmFhYdi7dy8+fPiA2bNn5+n4qTukZ7YxWur6xzl9CXB2dlZpX6NGDTRv3hyNGjXCjBkz8Ouvv2bZdtWqVZg3b16mj0VHRyMiIiLH16AJMfFJUCTEQC6TICnvl0nnKBUKJCekrOsmKaR1rq9euoBlC+YKZWsbW/gErIRUIUdSXEyhnLO4KorrSUUoMRHy5ETI42J5PXUc35uFx8XTB/dv38Tjhyk3oXduXIc69erji+5fF9o59el6KhMTkSwxQGSkAaRFsDludHS0xo5V2H1jAPj555/RoUMHIZkMAEZGRujduzeCgoLw/v37TAdjUPH1JuwVXJ1GCt+bZAYGWLgqBHYlS4kcGRER6YM6DRrBZb4/vFwmAwAiIz5g5vhhWL/rMIxNTESOjopCnjOVGafUubq6olOnTnByclKp7969O1asWIFffvkFDRs2zPXxra2tAQCR6da3TZWazM1pQ5PMktG1atVC06ZN8ccff2Tb1tHRUW25jfv376N3794wNzcvsmmBSsNESN8rIJMBhqaGRXJOMSk/jhA2NC0BiVSm8eO/fvUSs6c5QZ6cDCDl5sSCoB9RoVpNjZ+LCv96UtGSfZy2L+X11Hl8bxYea1MzLArejKFff47YmJSblH5zXFG7YVNUq1m7UM6pT9dTkmgAAxMDWFpaQSYt/ISyJkfpFnbf+OXLl3j9+jXq1lXf66Fu3bpQKBS4ceMGOnXqlOOxmjZtqrIcnKurKypW5EgifZOUlIRZE0bg3Zu09bGnuM1D4xatRYyKiIj0Te/+g/HP5YvY9/M2AMDtG9eweJ4r3H2XihwZFYUCD30NCQnBf//9l+ljgwYNQt26deHl5ZXr41WoUAEWFha4d++e2mN3794FkLKWcn6YmJggMTEx2+eUKlUKpUplfudeJpMJG+8UNplMBolUColUAqmOf0nMDQXw8fXKNP56ExMSMMtxBN6/TVvKZJKrJ1q176zR81CawryeVPQkkpS/ReD11Hl8bxauqjVqY+6iILg4DgcAxMfFwmX8cGzefxzmFpYaP58+Xc/U1yGTyYokoVyY/TlN943fvn0LIPMBFbldDs7S0hLe3t4qy8G5uLjg559/xrlz51C7dtY3PbgcnOYU1TI1gX7zcO1/54XyZ1/1wvfDRnOJt0KgT0sPUQou9aaf+F4tPM7zfHH73+u4e+tfAMCe7ZtQv0mzQt+nSt+vqS4sB1fghHJkZCTCwsIy7eSGhYVlOtI4OxKJBD179sTBgwcRExMjbByiVCpx6NAhtGjRIl8b67179w5XrlxBs2bN8tyWdNuS+e64cfWSUP68R28MGj1BxIiIiEhffdb9awwc5YhtP64CADx5eB9zpjli8drNwtJdpN803TfObjm41LrU52Rl2rRpKuUKFSqgcePGqF+/PmbOnIn9+/dn2ZbLwWlOUSxTc/LYYeHvDwBUrFIVLvMWIDk+tlDOV9zp09JD9BGXetNLfK8WHhkAn6WrMKJfD0R97OP4ecxAtWrVUaO2+uwqTdH3a6oLy8EVuDfWtm1bjB8/Hr/88gvs7e2F+jdv3mDcuHFo165dno85e/Zs7Nu3D+PGjcPatWthZGQEb29v/Pfffzh27JjKcxcuXIilS5di//79aNmyJeRyOQYNGoRp06ahUaNGMDIywu3btzFu3DjExsbCx8enoC+ZdMj+n7dh19YQoVytZm3M8V+R6ZcyIiIiTZg4ay5u/XMVVy/+DQA4/dth/Bi4GGMmzxQ5MioKmu4bpyamM0vc5nY5uMxUq1YNTZs2xenTp7N9HpeD05zCXqbmycMH8J2d9nfGtEQJLFq7Gdb2pbNpRQWhT0sPUQou9aaf+F4tXJVr1oHnklWYPjpl0+HEhAS4T3XE5v3HYWFZOP0Efb+murAcXIETygEBAejYsSMqVqyI1q1bCxuPnD9/HkZGRvjzzz/zfMzatWvj+PHjmDRpEmxtbSGTyVC+fHns3r0bn376qcpzo6KiEBYWJixlIZPJ8O2338LZ2RmXL18GACQnJ6Njx444ffo02rRpU9CXTDri9o1r8PVwFspmFhZYHLwFJcy4uzkRERUeQ0ND+K0MweCen+D1q5cAgLUBfqhdryE6ftZN5OiosGm6b1y+fHlYWVkJS7+ld+fOHQD5Xw7OyMgISUlJ2T6Hy8FpTmEuUxMTHQWX8cMQk26EkYffclSvVXijw0i/lh6iFFzqTT/xvVr4On/xFUZMmIaQlQEAgNAnj+E53QlL1m0tlFl6+n5NdWE5uAJf1UaNGuHq1asYOHAgnjx5giNHjuDJkycYNGgQrl69igYNGuTruK1bt8bFixfx/v17vHr1Cnfu3MHXX6vvlD5r1iy8fPkSrVq1Eur69u2L06dPIzIyEk+fPkVMTAx+++03JpOLkfD37+A8dggSExKEOu9la1GxSjURoyIiouLCvlRpLFq7GYYfRzoBgMeUsXh0Xz0pSPpF031jiUSCXr164fTp04iKihLqlUolDhw4gNatW8PBwSHPcb5+/RqXL19Gy5Yt89yWtItSqcQ8Zyc8uJu2dvf3w8ag29d9RYyKiIiKm3HTXdGyXdomwWeOH8WPgYtFjIgKk0ZuE1StWhXr1q3DgwcP8OHDBzx48ADBwcGoUqVKgY9dokQJWFhYZPm4ubk5ypQpA0ND9alvUqlUGOFMxUdycjLcnEbh1fNQoW705BkcFUZEREWqfuNmcPNZIpRjoqIwfcxgYX050l+a7hvPnj0bUqkUo0aNQmRkJBITE+Hq6or79+/Dz89P5bk+Pj6wtrbG+fMpm7LJ5XL06dMHf/75J2JiYqBQKHD16lX07t0biYmJWLBgQYFfL4lr4+rlOHHkgFBu1KwlprrPFzEiIiIqjmQyGXxWBKO0Q9q+Z2sD/HDm+FERo6LCon8rV1Oxt2qxDy6eS1sPsF2XzzFmiouIERERUXHV67uB+G7IKKH85ME9zJk6DgqFPu5HTYWlRo0aOHnyJF6+fAl7e3uYm5vjyJEjOHDgADp16qTy3Li4OERERCA5ORlAype7UaNGwdvbG+XKlYOJiQk6duwIe3t7nDt3Dm3bthXjJZGG/HXqOFb6ewll+1JlsHD1RpXZEUREREXF1r4kFq3dDCNjY6HOY8pYPH5wT8SoqDAwoUx65dj+3di0erlQLlexMryXry2UNXuIiIhyY9ocHzRtlZa0O3P8KNYt9xcxItJFzZo1w5kzZxAbG4vo6Ghcv34d3bqpz77y8PBAeHi4ylJv3bt3x7Fjx/Dhwwd8+PABUVFR2L9/P5o1a1aUL4E07NmTR3CbOBpKpRIAYGBoiEVrNqJk6TIiR0ZERMVZvUZN4Zpxlt7oQYiO4iw9fcIsG+mN2zeuY96MSULZ2MQUS4K3wNLKWrygiIio2DM0NMTCVRtUpv8FL/PHH8cOiRgV6SoDAwMYZTP61MTEBNbW1lku+VaiRInCCo2KUFxsDJzHDEZUZIRQN3PeQjRsxjWxiYhIfL36DVCZpff4wT3MnT6Bs/T0CBPKpBfev32D6WMGISE+TqjzXBKEGnXyt+s5ERGRJtnal8Ti4C0q0//mTB2PR/fuiBgVEekipVKJ+TMn4f5/t4S63v0H45sBQ0WMioiISNW02d5o3KK1UD517BA2rFwqYkSkSUwok85LSkzEjHFDEfbiuVA3YsI0fNGjj4hRERERqarbsAncfdM60bEx0Zg2ehCiIiKyaUVEpGpLcBB+O/CrUK7fpBlc5vtDIpGIGBUREZEqQyMjLFy1ASVLOwh1q5cswNmTv4sYFWkKE8qk8xZ5zsK1/50Xyh0/64bxzm4iRkRERJS5Hn37o//wMUL56aMH8JgyltP/iChXLpw9hUC/eULZrmQpLFqzSWX2AxERkbawL1Uai9akbRarVCrhPnk0nj1+KHJkVFBMKJNO+2VLCHZv2yiUq1SvCa9la7gJHxERaa0p7l5o1rq9UD578jesCfAVMSIi0gXPnz6B64SRwg0omYEBFq7agFJlyubQkoiISDwNmraAy/y0DamjIyMxffQgxMZEixgVFRSzbqSzLp8/h0Wes4SyhaUVAn7cBnMLSxGjIiIiyl7KJn0hKFOuvFC3PnAJfjv4azatiKg4i4uLxYyxQxDxIVyoc56zAE1athExKiIiotzp88MQ9B04TCg/uPsfPJ2doFQqxQuKCoQJZdJJL0OfYeb4YZAnJwMApFIpFgT9iIpVqokcGRERUc5s7OyxeO0WGBubCHWe051w+8Y18YIiIq2kVCrhM2sq7ty6IdT1+PYH9BsyUsSoiIiI8sZ5ri8aNm0hlE8c3o9Na1aIGBEVBBPKpHPiYmMwbfRAfHj/Tqib5OqJtp0+FTEqIiKivKnToBHmLAoUygnxcZg2aiDehL0SMSoi0jZb163Ekb2/COU6DRrD1WcxN+EjIiKdYmRsDP81G2FXsrRQF7RwPv46dVzEqCi/mFAmnaJUKuHp7IS7t/4V6rp/8z0GjZ4gYlRERET50+3rvhjpNF0ov371EtPHDEJ8fJyIURGRtjj3x+9Y4esplK1t7bBo7WaYmJiKFxQREVE+lSztAP/VG2BgaAggJcczy2kkHt2/K3JklFdMKJNOWR+4BMcP7RPK9Ro1hZtvAEdoEBGRzho33RVduvYQyjevXYHXzMlcU46omHt0/y5cJ45S24TPId3660RERLqmcYvWmDlvoVCOiYrCtFEDERnxQbygKM+YUCad8fuhvVi9ZIFQtitZGouDOUKDiIh0m1Qqxfylq1Czbn2h7ui+Xdiwapl4QRGRqCIjPmDaqIGIiYoS6mbOW4jmbdqLGBUREZFm9B04DP0Gp+0F8PTRA8yaMALJH/fJIu3HhDLphFv/XMXcaWnLWhgZG2Px2k0oVaasiFERERFpRgkzcwSs2wZb+5JC3Up/L5z67bCIURGRGJKTkzFrwgg8ffRAqOs3eCS+HTRcxKiIiIg0a/rcBWjepoNQvvDnKSzzmS1eQJQnTCiT1gt7+RxTRw5EQrr1JGcvXIGGzVqKGBUREZFmOZSvgMVrN8PQyEio85g8VmXfACLSf8sXzMGFP08J5eZtOmD63AVZPp+IiEgXGRoaYuHqDShXsbJQ91PIWuzdsUW8oCjXmFAmrRYXG4OpIwfi7eu0He9HTpyOr/r0EzEqIiKiwtGoeSu4+y4VynGxMZg6agDev30jYlREVFT27dyK7evXCOVyFStj4eoNMPy4eREREZE+sbaxxdL122Bmbi7U+Xo44+r/zosYFeUGE8qktRQKBTymjMOdm/8IdZ9+1QvjprmKGBUREVHh6vntDxgydqJQfvU8FM5jhyAxIUHEqIiosF3733kscJ8ulEuYmWPp+m2wtrEVMSoiIqLCVa1mHXivWAeJRAIASE5KwoyxQ/Ay9JnIkVF2mFAmrbVykTdOHTsklOs2bIJ5AasglfLXloiI9JuTyxy0/+QLoXz90gX4uE2DUqkUMSoiKiwvn4dixrihSE5KAgBIJBL4rAhGtZp1RI6MiIio8HX8tCucXOYI5fB3bzFt9EDExcaIGBVlh5k50kr7f9mOjel2ty9VxgEBP26DqWkJ8YIiIiIqIjKZ7GMyqbZQd3DXTwgJChAxKiIqDHGxMZg+aqDK0jYTZs5Gx8+6iRgVERFR0Ro6bhK+7J22vOndW/9i7jRHKBQKEaOirDChTFrn6sW/4eM6VSibmJbA0vU/oWTpMiJGRUREVLTMLSwRsH47rG3thLpVi31wdN9uEaMiIk1SKBSYO30C7ty6IdR1+/pbDBs/WcSoiIiIip5EIoHHwmWo16ipUHfiyAGsW+4vYlSUFSaUSauEPnmE6WMGC9P9AMBr2RrUrt9QxKiIiIjEUb5iZQSs2wojY2OhztN5Aq5xoxIivbB68QKcOLxfKNdr1BSz/ZcL60gSEREVJyYmpliybgtKlnYQ6oKX+XNAhRZiQpm0RlRkBKaNGoSI8PdCnZPLHHzSrYeIUREREYmrUfNW8FwcJJSTEhMxbfQgPHv8UMSoiKig9v+yHSEr05axsS9VBouDN8PExFTEqIiIiMRVsrQDlqzbAmNjE6Fu3gwnXL90QcSoKCMmlEkrJCUmwnXyODy6f1eo6/HtD5zuR0REBKBrr76YMMNDKEeEv8eUEQMQ+eGDeEERUb5dPn9ObYm3ZSHbUapMWRGjIiIi0g71GjWF55K0ARWJCQmYPmYwQp8+Fi8oUsGEMolOqVTCe9ZUXLnwt1DXuEVruC8I4HQ/IiKij4ZPmIpe/QYI5aePHsB18lgkJSaKGBUR5dWThw/gnG6JN4lEAu/la1GnQWNxAyMiItIiX/T8Bo7O7kI5/N1bTBneH1ERESJGRamYUCbRrV3qh8O//iyUK1WtjiUZ1oskIiIq7iQSCdwWBKB5mw5C3dX/XYCP61QolUoRIyOi3Ir4EI5pIwcgMuKDUDfJ1RNdunYXLygiIiItNcJpGrr37S+UH92/CxfHYSr7bpE4mFAmUe3/eRvWLV8klG3s7LF8405Y29iKGBUREZF2MjQywqK1m1C5Wg2h7tCen/HjisUiRkVEuZGUmAjXSWPxNN365737D8bgMU4iRkVERKS9JBIJPHyXoknLNkLdhbOnsXCOCwdUiIwJZRLNhbOn4J1u7TgjY2MsWbcFFSpVETEqIiIi7WZpZY3lG3fCxs5eqFsT4Isje3eJGBURZUepVMLHbRquXboo1LVo2xGzvBdziTciIqJsGBkbY3HwFlSoXFWo27tjC37a+KOIURETyiSKe//dwoxxQyFPTgaQctdp3qLlaNCkuciRERERab/yFStjybotKstDzZvhhCsX/hIxKiLKyoZVy3Bo906hXLlaDfiv2QhDQ0MRoyIiItIN1ja2WBbyEyytrIW6lYsX4NRvh8ULqpjT2oTy/fv38e2338Le3h7W1tb47LPPcPHixZwbZqJfv34wMDBA586dNRsk5cvrVy8wedh3iImKEuqmeMxHp8+6iRgVERGRbmnQpDnm+AYI5aTEREwbNRAP7t4WMSoiyuj3Q3ux0t9LKFvZ2GLZhh0qX4qJiIgoe5Wr1cCitZshMzAAkDL7Z/aU8bh945q4gRVTWplQfv78Odq1a4fo6GhcunQJ9+7dQ7Vq1dC5c2dcuXIlT8favHkzjh49CmNjYyR/HA1L4omJjsLk4f0R9vKFUNd/+BgMGDFOxKiIiIh00yfdusPJZbZQjoqMwMQh/RD28rmIURFRqhtXL2HuVEehbGhohMVrN3GJNyIionxo3qY9PHyXCuX4uFhMGTGAfV8RaGVCef78+YiKisK2bdtQuXJllCxZEitXroSDgwNmzJiR6+M8ffoUkyZNwoIFC2BhYVGIEVNuJCUlYZbjCNy99a9Q17lrd0yb7SNiVERERLptyNiJ+G7IKKEc9vIFJg79DlERESJGRUTPHj/E1JEDkJAQL9S5efujcYvWIkZFRESk23p9NxDDHCcL5bevX2Hy8P6IiowUMariRysTyrt370bHjh1hZ2cn1BkYGKBXr174448/8Pbt2xyPoVQqMWzYMNSrVw8TJkwozHApF5RKJRbOnoG/Tp8Q6uo1bgrv5Wshk8lEjIyIiEi3SSQSOHv6okvXHkLdgzu3MX3MICQmJIgYGWlacnIy/v33X1y7dg3x8fE5N8jCy5cvcfbsWdy4cUOD0VF64e/eYuLQ7xD+Lu17y6hJzujas7d4QREREemJ8dPd0OWLr4Tyvds3MXPcUCQlJooYVfGidQnlFy9e4N27d6hdu7baY3Xr1oVSqcxV53fZsmX466+/8OOPP0Iq1bqXWeysXeqHX3/aLJTLVayMZet/gqlpCRGjIiIi0g8ymQzeK9aiUfNWQt3l8+cwZ9p4KBQKESMjTdmzZw/Kly+Pnj17YuDAgShVqhQCAwPzfJyEhAR07doVHTp0wPjx4wshUoqLi8XUkQPw7PFDoa573/4YM2WmiFERERHpD6lUijl+AWjQtIVQd/HcacyfOQlKpVLEyIoPA7EDyCh19LG1tbXaY6l17969y/YYt2/fhpubGzw8PFCnTp08nf/169d48+aNSt39+/cBAHK5HHK5PE/Hyy+5XA6lQgGlBFAodDshvmvrBqxbvkgoW1nbYFnIT7C2tYVCkfLzVCo+vl6FHPzaq/t4PfWLRKmAUqHk9dQDfG/ql4zX08jICIuDN2N0vx54/OAeAOD3g3tRsnQZTPXwyv5gIkt9HXK5HFBKCv18RdWf05SLFy/i+++/x4wZM7BgwQIAKfuEDB06FKVLl8Z3332X62N5eHggOjo60742FZxcLof7pDG4cfWSUNeqfSfM9lsGiaTwf7eJiIiKC2MTEwSs24KR33bH00cPAACHf/0ZZcqWx4SZHiJHp/+0LqGcKrMOV2pddncbkpKSMHjwYNSsWRMuLi55Pu+qVaswb968TB+Ljo5GRBGtRxgTnwRFQgzkMgmStPcy5eiP347Af07adTA2MYH/qh9RrqwDkuJihHqlQoHkhDgAgIQjynUer6eeSUyEPDkR8rhYXk8dx/emfsnsepqZGGPJmg0YO+AbvH3zGgCwff0a2NnZ44dho7I8ltiUiYlIlhggMtIA0iJIukVHRxf6OTTJ29sbNjY2Kn3UIUOGYPXq1Zg7d26uE8p//vknli5diqNHj2LQoEGFFW6xpVQqsdjTFad/OyzU1ahTD/5rNsHQyEgYSEFERESaYW1rh8DNv2B4n654/zZlcGjIygCULlsO3w4aLnJ0+k3rMpW2trYAgA8fPqg9Fh4eDgAqaytnFBwcjGvXruH8+fMwNDTM8/kdHR3Rr18/lbr79++jd+/eMDc3h5WVVZ6PmR9Kw0RI3ysgkwGGpnl/Hdrg8vlz8Jw5WbgBIJPJ4Bu0Hk3bdFR7rvJjB9vQtAQkUq6prOt4PfWLzMgIACDl9dR5fG/ql6yuZ8XqtbBs4w6M/b4XYj4mTgP9vVGmfEV80bOPKLHmRJJoAAMTA1haWkEmLfyEsrm5eaGfQ1Pkcjl+//13dO/eXa1v+8UXX2D+/Pl49OgRqlSpku1xoqKiMHToUAwZMgSfffZZYYZcbG1ZG4ifN/8olEuXLYcVG3fC3MJSxKiIiIj0W/mKlbEs5CeM+b4X4uNiAQALZ89AqTIO6PhZN5Gj019al1AuX748bGxs8N9//6k99t9//0EikaB+/fpZtn/z5g3kcjlat1bdPVkulyMsLAwGBgZwcXGBj49Ppu1LlSqFUqVKZfqYTCYrsg3kZDIZJFIpJFIJpDr4pf/e7ZuYPmawyoLoHn7L0OnzLzN9vgL4+HplOvl6SRWvp36RSFL+FoHXU+fxvalfsruedeo3xuK1WzBx2HdITkoCAMydPgG29qXQsp36jV2xpb4OmUxWJAllXdoQ+MmTJ4iPj0fVqlXVHktNIt+5cyfHhPKUKVMQFxeHJUuW5On8XA4ud47t34Plvp5C2dzCEss37IB9qdJc4k2P8ZrqHy71pp/4XtU/Ga9pnQaN4Bu0DtNHD4ZCoYBCocCsCSOx9qe9qNe4qdjh5pkuLAendQllAOjTpw927NiB8PBw2NjYAAAUCgX279+P9u3bZ5nwBYA5c+bAw0N9rZTy5cujevXqOHXqFDfpK2Qvnj2F05B+iImKEuomzPBAr+8GihgVERFR8dKyfSfMXRSE2VPGAgCSk5LgPGYQ1u44gDoNGokcHeVWZGQkgMxHVVtYWABAjkuyHThwACEhIdi9e7fQt84tLgeXsysX/4bndCehbGhoBL/AtahYoQKXeNNzvKZ6iEu96SW+V/VPZte0VZt2cJ7jDX9PNwBAQnwcpowcgOBtu1G+UmWxQs0XXVgOTrt6Yx/Nnj0be/fuxdChQ7Fu3TqYmprCw8MDT58+xZYtW1SeO3fuXPj4+OD06dNo164dpFJptgljAwOtfMl6I/zdWzgN+RZvX78S6r4fNgbDJ0wVMSoiIqLi6as+/fA27KUwcjImOhoTh/bD+l2HUalqdXGDo1xJXeYis5EjycnJAFI2ZMzKmzdvMHr0aHzzzTf45ptv8nx+LgeXvQd3/8OsiWORnJwk1M1dHIhWndSXFeGyQ/qH11T/cKk3/cT3qv7J6pr2Gzoab9++RUhQAADgw/t3mD5+ONbvOgwbO3tRYs0PXVgOTiuzq5UrV8aZM2cwY8YMVK1aFXK5HM2aNcPvv/+e6VIWcrk82436qGjExcZg8oj+ePLwvlD3eY/ecJ67gLtaExERiWTw2Il48/oVtq9fAyDl5q/joG8QsvsISjuUEzk6ykmZMmUAQG3ZifR1qc/JTHBwMN6+fYsffvgBZ8+eFeqTkpIQGRmJs2fPokKFCqhUqVKm7bkcXNbCXj7H5OH9ER0VKdRNdvXEl737Zfp8Ljukf3hN9Q+XetNPfK/qn+yuqaOzO8JevsCh3TsAAM8eP8K00YOwZvtemJYwEyHavNOF5eC0MqEMAPXq1cPhw4dzfN78+fPh6emZ48jj58+fayo0ykRiQgKmjxmMm9euCHUt2nbE/IDVXGKEiIhIRBKJBFM9vBERHo5De3YCAF49D4XT4G+x7pdDsLaxFTlCyo6dnR2qVKmC69evqz127do1GBsbo0GDBlm2d3BwQOvWrbFs2TKV+qioKDx58gSzZs3CoEGDMG7cOE2Hrtc+hL/HhEF9EfYi7TvG90NHY/DYiSJGRURERBKJBLP9luHt61e48OcpAMC/Vy/DZfxwLPlxm9omx5Q/Op/pk0qluVrGoihHUBQ3ycnJ8Jg8RnijAkCtug2wOHgzjIyNRYuLiIiIUkilUsz2X6Gy0/XDe3cwadh3iI3J+5ppVLQGDx6M8+fPq2xa/eHDB+zZswd9+/ZVmab49OlTnD17Vlh7ecSIETh79qzaP1tbWzRo0ABnz55lMjmP4mJjMGV4fzy6f1eo69y1O6ZzVh4REZFWMDQygv/qjahZt75Qd+7UccyfMREKBbdm1ASdTyiTuBQKBXxmTcGJIweEuvKVqmDFpp9hbmEpYmRERESUnqGhIXxXrkeTlm2EupvXrsB5zGAkJiSIGBnlZMaMGWjUqBF69uyJnTt34sCBA+jevTvMzMzg7++v8tzg4GB06NAB//zzj0jR6rekpCTMHDcMN65eEuqatW6HBSvWcfAKERGRFjG3sMSKjT+jXIW0Zb0O//ozlnp7cNlcDWBCmfJNqVRiqbcH9v+yXagrVcYBq7f9CvtSpUWMjIiIiDJjYmKKpet/Qq26aUskXDh7Gh5Txma66RtpB3Nzc5w9exajR4/G5s2bERQUhM6dO+Pq1asoV051HexKlSqhXbt2OW6W16pVKzRs2LAww9Y7CoUCntMn4K/TJ4S6mnXrY8m6bTA2MRExMiIiIspMydJlsHLrbtjalxTqtq9fgw2rlokXlJ5gQpnybd2KRcIGPwBgZWOLVVv3oGyFiiJGRURERNmxsLRE4OZfUKFyVaHuxOH98HWfztEaWqxEiRKYOXMmDh06hGPHjsHHxwf29uq7lY8ePRpnz57Ndl1lANi3bx9WrVpVWOHqHaVSiSXz3XB03y6hrlzFygjc9AssLDkrj4iISFtVqFwVgZt+gZmFhVC30t8Lv/60WcSodB8TypQv20PWYG2An1A2MzfHyi27UKVGLRGjIiIiotywK1kKq7buQcnSDkLdrz9txgpfTyaViTKxYeVS7NgQLJTtSpbCyq27OSuPiIhIB9Su3xAB67ap7PO1wG0aTqZbvpXyhgllyrMDu37CknluQtnY2ATLQnagToPG4gVFREREeVK2QkWs3LoLVtY2Qt3mtYFYt2KRiFERaZ89P23CykXeQtnMwgKBm35BhUpVRIyKiIiI8qJ5m/bwWbEOUmlKKlShUMBt0mj8768/RY5MNzGhTHly8uhBzJ8xUSjLDAywcPVGNG3VVsSoiIiIKD+q1ayD5Rt3wrSEmVC3NsAPm9cGihgVkfY4eeQAfN2mC2UjY2MErNuGWvWyX1KEiIiItM8n3XrAbUGAUE5KTMT00QNx+8Z1EaPSTUwoU679deo43CaOgkKhAABIJBJ4LV2NDp9+IXJkRERElF8NmjTHspCfYGyctqnY8gVz8fPm9SJGRSS+v8+chNuk0ULfVyqVwmfFOjRv017kyIiIiCi/+vwwBE4uc4RyTHQ0Jg7th0f374oYle5hQply5eLZ03AeMwRJiYlCnduCJejaq6+IUREREZEmNG/THouDN8PA0FCoWzh7Bvb/vE3EqIjEc/Xi35g+enCGvm8APunWQ8SoiIiISBOGjZ+MgSPHC+Xwd2/hOLAPnj99ImJUuoUJZcrR1Yt/Y+qogUhIiBfqJrt64psBw8QLioiIiDSqbefP4LcyBDKZTKjzcpmMY/t3ixgVUdG79c9VTB7+PRLi44S6ya6e6PPDEBGjIiIiIk2RSCSY4uGFHt/+INS9fvUS4wf2wZuwlyJGpjuYUKZs/XP5IiYN+x7xcbFC3bhprhgybpKIUREREVFh6NK1O+YvXQOJRAIgZbOS2VPG4dRvh0WOjKho3L9zC06Dv0VMdLRQN2qSM/u+REREekYqlWL2wuX49MueQt3zp48xfkAfhL97K2JkuoEJZcrSrX+uwmloP8TGpHWoR06cjtGTZ4gYFRERERWmbl/3xeyFy4WyXC7HrAkj8NfpEyJGRVT4nj1+CMeBfRHxIVyoGzByHMZNcxUxKiIiIiosBgYG8Fmx7v/s3XV4FFcXBvB3N+4hOASCBHenQKFYcWlxCg3uLkWLFdfi7u7ufBQKFLfiJWiChrjL7v3+CBmy2d1kk2wykff3PC2ZM3ZmZuXumTszqPlDAyn2yv0/DOzaFoH+/jJmlvaxoEw6PXv0AAO7tEFwYKAU+7XvYPQfOV7GrIiIiCg1tOrQBWOmzZWGIyMiMKp3V9y48reMWRGlnI/vPdGvc2t4e32SYq06dMGI32dIPfaJiIgo4zEzN8fc1ZtRqXpNKfbs0b8Y2r0DQkOCZcwsbWNBmbS8+O8JBnT5GQH+flKsU4++GDJuChvUREREmUR7t14YOm6KNBweHoZhPTqxqEwZjrfXZwz45Wd8fOcpxRq1/BkTZi1i25eIiCgTsLS0wqL1O1CqfEUpdv/2DYzo1QXhYWHxzJl5saBMGl6/eI7+nX+Cn4+3FGvTpTtGTprJBjUREVEm82u/Ieg7fIw0HB4WimE9OuHmP5dkzIrIePz9fDGwaxu8eekuxWo3aIypC1dqPKCSiIiIMjYbWzss3bwXRUqUkmI3rlzE2IE9EBkZKWNmaRMLyiR5/eI5+nZsBW+vz1KsVftfMPaPeSwmExERZVK9h/6m8fyE8LBQDO3eEbeuXpYxK6LkC/T3x8AubfD8ySMpVrVmHcxevgFmZmYyZkZERERycHDMguVb98OlkKsU+/vcKUwc2gdRUVEyZpb2sKBMAIBXz5+hT4eW+PL5oxRr+lN7TJj9J5RKvkyIiIgyK4VCgb7Dx6LXkFFSLDwsFEO6dWBRmdKtwIAADPy1DZ48uCfFylasggVrt8LC0lK+xIiIiEhWWbPnwIrtB5DbOZ8UO3f8MCYN78eiciysFBJe/vcUfTu10ngIyY8tfsLk+ct4qR8RERFBoVCg34hx6Dl4pBSL6al8+9oVGTMjSrygwAAM/rUtHt27I8VKlCmPJZv2wNrGVsbMiIiIKC3IlccZK3ccQvacuaXY6SMHMGXkQKhUKhkzSztYUM7k3J89Rp+OLTVuc9G4VVv88edqmJqaypgZERERpSUKhQL9R47XKCqHhYZgSLcOLCpTuhESHIQh3Trgwd1bUqxYqbJYvm0/7BwcZMyMiIiI0pJ8LgWxetdhZMuRS4qdPLQXU0cPZlEZLChnas+fPELfjq3g6/1FijX9qT2mLVrJYjIRERFpkYrKg7SLyneu/yNjZkQJCw0JxtDuHXH/1nUpVrRkaazYfgAOjllkzIyIiIjSIpdCrli18xCyZs8hxY7v34XpY4ZCrVbLmJn8WFDOpJ49eoC+nVrBz8dbijVv2wlTFiznbS6IiIhIL4VCgf6jxqPHwBFSLCw0BIPd2uPG5YsyZkakX2hoCIb16Kxx4sO1eEms2H4QjlmcZMyMiIiI0rKCrkWxaschZMmaTYod2bsDM8YNz9RFZRaUM6EnD+6jf+fW8Pf1kWIt23XGpLlLWEwmIiKiBCkUCgwYPUGrqDy0R0dcPn9WxsyItIWFhWJEz19w6+olKVaoSDGs3H4QWZyyypgZERERpQeFihbHqp2H4Bir3XBo11bMnjgKQggZM5MPC8qZzON/72LALz/B389XirXq0AW/s5hMREREiRBTVI59T+WI8HCM7NMF508elTEzom/CwkIxsncX3Ljyrfd8gcJFsGrnYThlyy5jZkRERJSeuBYriVU7DmncJmv/9k2Y8/tvmbKozIJyJnL3xlX069QKAf5+Uuznzm6YOPtPKJV8KRAREVHiKBQKDBg1AQNHT5RiUZGRGDuwB04e2idjZkTR90we3qMzrv39lxSLvhfiYY17IRIREREZokiJUli54xDsHRyl2N6t6zFr4qhMd/sLVhEziX8u/g8Du7ZFcFCQFGvTpTvGzVjAYjIRERElS49BIzBy0gxpWKVS4fdhfXF49zYZs6LMLDgoEEO6ddDomezsUhCrdh5C9py54pmTiIiISL9ipcpgxfaDsLN3kGL7t23E9DFDoVKpZMwsdbGSmAmcP3UMw3t2RnhYqBT7pWd/jJs+n8VkIiIiMorOPftj/MyF0rAQAtN+G4I9m9fJmBVlRoEBARj8azuNB/C5FHLF2j1HkSNXHhkzIyIiooygRJlyWLH9gEZP5cN7tmPKyIGIioqSL7FUxGpiBnfiwB6MHdAdUZGRUqz30NEY/vt0KBQKGTMjIiKijKbNL90wZcFyjRPWcyb9hi2rl8qYFWUmAf5+GNj1Z9y/fUOKFSpSDKt3HWExmYiIiIymZNkKWLXzsMaD+k4c3IPfh/VFZKwaXEbFgnIGtm/bRkwa0V+jy/3Q8VPRb8Q4FpOJiIgoRbRo2wkzlq6FiampFFs8czJWLZyVKR9YQqnHz9cH/Tv/hEf37kgx1+IlsXrXEd7mgoiIiIyuWKkyWL1L89kMZ44exLiBPREZESFjZikvzReUIyIiEBoamvCEeoSGhma6G2MDwJZVSzBrwkjph5tCocD4mQvxa9/BMmdGREREGd2PzX/C3JWbYGZuLsXWLp6HuZPGZMp2GaU8ny9e6NepFZ4+vC/FipUqi9W7jsApW3YZMyMiIqKMzLVY9MnrbDm+nbz+6/QxjO7nhojwcBkzS1lptqB8+/Zt1KpVC7a2tsiSJQtKly6NkydPGjSvu7s7hg0bhvz58yN79uywsrJChQoVsHnz5hTOWn5CCKxcMBOLZ02RYiYmJpi2cCXa/NJNtryIiIgoc/nhx6ZYuG47LCytpNieLeswcWjfDN9jg1LXl8+f0LdjSzx/8kiKlSpXEat2HoJjFicZMyMiIqLMoKBrUazdcxQ5c3+7vdal/53GiN6/ICws6Z1k07I0WVB2d3dHvXr1kD9/fvj4+CAwMBCtW7dGy5YtcfHixQTn3759O8qWLYvr168jKCgIXl5eqFu3Lrp164YlS5akwhbIQ6VSYc7vv2HdkvlSzMzcHLNXbETTn9vLmBkRERFlRjXq1Mfybftha28vxU4f2Y+RvbsgNDRExswoo/jwzhO92zfHy+fPpFjZilWwfJvmg3KIiIiIUlL+goWxds9x5HHOL8WuXjyPod06IjgoUMbMUkaaLChPnToVQgisXr0atra2MDMzwx9//IGiRYti7NixCc4/efJk9OjRA7lz5wYA2NvbY+HChciePTu2bt2a0unLIiI8HOMG9cTereulmIWlFRat34F6jZvLmBkRERFlZhWqVMe6Pcc17i135cI5DPzlZwT4+8mXGKV7r9z/Q882jfH21QspVrFaDSzbug92sU5iEBEREaWGvPldsGbPMTi7FJRit65eQv9ffoKfr4+MmRlfmisoCyFw5MgR1KlTB3Z2dlJcoVCgefPmuHbtGj58+JDo5apUKkRFRSFr1qwJT5zOBAUGYEi3DvjfiSNSzM7eAcu37cd3tevJmBkRERERUKREKazffxJ587lIsfu3b6B3++bw+vRRxswovXr68F/0bt8cnz68l2JVa9bBkk27YWNrF8+cRERERCknd15nrN1zFC6Fi0ixR/fuoE/75vD6lPh6ZlqV5grKnp6eCAgIQNGiRbXGxcQePXqkNU4XX19ffP78GXfv3kXXrl2hUCjwxx9/GDVfuXl7fUafDi1x85+/pVj2nLmxbt9xVKhSXcbMiIiIiL7J51IQ6/efhGvxklLM/elj9GzTBB88XsuXWDpy+fJlTJ8+HVOmTMGxY8ekhy8bwsfHBzt27MCUKVMwf/58nD59OlHzpyV3b15Dn44t4Ov9RYrVbdQcizfugpW1jYyZEREREQE5cuXB+r3HUbx0OSn24r+n6NmmKTzfvpYvMSMylTuBuHx9fQFE36YiLgcHB41pElKnTh14eHjAz88PWbJkwZo1a1ClSpV45/n8+TO8vLw0Yu7u7gCiezmrVCqD1p1cKpUKQq2GUABqte66v+fb1xj8azt4vnktxVwKuWLp5j3I7ZwPanXq5GoMQv11e9Uq8Nnv6R+PZ8aiEGoIteDxzAD43sxY0uPxzJo9O1bvOozhPTvj39s3AQDvPN5g1K+tsGj9TlR2qQMIRYrnkVrtOWMRQqBXr17YvXs3evXqBRsbG/Tu3RulS5fG0aNHYWlpGe/8CxcuxIoVK1CzZk24urri5cuXmDFjBvLkyYMTJ07AxcUl3vnTkhuXzmPS4B4Ij/WAm+ZtO+H3OYthaprmftoQERFRJpUlazas2nkYw3t2wt0bVwFEt3t7tmmK5dv2wbVYyQSWkLaluVaXQhH9I0JXjwm1OvrnklJpWMfqf//9FwDg7++PefPmoV27dpg3bx5GjRqld54VK1Zg6tSpOscFBQXB39/foHUnV3BYJNThwVCZKBCp4zD99+QRRvRxg0+snhklypTDglUb4ZjFCZGhwamSp7EItRpR4dE/DBQGHl9Ku3g8M5iICKiiIqAKDeHxTOf43sxY0uvxtDI3w6LVmzFhWH9cuxz9sGU/by8M/KU1HLZuQZ06dVI8h6CgoBRfhzFt2LABGzZswKlTp9CoUSMAgJubG8qWLYupU6di1qxZ8c5fr149DBkyRKPgOmTIEJQuXRqjR4/Gnj17UjR/Y7l4+ghmjxmIqMhIKdahWx+MmjzT4N8HRERERKnFzt4eS7fsxZh+3XDlwjkAwJfPH9GnfQss2bwHpctXkjnDpEtzBeVs2bIBiL4sL66Ynskx0xjKwcEB06dPxz///INJkyZh2LBhenswDBgwAO3atdOIubu7o3Xr1rC1tZV6Sac0YRYBpY8aJiaAmZWZxrjb165gZJ+uCA789pTI6t//gDkrN8LaxjZV8jM28bU3tZmVNRRKE5mzoeTi8cxYTMzNAQBKHs90j+/NNE4IiOh/YoWiY7EmgYAARPSJdnWUGipTSygUSgiIOPN+nTbWMrWWH3vc1z/E1/8JfJtfGpYG4yxXaE4b659vy/26DAUAAWsMm7sOy6aOxOXT0c+ACAkOwsyZM9G8efMULw7a2qav9tKKFStQsmRJqZgMRN8KrlmzZli9ejWmTZsGMzMzvfOXL19eK1a8eHE4OzvjxYsX2jOkQVs3b8TMUQOkDiYA0HvoaPQdPlbqkEJERESU1lhZWWPB2m2YNKI/zhw9CADw9/NF/84/YcHabahas7bMGSZNmiso586dGzlz5sTjx4+1xj169AhKpRJlypRJ0rKLFCmCv/76C97e3siZM6fOaXLkyIEcOXLoHGdiYgITk9T5AW5iYgKFUgmFUgFlrB/9Jw7uxdTRgzR6ZjRu1RZT5i+D2deiT3qkBr5ur4nG9lL6xOOZsSgU0Z9F4PFM99LKezN2kTR2MVNf4VSa9uv/4i1m4lvhMvbVTkktnOLruNjDahFn2ljbFXv9QHTxVHsHRI8QUnVVQAEFoAAUCkABBWLXxxSx/x89GRSK6B7KiFRBARWUSiBuTU2hUEjLUsRk8nUdiFnP1yUrYsZJ80LKKfZ2RC8z9roUUk7RNeDo8UqFAkqF5vRKhSJ6WPFt/jUbtmDO1PHYvGYFnPPlx+HDh+MtjBpLarXnjCE4OBh3795Ft27dtMZ99913OHDgAB49eqSzaByfhw8fwtPTE+3btzdOoilowYIFWlcYjvh9On7pNUCmjIiIiIgMZ2ZujumL18DWzh4HdmwGEN2ZYmj3DpixZC3qNW4uc4aJl+YKygDQrl07rFmzBh8/fkSuXLkAABERETh48CAaNmwIJycnadqQkBCEhITA0dFR6nUshNDqqRAVFYV//vkHuXPn1lswTsuEEFi/dAFWLpipEe/csx+GT5zOy/yIiNIgfYVTABpxrWKm9HfyCqcaOcT0aI2IgCLK9GuP1vgLp3GLuskqnH6dSPE1ppAKlYpYf8f6f6zCaQxdxdbouGaRUyrOxqxTKnJ+jSm/rSlu4fRbXH/hNKYgqoBCKpzG5KGMUziNm3fsgi20YgrNnGJvU6zibMywUKkREmwKG1u76AJpnP2jazs0C9ffbjUWe73x7XOdy0xm79CNq5ahYqliaNSoEfLkyZOsZWVEHh4eEELo3DcxsTdv3hhUUJ4+fTp8fHzw7t07XL58GWPGjMHkyZPjnUfu54t4eHhg0qRJ0rBCqUT/CbNRs2UHvPocqPHZE/vzSPN1HPvTJdZE0H6vxf5bEWsOjc+p2J9fWsvTfk/EvN+EWg11eDiUKlMoTZRxPg9iLVPrMy5WPnFP8sTNJ+680jI0t0URe4CSLD3ey57ix2eHZEx8r2Y86fGYKhTA2OnzYGtvjy2rlgIAIsLDMaZ/N/w2bQ7a/NJNmjZm21QqVZp9vkiaLChPnDgRBw8eRPv27bF06VJYW1tj0qRJ8PPzw/z58zWmnTlzJmbMmIFLly6hVq1aCA8Px3fffYdhw4ahfPnyyJIlC9zd3TFv3jw8efIEu3fvTneXxUVGRmLWhJE4vHubRnzouCno2ndwutseIsqYYgqX+i7Xj69wGv2P0CicxsS+zWt44VQaH7sYmojC6bflaxdvNYoX4utv8SQVTr9GjVU4jTVeKpwqpKmi51EDKrUJTMxNoFCa6Cwoxi1yKpUJF06lnHUWH7WXmVDhNL5pYrYrsYVTXcs0tHD6bfvS1vetSqWCP8LhYG+ZrnrcxqVQKDBkyBC500izQkOj75NtYWGhNS7mYXwx0yQke/bsMDMzQ2hoKEJDQ3Hjxg18/vwZzs7OeueR+/ki9vb22LJlCzp16gQBYPbCZWjQuKnG90TcE3cQAuqvf6sBQET/1IyJfftu+jad9Gmv9d3wbR7omkbHbWW0bvvy9T+1Wg0REQYBBVQKhcb3X9xF63icjOZ3o9AVj/2v9gJ03rImJqb49rf0URfzvafno09fAT96GbHL8bHi8RTwNb4ddRbw406lub7Y82pNE+fzXCsfzSQ1xml8z8ZNVa2GKiIUUWp19BVdevLRlSelUXx2SIaUXp87Qfql52Pab8hIWFtZYdWiuQCi2wezJ47Gp3ce6DVoOBQKBUREBKIUpggIMIUyFb47kvJ8kTRZUM6ZMyeuXbuGSZMm4eeff0ZUVBQqV66Mq1evonTp0hrT2tjYIGvWrNLlkRYWFti+fTuWLVuGxYsX4/3793ByckK1atVw8+ZNVKhQQY5NSrLgoECMHd4L1y9dkGLmFhaYunAFfmz+k2x5EWVmxiycxp1e8z6n2j1ZNdalp3CquRw9PVwTUTh1CIoAVJHw/xICfO3VKolVOAX0FOq+/l+7d2bChdPY08T+QRdTUFTGjim1C5C6eozFLXLGxGIXTqXlxCqcxi5ixlc4/Zaz7iJnUgqnCS0zvsJp7PWqVCoEBZrCzt4epiamWsuU1sMfvERpho2NDQDdReOYWMw0Cenbt6/094QJE1ChQgW0b98e//zzj9550sLzRVq3bo0dO3bA2tpa4z7SKUEIkeB3cdzvVugZF7cYHaVSISgwADa29jD5etshjZOpsXOItcyY6XQXkfWvL25bQ9f6dLc14sT0rS9WuyTmVkCxi/Yxw9/W8TWm1rO+ePZD7BG6tjluu0b6V6Ot9G06fftBY3VxtkW77RZd0EBUJAAz6eqXuMvQ9XqJuzZDvnVjF/AB3YVqfYXxhHrgx8TjFszj9sKPr2Aee2lxY4b0wtfVdpP+1bcthuSTyDYNnx2SMfE5IhlPej+mPYeMRracuTFrwiipd/DGlUvg4+2NsTPmQ2FuClNLU9jbO8BEx/eLsSXl+SJpsqAMAM7OztiwYUOC040bNw7jxo3TiJUoUQLLly9PqdRSjdfH95jY/xe8/O+JFHPI4oSFa7ehfJXqMmZGmUVChVNAd4FSrVYhKlKNKJOor9XGONPGWqZWMROaDfhkF06/zhjzQyKhHqcaiSB6ZMx9ThWxYkktnMaORf+tWeTUVzj9VlyERuH06xqTXDj9ujjNwunXZcQUToWjFSLDFcibN/oHcHyF09jbqtX7U0+RM77Cqa59o2u98RZsY61Ha5mZrHCqUgLhJkpYmJrAxCR9ncknyqzy588PpVIJT09PrXEeHh4AgIIFCyZ6uc7OzmjWrBk2b94MX19fZMmSRed0aeX5Im3btk2V9aQklUoFZVQ4HGzT91UFxhS3gB8TixnW186L+dvgE/IJFODjK+DrGhfzt0qlQnCQKaxt7DQKGgYV/DW2R9d+0NO+lYr12kV7qWe++NZ3X7uA/20t+vYDYk1meC987Xa3FIrnOMdema59HHfdWtsAXdPFyjkmpvj2t+Jr+z52sTqGQ3AURFQUAnzCIb72eozd1oyeX7OAH7foDa3pNcdpxjTn1S7qa7dXdfXCj9sW/7YYA/JRJGJbYq1P17ak1bZ1WnmOCBlPRjimP3VyQ9YcuTB2QA+Eh0V3Eji8Zzt8vL9g2MzlUNg4RLe1UqGgnJR2SZotKGd2D+7fx+BOLeH9+aMUc3YpiKWb9yB/wcIyZpYxpeQDogwtnGouT0ejLJGFU43lplDhFIhd2IM0LNRqICIKSqigUH7bZl09MaUGzNeYMnYsgcJp3GXGzSm+wmn0sEKjcBqzjrjbBMRTkIyncIo40yS3cKpvvToLtHH3dzIadz72lggNVSNXDjv+ACYiSmWWlpaoVq2azl7Ely9fRo4cOVCiRIkkLTs0NBRKpTJVHoRIpEvs+83HisqRSpKoVCr4KyLg4GCdLtpIcXtb6yvA6yvgR09nYJFcJDxt3PVpdCLRUVzW9Rsndn4JnWCIv6Ae/a/K0RJR4Qrkzm0HhcJEWp9GAf/rDJoF/OjlqtVxc9Vcn668vm2E/m3W+g2Kb79NY3LQdeIk7vo0VmdAAV9rG3QsQ28B/+sMilh/A5B+d8bEY56BFWt09L96ivjaRXcDeuELNUREBJSRptG3R4j9myzW8hIqmMfOR1eeBuUTt4Af37YkkE9aLeCT4WrXb4TVOw9haPeO8PfzBQBc+t9pePXugIXrdgDIKm+C8WBBOQ06deoU2rVrp3EPk7IVq2Dhuu3IkjVbspadqAdExf4C+vpHch8Qpa9wKtRqiPAIINIUUChjzSt0ryuBwikQT/E0VuEUQkjFu6QUTqPH6r9cP77CqQLGe0BUzC2DYoqnMYXTmOkTKpx+2x7jFE7VKhVCgkxhbWencVl9vPsmMeuNXUyNu0x+qRIRUQYzZMgQdOrUCbt27ULHjh0BALdv38apU6cwZcoUjULWqVOncO7cOQwaNAgFChRAVFQUzp49iyZNmmgs8+bNmzh69CiaN2+epMsciSj9if3cg1hRWXJJq3zsrRAaKpArl71RThLo652t9ZtcTxHf0KJ8fMVoQ3rh6+28BM3f2tCxnITXp71MndsXp2agjlW017iNztd1JtQLP/b61KooRAkTmFiYaN4eQc8+1lvA/xpQx9mYRN1GR8+x/jZvrG3QsQyt+WMX8b/+rYj1t763uKEFfOnfeOsfcQr4sZanNa80QeIK+HFzFWoVVBFRMFVEQWmiuafi7ZCW0LbEyefb9mnnaixlKlbB+v0nMditHT54Rl999vTf2+jXsRn+OncWhQoWMPo6jYEF5TTG09MTrVq1QkREhBT7rn5TDJu+GIEKKwR4BwNIZOH060QJPyBKd+H0Wyz+wmnMMmMXTmNiCRVOhVoFldoEphYmUJqYJqpwGpNHQg+I0lc4jbsdcS/tiV2kjFvk1Pww1F/k1Ng3RiqcItYy0hqVSgV/EQ4HO17SSURElFwdO3bEvXv34ObmhsOHD8Pa2hp79+5Fu3btMHbsWI1pL1++jAULFqB169YoUKAAFAoFVq9ejdGjR6Ns2bLSA6vPnTuHxo0bY9OmTfJsFBFRJpDee+GnN7o6mgkhon+fBpjB3t4eSqVJAkVdA4rkOorDhpww0ChUJ1CMTtYVAjq3QccJg1gF+8TcRifucpFAXt82RPdJC8TN9dtWRM8Ts09j7zO1gDpKjUilGgqV/iJ+Ygr4cfPVeRJAXwE/1vbF1Gm+luB09sKPW8A3cXLGjPUHMW1QV7x+Hn3b27cv3fFT61a4e/culGnwwYMsKKcxzs7OmD9/vvS08196DcSw8VNh+vVel4YUToHYL9B4enLqKJwinmliisSJLZzGt8yYf9UqNQIDoz/gTU1MtHu+ptHCKREREVFqmD17Nrp3746///4bUVFR6NevH6pUqaI1XZMmTeDo6CjdV9nExASHDh3CmzdvcOPGDbx//x7Vq1fHsmXLUKRIkdTeDCIiohSjrxe+AgKmSgXMTJR8joiRJfVhtokuksdZjkqlQlCQCaxtom/LqLOIr6NwbMyH2WoVoxNan54CfkzMJUshbNp/EiP7dMHta5dhZWWN1atXp8liMsCCcpo0ePBgvHnzBgULFsTAgQPlTidVqBTgBzwRERFRPIoVK4ZixYrFO03NmjVRs2ZNrbiLiwtcXFxSKjUiIiLKhLR74adOZ8DoB92awcHWIoNdFZ0Fl/86ix49eqBr166oXr263AnpxYJyGjV//ny5UyAiIiIiIiIiIqJUYmlpiR07dsidRoLYFZSIiIiIiIiIiIiIDMKCMhEREREREREREREZhAVlIiIiIiIiIiIiIjIIC8pEREREREREREREZBAWlImIiIiIiIiIiIjIICwoExEREREREREREZFBWFAmIiIiIiIiIiIiIoOwoExEREREREREREREBmFBmYiIiIiIiIiIiIgMwoIyERERERERERERERmEBWUiIiIiIiIiIiIiMoip3AmkB+Hh4QAAd3d3mTPJuFQqFYKCgmBrawsTExO506Fk4vHMWPw9PBAWFgrvR494PNM5vjczFh7PpItp08W08Shx2DZOGr5nMx4e04yH7d6Mie/VjIfH1LiS0jZmQdkAHh4eAIDWrVvLmwgRkZzGj5c7AyIio/Lw8EDFihXlTiPdYduYiDI8tnuJKBNKTNtYIYQQKZxPuufn54eLFy8iX758sLCwkDudDMnd3R2tW7fGoUOH4OrqKnc6lEw8nhkLj2fGwWOZsfB4Jl14eDg8PDxQp04dODo6yp1OusO2cdLwPZvx8JhmPDymGROPa8bDY2pcSWkbs4eyARwdHdGqVSu508gUXF1dUapUKbnTICPh8cxYeDwzDh7LjIXHM2nYMznp2DZOHr5nMx4e04yHxzRj4nHNeHhMjSexbWM+lI+IiIiIiIiIiIiIDMKCMhEREREREREREREZhAVlIiIiIiIiIiIiIjIIC8qUJmTPnh2TJ09G9uzZ5U6FjIDHM2Ph8cw4eCwzFh5PovSF79mMh8c04+ExzZh4XDMeHlP5KYQQQu4kiIiIiIiIiIiIiCjtYw9lIiIiIiIiIiIiIjIIC8pEREREREREREREZBAWlImIiIiIiIiIiIjIICwoExEREREREREREZFBTOVOgDIvPz8/HDp0CGfPnkV4eDi2bdsGS0tLg+cPDQ3Fxo0bcePGDZiYmOCHH35A586dYWJikoJZkz7v37/H+vXr8ezZMzg6OuKnn35C/fr1DZp3wIAB+Pz5s1a8QoUKmDBhgrFTpa/u3buH7du34/3798iXLx/c3NxQokSJVJufjOv48eM4duwYgoKCUKpUKfTq1QvZsmVLcL4rV65g0aJFOsdNnToVpUqVMnaqlIDAwEAcOXIEp06dQmhoKNasWQMnJyeD54+IiMCWLVvwzz//QAiBmjVrws3NDWZmZimYNRHpcvv2bezZswcvXrxA8+bN0a1bt0TNf+fOHezYsQMfPnyAi4sL3NzcUKxYsZRJlgwSFBSE9evX486dOzA3N0f9+vXRoUMHKBSKBOedO3cubty4oRXPkiUL1q5dmxLpUiwXL17E/v374ePjg6JFi6JHjx5wdnZOtfnJ+D5//oz169fj8ePHsLOzQ8uWLdG4cWOD5h0xYgTevn2rFS9RogT++OMPY6dKBoqMjMTZs2dx8OBB+Pr6Yty4cahUqVKilnHo0CGcPHkSISEhKFeuHHr27IksWbKkUMaZF3sokyzmzZuHYsWK4cKFC3j9+jX279+PqKgog+f39fVF1apVsWTJEtSoUQPlypXD2LFj0bhxY0RERKRg5qTL3bt3UapUKVy6dAmNGjWCk5MTmjVrZnAx+MSJE3j37h06duyo8V+dOnVSOPPMa9OmTahSpQqCgoLQpEkTfPr0CeXLl8fhw4dTZX4yrn79+qFDhw7ImzcvGjRogBMnTqB06dJ4/vx5gvN6eHhg//79qFq1qtZ7MEeOHKmQPcW2evVquLq64sSJE3j//j3279+PkJAQg+cPCgpC7dq1MWPGDFSqVAlVqlTBzJkzUadOnUQth4iSJzAwEK6urujbty8AYP/+/bh3716ilrF+/XpUrVoVoaGhaNKkCTw9PVG2bFkcO3YsBTImQ3z+/BmVKlXC+vXr8f3336NEiRIYMmQIWrduDZVKleD8//zzD65cuaL1fduqVatUyD5zmzx5Mho2bAhbW1s0atQI169fR6lSpXDr1q1UmZ+M7/HjxyhdujROnTqFBg0aIHfu3GjTpg2GDh1q0PxnzpzBixcvtN6PhnaKIuO7cOEC8uXLh+XLlyMkJAT79+/Hhw8fDJ5fCIEuXbrAzc0NBQsWRL169bBv3z6UK1dO58kDSiZBJIMXL16I8PBwIYQQbm5uAoAIDAw0eP5+/foJGxsb8e7dOyl29+5doVAoxLx584yeL+mnVqtFmTJlRPny5UVUVJQUX7ZsmQAg/vnnnwSX4eLiItq0aZOSaVIs79+/F9bW1qJfv34a8Y4dOwonJyfh5+eXovOTcR07dkwAENu2bZNiYWFhwtXVVdSpUyfB+Xfu3CkAiJs3b6ZglmSoV69eidDQUCGEEEOHDhUAhIeHh8Hzjx07Vpiamornz59LsefPnwtTU1MxYcIEo+dLRLqFh4eLFy9eCCGE8PDwEADE0KFDDZ7fw8NDWFpaikGDBmnE27ZtK7Jnzy4CAgKMmS4ZqEuXLsLR0VF4eXlJscuXLwsAYuXKlQnO36pVK1G4cOGUTJF0uHnzplAoFGL+/PlSTKVSiapVq4oSJUoIlUqVovNTyqhevbooVqyYVFcQQohNmzYJAOLs2bMJzl+qVCnRqFGjlEyREunjx4/Cx8dHCCHExo0bBQBx9OhRg+fftWuXACAOHTokxYKCgoSzs7No1qyZ0fPN7NhDmWRRqFAhmJubJ2nemNtjNG/eHHny5JHi5cuXR7Vq1Xi5WCq7ceMGHjx4gJ49e2rcbqR79+4wNzfHunXrZMyOdNmxYwdCQkKkXlMx+vXrBx8fHxw4cCBF5yfjWrduHRwcHNCxY0cpZmFhgW7duuHixYsG9VKmtKNAgQKJuv1TbEIIbNiwAfXr14erq6sUd3V1RYMGDbB+/XoIIYyVKhHFw9zcHIUKFUry/Nu3b0dYWJjO71ovLy8cOnQomRlSYgUEBGDPnj1o06aNxi2latasidKlS/M3SBq2fv16KBQK9OrVS4oplUr06dMHT548wZUrV1J0fjK+R48e4dq1a+jWrZtGXaFz586ws7Pj+zGdypkzZ7JuTbFu3TrkypULLVu2lGI2Njbo0qWLdFU0GQ8LypTuPHr0CEFBQTrvo1O5cmX8999/8PX1lSGzzOn69esAoHU8rK2tUbJkSVy7ds2g5Tx8+BDdu3dHly5dMGnSJDx8+NDouVK069evw8zMDGXKlNGIV65cGQASPGbJnZ+M6/r16yhXrpzW/eMTezzmz5+PTp06oW/fvti4cSNCQ0ONniulrFevXkmXY8dVuXJlfPz4Ea9fv079xIgo0a5fvw4LCwut+9jzu1Y+9+7dQ0REhN7P2Pv37yMsLCzB5fj4+KBv377o3LkzRo0ahQsXLqRAthTb9evX4erqCgcHB414Ytq+yZmfjE/fb1AzMzOUK1fO4GPy/Plz9OjRA7/88gsmTJiAu3fvGj1XSj03btxAxYoVte5pX7lyZQghdN7DnpKOD+WjZFmwYAGuXr2a4HTm5ubYsWOHUdbp6ekJIPrsVVwxsXfv3vGm60nQu3dvg4rxZcuWxaRJkwAkfDwM+TK3sbFBjRo1ULVqVYSEhGDfvn2YMWMGZs+ejdGjRydyKyghnp6eyJYtm1YB0sbGBjY2NgmeuU3u/GQ8arUaHz58QK1atbTGxf48TEi5cuVQqlQp5M+fH0+ePMHQoUMxc+ZMnDt3Di4uLkbPm1KGod+PBQsWTNW8iNKz9+/fY8iQIQZN26lTJ7Rp08Yo6/X09ESOHDm0fhQ7ODjAwsKC37XJNGPGDIMKR/b29tiwYQOAhD9jVSoVPn78iAIFCuhdnomJCerXr4/SpUvD3Nwc586dQ926deHm5oYNGzZAqWR/r5Tg6emJ4sWLa8UNbSsld34yvoTej4bUKCwtLVGlShVUr14dYWFhOHz4MGbNmoVJkyZhypQpxk6ZUlhAQAACAgISbAeT8bCgTMly9epV7N+/P8HpLCwsjLbOmIfumZpqv3xjnmBvSO8A0nb06FF8+vQpwen8/PykvxM6HoYciwsXLiB79uzS8JAhQ9C5c2f89ttv+OGHH1ClShUDsidDRURE6DxegGHHLLnzk/EY4/OwYcOGaN++vcaPWDc3N1SsWBE9e/bEuXPnjJgxpSR+PxIZX0BAgEFtXSD69mvGwu/alHXp0iWcPn06wemyZs0q/W2Mz9hVq1ZptHn79u2L2bNnY9y4cahVq5bGLRXIePS9nww9bsmdn4wvofejSqVCVFSU3s9RADh58qTWb9BevXph6tSp+OGHH/DDDz8YPW9KOWwHpz4WlClZRo0apXHfTn3i9mRMDltbWwDQ+bT64OBgAICdnZ3R1peZrFu3zqAP2dhn/RI6HoYci9hf5ACgUCgwZswY7Nq1CwcOHGBB2chsbW31XvYeEhKS4DFL7vxkPJaWljA1NU3W52HsH8sxSpQogVatWmHPnj3w8/ODo6OjUfKllMXvRyLjy5s3L/bu3WvQtKVLlzbaem1tbfHx40etuBACoaGhfC8n08SJEw0q3sbuFGOMz9i4bV4AGDlyJCZNmoT9+/ezoJxCbG1tk3Xckjs/GV9C78eYNnJ8dL0fx44di/Xr12P//v0sKKczbAenPhaUKVmqV6+e6ussUqQIAODNmzda4968eQNTU1NezptEzZs3T/Q8sY9H3EvB3rx5I41PrFy5cgGIvs8cGVeRIkVw8eJFBAUFSV+8APDhwwdEREQkeMySOz8Zl6urq97PQwDJeg8KIeDr68uCcjqR0Pdj7GmIyDB2dnZo27Ztqq+3SJEi+OeffxAaGgorKysp7unpCZVKxfdyMum6VVRCEvqMtbGx0XhguKHMzMzg5OTENm8KKlKkCJ4+faoVN/S7Mbnzk/HFfj/G3Ms6Bn+DZk6WlpZwdnZmOzgV8SZNlO4ULlwYBQoU0HqAhRACFy5cQO3atTWe9Eopq169elAoFFrH482bN3j58iUaNmyYpOXeuXMHQHSxjIyrfv36AICLFy9qxM+fPw8ACR6z5M5PxlW/fn08fPgQ3t7eGvHz58/D3NwcderUSdJy79y5A2tr6yT9OCZ5ZM2aFeXKldP5gKe//voLFStWhJOTU+onRkSJVr9+fQgh8Pfff2vE+V0rn9KlSyNHjhxan7FRUVG4dOkS6tevr3XPa0O8f/8enz59Yps3BdWvXx8fP37Es2fPNOKJafsmZ34yvjp16sDU1FTr/fj582c8evSIv0Ezqfr16+P27dsICgrSiJ8/fx62trb47rvvZMosgxJEMnNzcxMARGBgoM7xixYtEm3atBG+vr5SbMWKFQKAOHLkiBRbunSpACDOnDmT0ilTHJ07dxZZsmQRL168EEIIoVKpRKdOnYS9vb34+PGjxrR9+vQRgwYNkob/+ecf8ffff2tM8/LlS1GsWDHh5OQkPn36lPIbkMmEh4eLIkWKiCpVqojg4GAhhBB+fn6iZMmSonLlykKtVkvTfv78WbRp00YsW7YsSfNTynv+/LmwsLAQffv2lWIPHz4U1tbWYujQoRrT/v3336JNmzbi3LlzUmzNmjXCx8dHGlar1WLhwoUCgBg5cmSK50/6DR06VAAQHh4eOsevXr1atGnTRrx7906K7dixQwAQW7ZskWJbt24VAMTevXtTPGci0ubh4SEAaH0mx/jw4YNo06aNWLlypRQLCwsThQoVEtWrVxchISFCCCF8fX1F8eLFRfXq1fldK5O5c+cKpVKp8T06a9YsoVQqxZUrVzSmnTVrlmjTpo0IDQ0VQgjx6tUrsXfvXo1j5+PjI3788Udhamoqrl69mjobkQl9/vxZODo6ip9//llERUUJIYR4+/atyJ49u+jQoYPGtP/++69o06aN2LdvX5Lmp9TTp08fYWtrKx4/fizFevXqJaytrcXr1681ph0yZIjo1auXNHzr1i2N97EQ0ce0bNmyws7OTrx58yZlk6cEbdy4UQAQR48e1Tn+zJkzok2bNuLy5ctS7P79+8LU1FSMGDFCit26dUuYm5uLCRMmpHjOmQ0LyiSLmKJGmzZthIuLiwAgWrduLdq0aSPGjRunMW2HDh0EAPHhwweN+NixY4W5ubmoW7eu+O6774SVlZVYvnx5am4GfRUYGCiaN28ubG1tRdOmTUXx4sVF7ty5xYULF7SmzZs3ryhcuLA0/PjxY9GwYUNRsGBB8eOPP4oaNWoIS0tLUaFCBXH37t1U3IrM5enTp6JkyZIib968olmzZiJnzpyiUqVK4u3btxrTvXr1SgAQbm5uSZqfUsfRo0dF1qxZRenSpUXjxo2FtbW16NixowgPD9eYLqawuHbtWim2YMECUbBgQVGxYkXRpEkTUahQIWFlZSVGjRol/Wii1HPr1i3p+9HV1VUAEE2bNhVt2rQRQ4YM0Zi2b9++AoB48uSJRnzmzJnCwsJCfP/99+L7778XFhYWYvbs2am5GUQkogsYbdq0EU2bNhUAhKurq/T+vnXrljTd8+fPBQDRs2dPjfkfP34sihcvLpydnUXz5s1Fjhw5RJUqVYSnp2dqbwp9pVarxdChQ4W5ubmoX7++qFq1qrCxsREbN27UmrZZs2YanWa+fPkiOnfuLPLmzSvq1Kkj6tatKxwcHET+/Pk1OslQyrh06ZLImzevKFKkiGjatKmwt7cXjRs3Fv7+/hrTnT17VgAQf/zxR5Lmp9QTEhIi2rZtK6ytrUWTJk1EqVKlRPbs2cWpU6e0pi1WrJjImTOnNOzu7i6aNm0qXFxcRMOGDUXNmjWFlZWVKF26tLh27VpqbgbF4u3tLX1PVq5cWQAQNWvWlGIRERHStCtXrhQAxM6dOzWWsXfvXuHo6CjKlSsnGjVqJCwtLUX37t1FZGRkam9OhqcQQohU7RJNBODt27e4ceOGznFZs2ZF3bp1peHr16/Dw8MDLVq00HgwBgB8+vQJd+7cgYmJCapWrcr7fMrsv//+w3///QcHBwdUq1ZN561HTpw4AaVSicaNG2vE379/j8ePH0v34OX9jVKeEAK3bt3Cx48f4ezsjAoVKmhNExISghMnTqBAgQJa9yczZH5KPWFhYbh27RqCgoJQsmRJFCpUSGuamM/eypUro0CBAlI8IiICjx8/xtu3b+Ho6IgKFSrwoRUy+fjxIy5fvqxznL29PX788Udp+O7du3jx4gUaNWqkdby8vb1x8+ZNKBQKVK5cWefDF4koZZ05cwYBAQE6x9WqVUu6V2dwcDBOnjyJggULolKlShrTqdVq3L59Gx8/fkS+fPlQvnz5lE6bDPD+/Xvcu3cP5ubmqFatms7vzCtXruDDhw/46aefNB5Q7uPjg4cPH8Lf3x/58+dHmTJloFTyTpSpITIyEtevX4evry+KFCmi9fwXIPqWCX///TdKly6tNd6Q+Sn1vXjxAk+ePIGdnR2qV6+uVTMAoj+PIyIitJ4Z9PHjRzx+/BihoaEoXLgwj6nMQkNDcfz4cb3jf/75Z+nz8uXLl7hz5w6qV68OZ2dnjelCQkJw7do1hIaGokyZMsifP3+K5p1ZsaBMRERERERERERERAbhqVAiIiIiIiIiIiIiMggLykRERERERERERERkEBaUiYiIiIiIiIiIiMggLCgTERERERERERERkUFYUCYiIiIiIiIiIiIig7CgTEREREREREREREQGYUGZiIiIiIiIiIiIiAzCgjIRERERERERERERGYQFZSIiIiIiIiIiIiIyCAvKRESZyK1bt6BQKHDo0KFMuX4iIiIiohhyt03lXj8RUVKxoExElMFcvnwZCoUCx44dy5TrJyIiIiKKIXfbVO71ExGlBFO5EyAiotRTuXJlCCEy7fqJiIiIiGLI3TaVe/1EREnFHspEREREREREREREZBAWlImI9MiVKxe6deuGa9euoXr16rCyskKhQoWwYMECvdPevHkTtWvXhpWVFcaOHQsACAoKwpgxY1C4cGFYWFggZ86c6N27N7y9vaX5PT09oVAopP8sLCxQokQJzJgxA1FRURrrCgoKwrhx41C0aFFYWlqiUKFCGDFiBPz8/LBt2zZ8//33AIAWLVpIy5s/fz4A7fu0vX79GkqlElOmTNHaps+fP8Pc3BxjxowxOMfErj/Gu3fv0L17d+TKlQsWFhYoXLgwJk6ciPDwcGmac+fOQaFQ4Ny5c1i5ciUKFiwIS0tLfPfdd7h161aCx/PXX3+FlZUV7ty5oxFv3749bGxs8PDhwwSXQURERJRZsW3MtjERkUQQEZFOOXPmFPXr1xfNmjUTjx49Et7e3mLx4sXCxMRETJo0Se+0Dx48EO/evRMHDhwQwcHBomLFiqJAgQLi5MmTIiAgQNy/f19UrlxZlC5dWoSEhOhct7e3t9ixY4ewtbUVkydPluJBQUGifPnyIm/evOLgwYPCx8dHvH79WixatEgsX75cCCHEpUuXBABx9OhRreXevHlTABAHDx6UYg0aNBD58+cXKpVKY9p58+YJAOLp06eJyjGx6/fy8hL58uUTxYsXF1evXhX+/v5i3759wsHBQTRq1Eio1WohhBBnz54VAESnTp3E9OnTxadPn8SLFy9EtWrVRJ48eURYWJjOPGMEBweL0qVLi4IFCwofHx8hhBCLFi0SAMSWLVvinZeIiIgos2PbmG1jIqIYLCgTEemRM2dOYWVlJT5//qwR79Wrl7CwsBBeXl4a01pYWIhPnz5pTDtnzhwBQNy8eVMj/uLFC2FiYiJWrlwZbw6jRo0SuXLlkoanT58uAIgrV67onSexjdZdu3YJAOL06dMa05YoUULUqlUr3vx05ZjY9Y8fP14oFApx7949jWlXrlypsZyYRnP79u01prt48aIAIPbv359grs+ePRN2dnaiefPm4vLly8LMzEz07ds3wfmIiIiIMju2jdk2JiKKwVteEBHFo3r16siePbtGrHXr1ggPD8eVK1e0ps2RI4dG7OjRoyhQoAAqV66sES9UqBAKFiyIixcvSrFdu3bh+++/h4ODA5RKpXQ53MePHxEcHAwAOHHiBAoUKIAaNWoYbRtbt26NrFmzYsOGDVLs6tWrePLkCXr27KkxrSE5Jtb//vc/FC5cGOXKldOIt23bVhofW7NmzTSGS5cuDQB4+fJlgusqWrQo1q9fj2PHjqFBgwYoW7YsFi9enKS8iYiIiDIbto3ZNiYiAngPZSKieOXMmVNv7MuXLxrxvHnzak378eNHvH79GqampjA1NYWJiYnU2HR3d5fuFbdt2zZ06tQJdevWxb///ovw8HAIIaT7t0VGRgKIvnebrvUkh4WFBbp06YJDhw7Bx8cHALB+/XrY2dmhXbt20nSG5phY3t7eyJUrl1Y8W7ZsMDU11drPuXPn1hi2t7cHAPj5+Rm0vqZNmyJ37twICwvD8OHDYWFhkaS8iYiIiDIbto3ZNiYiAlhQJiKK16dPn/TGsmbNqhE3MzPTmjZbtmwoXbo0oqKiEBUVBZVKBbVaDRF9yyGcOXMGALBlyxaUKFEC06ZNg4uLi7SsV69eaSwve/bsePfunVG2LbaePXsiPDwc27dvR3BwMPbs2YOOHTvCxsZGmsbQHBPLyclJ53729vZGVFQUsmXLphFXKBTJWl/fvn3h6+uLcuXKYcSIEXj//n2ylkdERESUWbBtzLYxERHAgjIRUbyuXbum1Qvg8OHDMDc3R61atRKcv0WLFnj8+LFBT0mO2xsgICAAhw8f1og1b94cr1+/xj///KN3OTEN3dhPgU5ImTJlULVqVaxfvx579uxBYGCg1iV9huaY2PXXr18f7u7uWvto//790nhjWblyJbZv346lS5fi+PHjEEKgY8eOWk8LJyIiIiJtbBsnPke2jYkoI2JBmYgoHt999x169OiBJ0+ewNfXF8uXL8fGjRsxevRord4BugwdOhQVK1ZEq1atcOjQIXh7e8PPzw/Xrl1D//79sX37dgBAy5Ytce/ePaxatQpBQUF4+PAhfvrpJ/zwww9ayytXrhzat2+PQ4cOwc/PD2/fvsXixYuxYsUKAEDhwoVhZWWF06dPIyQkxOBt7dmzJ+7fv4/JkyejVKlSqFatmsZ4Q3NM7PqHDx+OPHnyoEOHDrhx4wYCAgJw8OBBjB07Fg0aNNC6L1xS3bp1C8OHD0e3bt3Qq1cv5M2bFzt27MCVK1cwfvx4o6yDiIiIKCNj2/gbto2JKFOT62mARERpXc6cOYWbm5u4cuWKqFy5srCwsBAuLi5izpw5Qq1W65xWl5CQEDFlyhRRqlQpYWlpKbJmzSpq1qwpVq9eLUJDQ4UQQqhUKvHHH38IFxcXYWlpKSpWrChOnjwpZs2aJQAIX19faXkBAQFi9OjRolChQsLc3FwUKlRIjBgxQvj4+EjTbNmyRbi6ugpTU1MBQMybN08IoftJ0rGXa2NjIwCIhQsXao1PTI6JXb+Hh4f49ddfRY4cOYSZmZkoUKCAGDdunLR/hPj2JOuzZ89qzBsZGSkAiAkTJujc/0II4ePjIwoUKCDKli0rQkJCNMZNnTpVKBQKcejQIb3zExEREWV2bBtrYtuYiDIzhRBCpHYRm4goPciVKxcaN26MTZs2yZ0KEREREZGs2DYmIqIYvOUFERERERERERERERmEBWUiIiIiIiIiIiIiMggLykRERERERERERERkEN5DmYiIiIiIiIiIiIgMwh7KRERERERERERERGQQFpSJiIiIiIiIiIiIyCAsKBMRERERERERERGRQVhQJiIiIiIiIiIiIiKDsKBMRERERERERERERAZhQZmIiIiIiIiIiIiIDMKCMhEREREREREREREZhAVlIiIiIiIiIiIiIjIIC8pEREREREREREREZBAWlImIiIiIiIiIiIjIICwoExEREREREREREZFBWFAmIiIiIiIiIiIiIoOwoExEREREREREREREBmFBmYiIiIiIiIiIiIgMwoIyERERERERERERERmEBWUiIiIiIiIiIiIiMggLykRERERERERERERkEFO5E6DM59atW7C1tUXx4sXlTiVFRUVF4e3btwgJCUH+/Plhb28vd0oEICAgAG/fvoW1tTUKFCgApTLh82qfPn3Cp0+fkD17duTOnduo+Xh7e+PVq1fIkSMH8ufPb9Rlv379Gl++fJGG8+TJgzx58iR6OVFRUXj06BEiIyPh6uoKR0fHRM0fGhqKR48e6R2fM2dO5MuXTyPm7u4OPz+/eJfr5OSEQoUK6R0fERGBt2/fQqlUGnysEyO5+yWuxOablPUb+vr38vLCmzdvpGF7e3sULVo0UdujT2BgIN6+fQtTU1MUKFAAFhYWRllufCIjI/H69WsIIVCgQAGYm5sne5kx710AKF26NCwtLfVOm5zPkMSsh4jSJ7aNKS1IzPdNUtrTic2DbeN8esen1XZFRt4vbBsnLDGvrZT8HUOpSBClosjISAFAVKtWTe5UUszOnTtF48aNhZmZmQAgAAiFQiFq1aolrl69Knd6mdL79+/FhAkTRKFChaRjAkA4ODiIESNGiKCgIK15/ve//4muXbsKBwcHjXlcXV3Fxo0bjZbbxo0bBQDRt29foy0zhpubm0buEyZMMHje0NBQsX79etGmTRthb28vLePgwYOJzuPu3bsaecT9b+jQoVrzNGvWLN55AIhOnTrpXN+XL19Enz59hK2trTSto6OjGD9+vAgODk50/rEZc78kJd+krD8pr/+VK1dqTFu/fv0kb1+MJ0+eiLp16wqFQqHx2dikSRPx9OnTZC9fF5VKJWbOnCmcnJykdWbJkkVMnz5dqFSqZC27Xr160jIfPHigNd5YnyEJrYeI0je2jdk2TisS+r5JSnsiKdg21t02TqvtisyyX9g2TlhC+zC1fsdQ6mFBmVJVZmg0x3yZmZubi6JFi4qCBQsKU1NTAUBYWFiIK1euyJ1iphPTMAUgnJycROnSpUW2bNmkWJ06dbS+QKtVqyaNz5EjhyhZsqSwtraWYrNmzTJqbinZaC5VqpSoVKmSWLFihcHzPn/+XNpWU1NTYWdnl+zGYdasWUWlSpW0/luwYIHWPMOGDdM5baVKlaRctm3bpjWfp6enKFy4sJS7i4uLKFWqlDA3NzdKw9mY+yUp+SZl/Ul5/e/fv19UqlRJlC9f3iiNZrVaLVxdXaXPwZIlS4qyZctKeRjr/RTXb7/9Jm1ngQIFNBqjI0eOTPJyd+zYIQAIGxsbva8rY3yGGLIeIkrf2DZm2zgtMOT7JintiaRg21h32zittisyy35h2zh+huzD1PodQ6mHBWVKVZmh0Tx06FBx6dIlERUVJcU+fPggmjRpIgCIZs2ayZhd5nTmzBnx559/Ck9PT4348ePHpV6hZ8+e1Rg3YcIEsXXrVuHn5yfFgoKCxIABA6QfRb6+vsnOLTUazUlpLHp6eooePXqIvXv3Cj8/P/HLL78ku3Ho5uaW6HnjCgkJEY6OjsLe3l6EhIRoja9fv74AIIoVKyZu374txf39/cWsWbPEy5cvk7V+Y+6XpOSblPUn5fUfw9fX1yiNZnd3dwFAlCtXTgQGBmqMu3nzpjh+/Hiylq/LkydPhImJiTA1NRWHDh2S4sePHxfm5uZCqVSKhw8fJnq5/v7+Infu3OLnn3+WfsToeo8l9zPE0PUQUfrGtjHbxnIz9PsmOe2JxGDbWLe02q7IbPuFbWNthu7D1P4dQymPBWVKVYY0msPCwsTTp0/F06dPRUREhNZ4f39/cfPmTfHq1Su9y3j9+rW4efOm8Pf31zn+/fv34v79++Lt27d687x586b477//pNi7d+/EnTt3klxE/PfffwUAUbx48STNH19uHh4e4sGDB1pfRvoktP3BwcHi5s2b4s2bNzrHP3z4UNy5cyfBvPTtM19fX/HgwQPh7u6u94yip6enxjEMDQ0Vjx8/Fi9evDBoGw3Vo0cPAUCsWrXKoOlVKpV0Jvf8+fPJXn9abTTHlVYKylu3bhUARK9evbTGnT9/XjrL7+7unux1GSI5+8UY+Sa3oJ3Q699YjWZvb29hYmIiHB0dxfr168WXL1+StTxDjB07VgAQAwcO1Bo3YsQIAUCMHj060csdOnSosLGxEW/fvk3SDzJDP0OSux4iSh/YNmbbWAh528bG+L5JbHs6PmwbJ05aa1dk9P3CtrG2pL62Uvp3DKU83sWa0oxdu3ahTp06sLa2RvHixVG8eHE4ODhg4MCBCAkJkaZTq9WoU6cO6tevDyGEzmU1btwYtWvXRlRUlEZ8x44dKFasGPLkyYNy5cohf/78KFy4MHbv3q0x3ZcvX1ClShUMHjwYt27dQvny5ZE3b15UrFgRFy5cSNL2vXv3DgBQpEiRJM2vK7cHDx6gcuXKyJcvH8qUKQMnJyeMHDkSarVa57yGbv/jx49RpUoVzJw5U+dy2rRpg++//15vXvr22bNnz9CoUSNkzZoVZcqUgaurK3LkyIEZM2Zo5bx06VJUqVIFV65cwZw5c5ArVy6ULFkShQsXRuHChXHp0qUk7kFNiT0uSqUShQsXBgCdD0y4desWbt26BZVKZZT8MppPnz7h33//xcePH5M0/4YNGwAA3bp10xq3a9cuAEDHjh2lY5SWpYV8jfW5lBAnJyeMGjUKfn5+6NmzJ7Jly4YSJUpgxIgRePHiRYqs8+LFiwCA9u3ba42LiSX28/z+/ftYtmwZJk+eHO9DYeKT0GeIsdZDROkf28aGYds46W1jY33fxHcs2TaOX3Lbxhm1XZHR9wvbxsmXWr9jSD9TuRMgijFs2DB8+vQJtra2cHFxQWBgIDw8PLBixQp4enri8OHDAABHR0d06tQJ69evx7lz59CwYUON5fz99994+vQpevbsCScnJyk+Y8YMTJw4EUD0E2KzZ8+Oz58/4+XLl+jYsSNCQ0O1ilSenp5o0KABAgMDUbhwYTg6OiJLliwGbc/Dhw8RFhaGwMBA3LlzB3PmzIGNjQ0mTZqUjL30zbt371CvXj34+vrC1dUVwcHB+PDhAxYuXAhnZ2cMHz5cY/qkbH9S6NtnL1++RI0aNeDj4wOlUglXV1d4e3vD29sbEydOxKtXr7Bu3Tqt5c2dOxcXLlyAg4MDSpQogdevX+Ply5do1aoVnj9/jqxZsxqcm7+/P54/fw6VSoWPHz9i9+7dOH36NBo1aoR69eoZtIyAgABcv34dTk5OqFSpktb4KlWqAAB8fX0T/WTj2IKDg/HkyZNEzVOkSBE4ODgkeZ0pbc+ePdi8ebM0XLx4cfz+++/o3LmzQfO/evUKFy5cQJEiRVCzZk2t8Tdv3gQANG/eHADg4+ODd+/eIUeOHMiZM6cRtsC4UjtfY7z+k8PNzQ0bNmxAZGQkoqKi8PTpUzx9+hSrVq3CypUr4ebmJk1rjNf/06dPAQBly5bVmrZMmTIAon/IG0oIgQEDBqB48eIYNmxYonKLLaHPEGOth4jSP7aNE4dt48S1jZP6fZPY9gTbxvolt20MZMx2RWbZL2wbG07u3zGkh3ydoykziu+yvtGjR4t79+5pxJ4/fy4qVKggAIi7d+9K8Tt37ggAol27dlrL6dKliwCgcdnZw4cPhVKpFMWLFxc3b97UmP7UqVPC3t5eZMuWTYSHhwshou/rhq83eq9bt654//59orc19oO2AIjvv/9ePH78ONHLiSt2bo0aNRIfPnyQxq1atUrnpYOJ3f6bN2/Ge6lZsWLFhI2Njd68dO2zn376SQAQP/zwg8a47du3CwsLCwFAXLp0SYqPGTNGABAmJiZi3bp1Qq1WCyGiLw+qUaNGki5vOXr0qMYxMTMzE9OnT9d5+ag+v/76qwAgli5dqnN8zEMjDL3EUt9lfVevXtXI1ZD/jh49qrGMtHZZn0KhEPnz5xeFChWSHsYDQEybNs2g5fz+++8CgJg+fbrO8Tlz5pQ+K7p16yZMTEykdZQvX94otyiJKzn7xRj5Jmb9SXn9G+uyvrt37wpra2vRsmVLERwcLIQQ4vHjx9KlaiYmJuLRo0fS9MZ4/SuVSmFqaqo3p5iHhhj6II9169YJAOLixYtSLCmXjCb0GWKs9RBR+sC2MdvGcrWNk/p9k9j2BNvG2ozVNhYi7bUrMvp+Ydv4m+S+tlL6dwylPPZQpjRj7ty5ACD1vggNDYUQAm3btsXdu3dx6dIllC9fHgBQoUIFVKtWDYcOHcLnz5+RI0cOANFnvvft24fvvvsOFSpUkJa9efNmqNVqDB48GKamprh79y4AQAiBbNmyoUmTJti9ezfu3LmD6tWrS/NZWFhg27ZtyJ07d6K3p0yZMnB0dIS3tzc8PT1x6dIlDBw4ELt27ZLyTQ47Ozvs3LlTo1dI3759MX/+fDx79gwREREwNzcHkPTtTwpd+ywkJATHjh2DtbU1duzYoTGuc+fOePDgAWbPno3du3ejVq1aGsvr378/evbsKQ07OTlh/PjxaN68Of79999E5ebo6IhKlSohMjISnp6e8PHxwcyZM2FnZ4chQ4YkOP/06dOxZcsWtG3bFoMGDdI5za1btxKVkz62trY6z6bHJzG9Pnx8fPDy5UuNmLm5uc4z1sllZ2eHxYsXo1u3brC3twcA+Pn5YerUqfjzzz8xdepUtGvXDsWLF9e7DLVajc2bN0OpVOLXX3/VOU1YWBgAYMyYMThz5gxy5coFR0dHvHr1Cvfu3cOPP/6IM2fOoG7dukbfxqRI7XyT+/pPjoEDB0KpVGLt2rWwtrYGAJQoUQLr16+HWq3Gpk2bsGHDBsyfPx9A8l//UVFRUKvVei9xBAAzMzMAQEREBCwtLeNdto+PD8aOHYtff/0VtWvXTlResSX0GWKs9RBRxsC2ceKwbWx42zg53zeJbU+wbazNGG1jIOO1KzLTfmHbOHHk/B1D8ZCzmk2ZT3y9MC5duiRq1KghFAqFzjNsEydO1Jh+8+bNAoCYM2eOFPvzzz8FALF9+3aNaRs2bGjQWbyYs2MxPQrKlStnlO3+8OGD6Nixo9RzIrnLAiBq1Kihc3yDBg0EAOHt7S3FErv9yemFoWuf3b9/XwAQNWvW1Lm8K1euCACiXr16UiymF8aePXu0pn/48KEAILp27apzeYY6c+aMyJ49uwAgTp48Ge+0c+fOlY5fWFhYstYbm1wPHol5uF3s//Lmzat3Wcl9aII+rVq1EoD+XscxTp8+LQCIBg0a6J0md+7cAoCwtrYWp06dkuK+vr6ibdu2AoCoVKmS0XIXInn7xRj5Jmf9hrz+jdELw9PTUwAQtWrV0jn+4sWLCR7bpDAzMxMKhUJvLwtzc3NhYmJi0LJ69+4tHB0dxadPnzTiiemFYchniDHWQ0TpC9vGbBvrktJtY2N+3ySmPW0Ito0Naxun1XZFRt8vbBtHM8ZrK6V/x1DKYw9lShPu3buH+vXrSz0H8ufPDzs7OyiVSgQGBuK///5DZGSkxjwdOnTAyJEjsW7dOowePRoKhQJr165Fzpw50bZtW41pg4KCAETfLyjmzJsuMWdCY+TKlcso25crVy5s3boV//vf/3D69Gl8+vQp2fdItbOz0xk3MTEBAI0HeSR1+5NC1z4LDQ0FAI379sUWc6+3mOli07WdurYxKRo2bIj58+fDzc0NmzdvRuPGjXVON2vWLIwfPx7169fHwYMH4z2zaywpfZ+4rFmzap3lluM+w40aNcLhw4fx6tWreKdbv349AN0P44vh7OyMDx8+YODAgWjUqJEUd3R0xMaNG3Ho0CHcvn0bgYGBet8/qUnufA19/SeXp6cnAOi9x2bMZ0ZwcLAUM8brP2/evHj9+jU8PDzg4uKiMe3Hjx8RERFh0MND7ty5g3Xr1qF///54+/Yt3r59q5EnADx69AhhYWEoVqyYzmNlyGeIMdZDRBkH28aJx7axYW1jY3/fpFZ7gm3jbzJjuyIj7Re2jZMvtT53KH4sKFOasHbtWkRERGD06NH4448/ND78d+7cqfMG/BYWFujRo4f0YAoLCws8evQIEydOlC5nixFzGd3ChQtRv379lN0YPUxNTZErVy54eXnhw4cPqdpASez2m5pGfzTE/hKLIYSQvgQNlT17dgCAu7u7zvHPnz/XmC41OTs7AwA+fPigc/zkyZMxbdo01KtXD0ePHoWVlVWq5PXgwQN89913iZrn6NGj0gPeEtKkSRM0adIkKakZlbe3NwDEe1mVj48PDh8+DHt7e/z88896pytXrhxu3rypsyFka2uLLFmywMvLC0FBQWmi0ZwW8k3o9W8MMQ3ZmPd5XDHxbNmySTFjvP7Lli2L169f4/Lly1qN5suXLwP49gCS+Dx+/BhCCKxYsQIrVqzQOU3Hjh0BAGfPnkWDBg00xhn6GZLc9RBRxsK2ccrKzG3jlPi+SY32BNvG0TJruyIj7ZfM3jY2ltT43KH4saBMaYKXlxeA6N6HsRvMarUaGzdu1Dtfv379MH/+fKxZswYWFhYwMTFB3759taarV68eDh8+jBkzZqBOnTpSozA2Y/QCDA4Oho2Njc5xd+/exePHj2FiYqL1AZ7SErv9MWdFb9y4AbVaDaVSKU23c+fOeLdTl0KFCiFXrlx48uQJTpw4gaZNm0rj1Go1FixYAACoUaNGkrYvIUFBQbC1tdU5bvv27VKOcY0ePRrz58/HDz/8kKrFZCDl7xOXmvTtf19fX6nnccWKFfXOv337doSHh6NLly7xHoOWLVti3bp1OHHiBAYPHqwx7ubNm/Dy8oKdnZ0sJy50Sa18k/r6N5aiRYvCyckJT58+1fn+j7k3XOx7RBrj9d+8eXMcOXIES5YsQceOHaXeW0IILF68GADQokWLBJerq8dSjKdPnyI4OBilSpWCpaWlVk+2xHyGJGc9RJTxsG2csjJz2zip3zdytyfYNs747YrMsl8yc9s4seT+3KH4saBMaUKJEiUAAN27d8fo0aORO3duvH79GitXrsT169f1zlewYEE0btwYBw4cgImJCVq1aiWdqYqtR48eWLBgAf766y9UrFgRAwYMQNGiRaFWq/H69WucP38ef//9d6J7F8S1d+9ezJ8/H127dkXRokWRI0cOeHt748qVK1izZg1UKhXatWundXmLh4cHPn36hDx58iBPnjzJykGXxG5/rly5ULx4cTx9+hQdO3ZEz549oVAocP78efz55586G90JGTx4MCZMmIC2bdti7NixqFmzJnx8fLBy5UpcuHABDg4O6N69u7E3HUD0jwZXV1fUr18fBQoUgImJCd6+fYvt27fjzJkzAKDxcBMAGDlyJBYuXAgXFxdMnToVjx8/1lpuwYIFpUsSY8Q8eKRChQrSl3RSlC5d2mgPMUmqhw8fSg+O8/HxAQC8ePFCyit37tzImzevNH1ISAgeP34MS0tLlC5dWor/+OOPcHZ2xo8//oiCBQtCCIF///0Xf/75Jzw8PJAnTx506NBBbx4bNmwAEP/tLgCgWbNmKFu2LE6dOoWffvoJ3bp1Q5YsWXDnzh3MmjULANClSxet129ij5mx9ktS803s+pPy+jeEoftNqVRi0KBBmDZtmt73f5YsWdCjRw9pHmO8/rt06YKpU6fixo0baNWqFYYMGQKFQoHly5fj8uXLyJUrl9YDHnUdq/h6LFWvXh3Xr1/Hrl27NI4tkPjPkKSuh4gyJraN2TZOqbZxUr9vktqeYNvYOG3j1GxXyNU2Tuv7JSFsGye8D9PK7xgyElnu3EyZlr4Hj3h5eUkPqIr9n52dnZg+fboAIMaMGaNzmceOHZOmP3/+vN51P3z4UBQqVEjvQzcKFSokTRvzEI3EPiRk3759eh+cAkDUqVNH+Pr6as03dOhQAUBMnjw5wXUklFujRo0EAOHl5ZXk7RdCiEOHDgmlUqk13ZAhQ+J98Ii+vCIjI0Xr1q11rtvKykqcOHFCY/qYB4/ousn+kydPBADxyy+/JLi/hBCidu3aerfbwsJCrFy5UmseFxeXBB/UsnHjRq35YsbpOs66yPXgEUMULlw43u2P+56MeWBN4cKFNeJ169bVu4y8efOK27dv683h7t27AoBwdXU1KOdnz57p/CwBIL7//nsRGBioNU9ij5mx9ktS803s+pPy+o8R34NHErPfIiIiRIsWLXTmYGNjI86ePZvgMpLi8uXLws7OTmudtra24u+//9aaPr5jpUt8Dx5J6mdIYtdDROkb28ZsG+tad0q3jfWJ7/smqe0Jto2N0zZOzXaFXG3jtL5fhGDbOCEJ7cPU/B1DKY89lClVKRQKVKpUSep1ESNbtmy4f/8+5s+fj7t37yIqKgrlypXDoEGD4O3tjYMHD+rsXQFEn8m0sLBAoUKFULduXb3rLlWqFB4+fIjt27fj7NmzePfuHWxsbFCgQAHUr18frVq1kqY1MzNDpUqVULRo0URtX5s2bfDixQts374d//77Lzw9PWFlZYWiRYuiefPmaNq0KRQKhdZ8ERERAAx78EdCuRUtWhRfvnzResBIYrYfAFq1aoW//voLq1atwuvXr5EjRw506tQJHTp0QNu2baV7zxmal6mpKQ4cOIADBw5g//79ePPmDaysrFC5cmX069cPBQoU0Jje2dkZlSpV0vkgDSsrK1SqVMngy1vOnz+PU6dO4cSJE3B3d0dgYCCyZ8+OqlWr4pdfftFaNxB9j6nY963SRdf4mMt/DO2pki1bNlSqVCnVL/U0RJkyZeK9VDDue9LGxgaVKlXSuifw2bNncerUKRw/fhzu7u4ICQlBnjx5ULduXXTp0iXey2kvXbqESpUqaZyhj0/RokXx8OFDrFixApcuXUJgYCCcnZ3RokULdOrUSeu4xLz3zM3NDX7YorH2S1LyTcr6k/L6T0hi95uZmRkOHz6M/fv3Y//+/Xj79i2srKxQpUoV9O/fH/nz5090DoaoWbMmHjx4gKVLl+L27dsAonuNDB48GAULFtSaPr5jpUuJEiUQFRWl89LKpH6GJHY9RJS+sW3MtrEcbWN94vu+SWp7gm1j47SNU6tdIWfbOC3vl4SwbRwtoX2YFn7HkPEohBBC7iSIkmP37t3o2LEjFi9ejCFDhsidTpJUrlwZz549w+vXr7VuoUCUVN26dcPmzZuxa9cuFC5cOMUuG03PLl++jO+//x79+/fX+1CJzMjLywtv3rxBUFAQ6tati/r16+PcuXPSeO43IqK0i21jIt3YNk4Y23i6sW1MpE2Z8CREaY+vry9u3bqFPXv2YPDgwbC2tkaXLl3kTitJAgICcO/ePQwcOJANZkoRHTt2RJUqVdi40eHixYswMzPD2LFj5U4lTdm/fz+qVKmit2cb9xsRUdrCtjGR4dg21o9tPN3YNibSxlteULr0v//9D+3atZOGf//9dzg5OcmYUdI9e/YMlStXxsiRI+VOhTKYggULajyBN/YDDiiat7c3fvvttxS7rCy9ypEjh8Zrp1ixYhrjud+IiNIWto2JEsa2ccLYxtONbWMibbzlBaVL58+fx2+//QZHR0c0a9YMQ4YMSdZTg4mIiIiI0iu2jYmIiCg1saBMRERERERERERERAbhPZSJiIiIiIiIiIiIyCAsKBMRERERERERERGRQVhQJiIiIiIiIiIiIiKDsKBMRERERERERERERAYxlTuB9MDPzw8XL15Evnz5YGFhIXc6RERERJQM4eHh8PDwQJ06deDo6Ch3OukO28ZEREREGUdS2sYsKBvg4sWLaN26tdxpEBEREZERHTp0CK1atZI7jXSHbWMiIiKijCcxbWMWlA2QL18+ANE71tXVVeZs4qdSqRAUFARbW1uYmJjInU6Gxf2cOrifU4f/mbMICwtFtmbNuJ9TEF/PKY/7OHVkhP3s7u6O1q1bS208Spy00jbOCK/F9Ij7XR7c7/JgO1kefL3Lg/tdHmlhvyelbcyCsgFiLuVzdXVFqVKlZM4mfiqVCv7+/nBwcOAHQArifk4d3M+pw+fJU4SGhiBXqVLczymIr+eUx32cOjLSfubtGpImrbSNM9JrMT3hfpcH97s82E6WB1/v8uB+l0da2u+JaRvzoXxEREREREREREREZBAWlImIiIiIiIiIiIjIICwoExEREREREREREZFBWFAmIiIiIiIiIiIiIoOwoExEREREREREREREBmFBmYiIiIiIiIiIiIgMYip3AkRERERElDB3d3ccO3YMQUFB6N27N3LmzJmo+R88eICLFy8iKioKlStXRq1atVIoUyIiIiLKyNhDmYiIiIgoDfvrr79QsWJFNGnSBNu3b8fvv/+ODx8+JGoZI0eORJUqVXDr1i08f/4cLVq0QNu2bREZGZlCWRMRERFRRpVmC8qhoaHYt28funXrhrZt2+L169eJml+lUmHbtm3o1asXunXrhmXLliE0NDRlkiUiIiIiSiEODg5Yt24dnj9/jlatWiV6/h07dmDhwoXYsWMHNm3ahOXLl+PixYs4fPgwZs6cmQIZExEREVFGliYLynv27IGLiwt27doFHx8f7N+/H35+fgbPHxERgSZNmmD06NEoXbo0atasiRUrVqBq1arw9fVNucSJiIiIiIysYsWKqFixYpLnX7JkCYoUKYKff/5ZipUtWxZNmzbF8uXLoVKpjJEmEREREWUSafIeyt999x1evXoFGxsbTJ8+HUePHk3U/EuXLsXZs2dx8+ZNVK5cGQDQokULFClSBOPGjcOqVatSIm0iIiIiojQlNDQUN2/eRJcuXbTG1axZE0eOHMHjx49RpkwZGbIjIkobfH198e7dO7x//x6fPn1CcHAwgoODkePhQ6hUanh//gxra2tky5YN2bJlQ+7cuVGgQAFYWlrKnToRkSzSZEE5X758yZp/3bp1qFSpklRMBoBcuXKhdevW2LZtG/7888908cEfEaWGWgiNmEIR628otOJqlRoqtUCUSg0olLGm1V6GIvbCiIiIiCjDefv2LdRqNZydnbXGxcRevXqlt6D8+fNneHl5acTc3d0BRN9iLjV7N0d+befGUKlUCAuPhGlYBExMTFItj8yO+10e3O/GIYTAq1cvce3qVdy4cR1PHj/B06dP4P3li87pW9s7AAAObd2iNU6hUCBvXmcULVYU5ctXQPkKFVG1alU4J7OeQXy9y4X7XR769rupUgFTk9S5sURS2nNpsqCcHP7+/nj69Cn69OmjNa5y5crYtm0bHjx4gCpVqsiQneGCw6Pw4J0/IqKiD6pUPNZTA44JC7UKqvBQmPpEQaE0keLatWOFVjz2tHEL1wqFZgFbGXv814kVsZanVCi0xmutT9okPeOhPS0MmlahI6a5PbriunOMNW2saYRahZCgcIQhDMqvb3h9+SBR22lg7jr2o75lxY4neT8kkLu+/aRvX1LqUqlU8PT0xIsXL/Dq1St8/vwZX758ga+vLyIjI1Hq0ydERanw6sIF2Nvbw9HREXnz5kX+/PlRoEABFCpUCKamGe7rgogoUwgJCQEAWFhYaI2L6WARM40uK1aswNSpU3WOCwoKgr+/vxGyTFhYpAruXkFQqb4VlIVaBVVEOEzMA6BQ8odvaknN/S4gEp4okxBqNdQRYVCaB0ChTJN3rkyz/P18cf3SBVz7+zz+vXMTvt66i8eJJYSAp6cHPD09cP5//5Pi+QoUQqXqNVGpek2Ur1Id1ja2RllfZsLXuzy43+Whb79bm5miSE7bVKmlBAUFJXqeDFchePfuHQAgZ86cWuNiYu/evdNbUE4rvTDCIiIRGBIB/7AI2FmYaTSlNBpWcf6MfiFGQKEy1foAiD2f0NE2E0L3tLracRr56FpYrOXFHpvct4G+JqWu5SameBrf0hTaNVxAqIHIEMAsUtrPSV9f/IVkBXRXjHUXrfWMjxlW6IhBf+66dmxiitaxxxh68iL2coVaBVVYCEz9IqFUmug8eQF8O4GR0MkLzXXo2SYduceePqHt15w24deBrisGEjqhoPc19TX+8f0HXP77Au7duY1/79/Fowf/xvtQUqnnxV/ndY63tLREsRIlUaZMWVSvUQM1atZCgQIFvh0PI+6H2NMn+cRLGj15EfMdwnuVphzu49SREfZzes49saysrAAA4eHhWuPCwsIAANbW1nrnHzBgANq1a6cRc3d3R+vWrWFrawsHBwcjZqufCIlAOKLgFxndNgYAATWEiRpRSqsM8cM3bX57aYvZ76oMst/Ti5j9ruB+N0iAnw/+OnEIV/53Ag/v3IA6EZ/7JqamsLK2haWVFRytrKBWq5DVwhLhYWEICkz4JJrH65fweP0Sh3ZthbmFJarVaYAfmrRGlVp1YW6R9q+UTgv4epcH97s8dO13n5AI5LeyhJ29A0yUKd9CsLVN/ImvDFdQjoiIAACdvenMzKIbnzGNZ13SSi+M4LBIqMOD4WiqgE0ijpJQqxGlVsHUPCpTfwDoKnLrqXtL0+orVuuKC5UKKlUUTEwigFg9M4SOmRIqzsfOTV8vDKFjETq3EXGL/bqWlXBPDxHn37gDCa0DST15EZdaQESFAabhgFKRqJMXQCJ+mOk56yF0xBV6pk1U4VeDjqJ87LEJFI8VUEClUuHJv3dw9cJZ3Ll2CR6vXhiyYoOFhYXh/t07uH/3DrZt2QQAyJErD6rXrofqdeqjbKWqMDf/1vNNofWHAScvEhC3SKz/BAa0DnzMoFLnMdLc/4o4C1bEWZxSz4p1nnCI8zpQCzUiQkNg4RcKhUIZZxHxn7yAvqK9rlgCr5noaXQU5fVMrDufmH907wd9Md3r03PSLBEncWKo1WqEBocgIkqt9zI9Xfs94atjdK8vrZ68SGlqtRrBwcFQKBRQptO2RlJ6YaRX+fPnh1KpxPv377XGxXTEcHFx0Tt/jhw5kCNHDp3jTExMUu2SWBMTEyiUSmS3s4K9VXSbXq1WITJUBTMrSyjZQznVcL/Lg/s9YWq1GtcvX8Dh3dtw4cwJRH6tDehTuGhxlChbAQVdi6Kga1Hkcc6PbDlzwTGLk/Qdrzx1FKrIcAxu9hOUShNERkbC39cHnz++h8frl3j76iWeP3mIxw/u4YOnh9Y6IsLDcOnMMVw6cww2dnZo3LIN2rv1gmuxkimyDzIKvt7lwf0uD137XQUFFMrodlZqFJST0p7LcAXlmKq6rkv3goODAQB2dnZ6508zvTDMIqD0UcPEBDD72mg2aD519JlXMytrXvqXgoRahchQE+7nFBa9n0PS9n7WcUIioZMXcafXNZ+uExGaVyoAL549wdE92/DXqaPw8fpsULpm5hbImj0HHLI4wc7BEWZmZigSGgohVKhkYoaQ4GAE+Pni88f3iIzQ7s0W4/PH9ziyZxuO7NkGW3sH1G7UAg1btEXJClU0qnA6T9joOkESz5UX35alnYfGPk3MCRt90+pYrq5lxT0VovsrPs4yhBqIVAPhakARPVahe8p4l2uMqy90Fon1LCMxPc0150ugJ7mOQrKeGrvewnfckxdCrYY6PAom4VFQKNUJ3jJIz+oMuPpC90kj6eSFMnYs9rTfZtB166jYMV1XXsReZ2JuS6T5tyGvA/3HTgEF1CoVQlWmUEeawsRU+7NZ1xUDhrwOLMyUcLQ211peSkhKL4z0ytraGhUrVsTVq1e1xv3zzz/ImjUrSpUqJUNmRETGERYWiqN7d2LrmmV49/a13umKlCiFWvV+RIUq1VGmYhXYOzgmel1mZmbIliMnsuXIiZJlK2iM8/nihZv//I3rly/i+uUL+PjOU2N8cGAg9m/fhP3bN6FS9Vro2K03ajdswlvLEVG6lOE+uVxcXGBmZoY3b95ojYuJFSlSRO/8aa0XhkKpSNSZITXwdT4TnlFKQdzPqYP7WVtoaAjOHj2IAzs248HdW/FOW9C1KEqWq4iSZcrDtXhJ5CtQENlz5tbqURjT86LD154XQHRB1df7C968egH3p4/x/MlD/HvnJtyfPtYqtgYF+OPE3m04sXcbXAoXQcdufdC8TQfeLy6O6DPPCphZ2STp9azvhESChXa9y4sZH//JC41hPSc9knIlgq4rL6Ljeor5X/9V6z1xICCEAmoBqIQCCqGI/hCJM79WLrpO2OjNQWjlrplDItaRwPr07R/pJIQQCfeQ1jGjrisvAOi8+kJv0Vq67ZPQqJ7HV7SOszqdJyWy2VmgiosllGm0F0Z6cfHiRVy6dAlubm7Sg64HDhyI7t2749ixY2jevDkA4PHjxzh+/DhGjx7NYgYRpUsB/n7Ys3kddm1ao/e+yOWrVEf9pi1Rp0ET5M2v/2oMY3DKlh2NWrZBo5ZtIISA+7MnOHPkAE4fPaBV6L597TJuX7sMl0Ku6DV4FH5s+TM/i4koXclwn1hmZmaoXbs2Ll26BLVarVE4OX/+PFxcXFC0aFEZMyQiSrxAf3/s3rwWOzasgr+vj85p7B0cUbthE3xXux6q1PgeWbPrPjlmCIVCAads2eGULTsqVKkuxf39fHH3xlVcPn8Gf587DW+vTxrzvXnxHHN+H43l8/7ATx1/RcfufZArj3OS86Bv9N7uQWftLbPeikGFSJNImFmZ8yRUMiR08kLEuiwPCqXOaaFrvniuRPjoHwaVio/f0ufdu3fYuHEjgOiCMQCsW7cOuXLlgrW1NUaMGCFNe/bsWcyYMQM//PCDVFB2c3PDtWvX0L59e7i5ucHa2hpbtmxB/fr1MWnSpNTfICKiZAgLC8WujWuwccUiBAUEaI3Pmj0nmrftiFbtf4FLIVcZMoxutxUpXhJFipfEgNET8PDebRzevQ0nDu5FeNi3Z5u8eemO34f3w7ql89Fr8Cg0atUmQ5/0JKKMI90XlHft2oV9+/Zh+vTpKF68OABgzJgx+PHHH7F48WIMHz4cAHDy5ElcvHgRK1askDNdIqJE8fXxxvZ1K7BnyzoEBwZqjbewtMKPLVqjcau2qFS9lnSv+JTi4JgFP/zYFD/82BRqtRoP7tzEiYN7ceboAQT4+0nTBQUEYOuaZdi1aQ1+7twNPQYOR7Yc2g9LJaK0J6GTF2oooFTG/GeceyhbmvHHc3yEENIzQKpVq4Zq1aoBiL7HfdzCww8//AAg+t7JMRQKBVatWoUePXrg4sWLiIqKwo4dO9CgQYNMey9wIkp/oqKicGzfTqxeNBufP37QGl+8dDm49RuMuo1bpHibODEUCgXKVKiMMhUqY/DYyTi8exv2blmP955vpWliCstb1y7DqMmzUKl6TRkzJiJKWJosKD9//hzjxo0DADx9+hQAMHr0aDg4OMDe3h4bNmyQpn348CH279+PYcOGSbGGDRtixYoVGDlyJPbs2QNra2tcvnwZv/32G/r165eq20JElBThYWHYuXE1NixfqLOQXKREKfzcuRuatG4HO3t7GTIElEolylWuhnKVq2HkpBn438mj2LlhFR7dvyNNExkRgd2b1uDQrq3o2K033PoPhYNjFlnyJSJKr5ydnTF9+nSDpm3QoAEaNGigc1zVqlVRtWpVY6ZGRJQq7t+6jlkTR+H5k0da46rVqgO3/kNRtWadNH+SzMExC37tOxide/bHqcP7sG7JfHi8fimN/+/xQ/Tp0AL1m7TA0PHTUvw2HURESZUmC8pZs2ZFx44ddY6zsLDQGO7UqRPKly+PEiVKaMT79++Pzp0748aNG4iKisL27duRK1euFMuZiMgY1Go1Th85gGVzp2k9yAMAqtSojR6DRqBKje/TVIPZ3MICTVq3ReNWbfDgzk1sX78S/ztxRLp0PjwsFJtXLcGh3dsw6LeJaNWhKy/nIyIiIqJ4+fn6YOnsqTi0a6vWuDIVKmPIuCmoWK2GDJklj6mpKZq36YjGrdri1OF9WLt4HjzfvJLG/+/kUVw6fwa9h4xG176D01SPayIiII0WlJ2cnNC2bVuDpi1VqpTeJ1M7ODigYcOGxkyNiCjFvHL/DzPGDcfdG1e1xtWq9yN6DR6JMhWryJCZ4RQKBcpWqoqylari5X9PsWrRbPzvxBFpvL+vD2aMG4GDO7fit2lzUKZCZRmzJSIiIqK0SAiBU4f3Y96UsVrPD3Ep5IpBv/2Ouo2bp6kOFkkRU1hu1OJn7Nq0BmuXzJOuTowID8fyedNx9vghTJq7BCXKlJc3WSKiWIxz4zsiIkqyyIgIrFk8F52a1NYqJpcoUx5rdh/F4o270nwxOa5CRYtj7spN2HbsL9SoU19j3ON/76Jb6x8xa8IohAQHyZQhEREREaU1vt5fMKZ/N0wc2kejmGxhaYVBYyZh9+nLqNekRbovJsdmZm6Orn0G4dCFW/ip068a2/bf44f4tWUDLJk1BRHh4TJmSUT0DQvKREQyenT/Djo3rYPVC2cjMiJCiufMkxd//LkaW46cS/cP5ShRphyWbtmLRRt2wtmloMa4fds2oGOj73Hr6mWZsiMiIiKitOLi2ZNo/2NN/O/kUY147QaNse/cVXQfMAxm5uYyZZfynLJlx8TZf2LjwdMoXLS4FFer1di8agncWjfEy/+eypghEVE0FpSJiGSgUqmwYdlCdP+5MV4+fybFlUolfunZH/vOXUXTn9pBqcw4H9O16zfCnjNXMHD0RFhYWknxdx5v0LdjS8ydPBahoSEyZkhEREREcggPC8OsCaMwotcv8PniJcUdnbJi7spNWLR+B/Lkyy9jhqmrTIXK2H78AvqOGAvTWPdP/u/xQ3RpUR/7t2+SnlVCRCSHjFOpICJKJz54eqBvx5ZYPm86VFFRUrxoydLYdOgMRkyaAWsbWxkzTDkWlpboMWgEdp+5rPUAld2b1sCtZQP2uiAiIiLKRDxev0T3nxtj37YNGvE6PzbFnjNXUL9pS5kyk5eZuTn6DP0NO05cQLFSZaV4eFgoZo4fgVF9f0Wgv7+MGRJRZsaCMhFRKvrr9HF0bPK9xr2STUxM0G/EOGw58j+UKldRxuxSTz6Xgli96whGTZml0Vv5xX9P0bVlAxzZu0PG7IiIiIgoNZw7fhi/NPsBzx79K8VsbG0xdeEKLFizFVmz55Axu7ShcNES2HTwNLr0HqgRv3D6OLq2rA/3Z49lyoyIMjMWlImIUoFKpcLSOdMwqk9XBAUESHFnl4JYv/8keg8dDbNYl7NlBkqlEp2698WuU3+jVPlvhfSw0BBMHTUIk0YM4C0wiIiIiDIglUqFxTMnY8yA7ggO+vaA5mIly2D78Qto3qZjhnroXnKZW1hg+MQ/sGzrPmTNnlOKe7x+CbdWP+L0kf0yZkdEmRELykREKczX+wsG/9oWm1b8qRFv1f4X7Dx5EWUqVJYnsTQif8HCWL/3hFavi+P7d6Fnm6b48M5TpsyIiIiIyNgCAwIwvGdnbFm9VCPetksPbDx4GvkKFJIps7Tvu9r1sOvU36j83fdSLCw0BOMH98bCPyZCpVLJmB0RZSYsKBMRpaDoB2fUw/XLF6WYhaUVpi1aiUnzlmbYeyUnlpm5OYZP/AOL1u+Ag2MWKf7s0b/o2qIe7t68JmN2RERERGQMb1+9QLfWDXHlr7NSzNLKGjOWrMW4GfNhYWkpY3bpg1O27Fi+bT9+7TtYI7593QqM7N0FwUGBMmVGRJkJC8pERCnk0v/OoGfbpvgYq4dt3vwFsOngaTT7uYOMmaVdtRs0xvYTF1GiTHkp5uv9Bf06tcKBnZvlS4yIiIiIkuXujatwa9UQr188l2K58jpj44FTaNyqjYyZpT+mpqYYOn4qZq/YACtrGyl+6X+n0atdM3x8zyv8iChlsaBMRJQCdm1cgxG9OiMk+Ns94WrV+xHbjp1H0ZKlZcws7cud1xnr9h1H41ZtpVhUZCRmjB2OJbOnQq1Wy5gdERERESXW/04cwYAuPyPA30+KVaj6HbYe+R/bxsnQsFlrbDp0GrnyOkux/x4/hFurH/HkwT35EiOiDI8FZSIiI1KpVJg7aQzmTRmrUfjs2mcQFq3fAXsHR/mSS0csLa0wffFqDBk3ReOBLJtXLsbvw/ohIjxcxuyIiIiIyFC7N63FmAHdNdpvP3X6FSu3H4RTtuwyZpYxuBYric2Hzmo85PrL54/o06ElbsS67R4RkTGxoExEZCQR4eEYP7gXdm9eK8VMTEwwYdZCDJswDUolP3ITQ6FQwK3fECxav1PjUr5Th/dhsFt7BPr7y5gdEREREcVHCIGlc6Zh7uQxEEJI8YGjJ2LCrEUwMzeXMbuMJVuOnFiz+ygaNGslxUKCgzCkewecP3lUxsyIKKNidYOIyAhCgoMwrEdHnDt+WIrZ2ttjyeY9+LlzN/kSywC+r/8j1uw+otGD5dbVS+jVvhm+fP4kY2ZEREREpEtUVBQmjxyITSv+lGImJiaYPH8ZegwaoXEFGhmHpaUVZi1br/GwvsiICIwZ0B0Hd26RMTMiyohYUCYiSiZfH2/069Qa12NdUpYtRy6s23sC1b+vK2NmGUfJshWw8eBpuBRylWLuTx+jd/vm+PCODx0hIiIiSisiIyMxfnAvHN+/S4pZWdvgzw070bJdZxkzy/iUSiWGjp+KIeOmSDG1Wo3pY4dh08rFGj3FiYiSgwVlIqJk+PL5E/q0b45H9+9IsXwFCmHjgVMoUrykjJllPM75C2DDgVMoW7GKFHv76gV6tWsKj9cvZcyMiIiIiIDoW8CN6d8N/ztxRIplyZoNa3YfQY0fGsiYWebi1m8IJs7+U+OWe0tnT8WyuX+wqExERsGCMhFREnl9+oi+HVvi5fNnUqxYqbLYsP8k8uTLL2NmGZdjFies2H4AVWvWkWIf33miZ9tmcH/2WMbMiIiIiDK30NAQjOj9Cy6ePSnFcuTKjXV7j6Nk2QoyZpY5/dTpV8xZsVHjXtWbVvyJpXOmsahMRMnGgjIRURJ4ffqAvh1b4vWL51KsYrUaWvf6JeOLuWSyTsMmUszb6xP6dmzFojIRERGRDEKCgzCseydcvXheiuV2zoe1e4+jQOEiMmaWudVr0gJLNu7WeMD15pWLsXT2VBaViShZWFAmIkqkzx/fo0+Hlnjz0l2KVapeC0s27Yatnb2MmWUeFpaWmLNyExq1bCPF/Hy80b/zT3gVq8c4EREREaWs0NAQDO3eEbeuXpJizi4FsXbPcTjnLyBfYgQAqFqrDpZu3qNZVF61BEtmTWFRmYiSjAVlIqJE+PThHfp0aIm3r15IsSo1amPxpl0ajTRKeWZmZvjjz1Vo0a6TFPP54oW+nVprFPuJiIiIKGWEh4VhZO8uuHP9HylWoHARrNt7DLnzOsuYGcVWoep3WkXlLauXsqhMREnGgjIRkYG8vT6jf+efNB4AV61WHSzasANWVtYyZpZ5mZiY4Pc5S9D0p/ZSLOb2F3xQHxEREVHKiYyIwJgB3XH90gUpVqhIMazZfRTZc+aWLS/STV9RecW8GTJmRUTpFQvKREQGCPD3w6Bf22r0fK1euy4WrmcxWW4mJiaYsmA5GrX8WYp5ffqAvp1a4d3bNzJmRkRERJQxRUVFYeKwvrj0v9NSLH/Bwlix/SCyZs8hY2YUnwpVv8PSLXthbWMrxTYsX4gtq5bImBURpUcsKBMRJSA0JBhDu3XAf48fSrGqNetg4drtsLS0kjEzimFiYoJpi1ahQbNWUuzT+3cY0OVnfPn8ScbMiIiIiDIWtVqNaaMH49zxw1Ist3M+rNxxENlz5pIxMzJEhSrVsWTzHljG6hSzeNYUHNi5WcasiCi9YUGZiCgeEeHhGNmnK/69c1OKla1YBQvWboWFpaWMmVFcpqammL54Deo2ai7FPN+8whC39ggMCJAxMyIiIqKMQQiBeZPH4viB3VIse87cWLXzMHLl4T2T04sKVapj/potMDUzk2Izx43AmaMHZMyKiNITFpSJiPSIiorC+MG9Ne4LV6REKSzetFvjMjFKO8zMzDBr2Tp8V6eeFHv2+AFG9v4F4WFhMmZGRERElP6tX7oAe7ask4adsmXHyh0H4Zy/gHxJUZJ8V7seZi5ZC6UyuiwkhMDEYf1w+fxZmTMjovSABWUiIh2EEJg1YST+On1MiuUvWBjLt+6HvYOjfIlRgszMzTFv1WaUrlBJit2+dgXjBvdCVFSUjJkRERERpV8Hdm7GygUzpWE7ewes2HYABV2LypgVJUf9pi3x+5zF0rAqKgq/9XPD/VvXZcyKiNIDFpSJiHRYt2Q+Du3aKg3nzJOXDxlJR6ysbbB4426NHzgXz5zAzHHDIYSQMTMiIiKi9OfCmROYNX6kNGxuYYFF63egSIlSMmZFxtCy/S8YOfnbiYLw8DAM79kZr188lzErIkrrWFAmIorjyN4dWLVwljTs4JgFy7fuR+68vC9ceuKYxQnLtu5Dzjx5pdjhPds1etYQERERUfzu3byG8YN6Qa1WAwCUSiVmLl2HClW/kzkzMpbOPfqh99DR0rC/ny+GuLWHt9dnGbMiorSMBWUiolj+ufg/TB87TBq2sLDEovU7eClfOpUrjzOWb90PhyxOUmz90gU4sneHjFkRERERpQ8v/3uKYT06ITz827Moxs1cgLqNmsmYFaWEvsPHolWHLtLwO483GNq9I0KCg2TMiojSKhaUiYi+evrwX4zp3x2qr/fZVSgUmL54NcpVriZzZpQcBV2LYsmm3bCwtJJi08cOw40rf8uYFREREVHa5vPFC8N6dEJggL8U6ztiLH7u5CZjVpRSFAoFxs1YgJo/NJBiTx7cw9iBPfkcEiLSwoIyERGAj+89MaRbB40z8CMnz0S9Ji1kzIqMpXT5Spi+eDUUCgWA6AeOjO73K17+91TmzIiIiIjSnvCwMIzs3QXvPN5IsTZduqP3kNHxzEXpnZmZGWav2IDipctJsSt/ncXsiaP4HBIi0sCCMhFleqEhwRjR8xd4e32SYl16D0Sn7n1lzIqMrV7j5hg2YZo0HBQQgKHdO/LecERERESxCCEwdfQg/HvnphSr+UMD/DZ1jnRynjIuaxtbLN64C3mc80uxgzu3YNva5TJmRURpDQvKRJSpqdVq/D6sP549fiDFGjRrhaHjp8qYFaWUX3oNQLuuPaXh955vMbxnZ4SFhcqYFREREVHasXrRbJw+ckAadi1eEjOXrYOpqamMWVFqypYjJ5Zu2QsHxyxSbPHMyfj7f6dlzIqI0hIWlIkoU1u1YBb+On1MGi5VriKmLlwBpZIfjxmRQqHAqCmzULNuQyn26P4dTBs9hJfxERERUaZ34sAerF08TxrOmj0H/tywC7Z29jJmRXIoULgI5q3eApOvJxKEEJgwuDfcnz2WOTMiSgtYMSGiTOvkoX1Yv2yBNJw9Z24sWLsVlrEe3kYZj6mpKWYtW4diJctIsdNH9mPzqiUyZkVEREQkr7s3rmLamCHSsIWlFRat34HceZ1lzIrkVKl6TYybPl8aDgkOwvAeneHr/UXGrIgoLWBBmYgypYf3bmPab4OlYQtLKyxctw3Zc+aWMStKLTa2dliwbjucsmWXYsvmTMPl82dlzIqIiIhIHu/evsHIPl0RGREhxf5YtBKlylWUMStKC37q9Ct+6dlfGn7v+Raj+v6KiPBwGbMiIrmxoExEmc6nD+8wolcXjUbQlAXLULJsBRmzotSWO68z5q7aDFMzMwDRl/GNH9ILr9z/kzkzIiIiotQTGhKMkX26wN/XR4oNHjsZ9Zu2lDErSkuGTpimccu4ezevYeaEEbxlHFEmxoIyEWUqYWGhGNm7K7y9PkmxPsN+w4/Nf5IxK5JLhSrVMeaPudJwcGAgRvbugkB/fxmzIiIiIkodQghMHT0Yz588kmIt2nWCW78h8cxFmY2JiQlmLl2LQkWKSbGje3di6+qlMmZFRHJiQZmIMg0hBGZNGIUnD+5JsQbNWqH30N/kS4pk93MnN7T/tZc0/OalO8YP7gWVSiVjVkREREQpb/OqJTh77JA0XKp8RYybvgAKhUK+pChNsrWzx6INO+GQxUmKLZ0zDdcu/SVjVkQkFxaUiSjT2LdtI47t2ykNFytVFlMWLIdSyY/CzG7EpBmoVL2WNPzPxf9h6eypMmZERERElLL+uXAOy+ZMk4azZs+B+au3wMLSUsasKC1zzl8A81dvgYmpKQBArVZj/KBeeO/xVubMiCi1sYpCRJnCv7dvYP7UcdKwg2MWzF+9BVZW1jJmRWmFmZkZ5qzciDzO+aXY1jXLcO74YRmzIiIiIkoZb1+9wPjBvaV74JqamWHuyk3IkSuPzJlRWlexWg2MmjxLGvb388Wovr8iLCxUxqyIKLWxoExEGZ6312f81r87oiIjAQAKhQIzlq5Fnnz5E5iTMpMsTlmxcN12WMY6yTB19GC8fvFcxqyIiIiIjCs46OszIwK+PTNizLQ5KF+luoxZUXrSrmsPNG/bSRp+9uhfzBzHh/QRZSYsKBNRhhYVFYVxg3rC69MHKTZg1AR8V7uejFlRWlWkRCn8PmexNBwSHITRfX9FSHCQjFkRERERGYcQApNHDMTL58+kWJtfuuHnzt3kS4rSHYVCgXEz5qN46XJS7PiB3di9aa2MWRFRamJBmYgytCWzpuD2tSvScJ0fm6LbgGHyJURpXuNWbdChWx9p+OXzZ5g+djh7XBAREVG6t23tcvx1+pg0XK5yNYyeMlvGjCi9srS0wvzVW+DolFWKLZw+EXdvXJUxKyJKLSwoE1GGdebYQWxft0IadinkiqkLVvAhfJSg4ROmoUyFytLw6SP7sXfLehkzIiIiIkqeuzevaTx0OFuOXJi7chPMzM1lzIrSs9zO+TBr2Xrp95UqKgpjBnTH54/vZc6MiFIaqypElCG9fvEcf/w2VBq2srbBvNWbYWdvL2NWlF6YmZtjzsqNGj0uFvwxAQ/u3JQxKyIiIqKk8fb6jLEDekClUgEATExMMHvFBmTLkVPmzCi9q1qzNgaPnSwNe3t9xrhBvRD59fk1RJQxsaBMRBlOWFgoxg7soXHf20lzF6Nw0RIyZkXpTc7ceTFr6Tqpx0VUZCTGDOgOX+8vMmdGREREZDiVSoUJQ/rgy+ePUmzw2MmowIfwkZF07TMIP7b4SRq+d/MaVs6fIWNGRJTSWFAmogxnwdTxeP7kkTTcoVsf/NjiZxkzovSqaq066D9yvDT86cN7TBjaB2q1WsasiIiIiAy3etFs3Pznb2n4h0bN0KX3QBkzooxGoVDg9zmL4VK4iBTbvGoJ/v7faRmzIqKUxIIyEWUoJw/tw4Edm6XhEmXKY9j4qfHMQRS/bgOGoXaDxtLw9UsXsGnlYvkSIiIiIjLQlb/OYv3SBdJw3vwFMGXeMigUChmzoozI2sYWc1ZshIWllRSbMmIAPrzzlDErIkopLCgTUYbx+sVzzBw/Qhq2sbPD7BUbYG5hIWNWlN4plUpMXbgCuZ3zSbFVC2bi3s1rMmZFREREFL8Pnh74fVg/adjcwgJzV26CnYODjFlRRlakeEmM/WOuNOzv54txA3sgMiJCxqyIKCWwoExEGYKu+yZPnrsUzvkLyJcUZRj2Do6YtXQdTExNAUTfi3D8kN7w9/OVOTMiIiIibRHh4RgzsLtGW+W3qXNQvHRZGbOizKBl+1/QvG0nafjB3VtYOptXjBJlNCwoE1GGMH/KOK37Jtdv2lLGjCijKVOxCgaOnigNf3r/DtN+GwIhhIxZEREREWlbPGsyHt27Iw03b9sJrTt2lTEjykzG/jEXhYoUk4a3r1+Jv04flzEjIjI2FpSJKN07dXg/Du7cIg2XLFuB902mFNG1zyB8V6eeNHzh9HHs3rRWxoyIiIiINF08exK7Nq6Rhl2Ll8TY6fN432RKNVbWNpi7chMsrayl2JRRA/Hu7RsZsyIiY2JBmYjSNc+3rzFz/HBp2NbeHrOWr+d9kylFKJVKTFu4Etly5JJif86chCcP7suYFREREVG0Tx/eYeqoQdKwlbUN5qzYCKtYhT2i1FCwSDFMmLlQGg4KCMDYgT0QER4uY1ZEZCwsKBNRuhUZGYmJQ/ogOOjbfZMnzVnC+yZTinLKlh1//LlK6uUTGRGBcYN6IjgoUObMiIiIKDNTqVSYOLSvxn2Tx/wxFwUKF5ExK8rMmv7cXuNWK4//vYtlc/+QMSMiMhYWlIko3Vq7eC4e3L0lDf/c2Y33TaZUUbVmbfQcPFIa9nj9ErMmjOL9lImIiEg265fOx53r/0jDTVq3Q/M2HWXMiAgYPXU2ipQoJQ1vX7cC/1w4J2NGRGQMLCgTUbp0+9oVbFj27RKqAoWLYMTv02XMiDKb3kN/Q4Wq30nDJw/txdF9O2XMiIiIiDKrO9f/wdrF86RhZ5eCGDdjPu+bTLKztLTCrKXrYGFpJcUmjxyIL58/yZgVESUXC8pElO74+/ni92F9pd6gZubmmLl0HaysbWTOjDITU1NTzFiyBg5ZnKTYvMlj4fHmlYxZERERUWbj5+uDCUN6Q61WAwBMzcwwa9k62NjayZwZUbSCRYph1JSZ0rDPFy9MHjlAes0SUfrDgjIRpStCCEwfMxSfPryXYkPGTkaxUmVkzIoyq5y582LK/GXScEhwEH4f2hdRUVEyZkVERESZhRAC00YPxuePH6TY4DGTULJsBRmzItL2U8dfNW5PeO3vv7B93QoZMyKi5EizBeUnT56gefPmsLW1hZWVFWrVqoXLly8bPP/GjRtRpUoV2Nvbw8bGBqVLl8b8+fMRGRmZglkTUUo7tGsrzp86Jg3XqFMfHbv3lTEjyuxqN2iMNl26S8MP7t7C+qXzZcyIiDKyL1++4OPHj0m+Z7uPjw8+f/5s5KyISC57Nq/DxbMnpeGaPzRA5579ZcyISDeFQoGJs/9ErrzOUmzZ3D/w5ME9+ZIioiRLkwXlt2/f4vvvv4dSqYS7uzs+ffqEihUrokGDBrhx40aC88+cORM9evRAkyZN8PbtW3z+/BnDhg3DmDFj0K9fv1TYAiJKCa/c/8P8qeOlYads2TFlwXIolWnyo4wykeET/4BLrCeor1syH//eTvj7iojIUBcvXkTp0qVRuHBhlC5dGi4uLti9e7dB80ZGRmLy5MnInj07ChUqhFKlSsHW1haDBg2Cv79/CmdORCnlv8cPsWjG79Jwthy52DamNM3ewRHTF6+RXqNRkZEYP7g3QoKDZM6MiBIrTX7TTJs2DaGhodi8eTNy5coFe3t7LFq0CPny5cPo0aMTnH/r1q0oUKAApk2bBkdHR9jY2KBXr15o2LAhtm3bluQeHUQkn4jwcEwY3BthoSFSbOqC5ciaPYeMWRFFs7KyxozFa2BqZgYAUKvVmDi0L4ICA2TOjIgygkePHqFJkyaoUaMGvLy84OXlhcGDB6NTp044depUgvOPHTsW06ZNw5w5c+Dn5wcvLy8cPHgQ69atw6+//poKW0BExhYWFoqJQ/sgMiICQHTvzz/+XAWnbNllzowofhWqVEfvod/qOm9fvcDcyWNkzIiIkiJNFpQPHjyI2rVrI0uWLFLMxMQELVu2xKVLlxK8TM/R0VFn0VitVsPBwYFPuiVKh1YtnIVnjx9Iw5179kONHxrImBGRphJlyqH/yG896N95vMG8KeNkzIiIMopp06bBwsICixcvhrm5ORQKBUaPHo2yZcti/PjxCc5/9OhRFC1aFD169JBiDRs2RKNGjXD8+HF2tiBKh5bNmYYX/z2VhrsNGIaqNWvLmBGR4XoMGonyVapLw0f37sSpw/tlzIiIEivNFZTfvXsHHx8fFC9eXGtc8eLFIYTAw4cP413GH3/8AS8vL4wZMwYfP36Ej48Pli1bhr/++gtz5sxJqdSJKIXcuf4PtqxeKg0XLVkag8dMljEjIt269hmEStVrScPH9u3E2eOH5EuIiNI9tVqNEydOoE6dOrCystIY17RpU9y9exeenp7xLiNfvnwICAiASqXSiHt7eyNv3rzsbEGUzlz9+zx2blgtDZcoUx59h7GHJ6UfpqammL54DezsHaTYzAkj8MHTQ8asiCgxTOVOIC5vb28A0b2M44rpsRwzjT4NGjTAli1b0LNnT8ydOxcAYGZmhjlz5qB79+7xzvv582d4eXlpxNzd3QEAKpVKqyGeUlQqFYRaDaEA1GrD6/5C/XU+tQrqFMwvs+N+Th1CrUJQgD+mjBok9Z4yN7fA1IUrYGpmCrU6dd6PGZ1CqCHUgq9nI1AogCkLlqFzkzoIDIi+L+mMscNRqlxF5MyVi58bKYyfzakjJfZzzPJUKhUgUr64mVrtOWPw8PBAUFAQXF1dtcYVKRJ97/bHjx/D2dlZa3yMefPmoWnTpujatSuGDBkCc3Nz7NixA/fu3cOePXtSLHciMj4/H29MHjFQGrawtML0xathZm4uY1ZEiZc7rzMmzv4TYwZE12iCAwMxaUR/rNp5GDzPSZT2pbmCcgxdl97FxBLqRbFs2TIMGTIEU6dOxYABA2Bubo4DBw6gb9++8PDwwKJFi/TOu2LFCkydOlXnuKCgoFR7cElwWCTU4cFQmSgQmYjDJNRqRIWHAgAUfBhDiuF+Th1CrcaimVM0zlT3HTYaLvnzIzI0WMbMMpiICKiiIqAKDeHr2QiyZnHE6MnTMWnkYABAYIA/Jo/oj8Vrt0IdGQ6AnxsphZ/NqSMl9rOIiECUwhQBAaZQpsKvyKCg9PPwHz8/PwCAnZ2d1riYWMw0+lSsWBFz587FkCFDcPToUZibmyMwMBBTpkxBo0aN4p03LXe24EkkeXC/y0OoVVCrVJgxbgS8vT5J8eETpyF/wULsaJFC2PEiZdVr0hwt23fGkT07AERfmbptzTJ06d2fnzMy4Oe7PHTt9/TQ2SLNFZSzZcsGQHfDOCaWNWtWvfOHhYXht99+Q8OGDfH779+eeOvm5ob79+9j0aJF6N+/P4oWLapz/gEDBqBdu3YaMXd3d7Ru3Rq2trZwcHDQOZ+xCbMIKH3UMDEBzKzMDJ/va0PCzMoaCqVJSqWX6XE/p46/Th3DySMHpeHK39VCl76D+eRqIzP52qNFydez0TT5uSOuX7mE4weie/7duX4V+3dvR7tOXfm5kYL42Zw6UmI/KyJMYWppCnt7B5goU77RbGtrm+LrMBYTk+h9rFZr/7SLicVMo8/48eMxZ84cbNq0CV27dgUAXLx4ES1btsTdu3exd+9evfOm5c4WPIkkD+53eQi1Gkf27sCFMyekWM0f6qPFT23Z0SIlseNFihs8ejxuXb2M9x5vAQAr5s9ExcpVUfD/7N13WBRXGwXws7v0KmLvvffejcYau8beu6KgSJEiVRQERRDEjr13Y43RL5ZETSwxxo69IiqCdNjd7w9kYAUVpAwL5/c8yeN7d2fnMOzC5c7MvZUrAeDPmbzEn+/iyOi4q8PFFvluQLlMmTIwNTXFnTt30j1269YtSCQS1KtX74vbP3/+HLGxsahbt266x1K2u3fv3hcHlEuUKIESJUpk+JhMJvtmhz2nyGQySKRSSKQSSLPwx5oC+LSdLEvbUdbwOOe+d2FvsNDRWqj1DQ3huiQIGhqZP8FCmSORJP+sAd/POcrW3Rv//H0JL549AQAE+SxE85ZtUKN+Ix7nXMKfzXkjN45zyuvJZLI8GVDOq/5cTkjpl2Y05VtK25f6rkDyHwg+Pj7o16+fMJgMAB07dsSsWbMwf/58XL9+HQ0bNsxw+/x8sQVPIomDx10czx49QICPp1AXNS0OZ58AaOmpzwkydcQLL3JfEV19uPsGYcrQvlAoFEhKSoS7/Rys3XEAujzueYo/38WR0XFXh4st8t2AMgAMHDgQW7Zswbt374SrkZOSknDo0CF07NgRxYsXF56rUCigUCigoZH8pZQuXRpaWlq4ceNGute9fv06AKBSpUq5/0UQ0XdTKpXwsJuND+9T/3ie6+6N0mW/PD8kUX5jYGgEN98gTB7SG0qlEgkJ8ZhvPwfr95+AljY7aESUOSVKlECZMmUyXJT6xo0b0NDQQIMGDb64/atXr5CUlISKFSumeyyl7dmzZ18cUM7PF1vwJJI4eNzzXlJSElysZiI2NkZoc/YJQLESpURMVTjwwou80bhFG4ybPhvBy30BAA/v38WqZYth7erF456H+PNdHBkdd3W42CJfXsPu5OQEAwMDjBw5Ek+ePEFYWBhmzJiBly9fwsfHR+W5zs7O0NTUxPnz5wEA+vr6mDlzJn777Tc4Ozvj5cuXePv2LVatWoUVK1agb9++X73CmYjEd2DHZpz97bhQ/9izD34aMETERETfp3GL1hg1OXXhnDs3byB4+Zfn8Sciysjw4cPxxx9/4OnTp0JbTEwM9u/fj169eqlcJfz+/XuEhIQgLi4OAFC+fHno6+vjwoUL6V43pf9cu3btXP4KiCg7ggOX4Ma1y0L986gJaP9jNxETEeW8KbNtUate6snNnRvX4a8/zoqYiIi+Jl8OKJcvXx7nz5+Hnp4emjRpgmrVqiEkJAT/+9//0KxZM5XnplwZkXahPh8fH2zatAmnTp1C48aNUaNGDaxevRre3t5cyZoon3v25BGWuDsKtWmx4rBbsPibi3ES5VfTrRxQtUYtoQ4O9MWtf6+JmIiI1I2DgwMqVqyIAQMG4MyZM/j7778xdOhQJCQkpLvYwtfXF9WrV8fly8mDTzo6OnB1dcWlS5cwduxY/Pnnn7h8+TLs7e2xceNGTJkyBVWrVhXjyyKiTLhx9W+sXbZYqCtWqYbZ89xFTESUOzS1tODhtxLa2jpCm5v1TERGfBAvFBF9Ub4cUAaAGjVqYN++fXj37h0iIiJw6tQptGrVKt3z3NzckJSUhLZt2wptUqkUo0ePxh9//IHQ0FC8f/8eV65cwezZs6GtrZ2XXwYRZYFcLofLHDPExqQuLOLg4Y0iJkVFTEWUPdo6OnDzXQHZp6mZ5HI5nGZPQ1xcrMjJiEhdFC1aFBcvXkTHjh0xZ84cTJo0CSVKlMDff/+N6tWrqzzX1NQUVatWha6urtBmbW2Nc+fOISkpCZaWlpgyZQru3buHffv2YdWqVXn95RBRJsVER2He7GmQy5Pn15RpaGC+30ro6uqJnIwod1SuXhOzHFIXgn3z+hW85ll/ZQsiEku+nEOZiAqnTasCcP3yJaEeNHIcWnfoJGIiopxRu35DTDK3xqqlXgCAxw/uI8jbA3OcF4icjIjURbFixeDr6/vN51laWsLS0jJde7t27dCuXbvciEZEucR/oSueP3kk1JPN56B2/YznOycqKIaMnYSzp47j4tn/AQBOHNqH9j/2QM/+P4ucjIjSyrdXKBNR4RJy9xZW+qauXF2+UhXMcnAVLxBRDhtnNgt10vwRuHXdCly+cF7ERERERJRfXTh7Gnu2BAt1o2YtMXLCVBETEeUNiUQCZ+9lMDIuIrR5OVnj1Yvn4oUionQ4oExEoktKSoKbtTmSEhMBJE9bM3/pCujq6YucjCjnaGhoYJ7nEpV54VytZyDqY6SIqYiIiCi/+RgRAXcbc6HW1dOHy+IAyGQyEVMR5Z3iJUthrlvqxUZRkZFws54JhUIhYioiSosDykQkui1rlqssUjZ6qjnqN2kuYiKi3FGpSjXMnOsk1K+eP4NvmkUoiYiIiHzc7PHm9Suhnu3ojnIVK4uYiCjvderWE70GDRXqv/88iz2bg7+yBRHlJQ4oE5GoHoXcE+aVBYCKVatj6uy5IiYiyl1Dxk5Cs9bthfrgrq04+9txERMRERFRfvG/E0dwZO8OoW7VoRMGjRwnXiAiEVm7eKJU2XJC7e/pimePH4qYiIhS5MiAclJSEvbs2QNzc3OMGDFCaD969CgSP93CTkT0OblcDncbcyTExwNIni/LxXsZtHV0vrElkfqSSqVwXbIc+oaGQpuH3Wx8CH8vYioiyknsGxPR93j/NgwL7FMX1TQwMoKz9zJIJBIRUxGJx8DQEC7eAUIdFxsDF6sZkMvlIqYiIiAHBpQjIyPRvn17DB48GFu2bMH27duFx3bu3Indu3dndxdEVEDtWL8K/179W6hHTJiGhs1aipiIKG+ULlsO1i6p88K9C3sDHxc7ERMRUU5h35iIvodSqYSnoxXC370V2ua6eaNk6bIipiISX4t2HTFkzCShvn75EratWyFiIiICcmBA2cHBAZGRkbh8+TLCw8NVHps4cSLWrFmT3V0QUQH07PFDLPdZINTlK1XBdBvOJUuFR5+fh6P9j92F+vjBPfjfiSMiJiKinMC+MRF9j2P7d+P08cNC3al7b/QcMFjERET5h4W9i8o84kGLF+DhvTsiJiKibA8o79u3D+vXr0fTpk3TPVanTh1cuXIlu7sgogJGoVDA3dYC8XGxQpuz9zLo6uqJmIoob0kkEjh6+sLQyFho83S04tQXRGqOfWMiyqrQVy+wyMVWqE1Mi8Fh4RJOdUH0ia6ePlyXLBc+Ewnx8XCxmoGkpCSRkxEVXtkeUA4LC0OdOnWEOu0vPQ0NDSQkJGR3F0RUwOzetA5XL/0p1EPHTkaTlm1ETEQkjuIlS8PalVNfEBUk7BsTUVYolUrMnzsLUZGRQpvDQl8ULVZcxFRE+U/j5q0wavIMob717zVsCPITLxBRIZftAeXSpUvj2rVrQp2203zx4kVUrlw5o82IqJB68fQJAha5C3WZchUwc66TiImIxNVr4FBOfUFUgLBvTERZsW/bRlw4c1qoew0cis49eouYiCj/mm7lgMrVagj1an9v3L15Q8RERIVXtgeU+/fvDzMzM4SEhKi0P3nyBFZWVhgwYEB2d0FEBYRSqYSH3WzExkQLbU6L/KGnbyBiKiJxceoLooKFfWMiyqxnTx5hqUfqhRUlSpWGjauXiImI8jdtHR24+QZBJpMBAORJSXCxMkNCfLzIyYgKn2wPKLu4uCA+Ph61a9dG8+bNoVQq0alTJ9SuXRsAYG9vn+2QRFQw7N++EX/9cUaoB44YixbtOoqYiCh/4NQXRAUH+8ZElBlyuRxu1jNVLrRw9gmAobHxV7YioroNm2D8DEuhvn/7Jtb4+4iYiKhwyvaAsqmpKS5dugRLS0tERUXB2NgYz58/x7Rp03D+/HkYGhrmRE4iUnOvXjyH3wJnoS5ZpixmObiJmIgof+HUF0QFA/vGRJQZ24NX4tpfF4T651ET0LpDZxETEamPSebWqFmnvlBvWOGHG9cui5iIqPDJ9oAyAJiYmMDb2xu3b99GeHg47t+/D19fX5iYmOTEyxORmlMqlVhgPxvRUVFC2zzPpTAwNBIxFVH+wqkviAoO9o2J6GuePAxBkM8CoS5boRJmObiKF4hIzWhqacHNNwgampoAAIVCARerGYiLixU5GVHhkSMDykREX/PLnu0qi430GTwcbX7oImIiovwpo6kvvJ3nipiIiIiIcpJcLoebjTni4+MAJJ9QdluynGuKEGVR9dp1MXV2aj/5yYP7CPL2EDERUeGikd0X2LFjxzefM2zYsOzuhojUVFjoK/i6Owp1sRKlMMdpwVe2ICrceg0cit+OHMS5UycAACcO7UWXXv244juRmmDfmIi+ZufGNbh++ZJQDx03GY1btBYxEZH6GjPNAr+fPIqb/1wFAGwLXolOPXrzM0WUB7I9oDx8+PBvPoedZqLCSalUYqGDFT5GRghtjp6+MDIuIl4oonwuZeqLwV3aCJ8dT0crNGnZBkVMioqcjoi+hX1jIvqSZ48fInDRfKEuW6ESZto6iZiISL1paGjAbUkQRv70A+Lj46BUKuFmY47tx89CV1dP7HhEBVq2B5SvXbumUiuVSrx48QKHDx9GdHQ0rKyssrsLIlJTxw7swdnfjgt1z/6D0aFLDxETEamHlKkvXOaYAQDevw2Dt/NcLAxYI3IyIvoW9o2JKCMKhQLuthaITzPHq7P3Mujq6YuYikj9Va5WA2Y2jljqkXxy5tnjh1jhswBznHlXLFFuyvYcyo0aNVL5r3HjxujduzdWrlyJJk2a4OTJkzmRk4jUzNs3oVjsaifURYsVV5kbloi+rtfAoWj/Y3ehPnFoL04fPyxiIiLKDPaNiSgjezYH4+qlP4V68OiJaNa6nYiJiAqO4ROmoUGT5kK9LXilytQyRJTzcnVRvjFjxsDf3z83d0FE+ZS3sy0iPoQLtd18H96uT5QFKVNfGBoZC21e82wQGfFBvFBElC3sGxMVTi+ePsEyLzehLlOuAizsXURMRFSwyGQyOPsEQEtbGwCEqS/i0twRQEQ5K1cHlN++fYt3797l5i6IKB86eeQATh37Rai79OqHH3/qK2IiIvWUMvVFindhofCd7/iVLYgoP2PfmKjwUSqVmD93FmJjooU2p0X+0NM3EDEVUcFTuVoNTJtjL9RPHoZg5RLeIUuUW7I9oPz27dt0/z169AiHDh3C4MGD0bJly5zISURqIvzdWyxyshXqIkVNMdfdW8REROqt18ChaNPxR6H+Zfd2/Pn7byImIqKvYd+YiNLat20j/v7zrFAPGjkOLdp1FDERUcE1cpIZ6jZqItRb1wbhxtW/RUxEVHBle1G+4sWLf/Gx6tWrIzAwMLu7ICI14uNqh/B3b4Xa1s0LRYt9+ecEEX2dRCKBg+dSDO3WBtFRUQAAD3tL7Pr1DxgYGomcjog+x74xEaV49fwZ/BY4C3XJMmVhYe8qXiCiAk5DQwOuPoEY0esHJCYkQKFQwNV6JrYdPQNtHR2x4xEVKNkeUPbx8UnXZmxsjKpVq6Jjx46QyWTZ3QURqYn/nTiCE4f2CXXHbj+hW5+BIiYiKhhKly0HC3s3eDpaAQBCX75AgJc77BcsFjkZEX2OfWMiAj5NdWE3CzHRUUKb0yJ/ngwmymVVatTClNlzsdx7PgDg8YP7WOW3CBZ2nLecKCdle0DZ2to6J3IQkZqL+BAOT8fUnwdGxkVg77EYEolExFREBcfAEWPx6y/7ceXieQDAni3B6NZnAJq2aityMiJKi31jIgKAAzs249K534W639BRaN2hs2h5iAqTMVPNcfrYL7h94x8AwOZVAfixZx/Ubdjk6xsSUabl6qJ8RFR4LHF3xLuwUKG2clmI4iVLiZiIqGCRSqVw8vaHto6u0OZua4HY2BgRUxEREdHnXr98jqULnIS6ZOkysJznIWIiosJFQ0MDrksCoaGpCQDJU19YzUBCfLzIyYgKjixfoXz+/Pks76Rdu3ZZ3oaI1Me5U7/iyN4dQt22U1f0GjhUxEREBVP5ipUxw8YRvvPnAQCeP3mElUs8YTlvvsjJiAov9o2JKC2lUokFdpaI/vhRaHP0XApDI051QZSXqtWsg8kWNlixZCEA4OH9u1i7bDHMbBxFTkZUMGR5QLl9+/ZZ3olSqczyNkSkHj5GRmKhg6VQ6xsawtHTl1NdEOWSYeOn4uThA7hx7TIAYNu6FejSqx/qN24mcjKiwol9YyJK65c92/HnmVNC3WfwcLTt1FXERESF19jps3D6+GHcvfkvAGDDCj906tELtes3EjcYUQGQ5QHlY8eO5UYOIlJTSz3m4c3rV0Jt6TgfJUuXFTERUcEmk8ng7L1MZfVqdxtzbD3yO7S0tcWOR1TosG9MRCnevH6JJe4OQl2sRCnMcVogYiKiwk1TUxOuiwMxqk9nyJOSIJfL4Wo9E1t+OQ1NLS2x4xGptSwPKPfo0SM3chCRGrp47n84uHOLULds/wP6DxstYiKiwqFKjVqYbGGDoMXJf6Q+vH8XawMWw8yat/AR5TX2jYkI+DTVhf0cREVGCm2Onr4wMi4iXigiQo069TDJ3Aqrli4CAITcuYV1gUswbY69yMmI1BsX5SOi7xId9RHz584Saj19Azh5+XOqC6I8MmaaBWrWqS/UG1b44+7NGyImIiIiKryO7t+F86d/FeqfBgxBhy484USUH4w3s0SNOvWEOnj5Utz5718RExGpv2wPKCuVSgQGBqJ+/frQ1dWFhoZGuv+IqOBZ5umG1y+eC7WFvStKlysvYiKiwkVTUxPOPgGQyWQAAHlSEtxszJGYmChyMqLCjX1josInLPQ1FrumXu1oWrwErF09RUxERGlpamnBxScQsk+/g5P7zTORmJAgcjIi9ZXtAWU/Pz9YW1ujWbNmiIuLw/Tp09GjRw9oaGigR48emD17dg7EJKL85O8/z2HPlmChbtqqHQaNHCdeIKJCqla9Bhg3fbZQ3735LzavDhQvEBGxb0xUyCiVSnjOs0ZkxAehzX7BEhgXMREvFBGlU6teA4w3my3U9279h/Ur/ETLQ6Tusj2gHBwcjKCgIKxfvx4AEBAQgMOHD+Phw4eIiYlB//79s7sLIspHYmOiVaa60NHVg5O3P6RSzqBDJIZJFtaoXK2GUK/2W4RH9++KmIiocGPfmKhwOXFoH878elSou/cdiE7de4mYiIi+ZJK5NarVqiPUa5ctxr1b/4mYiEh9ZXsE6P79+yodY7lcDgAoU6YMAgICYGtrm91dEFE+stzHAy+ePhbqGbbzUL5iZfECERVyWtracPEJEOYvT0xIgLuthfD7mIjyFvvGRIXHu7A38HaZK9QmpsVg47ZIxERE9DWaWlpwXRyoMmWcq/VMThlH9B2yPaAcHx+PokWLAgD09fXx+vVr4bHKlSvj+vXr2d0FEeUT//x9ETvWrxbqhs1aYti4KSImIiIAqN+kOUZMmCbU/179Gzs3rP7KFkSUW9g3Jio8FjnZICL8vVDbzfeBSVFTERMR0bfUrt8IY6el3nF79+a/2LjCX8REROopR+9Rr1u3Lnbs2CHU+/fvh6kpf6ESFQRxcbFwt7WAUqkEAGhr68DFJ4BTXRDlE9NtHFEuzd0Cgd4eePbkkYiJiIh9Y6KC6+SRAzh17Beh7tKrH7r06idiIiLKrMmzbFClek2hXrPMByF3b4mYiEj95OhI0KRJk2BjY4MOHTqgS5cuGDNmDEaPHp2TuyAikaxc4oknD0OEepqVPSpWqSZiIiJKS1dXD06LUq+uiI+Lhcfc2cJJICLKe+wbExVM4e/ewmuejVAXKWqKue7eIiYioqzQ0taGy+JA4eKopMREuFmbIykpSeRkROoj2wPK165dE/49efJk+Pr6IioqCu/fv4ezszNcXV2zuwsiEtmNa5exdW2QUNdt1AQjJ5mJmIiIMtKsdTsMGjVeqC9fOIf92zeKmIio8GHfmKjg83aZiw/v3wn1XPdFKFqsuIiJiCir6jVqitFTZgr1rX+vYfPqQBETEamXbA8oN2rUSKWePXs2rl69iqtXr8LFxQWamprZ3QURiSghPh5u1jOhUCgAfFrIwCd1IQMiyl8s7FxQskxZofZb4IzXL5+LmIiocGHfmKhgO33sF/z6y36h7tS9N7r2HiBiIiL6XlMt7VCpanWhXrXUC4/u3xUxEZH6yPaAsre3N0JDQ3MiCxHlQ2v8ffAo5J5QT5lliyo1aomYiIi+xsDQCI4LfYU6OioKCx2sOPUFUR5h35io4PoQ/h6e86yF2riICew8fCCRSERMRUTfS1sneV2glM9wYkICXG1mQi6Xi5yMKP/L9oDy3LlzUa5cOQwYMACHDx/mB4+oALl94x9sXJk6J2uteg0xeqq5iImIKDPaduqKXoOGCfUf/zuJo/t3iZiIqPBg35io4PJxscP7t2FCbe3qhWIlSoqYiIiyq0HTFirTOf537YrKdI9ElLFsDyjfvXsXVlZWuHTpEvr06YMKFSrAwcEBISEh396YiPKtxIQEuFqnnp2VaWjAdXEgb9UlUhNWzgtgWryEUC9xc8C7sDciJiIqHNg3JiqYzpw8huMH9wh1x6490bP/zyImIqKcMt3aARUqVxXqlUs88fjBfRETEeV/2R5QrlGjBry8vPDs2TMcOnQIzZs3h4+PD6pXr44ffvgBmzdvzomcRJTHgpf7IuTOLaGeOHMOqteuK2IiIsoK4yImsJvvI9QRH8KxyNlWxEREhQP7xkQFT8SHcCywnyPUhkbGsF+whFNdEBUQOjq6cE4z9UV8fBzcbcx5lxHRV2R7QDmFTCZDnz59cODAATx//hze3t4ICQnBmDFjcmoXRJRH7t++iXWBqXOwVq9dFxNmzPnKFkSUH3Xu2QddevUT6lNHD+G3IwdFTERUeLBvTFRwLHF3xLuw1LnRrV09UbxkKRETEVFOa9y8FYaNnyLU16/8hZ0bVouYiCh/y7EB5RRxcXE4ffo0Tpw4gZcvX0Imk+X0LogoFyUmJiZPdZGUBCD5D2IXn0BoammJnIyIvoet2yIYFzER6kXOtvgQ/l7ERESFC/vGROrt3KlfcWTvDqFu26kreg0cKmIiIsotM2zmoWyFSkId6O2BZ48fiheIKB/LsQHlq1evYubMmShdujRGjBiBx48fw8PDA8+ePcupXRBRHti8KgB3/rsu1GOnzULt+g1FTERE2WFavASsXb2E+v3bMCx2tRcxEVHhwL4xkfr7GBGBBfaWQq1vaAhHT19OdUFUQOnq6cPFJ0Co4+Ni4W5rAYVCIWIqovwp2wPKAQEBaNy4MZo2bYp169ahV69eOH36NO7fvw8HBweULl06J3ISUR54cO82Vvt7C3XlajUweZaNiImIKCf07P8z2v/YXaiPHdiNs78dFzERUcHFvjFRwbHUYx7CQl8J9Zx5HihZuqyIiYgotzVt1RZDxkwS6quX/sTuTetETESUP2V7QNnCwgISiQSBgYF49eoVtmzZgk6dOvGsLZGaSUpKgpuNORITEgAAUqkULosDoaWtLXIyIsouiUQCh4VLYGBkJLQtdLDCx4gIEVMRFUzsGxMVDH/+/hsO7toq1K07dka/oaNETEREecXczhllylUQ6mVebnj+9LF4gYjyoWwPKF+5cgVXr17FjBkzUKRIkRyIRERi2LZuBW7+c1WoR04yQ/3GzURMREQ5qUSpMpgzz0Oow0JfYanHPBETERVM7BsTqb+PkZHwsJst1PoGBnD09OOJIaJCQk/fAE7ey4Q6LjYG7jac+oIorWwPKDdp0iQnchCRiJ48DMHKJZ5CXbFKNUyz4hyrRAVN3yEj0apDJ6E+uGsr/jxzSsRERAUP+8ZE6s9/oTNCX70U6tmO7ihdtpyIiYgor7Vo2wGDRo4T6isXz2Pv1g2i5SHKb3JsUT4iUk9yuRxuNuaIj48DkHxrvLP3Mujo6IqcjIhymkQiwTwvf+jpGwhtHnazEfUxUsRURERE+cel879j//ZNQt2yXUcMGD5WxEREJBYLe1eUSnMyaZmnK1495+K6RAAHlIkKvZ0b1+D65UtCPWz8FDRq3krERESUm0qXLYdZDm5CHfryBZZ5uX1lCyIiosIhOuoj3G0thFpXTx/zvPw51QVRIWVgaIR5Xn5CHRMdhflzLaBUKsULRZRPcECZqBB79uQRAhfNF+qyFSphhg3nVCUq6AaOGIumrdoJ9d4t6/H3n+dETERERCS+ZZ5ueP3iuVDPcnBDmfIVvrIFERV0rTuoLsh56fwZ7N+x6StbEBUOHFAmKqQUCgXm285CfFys0ObsvQy6evoipiKivCCVSuHk7Q8dXT2hbf7cWYiNiRYxFRERkXj++uMs9mwJFuqmrdqpzJ9KRIWX5TwPlCxdRqj9PJzwKs3JJ6LCKEcGlJOSkrBnzx6Ym5tjxIgRQvvRo0eRmJiYE7sgohy2d+sGXLl4Xqh/HjUBzVq3+8oWRFSQlK9YGTNsU+9IePH0MZb7eIiYiKjgYN+YSL3EREfBY+4sodbR1YOzzzJIpbz+iogAQyMjOHouFeroqCgssJ/NqS+oUMv2b8jIyEi0b98egwcPxpYtW7B9+3bhsZ07d2L37t3Z3QUR5bBXz59hmaerUJcuVx4W9i7iBSIiUQwbNwUNm7UU6h3rV+Ofvy+KmIhI/bFvTKR+AhfNx4tnT4TafK4zylWoJF4gIsp32nbqij6Dhwv1hTOn8cvubSImIhJXtgeUHRwcEBkZicuXLyM8PFzlsYkTJ2LNmjXf/drh4eE4fvw4Dh8+jJcvX37Xazx58gRHjhzB2bNnERMT891ZiAoKpVIJd1sLxERHCW3zvPygb2AoYioiEoNUKoWz9zJoa+sASP35EJdmKhwiyprc7BsTUc67cvEP7NyY+rls3KI1hoydJGIiIsqv5jgtQLESpYR6yXxHvHn9fWNVROou2wPK+/btw/r169G0adN0j9WpUwdXrlz5rtcNCgpC2bJl4eHhgYCAAFSpUgVz587N9PahoaHo1asXGjVqhOXLl8PLywsNGjTAgQMHvisPUUGxf/tG/PXHGaHuN3QUWrXvJGIiIhJTparVMW2OnVA/eRiCVb5eIiYiUm+51TcmopwXGxsDd1sLodbW0YWzN6e6IKKMGRkXgaOnr1BHRUZigf0cTn1BhVK2f1OGhYWhTp06Qi2RSIR/a2hoICEhIcuvefLkScyYMQOLFi3C+fPnceLECRw6dAg+Pj5YsWLFN7ePiYlB586dkZiYiCdPnuDo0aM4evQoLly4AE1NzSznISooXj1/hqUezkJdsnQZWM7jnKlEhd2ISWao27CJUG9Zsxz//cNBL6LvkRt9YyLKHUE+Hnj+5JFQz7BxRIXKVUVMRET5XYcuPfDTgCFCff70rzi6f5eIiYjEke0B5dKlS+PatWtCnbbTfPHiRVSuXDnLr7lo0SKUK1cOM2bMENq6deuGTp06wcvL65tnfxYvXowHDx5g8+bNMDIyEtqLFy+OXr16ZTkPUUGgVCox325WuqkuDNN8RoiocNLQ0IDL4gBofDrpqlAo4GY9Ewnx8SInI1I/udE3JqKc98/fF7E9eJVQN2zaAsPGTxUxERGpC2tXT5gWLyHUPi52CAt9LWIioryX7QHl/v37w8zMDCEhISrtT548gZWVFQYMGJCl10tMTMTZs2fRvn37dLcaderUCU+fPsW9e/e++hqbN29Gx44dYWpqij/++AP79u3D33//DYVCkaUsRAXJgR2bcenc70Ldb8hItPmhi2h5iCh/qVqjNiZb2Aj1w/t3sTZgsYiJiNRTTveNU8TExMDLyws9evRAly5dYGdnh7CwsCy9xu3btzFnzhx07doVAwcOxLp16yCXy78rD5E6i4uLhZuNuXChkra2Dpx9AiCTyURORkTqwLiICewXLBHqj5ER8HS04tQXVKhoZPcFXFxccPz4cdSuXRuNGjWCUqlEp06dcOnSJVSuXBn29vZZer0nT54gMTERFSpUSPdYxYoVAQAhISGoWbNmhttHRkYiJCQE5cqVQ40aNaCnp4fy5cvjwoULKFeuHPbs2YNatWp9cf9v3rxJ1zlP+YNALpfnWadbLpdDqVBAKQEUisyP+ysVn7ZTyMHh89yjbsf59YvnWOoxT6hLlCqNWY5uUCjy9x+R6nac1ZVEqYBSoeRxzmXq8H4eM3UmTh//BXdv3gAAbAjyQ6fuP6Fm3QYiJ8scdTjGBUFuHOeU15PL5YBS8u0Nsik3+3M53TcGgKioKHTo0AExMTHw8PCAvr4+PD09sXXrVly8eBFly5b95musX78eZmZmmDZtGmxskk8e7du3D7du3cKSJUu+sTVRwbJi8UI8ffRAqKdZ2aNS1eoiJiIiddOpey907zsQJw7tAwCcOXkMxw/uRc/+P4ucjChvZHtA2dTUFJcuXYKnpyd++eUXGBsb4/nz55g2bRqcnJxgaGiYpdeLikq+HV9fXz/dYyltHz9+/OL2Katp//7777CyssLixclXVz1//hytWrVCnz59cOvWrS/OpRwUFAQ3N7cvZouIiMj8F5MN0XGJUMRHQy6TIDEL3yalQoGk+FgAgISLSeQadTrOSqUSHnazEB2VOtWFrctC6GhqIDE2WsRk36ZOx1mtJSRAnpQAeWwMj3MuUpf3s727FyYO7Qd5UhLkcjncrGZi3a5DwnQY+Zm6HGN1lxvHWZmQgCSJBiIjNSCV5P6AclSa34k5Laf7xgDg4+OD69ev4/bt26hRowYAoE2bNqhatSpsbW2xdevWr25/5coVTJkyBStXrsTEiROF9m7duiEmJibLeYjU2b9X/sK2danr8tRr3BQjJ5mJmIiI1JWN2yL89cdZhL97CwDwcZmLFm07qEyHQVRQZXtAGQBMTEzg7e0Nb2/vbL+WlpYWAGS4YElKm46Ozje319bWhru7u9Berlw5zJ49GzY2Nvj999/RtWvXDLc3MzPD4MGDVdpCQkLQv39/GBgYwNjYOGtf0HdSaiZA+l4BmQzQ1M38H/HKT1ecaurqQSLlLVu5RZ2O86FdW3Hp/Fmh7v3zMHTs0VvERJmnTsdZnck+/dyU8jjnKnV5P9dt0gLjps/CuoDkKxbv372NrRvWYZKFlcjJvk1djrG6y43jLEnQgIaOBoyMjCGT5v6AsoGBQa6+fk72jQFg06ZNaN26tTCYDADGxsYYMGAANm/ejFWrVn31a1q4cCHKlCmDCRMmpHtMT08vRzISqYP4uDi42ZgLUyFqaWvD1SeQU10Q0XcxKWoKu/k+mGs2HgAQ8SEcXvOs4b1yo8oaCkQFUbYHlL29vTF27FiULFkyJ/IIt+yFhoame+z169cqz8lI8eLFoa2tjbJly6brIKdMk/Ho0aOMNgUAlChRAiVKZHw2SSaT5VlnQyaTQSKVQiKVQJqFP9YUwKftZFnajrJGXY5z6KsX8PVwEuriJUvDynlhvs6clrocZ3UnkST/rAGPc65Sp/fzZAsb/P7rUTy4exsAsC5wCTr37I1qNeuInOzr1OkYq7PcOM4pryeTyfJkQDk3+3M53Td+9+4dHj9+jJ49e6Z7rFGjRli7di1u3LiB1q1bZ7i9QqHAyZMn0bdvX+zbtw87duxAaGgoypcvjzFjxqB79+45kpNIHazyW4THD+4L9ZTZc1G5esZTKRIRZUaXXv3w4099ceroIQDA6eOHcfLIAXTr/X1rJhCpi2wPKM+dOxeOjo7o3bs3Jk6ciJ49e2ark25sbIzatWvjypUr6R67fPky9PX1Ub9+/S9ur6GhgRYtWuDff/9N99jbt8m3IZiamn53PiJ1oVQqscDOEtFppohx9FwKI+Mi4oUiIrWgqaUFF58AjOvfDQqFAkmJiXCzNsf6/SegoZEjNzcRFVg53TdOuaCiePHi6R4rVqwYAODVq1df3P7Nmzf4+PEjjh8/jpMnT8LDwwMVKlTArl270KNHD3h6esLOzu6r2+fX9UU4b7o41PW437x+DZtXBQh17QaNMHLS9Hy/pkgKdT3u6o5rjYhD3d7vtm5euHzhPCLC3wMAFjnZomnLNjAxLSZysqxRt+NeUGR03NVhfZFs/1V49+5dBAcHY9OmTThw4ADKlCmDsWPHYsKECahWrdp3vebEiRNhbW2Nv/76Cy1atACQ3FE+cOAARo0apTLlxY0bN3Dz5k106dJF6FRPnToVo0aNwqFDh9C3b18AyYNrGzduhKGhITp37pzNr5oo//tlz3b88ftvQt375+Fo/2M3ERMRkTqp27AJRk81x8YV/gCAW/9ew9a1QRg7zULkZET5W073jRMTEwFkfFV1ygmelOdkJD4+HkDylc5nz55F+/btAQDdu3fHu3fv4OTkhFGjRqFcuXIZbp+f1xfhvOniUMfjHh8fBzerGcJUFxoamnCYvwjKxHgkJsaLnC5z1PG4Fwhca0QU6vZ+N9TXwxxHV7hYJ/eTP7x/h0XzrDHfd7nIybJG3Y57QZHRcVeH9UWyPaBco0YNeHl5YcGCBTh69CjWrVsHHx8feHp6omPHjpg4cSJGjx6dpdc0NzfHiRMn0KdPH9ja2kJPTw/+/v6oXLkyFi5cqPLcnTt3YsGCBTh37hzatWsHABgxYgROnjyJ4cOHw9LSEhUqVMDevXtx8eJFbNu2DSYmJtn9svPEzZcRkEkl0NPWgFQCSCQSSABIJIAEEkgkgKZMimolVOfMe/EhDspoQCaVJT9XIoEUn7aXpG5f0khbZV6fj3GJSJQrv7gvaXIBbQ2pyhtaoVQCwKfnc56g/ODN65dY4u4g1MVKlIKV8wIRExGROpo6ey5+//Uonny6PXilryfad+6GKjVqiZyMKP/K6b5xyvodGXX0Uxaq/toaHymPmZqaCoPJKQYNGoSDBw/i7NmzGDFiRIbb5+f1RThvujjU8bivXLYEj9JMdTHJwgq1GjQRMVHWqeNxLwi41og41PH93nPgMPzv5An8fuIIAODU8SPo1u9ndFaT9YsA9TzuBUFGx10d1hfJsftWZTIZ+vTpgz59+iA0NBSbNm2Cv78/xowZk+UBZS0tLRw/fhw7duzAmTNnkJSUhFmzZmHcuHHQ1dVVeW6DBg0wdOhQldsAJRIJNmzYgKFDh+Lo0aM4f/48WrdujaCgIFStWjVHvt68sOx0COQK5VefU1RfC2vHNlNp8/3fU7yMTL+o4ed2Tm0FTVnqG9P/t/u4+vTDt3MNa4RyRVPnp1537hGO/Zd8O6YwCP1pQDplEFoKwLpHTTSpkDqYf+K/19j+91PVAWsg3YD2kGbl0alW6rzWN55HYN35R6r7kqQOaEskgBQStKteDL0alBa2ex0Rh5VnHnzaRwbbfNp/jZIGGNgk9SqdRLkCgadDVPIBgFKeCJmGJqRSKSQATA20MLhZeZVjteOvp4hLVKgM5H/+tWrIpPi5qepVQWfuhiE8JiHDrGmPzw81i0NbM/UH/a2XkQiNjMPWtaugLNcEekoFoFRi8DRz/Pc2CdK37wAJULeMEQx1Uhd7fPkhFqGRcRl+39Ie39JFdGGcZpHIj3GJeBuVkvPLJyIMtDVgoJP64yZRrkBUfFLq8f+0TepxSdm/6o02SqUSSvDkBVFe0dbRgYv3Mkz8+ScolUokxMfD1Xomgvcd59QXRN+QU33jChUqQFdXFw8fPkz3WEpbyjohGSlSpAhKliyZ4eJ7RkZGAICYmJgvbp+f1xfhvOniULfjfuPq39iyOvUqwVr1GmKcmaVaZE9L3Y57QcG1RsShru93e4/FuHbpT0R8CAeQPPVFs9btUcSkqMjJMkddj7u6y+i4q8P6Ijn+12BcXBxOnz6NEydO4OXLl9/dyZRKpRgxYsQXr5ZIMWTIEAwZMiTDx3r27JnhAibqQqn8+mAykDq4mVZm57r5fNNv7+3Tdp8N5Ck/+7dS+N9nj362g7gkOSJjk765v7hE1blcohOS8PT9l//wSVGjlOoZlthEOf59nvXbMpMUSpy7//abz6tYVC/dgPLxm6+/+TXqaKQfUD763yvcD/32LQctqxRVGVA+cfN1ctYq3VGsSuoiO0dfAUdf3RNqz4H1UbNU6sDw73fDsOfK82/ub1aX6uhYI/XkzV+P3mP5/x58c7thLcpjSJpjc+/1RzgdvPnN7TrVLIYprUoJ9buoBEzZnDy/+ucnL9L+u3ZpIzj3UV08bOKGv6FQfnk7qQQw0dPCwoGqc7QvPHIbYR/jkwe4v3DyAhJgfr+60JCl3ha05eIThLyJSr/NZ/uf1L4yTA20he1O33mDf599ULmjIGUfaU9E9KhXChVN9YXtbr6IwF+P3yfvL22+z07UNKlQBNVLGgrbvfwQizdPP0Apl+Pp9VfCCZLP91/RVA+1ShsJ20XFJeHvx+/TnUD6/Gs10ddCzVKp+5MrlLj+/AOkSHPHxOd3UQDQ1pShcrHUrw8AnryLhlKZ/vuX9kSEVAKUMNJR2S4yNhFypfKzjKr7lyD57ou0P99SfgYX9pMXDZu1xMhJZtiyJvmP8ZvXr2LTymWYMHOOyMmI1EN2+8YymQxdu3bFmTNnkJCQAK1PV8sBwK+//opatWqhcuXKX32N3r17Y+PGjQgPD1e5U+/69esAgDp18veCm0TfKzY2Bi5pprrQ1NKCm+9yaGpqfmNLIqKsK1aiJKxdPOFkOQ0A8P5tGHxc7LBg2WqRkxHlvBwbUL569SqCg4OxdetWfPjwAVWrVoWHhwfGjx+fU7sodMa0qgSJNHmQI2WMNuUKzZR/px1MTPFTbVPEKGQAJJ+em7qNQqkElIACSkg/G41uXqkoShnrJD+uVOLTU9PtU09bdZ9ViumjTVXT1Od+2k7Y16dBGSNd1Y6bqb426pYxEnKl3T5lG4Uy/XY6mjKULaKb5jmfZf30bz0t1be3VALoa8s+fR0ZHJdPr/X5IH1mBvYBpB+hR5px9dzYDukHumJjYzO53ef7y9wO0x+bTG2WPCCaRqZPemTp5EXqMxLl6fcQEZuIb1zwn+EdAS8+xOJVRFyWsz4Mi87UCYzRrSuq1PdDP+JsJk5gNK9UVGVAOSQsCr9c//KiTCmMdDRUBpSfvo/B1QfJi0f8pnz6xe36NCytMqAcFhWPgNMh39xfi8pFYdczdXqEhCQFPA7f/uZ2FYvqYemwRiptLgdvIjLu2ydotk1ppdK24Mht3H/z7RM0weOaoYhe6kDN0pP3cT4k+XuR0eB1yuC3W7+6Ksd075XnOPzvq4xPJEAJiSR52qDx7SqheaXUqxUuPnyHnX89y/jkBVIHv7vVKaly18ajt9HY+Odj1RMBSHOl/6fMDcsZo2f91Ls2ImMTse78o9R9feEuijJFdDHd2gHnT/+Kx59uF95+5RUif7mMoqbFMrxLRAJAX1sDfRqWUTnGJ2+FIjo+6YvHMuVr7Vq3pMrUSteffcD76AThuenuaPm0fb2yhiodm2fvY/AuKiHdyQt8tv9yJrrQ107d8kNMAsJjElW+75+faJFAAiNdDZXfNQlJCkQnpN59Ic3g5IUUgEwqUTkBxZMXBVNO9o0dHR3Rtm1bODk5wcvLCxKJBOvXr8elS5ewc+dOleeuWrUK69evx+rVq9GgQQMAgL29PXbs2IFZs2Zh3bp10NTUxL///gt/f3+0b98ebdq0yZGvObedvvMGv997A9mnz4oSyfMOSqSqPY2RrSqiReXUn69/PXqPrReffPP1e9QrpfJz8kFYFJb9dv8rWyRrXqkoRqX5ff4hJgEumThxXrmYPmZ3raHSZrXrOpIy6MekZaSrifn966m0LTx6G6GZ6K/4DG4ILY3Unz+rzz7EzRff7q/Y/VQLpY1T7xQ9cCMMfzx6kFE3VsWkDlVQv2zqtCi/332D/VdffHN/A5qUxQ8109yh+CICa8+mv0r/cx1qFMegNBdqLFnii4ROlijdKbkuWqw4Aq7GA1evqWxXt6wxpnSoItQJSQrY7L7+zf2VNNaBw0+1VdqcDvyHyNgvz2sOJN+huGRIQ5U2v5P38Oht9Be3SXm/u/evCxP91L/Htlx4gr8fv/9mVosu1VG1eOoFN8duvMLxT3eZfk1h/zy1vv8MSqUSF6L+Tfd+z6nP094rz3H2XthXtkgm9ufpVUQsvI7e+eZ2Ofl5cj54Cx+/0f//ns9TCrd+dVX6/znxeVKiOqpZbER0dPLfH9cBTFp7Dvr6Bvn+85Tyc6ZKCQNYdlW9+0ldfj+p4+cpo/5MkkKJ7nVLomUV02++tliyPaAcEBCA4OBg/PPPP9DR0cGgQYMwceJE/PDDD/zDKJvaVS8GDZkk3YDqt3StZQpNXf0s36LQo16pbz8pA13qlESXOiWzvF276sXQrnrWVz1tVL4IAkY0zvJ2FU31sXliyyxvp6spw86pyQNUKYPXcoUcCTHR0NDRAyRfnqx+yZCGqQPeSkAJZboB7YyYd66GuER5hgPeaQfP9bRSv8dKpRLPf12HN7fufRqZkWL8DEvUbtD408B+6v5LfXYFZ8sqpihhpKMyIJ/RgPvnV4xWLWGAES0qQIHUkwcZnfxIe4UqAJjqa+Gn+qUyPBGQ9qRHzc+uMtfSkKJNVdMMj0XanBVM09/WW6e0ERQZnLxI+7UWyeCzVtJIBxIJ0n0PP9//53Q0pcL3J6P3QErb5zJ/MuF7t/tskD6zJwXSbfedJyEyeS9ERr8+MnV+JoMNM3v3RbqvMc2Wn38W8NljacUmyhHxjT8kgfR3X3yMS8KTTNx90TTN1EFA8tXimTl5Yait+is/Lkmeqbsv6pUxQp+GZeC6ZDkmDOwBhUIBg6Z9cfZZAvDs5Re3K2aglW5A+cC1F5k6QdOlTkmVk20H/3mJf559+OZ2y4Y1QPHUC/5x9MYrnLgZ+s3t5vWurTIl0+93w7Dpwrc795M7VEbPeqmd+6tPwuF94u43t+vTsDTGt029ovTxuxhY7Ur+QyvlM5PuanoAzSsXhWWaP5ZjE+WYvuWKyl0MaU8+pLxWeRM9OPRS/QNt3v7/VAa/052IgATamlK49q2rst2asw/x4kMMoFBAKpN9yin5bOAdmNGpmsog/bEbr3AvNOqLJy+K6GliQKOy3zx26iA3+sYtWrTArl27MGPGDGzfvh26urp4/fo1AgIC0t2p9+zZM1y6dAmRkZFCW9WqVXH8+HFMmjQJJUuWRKlSpRASEoK+ffti9Wr1uWrqY1wiXn749s+Q6PikdPWz8G+fdP/8Z3dCkiJT21UprjrVnEKJTG2nr53+T7EX4bFI+MYf7Cbx6QdXQiPiMrXPz38Pv/0Yn6ntEuWq20XEJuF5Jrb7/HddVFzmvhdRnw0gxSXKM7Xdh5jU7+HVS3/iyP69KD0xdbqLKABRGbxOyc/6xkooM7W/jLz8EIvwmK/3A7Rk6f9+9FF38gAAiCNJREFUeJPJ78XnF0C8i07I1HYJSarvq4jYxExtV9g/TzWjk7+OjN7vOfV5+hCTue+FmJ8nIDl3ZrbLyc/Tq4i4XP08fX49UY59nnRNoaWbOhD4PgF4nxCrNp8nwwz+LlaX308F6fOUmTv6xZTtAWULCws0btwYgYGBGDlyJIoUKZIDsYjyD4lEojLXNAAoFICGlgya2hpfHbgvlmYqg6woXzT9gOi3HN2/C5d+2SLUPfsPxsQBXTO1bbUSBukWd8yMysX00w0yZ0aZIrqY1L7KN5+nUMiRGJt6ZtlYVxPW3b88T+TXuH92pjSzPp86I7Nse3zfomWTO1TBxPaVM77yPs2AtK6W6vuuW92SaFut2BevvE95LRM91c5B/bLGqNasLOSJCej0Qx1IJNLPBvaT9//5e7mkkQ7sf6qVevIgTdaUjplCqYSpgZbKdpoyKab/UFV4POXkwecnWgx10v96GtSkHOKT5OnuYkj7NWvI0g/WtKlqiuolDDLMl/aYamqodkZrlDRMff0MTn6kvJb+Z3dDlDTSQZ0yRqn7Er5GJRRyBSCRQInkucXT0tdKvfviaycvPv/eSyWAnpZM9aRVBlm/9yREysBu/cbNMHb6LKxfvjTjEf/PN8tocP+7T5hk9kTEd54w+eyao8wfms/vvsjsiZYvn6BJ+/78PM3nd18oFMpMdTQ/v2MHSL47ISqDTr/qdul/v90L/YgHYd++4mf6D6rH4tarSPwR8u6Lz69T2qjADCjnVt94wIAB6Nu3L+7du4ekpCTUqFED2trp+xnTpk1D7969Ubeu6smAdu3a4c6dO3jw4AEiIiJQqVIlFC2qHnM6pjDU0USZIjrfvEL584ElfW0NlDdRXYclI8af/QGtpSHN1Ham+qq/66QSZGq7Eobpv39lTXQzdQXY50oa62TwzPQ+/7lVzFA7U1k/7wsb62qgnInuN69Q1vnsbkoDncx9Lww+6wfoaMoytV2RT/2cmOgouFrNgCIpAQlvn0AikaJchUoqU8akVeyz74UEkkztL6PjXqaIbrrf8Z/TyGAArIShdrrBprRS3u+fz6Vpqq+Vqaxan/VzjHU1M7VdYf88FX2rCaVSmeH7Pac+T0X0Mve9EOvzlEJTlrnPRU5+nkob6+TK5ynF5xe/5OTn6WNkBN6Ept5BamBgmK5vld8+Tyk/Z4qr8e8ndfw8ZdSfSVIoYaSbv9eskSgzfT9/xq5evYomTdRrhdysunnzJurVq4f//vsvXec8t3yIScDlx+FZvkI5ZQDue65QpszLb8f5zeuXGNK1LT5GJl+laFq8BHad/FNtJv//kvx2nAsq6fFfIE+Mh7LXAB7nXJTf3s9KpRJJCqUwkK8AVKYpShnIlkklwh+TCfHxGN33Rzx6EQZIAJmGBnxWbUbVmrXTDX5LJUA5E9WTY4/eRiNRrshwCqe0+29QzlhlcPju64/ClRmfD7in/FuhBJpVMIaGPE44xrdfReJ5eGzqXQzC3RSq+29VpajK3Nt3XkXixouIL54oSdlny8qmKndgPAyLwqk7bzI80ZL2340rmKjcofM6Ig5bLj758r4+Za1VylBlrv7YRDkWHL4tfP2qJ2hSv76yRXQxp5vqbcBz9/yLqPikDE9EpOxfR1OKwBGqfTznA//h4dvoNMc09fuW9mvdMKG5yuKvi0/cxZ8PvjygXLeMERx/qo2WVUzzZOGR3OzbsW+cOzLqG+e3n6uFhTocd09Ha+zZEizUc5w8MHKSmYiJsk8djntBxH6yOArC+12pVMJq8iicOXlMaFsYsAbd+w4SMdXXFYTjro4yOu6vI+JQRFczX/eNsz3cXdA7zET5nVKpxHzbWcJgMgDYeSxW+8FkIspdGd198S1a2tpwWxKEMf26QJ6UBDmAZY4zseXw/6CVwZWSn/ueOxqA9NPmfElyZyy1rl3aCLXTzP2dWbVKG6nMGZ5ZVYoboErxrN/tUcpY57vuvtDVlMFjwPfdfbHo5wbftZ17/3rf9cfG7C7VYfFj9XQnAlJORLyNSvj2i6gJ9o2JxHXp/O8qg8mNW7TG8AnTRExERIWRRCKBw0Jf/HP5EiLCk+dl9ppngyYt26J4ye+bbpQoP8nygPL58+cBJN82l7b+mpTnElHO2799I/48c0qofxowBJ179BYxEREVZLXqNcAkcyusWroIAPDg3h2s8lsE87nOIiej/Cyj21HTio6Xf/Xx/Ix9Y6L842NkJNxszIVaR1cPLosDIZV+/WcQEVFuKFaiJOwXLIad2QQAQGTEB3jYzYZf8HauOUZqL8sDyu3btweQOt9fSv012ZxVg4i+4PnTx/Cd7yTUJUqVho2bl4iJiKgwGD9jDn7/9Rju3vwXALBp5TL80LUn6jdpLnIyorzHvjFR/rHUYx5CX74Q6lkObihfsfJXtiAiyl1de/XH//oexolD+wAA50//ikO7tqLf0FEiJyPKniwPKB87duyrNRHlDYVCAVerGYiNSV0cycl7GYyMi4gXiogKBU1NTbj5BmFU705ISkyEQqGAi/VMbDv6O3R0vr0oBVFBwr4xUf5w/vRJHNyZukB1i7Yd8fOo8SImIiJKZuvujcsX/sC7sFAAwBJ3B7Ro2xGly5X/xpZE+VeWB5R79Ojx1ZqI8sa2dStw7a8LQj1o1Hi06fijiImIqDCpXqsOplraYbn3fADAkwf3EeTtgTnOC0RORpS32DcmEl/KbeQp9A0M4OwTwKkuiChfKGJSFE6L/DF7wjAAQHRUFNxszBG0dR9/TpHayvY7d8OGDdl6nIiy7tH9u1ju4yHUZStUwmwHNxETEVFhNGaqOeo2Sl2AbFvwSpUTXUSFEfvGRHnPx8UOYaGvhHqO0wKULltOxERERKra/9hNZZqLv/88i92b1omYiCh7sj2gPH78128j+tbjRJQ1iYmJcJ5jhoT4eADJq8e6LVkOPX0DkZMRUWGjoaEBtyVB0NbWAZA8L6zLZ1PxEBU27BsT5a3/nTiCo/t3CXXbH7pwblIiypfmOHmgVJqTXf6ernjyMETERETfL1evrY+Li4NMJsvNXRAVOhuC/HDr32tCPXKSGRq3aC1iIiIqzCpXqwEzG0ehfvH0MZZ58o4Jooywb0yUs8Lfv8NChzlCbWhkjHmL/CCRSERMRUSUMQNDI7j4BAp1fFwsXOaYQS6Xi5iK6PtkeQ5lALh48eJXawCIj4/HsWPHUL48Jxknyim3b1zHmmU+Ql25Wg2YWTt+ZQsiotw3fMI0/O/EEfzzd3J/YNemtejUvRdatOsocjKivMG+MVHeUyqVWGhvifdvw4Q2GzcvlChVRsRURERf16JtBwwdNwU7N6wGANy4dhmbVwdi3PRZIicjyprvGlBu3br1V+sUMpkM/v7+37MLIvpMQnw8XOZMhzwpCUDy58t96Qpo6+iInIyICjuZTAbXxYEY1qMD4mJjAACu1jOw88QfMDQ2FjkdUe5j35go7x3ZtxOnjx8W6o7dfsJPA4aImIiIKHPM7Zxx4cwpPH30AACw0tcTbTt1RfVadURORpR53zWgvH//fuHfAwYMUKlT6Ovro3bt2ihXjoshEOWElb6eeHDvjlBPNLdCnQaNRUxERJSqfKUqmGXvikXOtgCA0FcvscjZFh7+q0RORpT72Dcmyluvnj+Dt8tcoTYxLYZ5nks51QURqQVdXT24Lg7EpMG9oFAokJiQAJc5Zth44FdoammJHY8oU75rQLl///7Cv5cuXapSE1HOu375EjatChDqWvUaYsJMKxETERGlN3jMRJw9dRwXzpwGABw7sBsdunRHtz4DRU5GlLvYNybKOwqFAi5WZoj++FFom+flh6LFiouYiogoaxo2a4nRU82xcUXynUt3b/6LdYFLMG2OvcjJiDIn24vyzZ49OwdiENGXxMZEw8VqBpRKJQBAU0sL7kuDoKmpKXIyIiJVEokELj4BMDIuIrR5OlrjzeuX4oUiymPsGxPlrq1rg3Dl4h9C3W/oKPzQ7ScRExERfZ9plnaoWqOWUAcH+uLGtcsiJiLKvGwPKCuVSgQGBqJ+/frQ1dWFhoZGuv+I6PstXeCMZ48fCrWZlQOq1qgtYiIioi8rXrI07BcuEerIiA9wtZ4JhUIhYiqivMO+MVHuCbl7C8t9PIS6bPmKsHJeIGIiIqLvp6WtDfelKyH71DeQy+VwtpyO2JhokZMRfVu2B5T9/PxgbW2NZs2aIS4uDtOnT0ePHj2goaGBHj168CoNomw4e+oE9m5ZL9QNm7XEyMkzRExERPRt3XoPQM/+g4X60rnfsXvTOvECEeUh9o2JckdCfDycZk9DYkICgOS7YtyWroC+gaHIyYiIvl+teg0wdXbqnPBPHz3AUg8nERMRZU62B5SDg4MRFBSE9euTB70CAgJw+PBhPHz4EDExMZxDjug7vQt7A3cbc6HW0zeA+9IVkMlkIqYiIsqcue7eKFm6jFD7L3TBo5B7IiYiyhvsGxPljpVLvXDv1n9CPXb6LDRu3krEREREOWPs9Flo2LSFUO/dugFnT50QMRHRt2V7QPn+/fsqHWO5XA4AKFOmDAICAmBra5vdXRAVOkqlEvPnzkL4u7dCm42bF8pVqCReKCKiLDA0NobrkiChjo+PS76yLDFRxFREuY99Y6Kcd+2vC9i0cplQ16hTD9Ms7URMRESUczQ0NODutxJ6+gZC23xbC7x/GyZiKqKvy/aAcnx8PIoWLQoA0NfXx+vXr4XHKleujOvXr2d3F0SFzr5tG3EuzRnJH3v2QZ+fh4uYiIgo61q07YCRE6cL9e0b/2DtMh8RExHlPvaNiXJW1MdIOFtOV1mger7fSmhqaYmcjIgo55SrUAk2rp5C/f5tGObPnSX87CPKb7I9oJxW3bp1sWPHDqHev38/TE1Nc3IXRAXek4ch8J0/T6iLlSgFB8+lkEgkIqYiIvo+M2ydUKV6TaEODvTFjat/i5iIKO+wb0yUfb7ujnj5/KlQz7R1QrWadURMRESUO/oMHoFO3XsL9dnfjmP/jk0iJiL6shwdUJ40aRJsbGzQoUMHdOnSBWPGjMHo0aNzchdEBVpiYiKcZk9DXGyM0OayOABFTIqKmIqI6Ptp6+hgvt8qaGhqAgAUCgWcuHo1FRLsGxNlz++/HsXBXVuFummrdhiR5s4XIqKCRCKRwNFrKUyLlxTafN3n4dnjhyKmIspYtgeUr127Jvx78uTJ8PX1RVRUFN6/fw9nZ2e4urpmdxdEhcbaZT64ef2qUA8bPwVtOv4oYiIiouyrVa8Bps2xF+pnjx9i6QJnERMR5R72jYlyxruwN/Cwmy3U+oaGcFuyHFJpjl4TRUSUr5gUNYWLT+qc8bEx0Zg3eyqSkpJETEWUXrZ/Gzdq1Eilnj17Nq5evYqrV6/CxcUFmp+uSCKir7t++RKCA32Fukr1mjC3cxExERFRzhkz1RwNm7UU6r1b1uPMyWMiJiLKHewbE2WfUqmEh91slQWqbd0WoXS58iKmIiLKG207dcWQMZOE+r9rVxC83PcrWxDlvWwPKHt7eyM0NDQnshAVWtFRH+E0exoUCgUAQENTEx7+q6GjoytyMiKinCGTyeC+dIXK6tVuNuYIC30lYiqinMe+MVH27d26AWd/Oy7UP/bsg14Dh4qYiIgob1k4uKJS1epCvdbfBzeuXRYxEZGqbA8oz507F+XKlcOAAQNw+PBhyOXynMhFVKgsdrXHi2dPhNrM2hE169YXMRERUc4rV6ESbN0XCXVE+Hu4zDETTqYRFQTsGxNlz6P7d1UWqDYtXhL2C325QDURFSq6unrw8F8FmYYGAEAul8OZ65BQPpLtAeW7d+/CysoKly5dQp8+fVChQgU4ODggJCQkJ/IRFXinj/2CQ7u3CXXTVm0xavIMERMREeWe3oOGoXvfgUJ96fwZbFmzXMRERDmLfWOi75cQHw8Hi8mIj4sV2tx8l8OkqKmIqYiIxFG7fiNMs7QT6qePHmCph5OIiYhSZXtAuUaNGvDy8sKzZ89w6NAhNG/eHD4+PqhevTp++OEHbN68OSdyEhVIb16/VFlsxMDICG6+KyCTycQLRUSUiyQSCew9lqjMg7ncxwO3b/wjXiiiHMS+MdH3C1zkjnu3/hPqkZPM0LpDZxETERGJa+z0WarrkGzdgLOnToiYiChZji2RK5PJ0KdPHxw4cADPnz+Ht7c3QkJCMGbMmJzaBVGBIpfLMW/WNER8CBfa7OYvRumy5URMRUSU+wyNjeHhtwpSaXI3JCkxEQ7mkxETHSVyMqKcw74xUdZcOHsaW9etEOqadepjpi2vxCOiwk0mk2H+0pWq65BYz0RY6GsRUxHl4IByiri4OJw+fRonTpzAy5cveaUl0RdsCPLDlYvnhbpHv5/Rs//PIiYiIso7jZq3wiQLa6F++ugBFrs5iJiIKHewb0z0beHv3sJlTuqUb9o6uliwbDW0tLVFTEVElD+UrVARNm5eQv3h/Tu4zJnOdUhIVDk2oHz16lXMnDkTpUuXxogRI/D48WN4eHjg2bNnObULogLj3yt/YdXS1F8IZStUgv2CxSImIiLKexPNrdGwaQuhPrhzC347clDEREQ5h31josxRKpVwszHHu7BQoc3KeQEqV68pYioiovylz8/D0a3PAKG+dP4MtqwOFDERFXYa2X2BgIAABAcH459//oGOjg4GDRqEiRMn4ocffuBKvEQZ+BgRAQeLycKq7zINDSwMWAMDQyORkxER5S0NDQ14+K/GsJ7tEf3xIwDAw2426jVuilJlOP0PqSf2jYmyZvfmYJxLMx9ox24/YeCIsSImIiLKfyQSCRwW+OLGtct49Tz55HSgjweatm6Hug2biJyOCqNsX6FsYWEBiUSCwMBAvHr1Clu2bEGnTp3YYSbKgFKpxELHOcIvAAAws3ZEvUZNRUxFRCSeMuUrwHGhr1B/jIyA0+xpwkk3InXDvjFR5j24dxt+HqnzJBcrUQpOi/z5eSEiyoChsTEW+K8Wps+SJyXBwXwyoqM+ipyMCqNsDyhfuXIFV69exYwZM1CkSJEciERUcB3atRW//rJfqFu264gxU81FTEREJL7ufQeh98/DhfrqpT+xIchPvEBE2cC+MVHmxMfFwcF8MuLj44Q296VBMClqKmIqIqL8rWGzlpg8y1aonz95hEXOc0VMRIVVtgeUmzThpfVEmfEo5B68XeyEukhRU7j5roBUmuNrYxIRqR1bNy+Uq1hZqFct9cK/V/4SMRHR92HfmChzAha5IeTOLaEeM9UcLdv9IF4gIiI1MWHmHDRp2Uaoj+zdgWMH9oiYiAqjLM+hfP78eQBAu3btVOqvSXkuUWGVEB8PR/PJiIuNEdrclgSheMlSIqYiIso/9A0MsWDZakwY1BPypCTI5XI4WkzBtqNnYGhsLHY8oi9i35go6/78/TdsD14l1LXqNYSZtaOIiYiI1IdMJsN8v5UY3qMDIiM+AAA8HeegfpNmKFehkqjZqPDI8oBy+/btASTPBZu2/pqU5xIVVgGL3HD31g2hHjFxGtp17ipiIiKi/Kdeo6aYbuWAwEXuAICXz5/C3dYc3is3cj5NyrfYNybKmrDQ13CeYybUOrp6WLBsNTS1tERMRUSkXkqVKQcn72WwmToGABAdFQUH80lYt+cYNDU1RU5HhUGWB5SPHTv21ZqIVJ079Su2rVsp1DXr1If5XBcRExER5V9jpprj7z/O4NL5MwCA08cPY9fGtRg6brLIyYgyxr4xUebJ5XLMmzUV4e/eCm1WLgtQqWp1EVMREamnzj16Y9DIcdi7dQMA4OY/V7FqqRdm2jp9fUOiHJDlAeUePXqo1I0aNYJUKkWJEiVyLBRRQfH65XO4WKlegbEwcC20tLVFTEVElH8l38K3CsN7dsC7sDcAgKULnNCgaQvUrt9Q5HRE6bFvTJR5wYG+uHzhnFB37d0fA4aNETEREZF6s3TywLW/LuDh/bsAgA1BfmjRtiNatO0gcjIq6LK9GliZMmVw4cKFnMhCVKAkJibCYeYkRIS/F9ps3bx4BQYR0TeYFi8BD79VwjQXiQkJsJsxAVEfI0VORvRt7BsTZezqpT+x2m+RUJetUAmOnn6c0oiIKBt0dfWwMHCdcNGaUqmEs+V0hL9/J3IyKuiyPaBctmxZtG3bNieyEBUoKxYvwPUrfwn1TwOGoO+QkSImIiJSHy3adcQkC2uhfv7kERbYW3LuWcr32DcmSi/8/Ts4WkyGQqEAAGhoasIzcC0MjYxETkZEpP6q16qD2Q7uQh0W+goultOFn7lEuSHbA8qTJk3C5s2bcyILUYFx9tQJbFy5TKgrVa0O+wWLeQUGEVEWTJ5li6at2gn1r7/sx/7tG0VMRPRt7BsTqVIqlXC1moE3r18JbRZ2LqjbsImIqYiICpYhYyehY9eeQv3H779h8+pAERNRQZflOZQ/16dPH3h6euLGjRvo27cvypYtm25FyUaNGmV3N0Rq49WL53BNs3K1to4uFq1YDz19AxFTERGpH5lMhgXLVmF4z47CAk6LXR1Qv3FzVK9dV+R0RBlj35hI1da1QTh/+lehbv9jd4yYOF3EREREBY9EIoHL4kAM79kBoS9fAACWe89Hw6Yt0Kh5K5HTUUGU7QHlpk2bCv9ev359hs/h7alUWCTPmzwRER/ChTa7+d6oVrOOiKmIiNRX8ZKl4b50BczHDAYAxMfHYe6MCdjyyymeqKN8iX1jolT//XMFAYtSb8MuUao0XBYH8q49IqJcYFzEBF6B6zBpSG/Ik5Igl8vhYD4J246dRRGTomLHowIm2wPK27dvz4kcRAXCcu/5+Pfq30Lda9Aw9Bk8QsRERETqr03HHzHObDY2BPkBAJ48uA8vJ1u4+waJG4woA+wbEyX7GBkJh5mTkJSYCACQSqVYGLAWJkVNRU5GRFRwNWjaAjNtneC/0AUAEPrqJVzmmGHpum2QSrM96y2RINsDysOGDcuJHERq7+xvx1XmKKpcrQbsPXx4BQYRUQ6YbuWAa39dwPXLlwAAR/buQNOWbdBv6CiRkxGpYt+YKPkqfA+7WXjx7InQNtXSDo1btBYxFRFR4TBq8gxcufiHMN3Q+dO/YsvqQIyZZiFyMipIcuT0RFJSEvbs2QNzc3OMGJF6NebRo0eR+OmMNFFB9ur5M7ikmTdZR1cPi1ash66evoipiIgKDg0NDSwMWAPjIiZC2yInW9z5718RUxFljH1jKuz2bFmP344cFOrmbTpg/AxLERMRERUeUqkUbr5BKFm6jNAW6D1fuDCDKCdke0A5MjIS7du3x+DBg7FlyxaV2/x27tyJ3bt3Z3cXRPlaYkIC7GZOQGTEB6HNzsMHVWvUFi8UEVEBVKpMObgvXSnU8fFxsJ02VuXnL5HY2Demwu7m9atY4u4g1CamxTDfbyVkMpmIqYiICpciJkXhGbhO+Nkrl8thbz4JH8Lfi5yMCopsDyg7ODggMjISly9fRnh4uMpjEydOxJo1a7K7C6J8zW+hM/67dkWo+wwejj4/DxcxERFRwdWuc1dMNLcS6hfPnsDZcjoUCoWIqYhSsW9MhVnEh3DMNRuPxIQEAIBEIoGH/yoUL1lK5GRERIVPw2YtMcPWSahDX76Ayxwz9pspR2R7QHnfvn1Yv369yorWKerUqYMrV65ksBVRwXD84F7sWL9aqKvWqIW5831ETEREVPBNtbRDy3YdhfrcqRPCgn1EYmPfmAorhUIBZ8vpePX8mdA2xXIuWrXvJGIqIqLCbfSUmWj7QxehPn/6V2xZs1zERFRQZHtAOSwsDHXq1BHqtAuQaWhoIOHT2Wmigibk7m142KXOBadvYIBFKzZAV1dPxFRERAWfTCbDgmVrVOaFW7FkIS6dPyNiKqJk7BtTYbVhhb+wABQAtO7YGZPMrUVMREREUqkUbktXoESp0kJb4CJ3XPvrgoipqCDI9oBy6dKlce3aNaFO22m+ePEiKleunN1dEOU7HyMj4DBrOuLjYoU218XLUblaDRFTEREVHiamxbAoaD00NDUBJF8ZN2/WVLx5/UrkZFTYsW9MhdHff57DisULhLpk6TKY77cKUmmOrAFPRETZYFLUFAs/m095rtkEhIW+FjkZqbNs/4bv378/zMzMEBISotL+5MkTWFlZYcCAAdndBVG+olAo4DJnBp4/fSy0jZ1mgc49+4gXioioEKrfpDmsnFIHMD68fwdHSzNh7k4iMbBvTIVNWOgrOJhPEubklGloYFHQepgUNRU5GRERpWjcvBVmznUW6ndhobCfORGJiYkipiJ1lu0BZRcXF8THx6N27dpo3rw5lEolOnXqhNq1awMA7O3tsx2SKD/ZEOSHc6dOCHWz1u1hZjNPxERERIXX4DET0aPfz0J98/o1+C90ETERFXbsG1NhkpSUBPuZk/D+bZjQZjlvPuo3aS5iKiIiysjoKTPxY5oL4a79dQGBi9xFTETqLNsDyqamprh06RIsLS0RFRUFY2NjPH/+HNOmTcP58+dhaGiYEzmJ8oULZ08jKM3tfCVKlYZn4FpoaGiImIqIqPCSSCSY57UUVWvUEtp2blyL4wf3ipiKCjP2jakwCfLxUJmHs0uvfhg2boqIiYiI6EskEgmcfQJQsWp1oW3LmuX47chBEVORusqRSa1MTEzg7e2N27dvIzw8HPfv34evry9MTExy4uWJ8oVXz5/B0WIKlEolAEBDQxNeQcEoWqy4yMmIiAo3XT19eK/cCD19faFt/txZuH/nloipqDBj35gKgzO/HsPGlcuEumKVanBa5K8ybzgREeUvBoZG8Fm5ATq6ekKbm405HoXcEzEVqaN8u0qCXC7Hhg0bMG7cOIwaNQp+fn6Ijo7+rtdavXo1+vfvDwcHhxxOSYVFfFwcbKaNRUT4e6FttoMz6jduJmIqIiJKUalqdTh7pw5sxMXGwGryKER8CBcxFRFRwfTs8SO4Ws8Uam0dXSxasQEGhkYipiIiosyoWqM2nBb5C3VMdBRspo5BTHSUiKlI3WT5Pv0tW7ZkeSejRo3K0vPj4+Px008/4c6dO7C3t4e+vj58fX2xevVqnDt3DqammV/g4Z9//oG5uTnkcjnevn2b1ehEUCqVWORsi9s3/hHaeg0cggFDs/a+JiKi3PXjT30xcsJUbA1eBQB48fQx7GdOxLINuzg1EeWavOgbE+Un0VFRsDOfgqiPkUKbw8IlqF6rjoipiIgoK3r0G4R/r/6NnRtWAwAehdyDh50lXBctFTkZqYss/3U1evToLO8kq51mf39/nD59GleuXEGTJk0AAL169UK1atVgb2+P1atXZ+p14uPjMXr0aIwcORJHjx7Ncm4iANi9ORgHd6b+sVijTj3YefhAAqWIqYiIKCPTLG0Rcu8uLp3/HQBw6dzvWO49H7Mc3ETNRQVXXvSNifILpVIJdxtzPHpwX2gbNHIceg8aJmIqIiL6HpaO7rj97zX8e/VvAMDJwwdQt35DjJwy8xtbEn3HgPKNGzdUarlcDg8PD5iYmKBfv34oWbIkQkNDceDAAXz48AFOTk5ZDhUcHIxmzZoJg8kAUKJECfTv3x9bt26Fv78/dHV1v/k68+bNw9u3b7FkyRIOKNN3uXzhPBa7pa7GbmhkDO+VG6Gjq4fE2O+bgoWIiHKPTCbDgoDVGNuvG148fQwA2LQqADXrNkCPfoPEDUcFUl70jYnyiw0r/HH6+GGhbtCkOWxcvURMRERE30tTSwuLVqzHyF6d8P5tGAAgwGcB6jVpjsYt2oicjvK7LM+hXK9ePZX/duzYgY4dO2L16tXo1asXmjVrhl69emHNmjVo3749du/enaXXj4iIwN27d1UGk1M0bdoUMTEx6TruGTl79ix8fX0RGBjIBVDou7x89hRzzcZDnpQEAJBKpfBcvg7lK1YWORkREX2NcRETLFmzRWWxEXdbC9z5718RU1FBldt9Y6L84s8zp7Dce75QmxYvAe+VG6CppSViKiIiyo4SpcpgYcBaSKXJw4PypCTMNZuAN69fipyM8rtsTygYHByMO3fuZPjYqFGjUKdOHcyfPz/DxzPy4sULAEDJkiXTPVaqVCkAwMuXX39jf/z4EWPHjkX//v0xaFDWrkZ68+YNwsLCVNpCQkIAJF9xIpfLs/R630sul0OpUEApARSKzI/7KxWftlPIocjFfAVdbEw05kwaiQ/v3wltFvauaNmuIxQKOY9zHuFxzhsSpQJKhZLHOZfx/Zz70h7jqjVqwnVxAOxmTAQAxMfFwmryKGw6dBImpsVETqrecuO9nPJ6crkcUEpy6FW/LDf7czndNybKD54/fQxH88lQKpOnfNPQ0MSioGAUL1la5GRERJRdzdu0x8y5zljm6QoAeBf2BtZTxmDNrsPQ1tERNxzlW9keUI6MjERoaGiGVwGHhoYiMjIyg62+LCEhITlYBovnaGpqAkieG/lrZs2ahQ8fPmD58uVZ2jcABAUFwc0t43kWo6KiEBERkeXX/B7RcYlQxEdDLpMgMQvfJqVCgaT4WACARJrlC9AJyXPDuVjOwP07N4W2Hn0HYvCI0cI0FzzOeYPHOY8kJECelAB5bAyPcy7i+zn3fX6MO3T6EWOmzMCm1cn9gdcvn8NuxgT4rd4EjU99Csq63HgvKxMSkCTRQGSkBqSS3B9QjorKvVXMc7pvnNbDhw9x9uxZJCUloVmzZmjUqNF3vc6lS5dw4cIFlC1bFoMHD/7uPFQ4xMZEw3rKaERGfBDaLB1c0LBZS/FCERFRjhoz1Ry3rl/Fb0cPAQBuXr+KhY5WcF0cCEke9M1I/WR7QLlNmzaYPn06du/ejWLFUq/4CQsLw7Rp09C2bdssvZ6BgQEAICYmJt1jKZ1/Q0PDL25/9OhRrF+/HuvWrROuaM4KMzOzdB3rkJAQ9O/fHwYGBjA2Ns7ya34PpWYCpO8VkMkATd3M/9GrVCRfcaOpqweJVJZb8Qq04EBf/O/X1Dm36zRoDMdFftDSSZ23m8c5b/A45w3Zp1tVpTzOuYrv59yX0TE2s3XCg/t38cf/fgMAXL10Act9F8Ha1VO0nOouN97LkgQNaOhowMjIGDJp7v/RktLfzA053TdO4eLiAi8vL/Tp0wf6+vqwtLTEwIEDERwcDJks89+HsLAw9OvXD6GhoWjbti0HlOmrlEol5s+dhfu3Uy+06DtkBPoPHSliKiIiymkSiQTOPsvw+ME9hNxNvtPq8J7tqFWvAYaPnypyOsqPsj2g7Ovriw4dOqBChQpo1aqVsPDIxYsXoaWlhXPnzmXp9SpWrAgtLS08fvw43WMpbdWrV//i9n/99RdkMhkOHTqEQ4cOCe0fPnzA7du3hWkwvrQid4kSJVCiRIkMH5PJZFnqsGeHTCaDRCqFRCqBNAt/rCmAT9vJsrQdJTtz8hhWLEkdZDAtXhJL1myGnp7qH548znmDxzlvSCTJP2vA45yr+H7OfRkdY6lUhgX+azCmXxc8ffQAALBz41pUrVkHg0aOEy+sGsuN93LK68lksjwZUM7N/lxO940BYO/evXB3d8eWLVswcmTyQN6MGTPQpk0b1K5dG3Z2dpl+ralTp6Jq1apQKDj5Dn3b1rVBOHFon1DXbdgEtu6LIFHkzTSARESUd3T19OEVsAYTh/ZDRPh7AMDS+fNQtUZttGjbQeR0lN9k+z7Fhg0b4tq1axg5ciSePHmCY8eO4cmTJxg1ahSuXbuG+vXrZ+n1NDU10bFjR5w7dy7d/HanT59GpUqVvjqgPGzYMOzZswfjxo1T+U9XVxclS5bEuHHj0LRp0+/6Wqlge3jvDubNSj3zpqmlhcWrNqJEqTIipiIiouwyNDaG75ot0E9zVeoiJxv89cdZEVNRQZXTfWMAWLp0KapUqSIMJgNAixYt0L17d/j7+2d6cHjjxo04evQo1q5NXXyH6Ev++uMs/Be6CHXRYsXhvXIDtLU5nyYRUUFVplx5LAxYI5x8l8vlsJsxAS+fPRU5GeU3OdKTrFKlCtasWYMHDx7gw4cPePDgAVavXo3KlSt/1+vNnTsXr169wtKlS4W2w4cP48yZM7C3t1d57rZt29C/f3/cvn0bAFCrVi30798/3X/a2tooWrQo+vfvjzp16nz/F0sFUvi7t5g9YThiolPnVHRYsAQNmrYQMRUREeWUytVrwjNwXeoK1nI55k4fhycPQ0RORgVRTvaN4+LicPHixQynymjfvj1ev34t9IO/5unTp7CwsMC8efNQu3btLOegwuXZ44eYO32ccLJCJpPBa3kwSpUpJ3IyIiLKbS3adsBsR3ehjgh/D6spoxAbEy1iKspvsj3lRW748ccfsWrVKlhaWmLnzp3Q09PDpUuX4OjoiClTpqg899atWzh48CCsra1FSkvqLiE+HlZTRuPFsydC27DxU9B3COeGIyIqSNp26grLefOxxN0RABAZ8QGWE0dgw4FfYWRcRNxwRF/w9OlTyOVylCuXfiCvfPnyAJIX66tbt+4XX0OpVGLs2LGoVKkS5s6dm6X9v3nzBmFhYSptISHJJ2Lkcnm6Owpzi1wuh1KhgFICKBTJJ4aUik9tCjk4gUfOiYqMhOXEESqL8M12dEfjFq2gUMh53EXC4y4OiVIBpULJ457H+H4XR9rjPnTcZNy5+S+O7N0JALh36z+42ZhjwbLVXKQvh2X0fk+p5XI5oMz94/09/bl8OaAMAJMnT8awYcNw+fJlJCUloVGjRihevHi6540cORLNmjX75lXHGzZsgL6+fm7FJTWVstDI9cuXhLbWHTvDcp6HiKmIiCi3DJ8wDY9C7mHfto0AgCcPQzB3+ngs27gLmpqZXwSXKK9ERydfDaSrq5vusZS2lOd8ydKlS3Hu3DlcvHgxy+/zoKAguLm5ZfhYVFQUIiIisvR63ys6LhGK+GjIZRIkfvoTRqlQICk+FkDyPNyUfXK5HA7mk/Ao5J7Q1nvgEAwcOgKJscnvMx53cfC4iyQhAfKkBMhjY3jc8xDf7+L4/Lhbz3PDw3t3cPvGdQDAycMHUK1GTYyeNF3MmAVORu93ZUICkiQaiIzUgDQPBvCjoqK+/aTP5NsBZQAwNDREp06dvvqc2rVrZ+q2vR49euRULCpA1gUswdH9u4S6SvWa8AwMhoZGvv5oEBHRd5JIJLB198aThw9w5eJ5AMBff5zBYld72Hn48IoLyndSBo3j4+PTPZbSltFgc4pbt27B0dERlpaWaNasWZb3b2ZmhsGDB6u0hYSEoH///jAwMICxsXGWX/N7KDUTIH2vgEwGaOomD4orPy0Mp6mrBwkXO80RQQtdceHs/4S6UbOWsF/oCy1tbaGNx10cPO7ikGlpAQCkPO55iu93cXx+3DV19eGzahPG9O2C92+T71ZaudQb1es0QPsfu4kZtUDJ6P0uSdCAho4GjIyM82TBaoM0a81kFkfNqND69fB+rFiyUKhNTIvBL3gHDI2MRExFRES5TVNTE94rN2Bc/2549vghAGDPlmBUqV4TQ8dNFjkdkapy5cpBIpHg5cuX6R5LaatQocIXt9+9ezcSExNRtGhR+Pn5Ce0xMTF4+fIl/Pz80Lx58wznaAaAEiVKoESJEhk+JpPJhEV7cptMJoNEKoVEKoH00x9bCuBTm0xoo+/3y57t2LJmuVCXKlsOPqs2QUdXT+V5PO7i4HEXh0SS/HMHPO55iu93cWR03EuXLQ+flRsxdXg/JCUmQqlUYt6sqQjeewzVa395ui3KvIyOe0otk8nyZED5e/pzvHeACqUb1y7Ddc4ModbS1obvmi0oW6GiiKmIiCivFDEpiqXrtsEgzUnExW72+PP330RMRZSegYEBGjZsiL/++ivdYxcvXkSRIkVQr169L27fsmVLzJw5E6GhoXj8+LHwn1wuR3x8PB4/fozw8PDc/BJIDVy/fAkL7C2FWldPH0vXbkPRYumnHCQiosKlUfNWsJvvI9Qx0VGYPXE43oW9ETEViY1XKFOh8+r5M1hNHoX4+DihzcUnEA2athAxFRER5bXK1WpgUdB6WIwdArlcDoVCgblmE7B29xHUrFtf7HhEgmnTpmHatGn47bff0KVLFwDAgwcP8Msvv8Dc3FxlXuQLFy7g0qVLGDJkCMqUKYMePXpkOPXbjh07ULlyZZWrlqlwevXiOaynjkFiQoLQNt9vJWrU+fKJCiIiKlwGDB+DR/fvYuu6FQCA1y+ew3rKaKzcfhDaOjoipyMx8AplKlSiPkamO5M2ZbYtevQbJGIqIiISS6v2nWDt6iXUMdFRmDV+GF6/fC5iKiJVkydPxsiRIzFw4EDY2NjAxcUFHTp0QMuWLdMtmHfkyBFYWlri4cOHIqUldRIbEw2rSSOFuTEBwMzaEZ269xIxFRER5UezHN3RrnPq3Mn/Xv0b7rYWUCqVIqYisXBAmQqNxIQE2E4fh5A7t4S27n0HYcrsuSKmIiIisQ0ZMxGjJqdOgxQW+goW44biY2SkiKmIUkmlUmzZsgUHDhyAnp4ekpKSEBgYiP/973/Q01Od37ZNmzaYNWsWypYt+9XXnDJlCoYMGZKbsSmfk8vlcLSYgru3bght3fsOxISZc0RMRURE+ZVMJsPCgDWoVquO0Hb84B6sXbZYxFQkFk55QYWCUqnEfLvZuHTud6GtQZPmcPEJgESS+xOcExFR/jbLwQ2vXjzDqaOHAAAP7t7G3Olj4b9hl8p0AkRi6ty5Mzp37vzV5/z000/46aefvvla7u7uORWL1JTfAiecOXlMqOs0aAxn9o2JiOgr9A0M4bduO8b06yLc3bLS1xMVq1ZDt94DRE5HeYlXKFOhsGLJQhzZu0Ooy1eqAt+1WznXDxERAUi+AtR96Qo0aNJcaLt0/gwW2M/mbXxEVODs3LAG29atFOqSZcrCd+0W6OjoipiKiIjUQely5bFk9WZoaWsLba5zZuDm9asipqK8xgFlKvD2bt2AdQFLhNrEtBgCNu2GiWkxEVMREVF+o6OjC99121C+UhWh7Zfd27Fmmc9XtiIiUi/nTv2KxW72Qq1vYAD/4B0oXrK0iKmIiEidNGjaAs7eAUIdHx8Hy4kj8er5MxFTUV7igDIVaGdPnYDXPGuh1tbRhV/wdpSvWFnEVERElF+ZFDVFwMZdMDYpKrSt8vXCL3u2i5iKiChn3L15A/YzJ0KhUABIng/TK2g9qteuK3IyIiJSNz37/4xJFqnjLe/CQmExbgg+RkSImIryCgeUqcD6758rsJ+R2mGWSqXwWr4O9Ro1FTkZERHlZ+UrVYHfum3Q1k6dFmn+3Fn488wpEVMREWVP6KsXmDV+GGJjooW2ufN90KbjjyKmIiIidTbV0g7d+qTOnfzw/l1YTRmNhPh4EVNRXuCAMhVIzx4/xOwJwxEXGyO02XksRocuPURMRURE6qJB0xaY77dSWJxKnpQE22nj8N8/V0RORkSUddFRH2E5YQTCQl8JbaOnzMSgkePEC0VERGpPKpXCdfFyNG7RWmi7cvE83G0tuA5JAccBZSpw3oW9gfnYIQh/91ZomzBjDjvMRESUJT/+1BdznBcIdWxMNCzGDcWjkHsipiIiypqkpCTYz5yEu7duCG2de/SGhb2reKGIiKjA0NbRwZI1W1CxanWh7diB3VixeKGIqSi3cUCZCpSPkZEwHzMYzx4/FNp6DRwKMxtHEVMREZG6GjFhGsaZzRbqiPD3mDF6EEJfvRAvFBFRJimVSix0mIM//ndSaKvbsAnc/VZCKuWfgkRElDOMi5ggYMMuFC1WXGhbF7gE+7dvEjEV5Sb2IqjAiIuLheXE4SpXX7Rs1xFOi/yFW5aJiIiyaqatE/oNGSnUoS9fYObonxHxIVzEVERE37ZqqRcO7twi1KXLlYfv2q3Q1dUTMRURERVEZStUhF/wdmjr6Aptno5W+PP330RMRbmFA8pUICQmJsJ+xkRc++uC0Fa3URMsXr0ZmlpaIiYjIiJ1J5FI4OC5FB27/SS0Pbx/F7PHD0Nsmrn6iYjyk71bN2CNv49QGxcxQeCmPShWoqSIqYiIqCCr27AJPAPXCnfByOVyzDWbgDv//StyMsppHFAmtadQKDDf1gJnfzsutFWuVgPLNuyCnr6BiMmIiKig0NDQwMKANSoLjvx79W/MnT4eiYmJIiYjIkrvfyeOwGuetVBr6+jCL3g7KqWZ35KIiCg3dOzaEzauXkIdEx0Fi3FD8PzpY/FCUY7jgDKpNaVSiaUe83Bk306hrVTZcli+ZS+KmBQVMRkRERU0Ojq6WLp2G6rXriu0/fG/k3C3MYdCoRAxGRFRqn/+vghH88nCzyWpVArPwLVo0LSFyMmIiKiwGDJ2EkZPmSnU78LeYMaoQXgX9kbEVJSTOKBMai040Bfb1q0UahPTYgjasg8lS5cVMRURERVUhsbGCNi4G2XKVRDaju7fBS8nGyiVShGTEREBj+7fheXEEYiPjxPa7BcuQceuPUVMRUREhZGFvSt69PtZqJ8/eQTzMYPxMTJSxFSUUzigTGpr9+ZgBC1eINT6BgYI2LgbFatUEzEVEREVdMVLlsLyLXtVVrHeu2U9/BY4c1CZiETz5vVLzBw7GJERH4S2qZZzMXD4WPFCERFRoSWVSuG6OBCtO3YW2u7eugGryaMQHxf3lS1JHXBAmdTSoV1bVeeF09bB0nXbUbt+QxFTERFRYVGhclUEbdkHI+MiQtuWNcux2m+ReKGIqNCK+BCOmWMG4/WL50LbgOFjMHmWrYipiIiosNPU0oLPyo2o17ip0Hbl4nnMmzUFcrlcxGSUXRxQJrVz7MAeuNtaCLVMJoPn8nVo2qqtiKmIiKiwqV67LgI27Ya+QeoCsKv9vLFp5TIRUxFRYRMd9RHmYwfjwd3bQlvHrj1h57EYEolExGRERESArp4+/NfvVFkY9vTxw/CaZ827+9QYB5RJrZw6egguc6YLP3SkUinm+63kvHBERCSKeo2awi94B7R1dIU2f09X7Nq0TsRURFRYxMfFwWryaNz856rQ1rBZSywIWAMNDQ0RkxEREaUqYlIUgZv3omTpMkLbvm0bsdLXU8RUlB0cUCa1cfa347A3n6RyW4SzTwC69x0kYioiIirsmrRsg8WrN0FTS0toW+Rkg1/2bBcxFREVdElJSXCwmIy//zwrtNWsUx/+wTugq6snYjIiIqL0Spcth8BNe1SmjFu7bDG2rg0SLxR9Nw4ok1q4cPY0bKePgzwpSWhz9PRFn5+Hi5iKiIgoWZuOP8JreTBkMpnQ5m5jjhOH9oqYiogKKoVCAXdbC/x+4ojQVrFKNQRu3gNDY2MRkxEREX1ZlRq14L9e9e4+3/nzsG/7RhFT0ffggDLle5cvnIfV5NFITEgQ2qxdPTFwxDjxQhEREX3mh24/wX3pSmHOUoVCgXmzpuLXX/aJnIyIChKlUokl7g44sneH0FayTFks37IPRYsVFzEZERHRtzVo2gI+KzdAQ1NTaFtoPwfHDuwRMRVlFQeUKV+79vdFzJ4wHPFxsUKbhb0rho+fKmIqIiKijPXoNwjzvPyEWqFQwNFiCq9UJqIcs9pvEXasXy3UJqbFsGLrfpQuW07EVERERJnXtlNXLFy2BlJp8rCkUqmEy5zp+F+aO28of+OAMuVbVy7+AfMxgxEbEy20TZ1jh7HTLERMRURE9HX9h42Go6evUKdcqcxBZSLKrs2rA7Haz1uo9Q0NEbhpDypWqSZiKiIioqz78ae+cFkcKNRyuRz2MyfiwtnTIqaizOKAMuVLf/1xFuZjh6gMJo8zm43JFjYipiIiIsqcgSPGZTiofPwgB5WJ6PtsX78KfguchVpbRxf+63eiVr0GIqYiIiL6fr0HDYOdx2KhTkxIgNXk0bj21wURU1FmcECZ8p2L5/6H2eOHqUxzMX6GJWbaOgnzUhIREeV3GQ0qO83moDIRZd3uzcFY7Gov1JpaWli8aiMaN28lYioiIqLsGzx6AmY5uAl1fFwsZo0fipvXr4qYir6FA8qUr/z5+2+wnDgC8fFxQtvkWTaYYTOPg8lERKR2OKhMRNl1YMdmeM2zFmqZhga8V2xAmx+6iJiKiIgo54yZao7Js1LvSI+OisLM0T/j9o3rIqair+GAMuUb5079ijmTRyEhPl5omzrHDtPm2HMwmYiI1NbAEePg6LVUqFMGlX/Zs13EVESkDg7v3QEPu9lCLZPJ4BW4Dh269BAvFBERUS6YammHkZPMhDoy4gPMRg7Anf/+FTEVfQkHlClfOHPyGKynjkZiQoLQNsNmHqbMshUxFRERUc4YOHxsukFlV6sZ2LVxrYipiCg/O35wL9ysZ0KpVAIApFIpPPxXoXPPPiInIyIiynkSiQSW8+Zj0KjxQlvKoPLdmzdETEYZ4YAyie7Eob2wmTYWSYmJQpuFvSsmzJwjYioiIqKcNXD4WDgt8le562aRsy02rPAXMRUR5UcnjxyAs+U0KBQKAMl/ZLstCUK3PgNFTkZERJR7JBIJ7Ob7YOCIsUJbxIdwTB/Rn4PK+QwHlElUe7dugKPFFMiTkoQ2y3nzMXaahYipiIiIckf/YaPh4b8aMplMaAvwckOQzwLhKkQiKtxOHjmQ3D+Wy4U2Z+9l+GngEBFTERER5Q2pVAr7BUswYPgYoY2DyvkPB5RJNBtW+GOhwxyVP6BtXL0wavIMEVMRERHlrh79BsF75UZoamkJbesCl2CJuwMHlYkKuaP7d8Nh5iSViy0cPX3Rd8hIEVMRERHlLalUCoeFvug/bLTQljKofO/WfyImoxQcUKY8p1QqEbDIHQFebkKbTCaDm28Qho2fImIyIiKivPFDt5/gF7wdOrp6Qtv24FWYP3eWylWJRFR4/LJnu8o0FwAw190bA0eMEy8UERGRSKRSKRw9l6YbVJ7GQeV8gQPKlKcUCgU851ljQ5Cf0KappYVFKzag96Bh4gUjIiLKY63ad0Lg5j3QNzQU2g7u3AJHi8lIiI8XMRkR5bV92zeqLMAnkUjg6OmLIWMniZyMiIhIPCmDyv2GjhLaIsLfY+qwvrh5/aqIyYgDypRnEhMTMW/WVOzdsl5o09XTh//6HejUvZeIyYiIiMTRuHkrrNp+EMYmRYW2k4cPYNb4YYj6GCliMiLKK7s2rsUCO0uVwWRnnwBemUxERITkQeV5Xn4qg8qRER8wfUR/XPv7oojJCjcOKFOeiImOgtXkUThxaK/QZmRcBCu27kPLdj+IF4yIiEhktes3wppdv8C0eEmh7a8/zmDK0L54F/ZGxGRElNu2rg3CImdboZZKpXBfuhJ9B48QMRUREVH+kjKoPHDEWKEtOioKM0f/jEvnfxcvWCHGAWXKde/fhmHqsH74438nhTbT4iWxetdh1G/SXMRkRERE+UPVGrWxft9xVKhcVWi7e/NfjB/YA88ePxQxGRHllg0r/OE7f55Qy2QyLAhYg58GDBYxFRERUf6UslDfiInThLa42BjMnjAcZ0+dEDFZ4cQBZcpVzx4/xPiBPXDr32tCW9nyFbFu71FUr1VHxGRERET5S9kKFRG89xjqNmwitL14+hjjB/bA7Rv/iBeMiHKUUqnEMi83lQWqNTQ14RW0Ht16DxAxGRERUf4mkUgwx2kBJs60EtoS4uNhPWU0Th45IF6wQogDypRr/vvnCsYN6I7nTx4JbbXqNUTwvuMoX7GyiMmIiIjyJxPTYli5/QBad+wstIW/e4spQ/vi4rn/iZiMiHKCXC7HQoc52LjCX2jT1NKCz6pN6Nyjt4jJiIiI1INEIoGZjSNm2DoJbfKkJDjMnIRf9mwXMVnhwgFlyhVnT53AlKF98eH9O6GtdcfOWL3zEIqVKPmVLYmIiAo3PX0DLF23Hb0GDhXaYqKjYDFuKI7u2yViMiLKjsSEBDhaTMa+bRuFtpQFqjv82F3EZEREROpnwgxLWLksFGqFQgFXqxnYsX61iKkKDw4oU47bt20DrCaNRHxcrNDW++fhWLpuO/QNDEVMRkREpB40NTXhumQ5xkw1F9rkSUlwspyGVUu9oFQqRUxHRFkVGxuDOZNG4uThA0KbkXERrNi2nwtUExERfacRE6ZhnpcfJBKJ0ObjaoegxQvYX85lHFCmHKNQKBDoPR8L7OdAoVAI7RPNreC6OBCampoipiMiIlIvUqkUsxzcMMfJQ6V9tZ83nGZPQ0J8vEjJiCgrPkZEYMaoQfjzzCmhrViJUliz+zDqN24mYjIiIiL1N2D4GMz3WwWZTCa0rQtYgoUOcyCXy0VMVrBxQJlyRGxMNOZOH4f1y5cKbSkrcJpZO6qcLSIiIqLMGznJDIuC1kNbW0doO3ZgN6aPHIDwNFNLEVH+8y7sDaYM64Prly8JbWUrVELw3mOoVpMLVBMREeWEnv1/hu/abdDW0RXa9m3bCDuz8YiPixMxWcHFAWXKtjevX2LS4N44ffyw0KatowufVZswaOQ48YIREREVEF169cOqnYdQtFhxoe2fvy9iXP9uePzgvojJiOhLHj+4j/EDuuPerf+Etmq16mDdnqMoW6GiiMmIiIgKnnadu2Lltv0wMi4itJ0+fhjmY4fgY2SkeMEKKA4oU7bcvnEdY/p2xZ3/rgttxUuWxro9R/BDt59ETEZERFSw1G/cDBsPnESV6jWFtudPHmH8gO64fOG8iMmI6HP/XvkLEwb1xItnT4S2+o2bYfXOX1C8ZCkRkxERERVcDZq2wNo9R1CiVGmh7crF85g6tA/ehb0RMVnBwwFl+m6nj/2CSYN7ISz0ldBWq15DbDp0ErXrNxIvGBERUQFVpnwFBO89jpbtOgptkREfMGP0IOzbvlHEZESU4n8njmDa8P6ICH8vtLXt1BVBW/fBuIiJiMmIiIgKvqo1aiN43wlUqlpdaLt76wbGD+yBJw9DRExWsHBAmbJMqVQiePlS2Ewbi7jYGKG9c4/eWLv7MEqUKiNiOiIiooLN0NgY/ht2YcDwMUJbUmIiFthZwtPRGokJCSKmIyrcdm1aB9tpYxEfnzpfY7+ho+C7div09A1ETEZERFR4lC5bDmv3HEW9xk2FthdPH2P8gO649vdFEZMVHBxQpiyJjYmG/cyJWO49X6V9wow5WLRiA3T19EVKRkREVHhoamrC0XMpZjm4qSx8u2dLMMxGDcT7t2EipiMqfBQKBZZ5uWGRkw0UCoXQPtVyLpwW+UNDQ0PEdERERIWPSVFTrNx2AG06/ii0RXwIx/QR/XHi0F4RkxUMHFCmTHv25BHGDeiOk4cPCG2aWlpwX7oCM2znQSrl24mIiCivSCQSjJlqDr/gHTAwMhLar176E6P6dMbtG9e/sjUR5ZSE+Hi4zDHDxhX+QptMJoOz9zJMmT1X5aQPERER5R1dPX34rtuG/sNGC22JCQlwMJ+M4OVLoVQqRUyn3jgCSJny5++/YXTvzgi5c0toMy1eAiu2HUCvgUNFTEZERFS4tevcFRsPnETFNPPEhb58gYk//4TjB3n1BVFuCn//DtNHDsDR/buENl09fSxdtw39ho4SMRkREREByXf2zfPyw8y5zirty73nw8NuNhITE0VKpt44oExfpVQqERzoC4txQ/ExMkJor9e4KbYcPo3GzVuJmI6IiIgAoFLV6th44CTade4mtMXHxcLRYjL8F7ogKSlJxHREBdPDe3cwtm8X/JNmLsaixYpj9c5DaNupq4jJiIiIKC2JRILxZrOxMGANNLW0hPYDOzZj9oRhiPoYKWI69cQBZfqi6KiPsJ02Fst9PFRuAxgwfAzW7OTie0RERPmJoZERfNduxYQZc1TaN60KwPQR/REW+lqkZEQFz59nTmHcwO548eyJ0Falek2s338CdRo0FjEZERERfUn3voOwYut+GBcxEdounv0fJgzsgedPH4sXTA1xQJkyFHL3Fsb07YLTxw8LbRqamnD09MU8Lz9oaWuLmI6IiIgyIpPJMMN2HryCgqGjqye0X730J0b2+gGXL5wXMR1RwbBzwxrMGjcU0R8/Cm2tO3ZG8L4TKFehknjBiIiI6Jsat2iN9ftPoFzFykLbg3t3MKZvF/aVs4ADypTOoV1bMaZvVzx+cF9oK1aiFNbs/AUDR4wTLxgRERFlStde/bFh/wlUqFxVaHsX9gbTR/TH+iA/KBQKEdMRqaekpCQscrKFt8tclc/Q0HFT4Be8A4ZpFsckIiKi/KtilWrYsP8EGjZtIbRFhL+H2aiB2LNlvYjJ1IdaDChnZ9VFrtiYebEx0XCxmgE3G3PEx8UK7Q2btcSWw6fRIM0HjYiIiPK36rXrYtOhU/ixZx+hTaFQIHCRO+ZMGonIiA/ihSNSMxEfwmExbgh2bVortMlkMsyd7wNbNy9oaGiImI6IiIiyysS0GFZuP4g+g4cLbfKkJHg6WsFrng0X6/uGfDugfPPmTfTo0QO6urrQ0dFBq1atcObMmUxt++eff2L8+PGoVKkStLS0UK5cOYwZMwaPHj3K5dTq6+Gny/sP79mu0j5mqjlW7TiE4iVLiZSMiIiIvpehkREWrdgAK+cFkKUZ8Dp36gRG/vQDbl6/KmI6IvVw79Z/GN27My6d+11o0zc0hP+GnRgyZqJouYiIiCh7tLS14eITiDlOHpBKU4dId29eh5mjf8aH8Pcipsvf8uWA8uPHj9GhQwfo6uri2bNneP/+PVq3bo1u3brh4sWL39y+bdu2ePnyJfbs2YO4uDgcP34c9+7dQ9OmTfHgwYM8+ArUy+G9OzC6bxc8vH9XaDMyLoKlwdsxy8ENmpqaIqYjIiKi7JBIJBgxcTrW7PwFJUunLqj78vlTjB/YA5tWLuMUGERfcOLQXowboLr4XtkKlbBh3wm07tBZxGRERESUEyQSCUZOMoP/+h0wSDN91eUL5zCmbxfcv3NLxHT5V74cUHZ3d0d8fDzWr1+PYsWKQV9fH0uWLEHFihVha2v7ze0dHR1x4sQJNGvWDDKZDPXq1cOWLVsQHh4Of3//PPgK1EPUx0g4zzGDyxwzxMXGCO31GzfDtmNn0eHH7iKmIyIiopzUsFlLbD3yO1q2/0Fokyclwd/TFTPH/Iyw0NfihSPKZ5KSkuC3wBkO5pNVpoJr1aETNv9yClVq1BIxHREREeW0Nj90wcYDJ1XWIHnx9DHG9e+GYwf2iJgsf8qXA8oHDhxAhw4dUKRIEaFNKpWib9++OH/+PEJDQ7+6vYeHR7q2ihUrQiKR4MmTJxlsUfj8e+UvjOjZEUf27lBpHznJDGt2HUbpsuVESkZERES5xcS0GAI27sa0OfaQyWRC+6Vzv2NYj/Y4d+pX8cIR5RPh79/BfMxgbF4dqNI+dvosLNuwC8ZFTERKRkRERLmpUtXq2HjwpMoFGHGxMZg3awq8XeyQmJAgXrh8Jt8NKD9//hzh4eGoWbNmusdq1aoFpVKJmzdvZvl1Dx06BKVSidq1a+dETLWVlJSEVUu9MGlwL5Vb9wyNjLFkzRbMcfKAppaWiAmJiIgoN8lkMkyeZZN8ArlceaH9w/t3mD1hGLxd7BAfFydiQiLx3L5xHaP7dMZff6Su3aKjqwfP5etgYeeiciKGiIiICh4j4yJYtmEXRk+ZqdK+c8NqTB3WF2Ghr0RKlr/ku+WI379PnvA67dXJKVLa3r17l6XXfPfuHebMmQNDQ0PMnDnzq8998+YNwsLCVNpCQkIAAHK5HHK5PEv7/l5yuRxKhQJKCaBQZH7cX6n4tJ1Cjs9nQ3z+9DGcLc1w4+rfKu1NW7WF65LlKFWmLBSKvPn61N3XjjPlHB7nvCFRKqBUKHmccxnfz7mPxzhr6jdphq1H/gdPR2ucPHxAaN+5YTWuXvwD7n4rUK1m+hPxuXGcU15PLpcDSkkOveqX5VV/jtSHUqnE3q0bsNjNXuXqo3IVK2Px6s2oXquOiOmIiIgoL2loaGC2ozvqNWoKNxtzxERHAQCuX/kLI3t1gmfgOjRt1VbklOLKdwPKKZRK5RfbJJLM/6ERGxuLAQMG4Pnz59ixYwfKlfv6VA5BQUFwc3PL8LGoqChERERket/ZER2XCEV8NOQyCRKz8G1SKhRIik+e503yaYVKpVKJ44f2YYmHi/AhAAANDU1MmWWF4eMmQyaTITE2Ome/iAIso+NMOY/HOY8kJECelAB5bAyPcy7i+zn38RhnnY6mBlwXLUXzlm3gu9AFcbHJx+/+nZsY06cLJlvMEfoJKXLjOCsTEpAk0UBkpAakWejnfa+oqKhvP4kKjeioj1hgPwcnDu1VaW/dsTMWBqyFkXERcYIRERGRqLr06oeqNWvDZuoYPAq5BwB4F/YG00f0h7mdC0ZNnpGlMcqCJN8NKBcrVgwAEB4enu6xlDZTU9NMvVZCQgIGDhyI8+fPY82aNRg8ePA3tzEzM0v3vJCQEPTv3x8GBgYwNjbO1L6zS6mZAOl7BWQyQFNXM/PbfbrCWFNXDxKpDO/C3sBrng1+//WoyvMqVqmG+X4rUbt+wxzNXVh8fpwpd/A45w3Zp2lupDzOuYrv59zHY/z9Bowaj8at28PRYjLu3foPAJCYmICgJV44//spuC4ORPlKVQDkznGWJGhAQ0cDRkbGkElzv1NuYGCQ6/sg9XD/zi3MnT4OTx6GCG0SiQQTza0wZfZcTnFBRERUyFWuVgMbD56Eu60FfjtyEEDy3W5+C5xx7a8LcPYJQBGToiKnzHv5bkC5TJkyKFasGO7cuZPusVu3bkEikaB+/frffJ2kpCQMGTIEJ06cwMqVKzFx4sRM7b9EiRIoUaJEho/JZLI861TKZDJIpFJIpBJIs/DHmgKfrhaSSHHy8AEscp6LiPD3Ks8ZNGo8LB3doaunn8OpC4+U4yyRyrL0/aGs4XHOGxJJ8s8a8DjnKr6fcx+PcfZUqV4TGw+cxMqlXti8KgAKRfKEFv9e+RsjfuqEWQ5u+HnUeEikshw/zimvJ5PJ8mRAWZ0HCRMSEiCXy6Grq/td28fExEBPTy+HU6mnQ7u2wmueDeLjU+cML1LUFB7+q9C6Q2cRkxEREVF+om9gCK/lwdjaOAjLPF2F6dPOnDyGET07YEHAWjRu3krklHkrX94POmjQIJw9exZv374V2pKSknDw4EF06tRJuIo5pT0uLk5ligy5XI4RI0bg0KFDCAoKwpQpU/I0v9jev3uLuWbj4WA+WWUw2cS0GJas2QKHBUs4mExERETpaGlrw8LOBWt3HxGuSAaSV7de5GSDmaMH4fXLFyImLLyuXLmCdu3awcDAACYmJqhXrx6OHTuWqW1//fVXDB48GMWKFUOxYsVQpEgRDBw4ELdv387l1PlTTHQUXKxmwM3GXGUwuVHzVth+7AwHk4mIiCgdiUSCUZNnYOX2gyhWopTQHvrqJaYO7YN1AUsK1Tod+XJA2cnJCUZGRhgxYgQePHiAV69eYerUqQgNDYWPj4/Kc11dXaGrq4s//vgDQPJ8wRMnTsTu3bvh6+uLcePGIS4uTvgvIc0iGwXRycMHMKpvN/zv+BGV9q69+2P3yT/xQ7efREpGRERE6qJhs5bYfuwMho6drNJ+6fwZDOvRHvt3bBGuYKbcFxISgs6dO6NChQp4//49Pn78iP79+6Nv3744c+bMN7fv3r075HI5zp8/j5iYGPzzzz949+4dWrRokeFdgQXZ7Rv/YGSvTji8Z7tK+9hpFli5/SBKlCojUjIiIiJSB01atsGO42fR9ocuQptcLkfQ4gUwH/Mz3r4JFTFd3smXA8ply5bFn3/+iSJFiqBNmzaoV68enj9/jjNnzqBJkyYqz9XU1IS2tjaknxaFef/+PXbs2AFtbW3Y2dmhSJEiKv917dpVjC8p14WFvobt9HFwMJ+MD2muSi5S1BReQcHwWh4ME9NiX3kFIiIiolS6evqwdV+EoK37ULJMWaE9+uNH+LjPw7Rh/fD4wX0RExYebm5uUCqVWLVqFQwMDKCpqYn58+ejRo0asLOz++b2np6e2LdvH2rVqgUAqFSpEjZu3IioqCgEBATkdvx8QaFQYOPKZRg3oDuePnogtBsZF8HS4O2wsHeFpmbm1y0hIiKiwsvEtBj81u/ALAc3yDRSZxO+dP4MhvfsgAtnT4uYLm/kywFlAKhatSp27dqF0NBQvHv3DidOnEDz5s3TPc/FxQVxcXFo06YNgOQF+9Jekfz5f5m5ikOdKBQK7NmyHj93aYVTRw+pPPZjzz7YffJPdO3VX5xwREREpPZatvsBO0/8gT6Dh6u0X/v7Iob37IB1AUuQmJgoUrqCT6lU4tChQ+jYsSMMDQ2FdolEgt69e+PixYt49erVV18jo0HnMmXKQCKR4Pnz5zmeOb958/olzEYOxDJPVySlea82bNoC246eQYcfu4uYjoiIiNSRVCrFmKnmWLfnKMqUqyC0v38bhpmjf8bi/7d353FRlfsfwD/DsK+GsrgjilqiKJJLrqmVa6K4pqZezaXbve6VaVkumdfKn5l5LSU1t8gRDMr0Zl3DBXNDEZfrLiAigggM68w8vz+QieMMMCAzA8zn/XrN6+U853nmfOfLEZ75njPP+Wgh8vJyzRihcVXbgjKV7/r/LmHqiIFYuWgesjMzte1udZ7BinVfY9WGLXCv52HGCImIiKg2cHF1xYefrse6bT+gfsPG2vaC/Hx89ekKjB/8IuLPnTFjhLVXYmIiMjMz0bJlS51txW3x8fEVfl2FQgEhhEE3u67Jfj/wE8a80gMnj/2hbbOyssL0Oe/g67Ao1G/UuIzRRERERGVr2yEIO38+jL4DX5W07wrdiAmD++BKfJyZIjMu6/K7UHWTn5eHzV9+hq3//kJylQUAvDI0BP+YvxBejZpCJjP+XdKJiIjIcrzQqy92H/gDX/1rGcK++1Z7U+Rrly9iUvDLGDVxKmbMfQ8urq5mjrT2ePjwIQDAVU9O3dzcJH0MlZKSgnnz5qFOnTr4xz/+UWbf+/fvIzU1VdJ27do1AEXrBZrq5jNqtRpCo4GQARpN0TUxQvO4TaPGkyt6K7OzsfbjJQjftU3SXr9hYyz7vw0ICOoMANBoLOfmOVWlrLyT8TDv5iETGgiNYN5NjMe7eTDvlefk4oyVX27C3p1bsWb5B8h/fGXyjatX8PrQfpg5dyHGvfEm5HK5zlh9eS9+rlarAWH82l5l5nMsKNcwJ478F58sXiBZ+w0AGjRqgoUff4YuPXqjMFdppuiIiIiotnN0csasdz9A/+BRWL5wDq5fuQSgaBmu3d9+jf9E7cPsRUsxIHgET25XgeIcFhfvSyq+MWLxvUQMkZ2djVdffRX3799HREQEvLy8yuz/1Vdf4aOPPir1tR49emTwvp+GMq8Qmnwl1HIZCh9/hBEaDVT5RR/YZCVycObP41ixaAGSk6TLebw8aCjmf7AMzi6unC8/hdLyTsbFvJtJQQHUqgKoc3OYdxPi8W4ezPvTe3X4SAR0CMRH78zB5QvnAQCqwkKsW7UU0Yd+wfsrP0f9ho0kY/TlXRQUQCWzRmamNaxMMJ/Ozs6u8BgWlGuI5MQErFm+GIf2R0ra5XI5xk19E9Nmvw0HRydeZUFEREQm4d+hI3ZE/Y5tG9fhmy9Wo7CgAACQlpqC92dPR8Tu7/Dusn/Bt2VrM0das9WrV3RT5fT0dJ1txVcmF/cpT25uLgYPHozTp0/ju+++w+DBg8sd8+abb2LkyJGStmvXriE4OBjOzs7aq6SNTdgUwCpdA7kcsHEounmeeDzvtXFwhMxKjrzcHKxfvQK7v/1aMtbJ2RlvL/0XBg4bqfO6VHFP5p1Mg3k3D7mtLQDAink3KR7v5sG8V40Wz7XDt3t/waZ1n+Hb9Wu0FwDEnvoTrw8fgLnvL8eQEWP/umhAT95lBdawtreGq6sb5FbGLyg7OztXeAwLytVcQX4eNn2zFqHr12gvmS/2XLsOWPzJ/6FVm9q99h0RERFVTza2tpjyj3noO/BV/OuDt3HiyF83Pz4dcwRjBvTEuCkz8casBXB0qvhElYD69evDy8sLFy9e1NkWHx8PKysrg9ZBzs/PR3BwMKKjo7F161aMHTu23DEA4OnpCU9PT73b5HK53q9uGoNcLofMygoyKxmsHn/Y0gCP2+S4cPY0lsz7u863+AI7v4Aln36JRk18TBKnJSiZdysWHEyGeTcPmazo9w6Yd5Pi8W4ezHvVsbWT4835i9D9xZfw/pyZSLx9EwCgzMrCsrdn4dBPP2LRJ2vg3aCR3rwXP5fL5SYpKFdmPsdr2KuxmP8exOTBPbHhs48lxWRnV1cs+PATbIk4yGIyERERmZ1Pcz+s374XK9dvhodXfW27WqXCto3rMKJvF+yP2KN32QYq38iRIxEdHY179+5p2woKChAeHo6XXnoJ7u7u2vacnBw8ePAAKpVK21ZYWIgRI0bg119/xZYtWzB+/HiTxm9MBQX5WLdqKaaMGCgpJtvZ2WPeByuwcfePLCYTERGRWbTr2Am79h9G8JgJkvZjhw9h1EsvYO+urTV2fsyCcjV07do1jA4Jxvt/fx13E25Ltg0dNQ7hv5/EmMnTTHZFCBEREVF5ZDIZXh48DIrfYjBh2luQW//1RbiU5LtYPGsaJg97BXFnTpoxyppp8eLF8PDwwKhRo3Du3DlcvXoVEydOREZGBj799FNJ348//hgeHh6IiYkBULTO8rhx4xAVFYXVq1djwIABePDggfZhqjWQjeHcqROYOHwgtv17nfbrpMDju63vP4zXpsys0PrSRERERFXN0ckZ769aizWhuyQXXiizs7Hi3Tl4a8JIJCclmDHCyuEMq5pJTU1FQEAADv6yX9Leyj8AX+7+GbOWfg65Ux1k5hZqH1l5fz2y8wqRU6CCMk+F7McPZb7+R05B0SO3QC19FBY98ko88osfqr8eBSqN9lGo1n2oSjzUGqH3oRF/PcTjBxEREdVcTs4umL1oKXbtP4yOXbpJtsWdPYVJw17Bon9O07lhGpXOy8sLMTExaNGiBYYPH45+/fqhoKAAx48fh7+/v6Svk5MT6tatCxubonWGMzIy8Ntvv6Fu3br4+OOP0bp1a8lj9OjR5nhLTyU7KxMrF83H1JGDcfvGX1cl29ja4q13PsCmPT/Dp7mfGSMkIiIikurZ9xWE/ecohoyULjv259HDmDC0P374LlRygry64xrK1YyHhwcmT56M9evXAwBc67hj6txF6D98LKysrKDWlCi4Sv5Z9ESjFhAqDVRWGu3pAkOKtELPaz3Zrr+vvn5Cz7/0v0ap+yq5D5lum+QmlyXHVWBpGX3DSrt5Zsm71MsAQKMBCvOAXKuiu3CWGCcr8UTf68lK6VvaGJnOP/4apzNaptNVErv+GPQrHid9rVL6FsdTav50+5Yab4l/C6GGSq2BUKlhZVXOPvTEXnoMpfcjIqKq0bzls9i4+0cc+HEvvvjkQ6TcTdJu+2XfHvz+SxTGT/s7Js2cBU5Jy9eoUSOEhoaW22/hwoVYuHCh9rm7uzsePHhgzNBM6uhvv+CLpe/i/r1kSXurNu3w0edfwa/1c2aKjIiIiKhsrm518OGn6/HSoGCsWDgHKcl3AQA5OUr864N3cODHvVi08nM4eTUzc6Tl4+y9Glq6dCl++OEHBA8PwcL3l+CZZ9ylHcooxqrUaiiz5XBydpV8xU9Sq9VXPH6i8quvUFzcV19Buai9IoVo/X3/2q/+orS+wnXp8ejuS9JectzjJwJC0q4R0u3FbRqNGoW5Gljb20NmJS+ORPu6Qk9aSy266+1bflG+rONAZ4y+fUj+rdtBU0piK3Ic6A27AicvNGoNUFAIFBYCVupS+pXMfiX2V87JC6BEMboKT15IXlfSJtPfV0/j0568KO7jlKeCUKmRk5lf9gkSyUA9MZZSoJfp6SvdXsp71veeKvueS4mhzBM2VXDyQt8+iSyJTCZD/6Eh6P3KQGz/ej22bFiL3BwlACA/Pw+b132GiN3b8drM+Rgxtvas60tVL+XePSyb+xb+OBApabe1s8OMOe9i3Bt/h7U1P9oQERFR9dftxZfw/cFj+L8V7yNi93fa9nOnTuC1gb0x4m9/x7R/zDNjhOXjrKsacnd3x9WrV+Hq6lrhsWq1GnKVDdycbLnGshGp1Wo8emQFNzc3g/NcWsFdXzG23BMA0F+ALa+vIUVrfVeKV8UJAAH9b7qskxeaxydIHJ1cICuR54qcvCj5esY6eVFym968P/neHj8vWbTXd/JCiBJxPj7Z8eQRUvyNmKc5eWFrbQWNkEFmJ4dM3918DTwOpH31b9d3HAhRorWSJ570MeQ4KK2/vuPyqU9eaDRAQS6glEGUXNOzxLhqdfLiiSeV/iaC3u26+yjv5EXJf5b67QKhgSa/EFaiEFaPT0JJYzDdyQudfejdXs4+JPsr+4RNyf7V8dsX9vYOmPrP+Rg6ehzWr16OyB92abelpaZg3dIFiNzxDdZ8+i8EBw81Y6RU3QghEBoainnz5+NRRoZk2/PdemLB+8vQrNVzvBs9ERER1Sgurq54f9Va9Bv4KlYsnKNdDk5VWIjdG/8PRw9G4ttN36BvnxfNHKl+LChXU5UpJlP1JilklPpZv/oVAcxJrVbjEQrg5ubAEyQVpD05YcCV5A89nJGbK4eHT11tnp/25EXJ/VXlyYuSr1eVJy8Mit2gQnTpedeo1chVymHv6PzECRLJq+m0G+3kRYkBJU9elOxfch/6Tl5I92lI7PqPhScbS/1mid6+0n7icZlWb18jnbzQHWfckxcl+z/t0lFC/PWXR0Bo/1YJUcbfKo0GKMyFUMqAxydHylo6SlLgtnHDlPdWo1fwBIR++hEunj2h3XTr+v+wdetWFpRJIjU1FXPnzkVmZqa2zdnVDX+btwQvDg6BrDAXSQ9zi75hQyYhNBqgIA/Is2LeTYh5Nw+XnEJAVYjMh7mQyZh3UxFPLjFJJsG8m0ejNp2xZvuP2B26ET/u3ASNuujCmISb17Fs2VL0ebF3tbxghAVlIqJaSLsGtgEnL+QyGeQywNbaCnI5Jw7Golar8chGBTc351p1gkRf0b20kxf6TgKUd/LCkL7F/1Sr1cjKlMPZxRVyuVx/sbacQnJlYy/lnwbHXnJ/T3vyorTYyzt5YWjsGo0a+Tky2Do4w8pKXu7Jiyfj0Qigcbcu6P3Cz/j94E9Y98lHuHPzOuRyOVau/FjPOydL5unpidWrV2P69OkAgJeHDMOc91egrocnhEaNwlw1bBzs9H/DhoyiKO+qWpj36vdhvaS/jnd7k+S9emfDdOQONlAXaODsZpq8UxFTH+9UhHk3PZlMBqFRo8DeHYs+Wo4Ro8dixcLZuBQXC1s7O3y9cWO1LCYDLCgTERHRUyj/2xemmwCp1TKoba3hYm9Tq4r21U3Rsk8Cbm6uT53njtMmYNbkMdi8eTOSkpLQunXrKoqSapOpU6fi8OHDGDt2LAYPHqxtLzoWrSu0BBk9PebdPIryLmfeTSy9rhNyc2XwburOvJsQj3fzYN7No+SSqkHNemH0Kyfx5ZdfQqVSoWXLluYOr1QsKBMRERGR2djY2GDGjBnmDoOqMSsrK+zYscPcYRAREREZnbW1NWbPnm3uMMrF7zYTERERERERERERkUFYUCYiIiIiIiIiIiIig7CgTEREREREREREREQGYUGZiIiIiIiIiIiIiAzCgjIRERERERERERERGYQFZSIiIiIiIiIiIiIyCAvKRERERERERERERGQQFpSJiIiIiIiIiIiIyCAsKBMRERERERERERGRQVhQJiIiIiIiIiIiIiKDsKBMRERERERERERERAaxNncANUF+fj4A4Nq1a2aOpHxqtRrZ2dlwdnaGXC43dzi1FvNsGsyzaTxKSEBeXi7S4uOZZyPi8Wx8zLFp1IY8F8/piud4VDHVZW5cG47Fmoh5Nw/m3Tw4TzYPHu/mwbybR3XIe2XmxiwoGyAhIQEAEBwcbN5AiIiM6b33zB0BEZFJJSQkIDAw0Nxh1DicGxORxeE8mYgsQEXmxjIhhDByPDVeRkYGDh8+jMaNG8POzs7c4ZTp2rVrCA4ORkREBFq0aGHucGot5tk0mGfTYJ5Ng3k2PubYNGpDnvPz85GQkIBevXqhTp065g6nxqkuc+PacCzWRMy7eTDv5sG8mwfzbh7Mu3lUh7xXZm7MK5QNUKdOHQwdOtTcYVRIixYt0KZNG3OHUesxz6bBPJsG82wazLPxMcemUdPzzCuTK6+6zY1r+rFYUzHv5sG8mwfzbh7Mu3kw7+Zh7rxXdG7Mm/IRERERERERERERkUFYUCYiIiIiIiIiIiIig7CgTEREREREREREREQGYUG5lvHw8MCSJUvg4eFh7lBqNebZNJhn02CeTYN5Nj7m2DSYZ6oueCyaB/NuHsy7eTDv5sG8mwfzbh41Ne8yIYQwdxBEREREREREREREVP3xCmUiIiIiIiIiIiIiMggLykRERERERERERERkEBaUiYiIiIiIiIiIiMggLCgTERERERERERERkUGszR0APb3Lly8jLCwMcXFx6NixI959912Dx44ePRpqtVqnvV+/fpgxY0ZVhlnjxcfH4/vvv8fFixfRvXt3zJ49u0LjL1++jK1bt+LOnTuoX78+xo0bhw4dOhgn2BosLi4O3333HZKSktCoUSNMmDAB/v7+5Y7LysrC5MmT9W4bMWIExowZU9WhVnuxsbHYsWMH7t69i8aNG2PixIl49tlnTTbeUpw8eRK7du1CSkoKfHx8MHnyZLRo0aLccXfv3sU///lPvdsmT56MQYMGVXWoNZYQAsePH8cPP/yAhIQEjB07FiEhIRV6jWPHjiEsLAwPHjxA8+bNMXnyZPj4+Bgn4BpKo9HgyJEj2LNnD+7evVuh4zA+Ph5LlizRu23OnDno1q1bVYZKFuz06dMICwvD9evXMXjwYEyaNMmgcampqZg5c6bebePHj0dwcHDVBVkLXbx4EWFhYbhw4QK6du2KefPmVWj8lStXsHXrVty+fRve3t547bXX0LFjRyNFW3skJCQgNDQUV69ehbu7O0JCQtCrVy+Dxr7xxht4+PChTnunTp3w9ttvV3WoNU56ejo2bdqEuLg4ODk5YeDAgXj11VdNNt5SKZVKhIaG4vTp07C2tkafPn0wZswYWFmVf63jZ599huPHj+u0u7q6IjQ01Bjh1hrZ2dmIjIzE/v37kZOTgw0bNsDDw8Pg8YWFhfjuu+9w5MgRCCHwwgsvYOLEibC1tTVi1DVfXl4efvnlF/z444/IzMzE8uXL0bp1a4PGRkZGYuvWrXq3ffnll/D29q7KUCuFVyjXcC+88AKGDRsGlUqFffv24ciRIxUar1AokJeXhzFjxkgezz//vJEirpnatm2L0aNHw8rKCgqFAjExMRUav2/fPgQEBCAxMREDBgxAXl4eOnXqhC1bthgn4Bpq165dCAwMRHp6OgYMGID09HQEBgYiLCys3LH5+flQKBSwt7fXOZ7btm1rguirly1btuD5559HdnY2BgwYgJSUFLRv3x779u0zyXhL8dVXX6Fr165Qq9UYMGAAbty4gbZt2+LgwYPljs3MzIRCoYCHh4fOMduqVSsTRF8zJCUlwcfHB/Pnz4dGo4FCocClS5cq9BqrV69Gr169YGNjg/79+yMuLg7+/v6Ijo42UtQ1z7Vr19CkSRMsXrwYarUaCoUCV69eNXh8amoqFAoF/Pz8dI7npk2bGjFyshRZWVlo0aIFpk+fDqBoDhsbG2vweKVSCYVCgTp16ugcozxZWrbAwECMHDkSQFHe9RV0yhIVFYWAgADcvn0bAwYMQGFhIbp06YJNmzYZI9xa48SJE/D398eJEyfwyiuvwMXFBS+99BKWLVtm0PjIyEjcv39f53jv0aOHkSOv/q5fv462bdsiPDwcffv2RdOmTTF+/HhMmTLFJOMtVWpqKoKCgrBx40Z069YNbdq0wdy5czFkyBCoVKpyxx8/fhx//PGHzjE9bNgwE0Rfc23atAm+vr748ccfkZycDIVCAaVSafD4nJwc9OrVCx999BE6duyITp064ZNPPkGPHj2QnZ1txMhrtqioKDRp0gRbtmzBo0ePoFAo8ODBA4PHX7lyBQqFAv3799c55l1cXIwYeQUIqtEuXryo/bednZ0YNGhQhcbL5XIxa9asKo6q9rl06ZIQQojCwkIBQIwePdrgsY8ePRJ169YVI0aMkLS/9dZbwtHRUSQlJVVprDXV/fv3hYuLi5g0aZKkfdKkScLNzU08ePCgzPGpqakCgFi5cqUxw6wR7t69KxwdHcWMGTMk7WPGjBHu7u4iIyPDqOMtxY0bN4Stra2YP3++pH3w4MGifv36Iicnp8zxly5dEgDEN998Y8wwa7ysrCyRkJAghBDi7NmzAoBYtmyZwePj4+OFXC4XH374oaT9xRdfFM2aNRMFBQVVGm9NlZGRIe7evSuEECI6OloAEGvWrDF4/O+//y4AiMjISCNFSJYuPz9fXL9+XQghREJCggBQoTnszZs3BQCxbt06I0VYe5X8vAFAhISEGDw2KytLeHh4iODgYEn77Nmzhb29vfb3O0mp1WrRqlUr0alTJ6FWq7Xtn376qZDJZOL06dPlvoaXl5cYN26cMcOssfr16yeaNGkilEqltm3Pnj0CgAgPDzf6eEs1adIk4erqKlJSUrRtMTExBv9uDgkJEU2bNjVihLXTrVu3tJ9L5s2bJwCImzdvGjx+8eLFQi6Xi8uXL2vbij8HvfPOO1Udbq2RmJgoMjMzhRBCrFmzRgAQ0dHRBo9fvXq1ACCSk5ONFeJT4xXKNRyvqDANQ7+WoE94eDjS0tK0V9QUmzFjBnJycrBjx46nDa9WCAsLQ1ZWlt48PXr0yKCrlKnIzp07kZOTozeX6enp2Lt3r1HHW4pt27ahoKBAb56Sk5MRFRVlpshqF2dnZzRq1KjS47/99luo1WpMmzZN0j59+nTcvHkTv/7669OGWCu4ubmhfv365g6DqFS2trbw9fU1dxgW6Wk+b+zbtw+pqal6/1bm5eXhu+++e9rwaqXo6GhcuXIFb7zxhmQpgKlTp8LKygqbN282Y3Q12+3bt/Hrr79iwoQJcHR01LYPGzYMnp6e+Oabb4w63lIplUrs3r1bm6dinTt3Rvv27Zk3I2ratCkcHBwqPX7z5s3o3bu35BuUzZo1w8svv4zQ0FBoNJqqCLPWadiwYfW5kthIuIYy4ciRI3j99dcBAC1btsRrr73GCXsVOnHiBADorBPXpk0bODg4VHj5jNrqxIkTkMlkOutKBwYGwsrKCjExMaWufVjSzz//jAsXLsDa2hpt2rTBuHHj0KBBA2OFXS2dOHECNjY2Okt9BAUFAQBiYmJKXW+6KsZbihMnTsDV1VVnveSSeSr+inBZwsLC8Pvvv8PW1hYBAQEYP3486tWrZ5SYLdGJEyfQoEEDnWJpyZ/TgAEDzBFarbRp0ybs3LkTjo6OCAoKwmuvvQZXV1dzh0WkFR4ejuPHj8PW1hZt27bF+PHjJcUNqlqlzYNbtWoFZ2dnzoNLUVre3Nzc4OfnZ3DeYmNjMWnSJKhUKjRv3pxLvKD03FpZWSEwMLDc3D7teEt17tw55OXl6V07PSgoCJs3b4ZSqYSTk1OZr5ORkYHp06cjKysLDRo0wMCBA9GnTx9jhW3x7ty5g+TkZEyYMEFnW1BQEKKionD9+nX4+fmZITrLsGjRIuTk5MDd3R09e/bE8OHDYWNjY+6wALCgXC1oNBqMGjXKoL4vvvgi/v73v1fZvp955hl07doVHTp0QHp6OrZv345ly5Zh06ZNen9p1GQ5OTnawnl5BgwYUGVrYCUmJsLGxgbPPPOMzjZPT08kJSVVyX6qi/T0dJ0rAUszbNgwjBs3DkBRnurUqQM7OztJn+LcGZInLy8vdOrUCf7+/khJScHXX3+NpUuXYvfu3RZ1g7PExETUq1cPcrlc0u7k5AQnJ6dyc/m04y1FYmIivLy8dNo9PT0hk8kMylPjxo3RsWNHtG7dGgkJCfjss8+wbNkyREREcI3DKlLaz6m4jcdz1fHz80O7du3QvHlz3LhxA++//z6WLVuGX375xSLXsif9yroh6ZMqcwPOsjRo0ABBQUF47rnncPfuXXzxxRdYunQp9u7dW+sLEnl5eRg/frxBfV9++WWD53LlSUxMhEwm03vzJy8vL4v4HTxx4kSD1isNCgrS3lw9MTERAEr9+3X58uVyX8/Z2Rndu3dHUFAQsrKyEBYWhhUrVuDzzz83+P9gbVRebtPT05GXlwd7e3ujjLdU5eVNCIHk5OQyb2wtl8vRt29ftGnTBvb29jh06BD69u2L8ePHY+vWrQbd2I8qpryfG1A0l2ZB2Ti6desGPz8/eHl54ezZs5g4cSJWr16NgwcPwt3d3dzhsaBcHRTfZMgQzs7OVbrvixcvSiZ4s2fPRr9+/TB16lT07NmzVt1Ip6CgwOA8V+UdMwsKCmBtrf+/mo2NDfLy8qpsX9VBTk6OwXkuOWF42jy5urriypUrcHNz07b985//RKdOnTBu3DjcunULderUMSiumu5pc2lpx2xllZYnmUwGuVxebp4aN26My5cvS74u+dZbbyEgIABjxozBjRs3dE6wUMWV9nMqPrPP47lqdOjQAfHx8ZIrJqZNm4Z27dph7NixuHDhghmjo+qk+Iakhmjfvn2V7dfLywtXrlyRzKXfeustdOjQAWPHjsXNmzclv49rG5VKZXDeq/JbMpxTFC378ejRo3L7lcxFQUEBAJT698uQvB0/flznc97w4cMxe/Zs9O7dG+3atTMk/FqnvNwCKLMg/LTjLZWheSvLl19+KTmmp02bhueffx4LFixAt27dMGPGjCqMmICq+blR5UycOBHz58/XPp88eTJCQkLQp08fzJ07F1u2bDFfcI+xoFwNyOVy/PDDDwb19fHxqdJ9P3m1gLW1NebPn48hQ4YgKiqqSq+GNjcnJyeD81zWmdGKcnZ2Rn5+PjQajc5ZU6VSWaXF6+qgbt26Bue55NrUzs7OyMnJ0dtPqVSWu/6Qra0tbG1tJW0ODg6YNWsW3njjDRw6dKhKr3KqzpydnXHr1i2923JycsrN5dOOtxTOzs5679RbWFgIlUpVbp70faWvTp06mDlzJhYuXIiYmBj06tWryuK1VKX9bim+WozHc9UoeTKvWMOGDTFhwgSsXbsWV65ckay9R5arYcOGBs8T/P39q2y/+taPdHFxwVtvvYU5c+YgOjoar7zySpXtr7pxcHAwOO/Nmzevsv06Oztr/y4+WZBQKpVo2LBhle2ruiq+50J5Si7NVHzio7S/X4b87Xryc55MJsM777yDiIgIhIeHW2xBubzcymSyMi/ietrxlqq8vAHlz8n0fdNh9uzZWLx4MRQKBQvKRlAVPzeqHH3He69evdC1a1coFAoWlKmITCbDiBEjzB2GVnGBMz093cyRVC0bGxuz5NnPzw8ajQYJCQmSK77z8vKQkpKC/v37mzwmY3JwcKhUnv38/HDgwAGkpaWhbt262vaHDx8iKyur0l+jqa3Hc1n8/Pxw+PBhZGdnSya0ycnJKCgoKDeXTzveUvj5+eH8+fMoLCyUXJVZXIznMVs9FB/PQgjIZDJt++3bt7XbyXh4PNOTXFxcOO81A7lcbrZ5MFD0O7dkobqgoADJycno3bu3yWMytVdffbXCY0rm7ckLim7fvl3pE3SWcryXpWRun1Sc79Kuqq+K8ZaqvLzZ29tX6ibM1tbWqFevnkUf08ZUfKFdaT83gHNpU/P29sbRo0d1PoOaAxeZIR1nzpwBULVX6Vqyvn37AgD++9//StoPHz4MjUaDl156yQxRVT+l5em3334DgErnyRKP5+JcHj58WNJuaC6fdryl6Nu3LwoLC3Hs2DFJO4/Z6qVv377IysrS5rUYj2fTOHPmDGQyGW/2S9UWf+caV2nzu+joaKhUKv4OLkVpefvf//6Hu3fvco7xFLp16wZ7e3ud3GZmZuLMmTPl5vZpx1uqZ599Fg0aNNDJm1qtxh9//IE+ffro3L/FECkpKbh7965FH9PGVKdOHXTs2FHn5wYUzaXbtWvHG9uakEajwblz59CkSROzF5MBAIJqDTs7OzFo0KBSt4eEhIgPPvhA+/zAgQPi7Nmzkj7nz58XXl5eolmzZiInJ8dYodZYhYWFAoAYPXq03u25ubkiJCRErFy5Utum0WhEUFCQaNWqlXj48KEQQoicnBzRtWtX4efnJ/Lz800RerVXWFgo2rRpI9q3by8yMzOFEEJkZWWJ9u3bi3bt2gmVSqXt+/DhQxESEiLWrFmjbQsPDxdXrlyRvOaRI0eEs7OzCAwMlIyv7fLz84Wfn594/vnnhVKpFEIIkZGRIZ577jkRFBQkNBqNtu/9+/dFSEiI+PLLLys13pIplUrRuHFj0bNnT5GXlyeEEOLBgwfC19dX9OrVS9L3zp07IiQkRGzatEnbtnv3bnHr1i1Jv/379wtbW1vRt29fo8dfE509e1YAEMuWLdO7/X//+58ICQkR27dv17ZlZGQIT09P0b9/f1FQUCCEECI5OVk0bNiwzL+Zliw6OloAkPyOLen8+fMiJCRE7NmzR9u2detWce/ePUm/7du3C5lMJsaNG2fMcMkCJSQkCABi1qxZercnJyeLkJAQsWHDBm3bDz/8IK5fvy7pd+jQIWFvby+6d+/Ov20GAiBCQkL0bisoKBAhISFi+fLl2jaNRiO6dOki/Pz8RFpamhCiaL7cvXt34evrq/37SbqGDx8uPDw8xO3bt4UQQqhUKjF8+HDh7u4uHjx4IOk7efJkMWfOHO3zP/74Qxw7dkzS5+rVq8LX11d4enpqfxaWau7cucLe3l6cOXNG2zZr1ixha2srLl++LOn79ttvi/Hjx1d6PP3l888/FzKZTBw4cEDbtnr1aiGTycThw4clfVetWiVCQkK0n0Vu374tvv/+e6FWq7V9Hj58KAYMGCDkcrmIjo42zZuo4ebNmycAiJs3b+rdvmnTJhESEiLu3LmjbQsLCxMARGhoqLZt165dAoDYtWuXsUOuFdasWSMAlHqchoeHi5CQEEl9bt26ddrjX4iiesmCBQvKnKObGr+LUcN98sknOHXqFICidTtPnz6t/Trb5MmTMWjQIG1fhUIhWe/T1dUVb775Jh48eIDmzZsjLS0NZ8+eRbdu3RAaGqp3rTlLtWTJEsTHx0MIAQA4evSoNs8zZ87UXkVQfMOTkgvTy2Qy7N27F8OHD0erVq3QqVMnxMbGwsXFBZGRkTrr/loqa2tr7Nu3T5unwMBAnDlzBp6enoiIiJCcsc7Ly4NCoZB8nczR0RFjxoxBbm4umjVrhnv37uHcuXMYOHAgvvnmm0qd8a6pbG1tERkZieHDh6Nly5Zo3749Tp06hUaNGmHv3r2Sr/0rlUooFArJ0hYVGW/JHB0dERUVhREjRqBly5Zo164dYmJi0LJlS3z//feSvo8ePYJCoZCsmW5jY6P9Hd20aVMkJCQgPj4eI0aMwMaNG036Xqq7v/3tb8jMzNTe1Gj37t2IjY0FAKxcuVL7Vbu0tDQoFArJVSpubm746aefMHLkSLRq1QrPPfccjh07hoCAAGzbts3k76U6e+2111BQUIC0tDQAwJYtW3DkyBEAwJo1a9C4cWMARVcDKRQKyQ3TVCoVevToAUdHRzRs2BDXr1/HjRs3MGXKFHzxxRcmfy9UO82aNQtJSUnIzc0FAPz000/aO9AvXLgQHTt2BABkZ2dDoVBIbsZrZ2eH4OBgqNVq+Pj4ICkpCXFxcQgODsbXX3/Nv21lWLp0Kc6fP699HhMTo50HT58+XXtFplqthkKhQHZ2travTCbDnj17MHz4cLRu3RqdOnXC+fPn4eTkhKioKN58tgyhoaEYPXo02rZti+7du+Pq1avIzc1FZGSkZHk4APjxxx8lc4w6depg1qxZuH37Nvz8/PDo0SOcOXMGAQEB2LdvH9zd3U39dqqVlStXIjU1FS+88AJ69eqFe/fu4c6dOwgLC9NZTuTQoUO4fPlypcfTX2bPno3ExEQMGTIE3bt3h1KpRFxcHL7++mv07NlT0vfYsWPYt28fNm3aBEdHR+1n57lz56JFixaQy+U4c+YMXFxcoFAo0L17dzO9q+ovNjYWy5cvBwDExcUBAN588004Ojqifv36WLdunbbv2bNnoVAo8OGHH2rnfSNHjsSqVaswc+ZMhIaGwsrKCidOnMCKFSswZswY07+hGiIhIQFz5swBAFy7dg0A8P7776Nu3bqwtbXFzp07tX0vX74MhUKBqVOnatuSkpLQunVrNGjQAM888wzi4uKQlZWFlStXYvbs2SZ9L6WRieIKGdVIR48eRXJyst5t7du3l3yo3rNnDzw8PHRu8nTr1i3tH8nnnnsOTZo0MV7ANdThw4eRmpqqd1tQUJB2bTO1Wo3w8HDUr18f3bp10+kbGxuLhIQEeHt7o2PHjjo36SNACIEzZ84gKSkJDRs2RGBgoM6HvPz8fERGRqJx48bo3LmzZNvVq1dx9epVWFtbo23btpIbnFgaIQROnTqFe/fuoVGjRujQoYNOn5ycHPz888/w8fFBUFBQhcdT0VePTp48ifv376Np06Z6b3KTmZmJgwcPokWLFpIinBACV65cwfXr12FnZ4eAgAC9N2CwdJGRkcjPz9e7rU+fPtoPxunp6fjtt9/QunVrnZt5qdVq/Pnnn3jw4AF8fX3Rpk0bo8dd04SHh0OtVuvd9vLLL8PV1RUAcP/+ffzxxx/w9/eX3FxVrVbj0qVLuHnzJpydndG+fXs888wzJomdLMPBgweRmZmpd1v37t21BTWlUon9+/ejWbNm2iIzUPQ7t3ieYGtri3bt2sHLy8sksddk0dHRSElJ0bstMDBQu6SNRqPB3r174e3trbe4c+7cOdy+fRve3t4ICgriPNhAly9fxtWrV+Hu7o5OnTrp/ZpzZGQk7O3tdZZbSExMxKVLl1BYWIhWrVpV6Q0Xa4Nbt27hwoULcHJyQpcuXfReUPXbb78hMzMTwcHBlRpPupKTkxEbGwtra2t07txZO78o6fjx40hKSsLQoUMlx/zDhw9x4cIFZGRkoHHjxmjbtq1FXThUGSkpKYiOjta7zdnZWXJPp9jYWFy7dk0y7yuWnp6OkydPQgiBoKAg1KtXz6hx13TFn//0kcvlGDZsmPb5lStXEBcXhx49ekjmJbm5uYiPj8fdu3dRr149dOjQoVr9nmFBmYiIiIiIiIiIiIgMwtPCRERERERERERERGQQFpSJiIiIiIiIiIiIyCAsKBMRERERERERERGRQVhQJiIiIiIiIiIiIiKDsKBMRERERERERERERAZhQZmIiIiIiIiIiIiIDMKCMhEREREREREREREZhAVlIiIiIiIiIiIiIjIIC8pEREREREREREREZBAWlImILMipU6cgk8kQERFhkfsnIiIiIipm7rmpufdPRFRZLCgTEdUyR44cgUwmQ1RUlEXun4iIiIiomLnnpubePxGRMVibOwAiIjKdoKAgCCEsdv9ERERERMXMPTc19/6JiCqLVygTERERERERERERkUFYUCYiKoW3tzcmTZqEmJgYdOnSBQ4ODvD19cVnn31Wat+TJ0+iZ8+ecHBwwLvvvgsAyM7OxjvvvIPmzZvDzs4OXl5eeOONN5CWlqYdn5iYCJlMpn3Y2dnh2WefxYoVK6BSqST7ys7OxsKFC9GyZUvY29vD19cXc+fORUZGBrZv344ePXoAAIYMGaJ9vU8//RSA7jptt27dgpWVFT788EOd93T//n3Y2trinXfeMTjGiu6/WFJSEiZPngxvb2/Y2dmhefPmWLx4MfLz87V9fv31V8hkMvz666/YsGEDmjVrBnt7e3Tt2hWnTp0q9+f5+uuvw8HBAWfOnJG0jxo1Ck5OTrhw4UK5r0FERERkqTg35tyYiEhLEBGRXl5eXqJv375i0KBBIj4+XqSlpYm1a9cKuVwuPvjgg1L7xsXFiaSkJLF3716hVCpFYGCg8PHxEfv37xeZmZni3LlzIigoSPj7+4ucnBy9+05LSxM7d+4Uzs7OYsmSJdr27Oxs0b59e9GwYUMRHh4u0tPTxa1bt8SaNWvE+vXrhRBCREdHCwAiMjJS53VPnjwpAIjw8HBtW79+/USTJk2EWq2W9F29erUAIC5fvlyhGCu6/9TUVNG4cWPRunVrcfz4cfHo0SOxZ88e4ebmJl555RWh0WiEEEL85z//EQDE2LFjxfLly0VKSoq4fv266Ny5s2jQoIHIy8vTG2cxpVIp/P39RbNmzUR6eroQQog1a9YIAGLbtm1ljiUiIiKydJwbc25MRFSMBWUiolJ4eXkJBwcHcf/+fUn71KlThZ2dnUhNTZX0tbOzEykpKZK+q1atEgDEyZMnJe3Xr18XcrlcbNiwocwY5s+fL7y9vbXPly9fLgCIo0ePljqmopPW3bt3CwDiwIEDkr7PPvus6N69e5nx6Yuxovt/7733hEwmE7GxsZK+GzZskLxO8aR51KhRkn6HDx8WAIRCoSg31itXrggXFxcxePBgceTIEWFjYyOmT59e7jgiIiIiS8e5MefGRETFuOQFEVEZunTpAg8PD0lbcHAw8vPzcfToUZ2+np6ekrbIyEj4+PggKChI0u7r64tmzZrh8OHD2rbdu3ejR48ecHNzg5WVlfbrcPfu3YNSqQQA/Pzzz/Dx8cELL7xQZe8xODgYdevWRWhoqLbt+PHjuHTpEqZMmSLpa0iMFXXo0CE0b94cAQEBkvYRI0Zot5c0aNAgyXN/f38AwI0bN8rdV8uWLbF582ZERUWhX79+aNeuHdauXVupuImIiIgsDefGnBsTEQFcQ5mIqExeXl6ltj148EDS3rBhQ52+9+7dw61bt2BtbQ1ra2vI5XLtZPPatWvateK2b9+OsWPH4sUXX8T58+eRn58PIYR2/bbCwkIARWu36dvP07Czs8P48eMRERGB9PR0AMDmzZvh4uKCkSNHavsZGmNFpaWlwdvbW6e9Xr16sLa21slz/fr1Jc9dXV0BABkZGQbtb+DAgahfvz7y8vIwZ84c2NnZVSpuIiIiIkvDuTHnxkREAAvKRERlSklJKbWtbt26knYbGxudvvXq1YO/vz9UKhVUKhXUajU0Gg1E0ZJDOHjwIABg27ZtePbZZ7F06VI0bdpU+1o3b96UvJ6HhweSkpKq5L2VNGXKFOTn52PHjh1QKpUICwvDmDFj4OTkpO1jaIwV5e7urjfPaWlpUKlUqFevnqRdJpM91f6mT5+Ohw8fIiAgAHPnzsXdu3ef6vWIiIiILAXnxpwbExEBLCgTEZUpJiZG5yqAffv2wdbWFt27dy93/JAhQ3Dx4kWD7pL85NUAmZmZ2Ldvn6Rt8ODBuHXrFo4dO1bq6xRPdEveBbo8bdu2RadOnbB582aEhYUhKytL5yt9hsZY0f337dsX165d08mRQqHQbq8qGzZswI4dO7Bu3Tr89NNPEEJgzJgxOncLJyIiIiJdnBtXPEbOjYmoNmJBmYioDF27dsXf/vY3XLp0CQ8fPsT69evx7bffYsGCBTpXB+gza9YsBAYGYujQoYiIiEBaWhoyMjIQExODmTNnYseOHQCAV199FbGxsfj3v/+N7OxsXLhwAcOGDUPv3r11Xi8gIACjRo1CREQEMjIycOfOHaxduxZfffUVAKB58+ZwcHDAgQMHkJOTY/B7nTJlCs6dO4clS5agTZs26Ny5s2S7oTFWdP9z5sxBgwYNMHr0aPz555/IzMxEeHg43n33XfTr109nXbjKOnXqFObMmYNJkyZh6tSpaNiwIXbu3ImjR4/ivffeq5J9EBEREdVmnBv/hXNjIrJo5robIBFRdefl5SUmTpwojh49KoKCgoSdnZ1o2rSpWLVqldBoNHr76pOTkyM+/PBD0aZNG2Fvby/q1q0runXrJjZu3Chyc3OFEEKo1WqxbNky0bRpU2Fvby8CAwPF/v37xcqVKwUA8fDhQ+3rZWZmigULFghfX19ha2srfH19xdy5c0V6erq2z7Zt20SLFi2EtbW1ACBWr14thNB/J+mSr+vk5CQAiM8//1xne0VirOj+ExISxOuvvy48PT2FjY2N8PHxEQsXLtTmR4i/7mT9n//8RzK2sLBQABCLFi3Sm38hhEhPTxc+Pj6iXbt2IicnR7Lto48+EjKZTERERJQ6noiIiMjScW4sxbkxEVkymRBCmLqITURUE3h7e6N///7YsmWLuUMhIiIiIjIrzo2JiKgYl7wgIiIiIiIiIiIiIoOwoExEREREREREREREBmFBmYiIiIiIiIiIiIgMwjWUiYiIiIiIiIiIiMggvEKZiIiIiIiIiIiIiAzCgjIRERERERERERERGYQFZSIiIiIiIiIiIiIyCAvKRERERERERERERGQQFpSJiIiIiIiIiIiIyCAsKBMRERERERERERGRQVhQJiIiIiIiIiIiIiKDsKBMRERERERERERERAZhQZmIiIiIiIiIiIiIDMKCMhEREREREREREREZhAVlIiIiIiIiIiIiIjLI/wPj+p5idsV0jAAAAABJRU5ErkJggg==" + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "worst_per_layer = (\n", + " tanh_prime_diagnostics\n", + " .sort_values(['hidden_layer', 'affine_residual_delta'], ascending=[True, False])\n", + " .groupby('hidden_layer', group_keys=False)\n", + " .head(2)\n", + " .reset_index(drop=True)\n", + ")\n", + "fig, axes = plt.subplots(3, 2, figsize=(12, 11), squeeze=False)\n", + "for axis, (_, row) in zip(axes.flat, worst_per_layer.iterrows()):\n", + " x = np.linspace(row.preactivation_lower, row.preactivation_upper, 600)\n", + " exact = 1.0 - np.tanh(x) ** 2\n", + " affine = row.affine_slope_p * x + row.affine_intercept_q\n", + " axis.plot(x, exact, color='black', linewidth=2, label=r'$\\tanh\\prime(x)$')\n", + " axis.plot(x, affine, '--', color='tab:blue', linewidth=1.6, label=r'$px+q$')\n", + " axis.fill_between(\n", + " x, affine - row.affine_residual_delta, affine + row.affine_residual_delta,\n", + " color='tab:blue', alpha=0.22, label=r'$px+q\\pm\\delta$'\n", + " )\n", + " if row.contains_zero:\n", + " axis.axvline(0.0, color='tab:red', alpha=0.45, linewidth=1)\n", + " axis.set_title(\n", + " f'layer {int(row.hidden_layer)}, neuron {int(row.neuron)}: '\n", + " f'I=[{row.preactivation_lower:.3f}, {row.preactivation_upper:.3f}], '\n", + " f'δ={row.affine_residual_delta:.3f}'\n", + " )\n", + " axis.set_xlabel('preactivation x')\n", + " axis.set_ylabel('derivative value')\n", + " axis.grid(alpha=0.22)\n", + "axes[0, 0].legend(loc='best')\n", + "fig.tight_layout()\n", + "plt.show()\n", + "\n", + "worst_per_layer[['hidden_layer', 'neuron', 'preactivation_lower', 'preactivation_upper',\n", + " 'affine_residual_delta', 'interval_radius', 'delta_over_interval_radius']]" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-refinement-intro", + "metadata": {}, + "source": [ + "## Four-step Dörfler mark-and-refine comparison\n", + "\n", + "This experiment tests whether domain refinement resolves the broad zero-crossing preactivation intervals identified above. Cells are marked by the usual Dörfler criterion with $\\theta=0.5$, using the local certified interval width of the integrated squared $W^{1,2}$ contribution.\n", + "\n", + "For each marked 100D cell, the edge is chosen by an exhaustive one-step interval look-ahead: every coordinate is tentatively bisected and the coordinate minimizing the sum of the two child contribution widths is selected. Equivalently, it maximizes\n", + "\n", + "$$g_j=\\eta_K-\\bigl(\\eta_{K_j^-}+\\eta_{K_j^+}\\bigr).$$\n", + "\n", + "The resulting partition is shared by interval arithmetic, Top-96, and PCA-64. Thus all three methods certify exactly the same cells. The table reports certification time separately from the shared edge-selection overhead.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "adaptive-refinement-run", + "metadata": {}, + "outputs": [], + "source": [ + "from intervalnets import Interval\n", + "from intervalnets.pytorch import (\n", + " _dorfler_marking, _lookahead_sobolev_split_dimension,\n", + " _weighted_sobolev_squared_contribution,\n", + ")\n", + "from intervalnets.pz_integration import (\n", + " _evaluate_squared_contribution_cache, _independent_pz_sum,\n", + ")\n", + "\n", + "REFINEMENT_STEPS = 4\n", + "DORFLER_THETA = 0.5\n", + "DOMAIN_VOLUME = 0.2 ** 100\n", + "PZ_REFINEMENT_CONFIGS = {\n", + " 'topk-96': dict(reduction_strategy='topk', max_terms=96),\n", + " 'pca-64': dict(\n", + " reduction_strategy='pca', max_terms=64, pca_rank=4, pca_candidates=32\n", + " ),\n", + "}\n", + "\n", + "def add_intervals(intervals):\n", + " total = Interval.point(0.0)\n", + " for item in intervals:\n", + " total = total + item\n", + " return total\n", + "\n", + "def normalized_norm_metrics(squared):\n", + " lower = sqrt(max(0.0, float(squared.lower) / DOMAIN_VOLUME))\n", + " upper = sqrt(max(0.0, float(squared.upper) / DOMAIN_VOLUME))\n", + " width = upper - lower\n", + " return lower, upper, width, width / max(abs(lower), abs(upper)) if upper else 0.0\n", + "\n", + "interval_start = perf_counter()\n", + "interval_cells = [{\n", + " 'box': box,\n", + " 'contribution': _weighted_sobolev_squared_contribution(model, box),\n", + "}]\n", + "interval_certification_s = perf_counter() - interval_start\n", + "\n", + "pz_cells = {}\n", + "pz_certification_s = {}\n", + "for label, config in PZ_REFINEMENT_CONFIGS.items():\n", + " start = perf_counter()\n", + " pz_cells[label] = [_evaluate_squared_contribution_cache(\n", + " model, box, integrand_kind='w12', **config\n", + " )]\n", + " pz_certification_s[label] = perf_counter() - start\n", + "\n", + "adaptive_rows = []\n", + "edge_rows = []\n", + "selector_s = 0.0\n", + "for refinement_step in range(REFINEMENT_STEPS + 1):\n", + " interval_squared = add_intervals([cell['contribution'] for cell in interval_cells])\n", + " metrics = normalized_norm_metrics(interval_squared)\n", + " adaptive_rows.append({\n", + " 'method': 'interval', 'refinement_step': refinement_step,\n", + " 'cells': len(interval_cells), 'marked_cells': None,\n", + " 'normalized_lower': metrics[0], 'normalized_upper': metrics[1],\n", + " 'normalized_absolute_width': metrics[2], 'relative_width': metrics[3],\n", + " 'certification_s': interval_certification_s,\n", + " 'shared_selector_s': selector_s,\n", + " 'end_to_end_s': interval_certification_s + selector_s,\n", + " })\n", + " for label, cells in pz_cells.items():\n", + " squared_pz = _independent_pz_sum([cell.integrated_pz for cell in cells])\n", + " metrics = normalized_norm_metrics(squared_pz.interval_enclosure())\n", + " adaptive_rows.append({\n", + " 'method': label, 'refinement_step': refinement_step,\n", + " 'cells': len(cells), 'marked_cells': None,\n", + " 'normalized_lower': metrics[0], 'normalized_upper': metrics[1],\n", + " 'normalized_absolute_width': metrics[2], 'relative_width': metrics[3],\n", + " 'certification_s': pz_certification_s[label],\n", + " 'shared_selector_s': selector_s,\n", + " 'end_to_end_s': pz_certification_s[label] + selector_s,\n", + " })\n", + " if refinement_step == REFINEMENT_STEPS:\n", + " break\n", + "\n", + " indicators = [\n", + " float(cell['contribution'].upper) - float(cell['contribution'].lower)\n", + " for cell in interval_cells\n", + " ]\n", + " marked = set(_dorfler_marking(indicators, DORFLER_THETA))\n", + " for row in adaptive_rows[-len(PZ_REFINEMENT_CONFIGS) - 1:]:\n", + " row['marked_cells'] = len(marked)\n", + "\n", + " selected = {}\n", + " for cell_index in sorted(marked):\n", + " start = perf_counter()\n", + " split_dim, children, _, candidates = _lookahead_sobolev_split_dimension(\n", + " model, interval_cells[cell_index]['box']\n", + " )\n", + " selector_s += perf_counter() - start\n", + " selected[cell_index] = (split_dim, children)\n", + " for rank, candidate in enumerate(candidates[:5], start=1):\n", + " edge_rows.append({\n", + " 'refinement_step': refinement_step + 1,\n", + " 'marked_cell': cell_index, 'rank': rank, **candidate,\n", + " })\n", + "\n", + " new_interval_cells = []\n", + " for cell_index, cell in enumerate(interval_cells):\n", + " if cell_index not in marked:\n", + " new_interval_cells.append(cell)\n", + " continue\n", + " _, children = selected[cell_index]\n", + " for child in children:\n", + " start = perf_counter()\n", + " contribution = _weighted_sobolev_squared_contribution(model, child)\n", + " interval_certification_s += perf_counter() - start\n", + " new_interval_cells.append({'box': child, 'contribution': contribution})\n", + " interval_cells = new_interval_cells\n", + "\n", + " for label, config in PZ_REFINEMENT_CONFIGS.items():\n", + " new_cells = []\n", + " for cell_index, cell in enumerate(pz_cells[label]):\n", + " if cell_index not in marked:\n", + " new_cells.append(cell)\n", + " continue\n", + " _, children = selected[cell_index]\n", + " for child in children:\n", + " start = perf_counter()\n", + " new_cells.append(_evaluate_squared_contribution_cache(\n", + " model, child, integrand_kind='w12', **config\n", + " ))\n", + " pz_certification_s[label] += perf_counter() - start\n", + " pz_cells[label] = new_cells\n", + "\n", + "adaptive_refinement_table = pd.DataFrame(adaptive_rows)\n", + "edge_choice_table = pd.DataFrame(edge_rows)\n", + "adaptive_refinement_table\n" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-refinement-observed", + "metadata": {}, + "source": [ + "### Observed four-step result\n", + "\n", + "| Method | Step 0 normalized width | Step 4 normalized width | Reduction | Certification time | Shared selector time |\n", + "|---|---:|---:|---:|---:|---:|\n", + "| Interval | 88.846805 | 87.815079 | 1.16% | 1.95 s | 191.04 s |\n", + "| Top-96 | 85.370153 | 84.143588 | 1.44% | 21.93 s | 191.04 s |\n", + "| PCA-64 | 85.900362 | 84.720192 | 1.37% | 19.31 s | 191.04 s |\n", + "\n", + "The four selected coordinate directions were 70, 34, 58, and 49 (zero-based); coordinate 49 was selected independently in both marked cells in the fourth round. The local look-ahead gains are only about 0.84%--0.98% of the marked-cell indicator per bisection. Four refinements therefore improve all methods only mildly and do not remove the derivative-enclosure bottleneck. The exhaustive selector is intentionally diagnostic: its cost is too high for production use, so a practical implementation should approximate this ranking with a cheaper sensitivity score once the useful coordinate pattern is understood.\n" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-refined-activation-intro", + "metadata": {}, + "source": [ + "### Derivative-approximation errors on the final six cells\n", + "\n", + "This final diagnostic reruns the Top-96 one-jet trace on the six active cells and reports the volume-weighted mean $\\tanh'$ residual, the worst residual, and the volume-weighted fraction of neuron intervals still crossing zero.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "adaptive-refined-activation-code", + "metadata": {}, + "outputs": [], + "source": [ + "refined_activation_rows = []\n", + "for cell_index, active in enumerate(interval_cells):\n", + " cell = PZIntegrationCell.from_affine_box(active['box'])\n", + " traced = model.eval_pz_onejet(\n", + " cell.domain, return_trace=True, reduction_strategy='topk', max_terms=96\n", + " )\n", + " activation_trace = [record for record in traced.records if record.layer_type == 'Tanh']\n", + " cell_volume = float(cell.volume)\n", + " for layer_index, record in enumerate(activation_trace, start=1):\n", + " lower = record.summary['preactivation_lower']\n", + " upper = record.summary['preactivation_upper']\n", + " delta = record.summary['tanh_prime_approximation_radii']\n", + " refined_activation_rows.append({\n", + " 'cell': cell_index, 'hidden_layer': layer_index,\n", + " 'cell_volume': cell_volume,\n", + " 'mean_tanh_prime_delta': float(delta.mean()),\n", + " 'max_tanh_prime_delta': float(delta.max()),\n", + " 'zero_crossing_fraction': float(((lower <= 0) & (upper >= 0)).double().mean()),\n", + " })\n", + "refined_activation_cells = pd.DataFrame(refined_activation_rows)\n", + "refined_activation_summary = pd.DataFrame([\n", + " {\n", + " 'hidden_layer': layer_index,\n", + " 'initial_mean_delta': tanh_prime_approximation_errors[f'hidden_layer_{layer_index}'].mean(),\n", + " 'refined_volume_weighted_mean_delta': np.average(\n", + " group['mean_tanh_prime_delta'], weights=group['cell_volume']\n", + " ),\n", + " 'refined_max_delta': group['max_tanh_prime_delta'].max(),\n", + " 'refined_volume_weighted_zero_crossing_fraction': np.average(\n", + " group['zero_crossing_fraction'], weights=group['cell_volume']\n", + " ),\n", + " }\n", + " for layer_index, group in refined_activation_cells.groupby('hidden_layer')\n", + "]).set_index('hidden_layer')\n", + "refined_activation_summary\n" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-refined-activation-observed", + "metadata": {}, + "source": [ + "| Hidden layer | Initial mean $\\delta$ | Refined volume-weighted mean $\\delta$ | Refined maximum $\\delta$ | Refined zero-crossing fraction |\n", + "|---:|---:|---:|---:|---:|\n", + "| 1 | 0.275129 | 0.270346 | 0.317705 | 100% |\n", + "| 2 | 0.305928 | 0.300190 | 0.353773 | 100% |\n", + "| 3 | 0.359351 | 0.352820 | 0.420270 | 100% |\n", + "\n", + "Every neuron interval on every final cell still crosses zero. The four bisections reduce the mean derivative residual by only about 1.7%--1.8%, so the central $\\operatorname{sech}^2$ bump remains present everywhere. This directly explains why four refinement rounds improve the final $W^{1,2}$ widths only mildly.\n" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-edge-rankings", + "metadata": {}, + "source": [ + "### Edge-choice diagnostics\n", + "\n", + "The table shows the five best coordinate candidates for every marked cell. Coordinate indices are zero-based, matching the network input convention.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "adaptive-edge-table", + "metadata": {}, + "outputs": [], + "source": [ + "edge_choice_table" + ] + }, + { + "cell_type": "markdown", + "id": "adaptive-refinement-plots", + "metadata": {}, + "source": [ + "### Refinement convergence\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "adaptive-refinement-plot-code", + "metadata": {}, + "outputs": [], + "source": [ + "fig, axes = plt.subplots(1, 2, figsize=(12, 4.5))\n", + "for method, group in adaptive_refinement_table.groupby('method'):\n", + " axes[0].plot(\n", + " group['refinement_step'], group['normalized_absolute_width'],\n", + " marker='o', label=method,\n", + " )\n", + "axes[0].set(\n", + " xlabel='refinement step', ylabel=r'normalized $W^{1,2}$ interval width',\n", + " title='Certified-width reduction on the shared partition',\n", + ")\n", + "axes[0].grid(alpha=0.25); axes[0].legend()\n", + "\n", + "chosen_edges = edge_choice_table[edge_choice_table['rank'] == 1]\n", + "edge_counts = chosen_edges['split_dim'].value_counts().sort_index()\n", + "axes[1].bar(edge_counts.index.astype(str), edge_counts.values)\n", + "axes[1].set(\n", + " xlabel='selected coordinate (zero-based)', ylabel='number of selected bisections',\n", + " title='Coordinates selected by one-step look-ahead',\n", + ")\n", + "axes[1].tick_params(axis='x', rotation=45)\n", + "axes[1].grid(axis='y', alpha=0.25)\n", + "fig.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "quadratic-derivative-intro", + "metadata": {}, + "source": [ + "## No-refinement hybrid quadratic $\\tanh'$ experiment\n", + "\n", + "This experiment keeps the original 100D cell and replaces the affine $\\tanh'$ enclosure only when the secant slope is flat relative to the derivative interval radius. The quadratic proposal interpolates $\\tanh'$ at the lower endpoint, midpoint, and upper endpoint. A fixed 64-bin, non-adaptive Taylor-form pass certifies and recenters its residual band; there is no fitting iteration, optimizer, root search, or domain refinement. A candidate is used only when its certified residual is smaller than the affine residual.\n", + "\n", + "The relative flatness score is $|p|h/r_I$, where $p$ is the affine secant slope, $h=(u-l)/2$, and $r_I$ is the interval radius of $\\tanh'([l,u])$. The broad threshold $0.1$ selects all 150 bump-containing intervals; thresholds $0.0025$, $0.005$, $0.01$, and $0.02$ select respectively $(1,2,10,26)$ neurons across all layers. The implementation streams quadratic monomials through Top-$k$/PCA instead of first materializing the full squared PZ. It separately records the approximation residual and the radius caused by reducing the quadratic polynomial core.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "quadratic-derivative-benchmark", + "metadata": {}, + "outputs": [], + "source": [ + "quadratic_configs = [\n", + " ('quadratic-all-topk-96', 'topk', dict(\n", + " max_terms=96, derivative_enclosure='quadratic_flat',\n", + " derivative_flatness_threshold=0.1, quadratic_compression_guard=False,\n", + " )),\n", + " ('quadratic-all-pca-64', 'pca', dict(\n", + " max_terms=64, pca_rank=4, pca_candidates=32,\n", + " derivative_enclosure='quadratic_flat',\n", + " derivative_flatness_threshold=0.1, quadratic_compression_guard=False,\n", + " )),\n", + " ('quadratic-superflat-topk-96', 'topk', dict(\n", + " max_terms=96, derivative_enclosure='quadratic_flat',\n", + " derivative_flatness_threshold=0.0025, quadratic_compression_guard=False,\n", + " )),\n", + " ('quadratic-guarded-topk-96', 'topk', dict(\n", + " max_terms=96, derivative_enclosure='quadratic_flat',\n", + " derivative_flatness_threshold=0.1, quadratic_compression_guard=True,\n", + " )),\n", + "]\n", + "quadratic_rows = []\n", + "quadratic_domain_volume = 0.2 ** 100\n", + "quadratic_layer_rows = []\n", + "for label, strategy, kwargs in quadratic_configs:\n", + " result = benchmark(model, box, strategy=strategy, **kwargs)\n", + " quadratic_rows.append({\n", + " 'method': label, 'total_s': result['total_s'],\n", + " 'normalized_W12_width': (\n", + " result['W12_absolute_width'] / sqrt(quadratic_domain_volume)\n", + " ),\n", + " 'J_mean_width': result['J_mean_component_width_before_integration'],\n", + " 'J_terms': result['J_terms'], 'J_degree': result['J_degree'],\n", + " })\n", + " for layer_index, record in enumerate(\n", + " [item for item in result['trace'] if item.layer_type == 'Tanh'], start=1\n", + " ):\n", + " summary = record.summary\n", + " quadratic_layer_rows.append({\n", + " 'method': label, 'hidden_layer': layer_index,\n", + " 'quadratic_neurons': summary['tanh_prime_quadratic_count'],\n", + " 'mean_selected_delta': summary['tanh_prime_approximation_radius_mean'],\n", + " 'mean_affine_delta': float(summary['tanh_prime_affine_radii'].mean()),\n", + " 'mean_quadratic_reduction_radius': (\n", + " summary['tanh_prime_polynomial_reduction_radius_mean']\n", + " ),\n", + " 'mean_propagated_J_remainder': summary['J']['remainder_mean_radius'],\n", + " })\n", + "quadratic_benchmark_table = pd.DataFrame(quadratic_rows).set_index('method')\n", + "quadratic_layer_table = pd.DataFrame(quadratic_layer_rows).set_index(\n", + " ['method', 'hidden_layer']\n", + ")\n", + "display(quadratic_benchmark_table)\n", + "quadratic_layer_table\n" + ] + }, + { + "cell_type": "markdown", + "id": "quadratic-derivative-observed", + "metadata": {}, + "source": [ + "### Observed no-refinement result\n", + "\n", + "| Method | Quadratic neurons by layer | Time | Normalized $W^{1,2}$ width | Mean Jacobian width |\n", + "|---|---:|---:|---:|---:|\n", + "| Interval | -- | 0.115 s | 88.8468 | 17.5488 |\n", + "| Affine Top-96 | $(0,0,0)$ | 2.03 s | 85.3702 | 16.8499 |\n", + "| Affine PCA-64 | $(0,0,0)$ | 1.67 s | 85.9004 | 16.9556 |\n", + "| Quadratic-all Top-96 | $(50,50,50)$ | 9.75 s | 480.6166 | 95.3806 |\n", + "| Quadratic-all PCA-64 | $(50,50,50)$ | 7.59 s | 485.4710 | 96.3479 |\n", + "| Quadratic-superflat Top-96 ($0.0025$) | $(0,0,1)$ | 2.30 s | 85.5927 | 16.8942 |\n", + "| Compression-guarded Top-96 | $(0,0,0)$ | 2.98 s | 85.3702 | 16.8499 |\n", + "\n", + "The quadratic fit itself succeeds: with all neurons selected, the mean certified local residual falls from $(0.2751,0.3059,0.3594)$ to $(0.0385,0.0520,0.0821)$. The failure occurs when the new degree-two support is reduced. For Top-96, the mean quadratic-core reduction radii are approximately $(0.545,0.613,0.717)$, and the propagated mean Jacobian remainder grows from $(0.0264,0.1738,1.0680)$ to $(0.0560,0.5814,6.0930)$. PCA-64 behaves similarly. Even switching only the single flattest neuron slightly worsens the final width.\n", + "\n", + "The compression guard compares, componentwise, $\\delta_2+r_{\\mathrm{quad}}$ with the original affine $\\delta_1$ and falls back to the affine enclosure unless the retained quadratic plus its certified reduction radius is locally better. At Top-96 it rejects all 150 candidates, reproducing the affine certificate exactly but with diagnostic overhead. Thus the negative result is not that the parabola is inaccurate; it is that the current monomial-wise Top-$k$/PCA representation cannot retain its structured quadratic form economically. A useful next algorithmic step would need grouped or factored quadratic terms rather than merely a larger $k$.\n" + ] + }, + { + "cell_type": "markdown", + "id": "66bc9230", + "metadata": {}, + "source": [ + "## Interpretation\n", + "\n", + "- Top-k is the cheapest reduction and gives a direct runtime/tightness knob.\n", + "- Both squared integrations return scalar PZs. Pointwise uncertainty becomes a fresh integrated global generator; intervalization occurs only after this integration step.\n", + "- For both $L^2$ and $W^{1,2}$, the primary final tightness diagnostic is $U-L$ for the certified norm interval $[L,U]$. The reported relative width is $(U-L)/\\max(|L|,|U|)$ (and is defined as zero when both endpoints vanish); it is not obtained by dividing by the lower bound.\n", + "- The mean, maximum, and relative mean Jacobian component widths are complementary pre-integration diagnostics: they show how much tightness has already been lost in the image enclosure, before squaring and integration can add further overestimation. The relative mean averages the entrywise width divided by the largest endpoint magnitude, so it is scale-normalized and lies between zero and two.\n", + "- In the target experiment every method currently has $L=0$ for both norms, hence every relative norm width is $100\\%$. Here a smaller upper endpoint happens to equal a smaller absolute width, but it does not constitute an improvement in relative precision.\n", + "- The target-network relative mean Jacobian widths are close to two. This says that most component intervals straddle zero and are nearly symmetric relative to their endpoint magnitude. The absolute mean and maximum widths therefore remain the more discriminating Jacobian diagnostics in this experiment.\n", + "- The full per-neuron tables separate the initial activation-value error from the derivative error. Across hidden layers 1--3, the mean $\\tanh$ radii are approximately $0.0742$, $0.0924$, and $0.1330$, whereas the mean $\\tanh'$ radii are approximately $0.2751$, $0.3059$, and $0.3594$. Thus the derivative enclosure is already the larger local error source before Jacobian multiplication and support reduction.\n", + "- Comparing with interval arithmetic explains the contrasting norm results. The mean full interval radii for $\\tanh(I)$ are approximately $0.7414$, $0.7818$, and $0.8473$, about an order of magnitude larger than the affine value residuals. In contrast, the mean interval radii for $\\tanh'(I)$ are $0.2795$, $0.3124$, and $0.3670$, only slightly larger than the affine derivative residuals $0.2751$, $0.3059$, and $0.3594$. The affine value enclosure therefore preserves substantial dependency information, while the affine derivative enclosure is already nearly as uncertain as replacing $\\tanh'$ by its interval range. This is the local mechanism behind the strong $L^2$ improvement but weak $W^{1,2}$ improvement.\n", + "- The derivative problem is systematic rather than caused by a few outliers. Every one of the 150 preactivation intervals crosses zero, so every interval contains the central maximum of $\\tanh'=\\operatorname{sech}^2$. The layerwise median residuals are approximately $0.276$, $0.306$, and $0.358$, and the mean ratios $\\delta/\\operatorname{rad}(\\tanh'(I))$ are $98.4\\%$, $97.9\\%$, and $97.9\\%$. The worst case, hidden layer 3 neuron 32 on $[-1.576,1.613]$, has $\\delta=0.42388$ versus interval radius $0.42649$. Its nearly zero secant slope makes the affine line essentially constant, so the band must cover almost the entire bump height.\n", + "- Four Dörfler rounds on the 100D Poisson cube do not yet change that geometry. The shared look-ahead partition has six cells, but every hidden-neuron preactivation interval on every cell still crosses zero. The volume-weighted mean $\\tanh'$ residuals fall only from $(0.2751,0.3059,0.3594)$ to $(0.2703,0.3002,0.3528)$. Consequently Top-96 improves from $85.3702$ to $84.1436$ in normalized $W^{1,2}$ width, while interval arithmetic improves from $88.8468$ to $87.8151$.\n", + "- Exhaustive 100-coordinate look-ahead is useful as a diagnostic but not as the production edge selector: it costs about $191$ seconds for four rounds, compared with about $22$ seconds for the Top-96 certifications on the final partition. The selected coordinates and candidate rankings can be used to validate a cheaper sensitivity-based proxy.\n", + "- The no-refinement quadratic experiment sharply reduces the local $\\tanh'$ approximation residuals, but ordinary monomial-wise reduction loses the gain. Quadratic-all Top-96 increases the normalized $W^{1,2}$ width from $85.3702$ to $480.6166$ because the new quadratic-core reduction radii exceed the old affine residuals. A compression-aware guard rejects every quadratic candidate at Top-96 and exactly recovers the affine result. This points to structured/factored quadratic retention, not a larger unstructured Top-$k$, as the relevant next representation change.\n", + "- Degree capping matters once higher-degree terms survive the importance ranking; on narrow boxes it can coincide with top-k.\n", + "- PCA is certified because the projected generators are intervalized in PCA coordinates and the orthogonal residual is bounded componentwise. Its SVD and added pointwise generators must earn their cost empirically.\n", + "- Extending the target sweep from Top-96 through Top-256 shows clear saturation: Top-256 remains just below three seconds in this run but improves the $W^{1,2}$ width by only about $0.49\\%$ relative to Top-96. Top-192 is the more robust sub-three-second accuracy-biased configuration.\n", + "- The unreduced polynomial path is intentionally limited to smaller networks: it diagnoses genuine monomial growth rather than hiding it behind interval propagation." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/reproduce_lp_w1p_experiments.ipynb b/notebooks/reproduce_lp_w1p_experiments.ipynb index 469fead..c3805c4 100644 --- a/notebooks/reproduce_lp_w1p_experiments.ipynb +++ b/notebooks/reproduce_lp_w1p_experiments.ipynb @@ -9,7 +9,7 @@ "\n", "This notebook reproduces the paper-style experiments for **Lp** and **W1p** and intentionally excludes **W2p**.\n", "\n", - "⚠️ Stability note: this version uses **chunked Monte Carlo** for W1p and configurable quick settings to avoid kernel OOM/kill.\n" + "\u26a0\ufe0f Stability note: this version uses **chunked Monte Carlo** for W1p and configurable quick settings to avoid kernel OOM/kill.\n" ] }, { @@ -59,8 +59,20 @@ "\n", "from intervalnets import IntervalTensor, enable_interval_eval\n", "\n", - "enable_interval_eval()\n", - "torch.set_num_threads(max(1, min(torch.get_num_threads(), 8)))\n" + "ENABLE_INTERVAL_EVAL_KWARGS = {}\n", + "enable_interval_eval(**ENABLE_INTERVAL_EVAL_KWARGS)\n", + "torch.set_num_threads(max(1, min(torch.get_num_threads(), 8)))\n", + "\n", + "print(\"\\n=== Configuration summary [IA] ===\")\n", + "print(f\"domain type: {IntervalTensor.__name__}\")\n", + "print(\n", + " \"enable_interval_eval settings: \"\n", + " f\"enclosure_mode={ENABLE_INTERVAL_EVAL_KWARGS.get('enclosure_mode', 'None')}, \"\n", + " f\"affine_tanh_mode={ENABLE_INTERVAL_EVAL_KWARGS.get('affine_tanh_mode', 'None')}\"\n", + ")\n", + "print(\"norm paths:\")\n", + "print(f\" lp -> model.lpnorm(..., domain={IntervalTensor.__name__})\")\n", + "print(f\" w1p -> model.sobolev_norm(..., domain={IntervalTensor.__name__})\")\n" ] }, { @@ -360,7 +372,7 @@ "id": "c3008279", "metadata": {}, "source": [ - "## Figure A — 1D W1p (untrained vs trained)\n", + "## Figure A [IA] \u2014 1D W1p (untrained vs trained)\n", "\n", "Short description of the experiment/test performed in the following code cell.\n" ] @@ -400,8 +412,8 @@ "plt.close(\"all\")\n", "fig, axes = plt.subplots(1, 2, figsize=(12, 4), sharey=True)\n", "for ax, deep_arr, wide_arr, title in [\n", - " (axes[0], w1p_deep_untrained, w1p_wide_untrained, \"Untrained tanh networks\"),\n", - " (axes[1], w1p_deep_trained, w1p_wide_trained, \"Trained tanh networks (Gaussian peak)\"),\n", + " (axes[0], w1p_deep_untrained, w1p_wide_untrained, \"[IA] Untrained tanh networks\"),\n", + " (axes[1], w1p_deep_trained, w1p_wide_trained, \"[IA] Trained tanh networks (Gaussian peak)\"),\n", "]:\n", " for label, arr, color in [(f\"deep (3x{DEEP_WIDTH})\", deep_arr, \"tab:blue\"), (f\"wide (1x{WIDE_WIDTH})\", wide_arr, \"tab:orange\")]:\n", " plot_ci_curve(ax, ITERATIONS, arr, label=label, color=color, ci_style=PLOT_CI_STYLE)\n", @@ -412,7 +424,7 @@ "\n", "axes[0].set_ylabel(\"normalized global bound gap\")\n", "axes[0].legend()\n", - "fig.suptitle(\"1D W1p reproduction\")\n", + "fig.suptitle(\"[IA] 1D W1p reproduction\")\n", "plt.tight_layout()\n", "finalize_figure(fig, \"figure_a_w1p_1d.png\")" ] @@ -422,7 +434,7 @@ "id": "e2d46b0d", "metadata": {}, "source": [ - "## Figure B — 1D Lp (untrained vs trained)\n", + "## Figure B [IA] \u2014 1D Lp (untrained vs trained)\n", "\n", "Short description of the experiment/test performed in the following code cell.\n" ] @@ -462,8 +474,8 @@ "plt.close(\"all\")\n", "fig, axes = plt.subplots(1, 2, figsize=(12, 4), sharey=True)\n", "for ax, deep_arr, wide_arr, title in [\n", - " (axes[0], lp_deep_untrained, lp_wide_untrained, \"Untrained ReLU networks\"),\n", - " (axes[1], lp_deep_trained, lp_wide_trained, \"Trained ReLU networks (Gaussian peak)\"),\n", + " (axes[0], lp_deep_untrained, lp_wide_untrained, \"[IA] Untrained ReLU networks\"),\n", + " (axes[1], lp_deep_trained, lp_wide_trained, \"[IA] Trained ReLU networks (Gaussian peak)\"),\n", "]:\n", " for label, arr, color in [(f\"deep (3x{DEEP_WIDTH})\", deep_arr, \"tab:green\"), (f\"wide (1x{WIDE_WIDTH})\", wide_arr, \"tab:red\")]:\n", " plot_ci_curve(ax, ITERATIONS, arr, label=label, color=color, ci_style=PLOT_CI_STYLE)\n", @@ -474,7 +486,7 @@ "\n", "axes[0].set_ylabel(\"normalized global bound gap\")\n", "axes[0].legend()\n", - "fig.suptitle(\"1D Lp reproduction\")\n", + "fig.suptitle(\"[IA] 1D Lp reproduction\")\n", "plt.tight_layout()\n", "finalize_figure(fig, \"figure_b_lp_1d.png\")" ] @@ -484,7 +496,7 @@ "id": "fe3e7864", "metadata": {}, "source": [ - "## 2D trained experiments (Figure C + D)\n", + "## 2D trained experiments [IA] (Figure C + D)\n", "\n", "Short description of the experiment/test performed in the following code cell.\n" ] @@ -566,7 +578,7 @@ "fig, axes = plt.subplots(1, 2, figsize=(12, 4), sharey=True)\n", "axes[0].plot(ITERATIONS, lp_curve_deep, marker='o', label=f'deep (3x{DEEP_WIDTH})')\n", "axes[0].plot(ITERATIONS, lp_curve_wide, marker='o', label=f'wide (1x{WIDE_WIDTH})')\n", - "axes[0].set_title('2D trained Lp (ReLU)')\n", + "axes[0].set_title('[IA] 2D trained Lp (ReLU)')\n", "axes[0].set_yscale('log')\n", "axes[0].set_xlabel('refinement iterations')\n", "axes[0].set_ylabel('normalized global bound gap')\n", @@ -575,7 +587,7 @@ "\n", "axes[1].plot(ITERATIONS, w1_curve_deep, marker='o', label=f'deep (3x{DEEP_WIDTH})')\n", "axes[1].plot(ITERATIONS, w1_curve_wide, marker='o', label=f'wide (1x{WIDE_WIDTH})')\n", - "axes[1].set_title('2D trained W1p (tanh)')\n", + "axes[1].set_title('[IA] 2D trained W1p (tanh)')\n", "axes[1].set_yscale('log')\n", "axes[1].set_xlabel('refinement iterations')\n", "axes[1].grid(True, alpha=0.3)\n", @@ -597,8 +609,8 @@ "h_lp = local_gap_heatmap(lp_deep_2d, 'lp')\n", "h_w1 = local_gap_heatmap(w1p_deep_2d, 'w1p')\n", "fig, axs = plt.subplots(1,2,figsize=(10,4))\n", - "axs[0].imshow(h_lp, origin='lower', extent=[-1,1,-1,1], cmap='magma'); axs[0].set_title('Lp local gap (deep)')\n", - "axs[1].imshow(h_w1, origin='lower', extent=[-1,1,-1,1], cmap='magma'); axs[1].set_title('W1p local gap (deep)')\n", + "axs[0].imshow(h_lp, origin='lower', extent=[-1,1,-1,1], cmap='magma'); axs[0].set_title('[IA] Lp local gap (deep)')\n", + "axs[1].imshow(h_w1, origin='lower', extent=[-1,1,-1,1], cmap='magma'); axs[1].set_title('[IA] W1p local gap (deep)')\n", "plt.tight_layout(); finalize_figure(fig, \"figure_d_local_gap_heatmaps.png\")" ] }, @@ -633,4 +645,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} +} \ No newline at end of file diff --git a/notebooks/test_suite.ipynb b/notebooks/test_suite.ipynb index 46e4a2e..40d4db2 100644 --- a/notebooks/test_suite.ipynb +++ b/notebooks/test_suite.ipynb @@ -74,7 +74,7 @@ "source": [ "# Notebook step 3: run the example/test logic for this section.\n", "def report_pass(name: str) -> None:\n", - " print(f\"✅ PASS: {name}\")\n" + " print(f\"\u2705 PASS: {name}\")\n" ] }, { @@ -98,7 +98,7 @@ "output_type": "stream", "text": [ "Interval(midpoint=(0.09829720301431345, 0.07495725832159605), radius=(0.044977176033279345, 0.009431324640595318), lower=(0.0533200269810341, 0.06552593368100074), upper=(0.1432743790475928, 0.08438858296219137))\n", - "✅ PASS: Random linear network interval eval executes and returns non-degenerate bounds\n" + "\u2705 PASS: Random linear network interval eval executes and returns non-degenerate bounds\n" ] } ], @@ -140,7 +140,7 @@ "output_type": "stream", "text": [ "Interval(midpoint=(0.0, 0.0), radius=(5e-324, 5e-324), lower=(-5e-324, -5e-324), upper=(5e-324, 5e-324))\n", - "✅ PASS: Zero network output encloses zero with outward rounding\n" + "\u2705 PASS: Zero network output encloses zero with outward rounding\n" ] } ], @@ -180,7 +180,7 @@ "output_type": "stream", "text": [ "Interval(midpoint=(-3.75,), radius=(1.2500000000000024,), lower=(-5.000000000000003,), upper=(-2.499999999999998,))\n", - "✅ PASS: Hand-computable linear network encloses exact corner evaluations\n" + "\u2705 PASS: Hand-computable linear network encloses exact corner evaluations\n" ] } ], @@ -226,7 +226,7 @@ "output_type": "stream", "text": [ "Interval(midpoint=(0.0, 4.5), radius=(1.0000000000000004, 0.5000000000000016), lower=(-1.0000000000000004, 3.9999999999999987), upper=(1.0000000000000004, 5.000000000000002))\n", - "✅ PASS: Identity-style network preserves interval endpoints\n" + "\u2705 PASS: Identity-style network preserves interval endpoints\n" ] } ], @@ -267,10 +267,10 @@ "name": "stdout", "output_type": "stream", "text": [ - "✅ PASS: ReLU negative interval rounds outward to zero\n", - "✅ PASS: ReLU positive interval preserves endpoint images\n", - "✅ PASS: ReLU mixed interval clamps only the lower endpoint\n", - "✅ PASS: Linear -> ReLU -> Linear network encloses endpoint evaluations\n", + "\u2705 PASS: ReLU negative interval rounds outward to zero\n", + "\u2705 PASS: ReLU positive interval preserves endpoint images\n", + "\u2705 PASS: ReLU mixed interval clamps only the lower endpoint\n", + "\u2705 PASS: Linear -> ReLU -> Linear network encloses endpoint evaluations\n", "All ReLU notebook tests passed.\n" ] } @@ -905,7 +905,159 @@ "id": "0f7f65a7", "metadata": {}, "outputs": [], - "source": [] + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from scipy import integrate\n", + "\n", + "mu = np.array([0.5, 0.5])\n", + "sigma2 = 0.005\n", + "f = lambda x, y: 10*np.exp(-((x - mu[0])**2 + (y - mu[1])**2) / (2 * sigma2))\n", + "\n", + "\n", + "def interval_extension_eval(x_int, y_int):\n", + " \"\"\"Natural interval extension enclosure for f over rectangle x_int x y_int.\"\"\"\n", + " x_lo, x_hi = x_int\n", + " y_lo, y_hi = y_int\n", + "\n", + " # Bounds for squared distance to mu in x and y\n", + " def sqdist_bounds(lo, hi, c):\n", + " vals = [(lo - c) ** 2, (hi - c) ** 2]\n", + " if lo <= c <= hi:\n", + " return 0.0, max(vals)\n", + " return min(vals), max(vals)\n", + "\n", + " sx_lo, sx_hi = sqdist_bounds(x_lo, x_hi, mu[0])\n", + " sy_lo, sy_hi = sqdist_bounds(y_lo, y_hi, mu[1])\n", + " s_lo, s_hi = sx_lo + sy_lo, sx_hi + sy_hi\n", + "\n", + " # Monotone map through exp(-s/(2*sigma2))\n", + " f_lo = 10 * np.exp(-s_hi / (2 * sigma2))\n", + " f_hi = 10 * np.exp(-s_lo / (2 * sigma2))\n", + " return f_lo, f_hi\n", + "\n", + "\n", + "def midpoint_rect(rect):\n", + " x0, x1, y0, y1 = rect\n", + " return 0.5 * (x0 + x1), 0.5 * (y0 + y1)\n", + "\n", + "\n", + "def area_rect(rect):\n", + " x0, x1, y0, y1 = rect\n", + " return (x1 - x0) * (y1 - y0)\n", + "\n", + "\n", + "def midpoint_integral(parts):\n", + " val = 0.0\n", + " for r in parts:\n", + " xm, ym = midpoint_rect(r)\n", + " val += area_rect(r) * f(xm, ym)\n", + " return val\n", + "\n", + "\n", + "def local_indicator(rect):\n", + " # enclosure-based indicator: width of interval enclosure times cell area\n", + " x0, x1, y0, y1 = rect\n", + " lo, hi = interval_extension_eval((x0, x1), (y0, y1))\n", + " return (hi - lo) * area_rect(rect)\n", + "\n", + "\n", + "def refine_longest_side(rect):\n", + " x0, x1, y0, y1 = rect\n", + " hx, hy = x1 - x0, y1 - y0\n", + " if hx >= hy:\n", + " xm = 0.5 * (x0 + x1)\n", + " return [(x0, xm, y0, y1), (xm, x1, y0, y1)]\n", + " ym = 0.5 * (y0 + y1)\n", + " return [(x0, x1, y0, ym), (x0, x1, ym, y1)]\n", + "\n", + "\n", + "def adaquad(parts, theta=0.3, n_iter=20):\n", + " \"\"\"Dorfler marking + longest-side bisection refinement.\"\"\"\n", + " for _ in range(n_iter):\n", + " indicators = np.array([local_indicator(r) for r in parts])\n", + " total = indicators.sum()\n", + " order = np.argsort(indicators)[::-1]\n", + "\n", + " marked = []\n", + " acc = 0.0\n", + " for j in order:\n", + " marked.append(j)\n", + " acc += indicators[j]\n", + " if acc >= theta * total:\n", + " break\n", + "\n", + " marked_set = set(marked)\n", + " new_parts = []\n", + " for i, r in enumerate(parts):\n", + " if i in marked_set:\n", + " new_parts.extend(refine_longest_side(r))\n", + " else:\n", + " new_parts.append(r)\n", + " parts = new_parts\n", + " return parts\n", + "\n", + "\n", + "true_val, _ = integrate.dblquad(f, 0, 1, lambda _: 0, lambda _: 1)\n", + "\n", + "# AdaQuad setup requested by user\n", + "partition = [(0.0, 1.0, 0.0, 1.0)]\n", + "partition = adaquad(partition, theta=0.3, n_iter=20)\n", + "\n", + "mid_x = np.array([midpoint_rect(r)[0] for r in partition])\n", + "mid_y = np.array([midpoint_rect(r)[1] for r in partition])\n", + "mid_z = f(mid_x, mid_y)\n", + "\n", + "# Surface grid for visualization\n", + "grid = np.linspace(0, 1, 400)\n", + "X, Y = np.meshgrid(grid, grid)\n", + "Z = f(X, Y)\n", + "\n", + "fig = plt.figure(figsize=(8, 6))\n", + "ax = fig.add_subplot(111, projection='3d')\n", + "ax.plot_surface(X, Y, Z, cmap='Reds', alpha=0.7, rcount=120, ccount=120)\n", + "\n", + "# Midpoints of final adaptive partition\n", + "ax.scatter(mid_x, mid_y, mid_z, color='black', s=20, alpha=0.9, label='AdaQuad partition midpoints')\n", + "\n", + "# Partition grid on xy-plane\n", + "for x0, x1, y0, y1 in partition:\n", + " ax.plot([x0, x1], [y0, y0], [0, 0], color='royalblue', linewidth=0.8, alpha=0.7)\n", + " ax.plot([x0, x1], [y1, y1], [0, 0], color='royalblue', linewidth=0.8, alpha=0.7)\n", + " ax.plot([x0, x0], [y0, y1], [0, 0], color='royalblue', linewidth=0.8, alpha=0.7)\n", + " ax.plot([x1, x1], [y0, y1], [0, 0], color='royalblue', linewidth=0.8, alpha=0.7)\n", + "\n", + "ax.set_xlabel('x')\n", + "ax.set_ylabel('y')\n", + "ax.set_zlabel('f(x, y)')\n", + "ax.legend(loc='upper right')\n", + "plt.tight_layout()\n", + "plt.savefig('spike_2d_surface_adaquad.pdf', dpi=150)\n", + "plt.show()\n", + "\n", + "# Certified integration interval from area-weighted local enclosures\n", + "cert_lo = 0.0\n", + "cert_hi = 0.0\n", + "for r in partition:\n", + " x0, x1, y0, y1 = r\n", + " lo, hi = interval_extension_eval((x0, x1), (y0, y1))\n", + " a = area_rect(r)\n", + " cert_lo += a * lo\n", + " cert_hi += a * hi\n", + "cert_width = cert_hi - cert_lo\n", + "\n", + "midpoint_val = midpoint_integral(partition)\n", + "rel_err = abs(midpoint_val - true_val) / abs(true_val)\n", + "\n", + "print(f\"True integral: {true_val:.6f}\")\n", + "print(f\"Midpoint estimate on AdaQuad partition: {midpoint_val:.6f}\")\n", + "print(f\"Relative integration error (AdaQuad midpoint): {rel_err:.3e}\")\n", + "print(f\"Number of final partition cells: {len(partition)}\")\n", + "print(f\"Certified integration interval: [{cert_lo:.6f}, {cert_hi:.6f}]\")\n", + "print(f\"Certified interval width: {cert_width:.3e}\")\n", + "\n", + "\n" + ] }, { "cell_type": "code", @@ -939,4 +1091,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} +} \ No newline at end of file diff --git a/src/intervalnets/__init__.py b/src/intervalnets/__init__.py index 72ff9cb..62b98ee 100644 --- a/src/intervalnets/__init__.py +++ b/src/intervalnets/__init__.py @@ -1,8 +1,149 @@ """Interval arithmetic utilities for neural network evaluation.""" from .interval import Interval +from .polynomial_zonotope import ( + PZOneJet, + PZTwoJet, + PolynomialZonotope, + collect_pz_diagnostics, + pz_to_latex, + pz_to_markdown_code, + twojet_to_latex, +) +from .pz_tanh import ( + AffineTanhEnclosure, + QuadraticTanhEnclosure, + TanhApproximation, + affine_tanh_double_prime_enclosure, + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + quadratic_tanh_prime_enclosure, + certify_tanh_residual_subdivision, + compute_tanh_polynomial, + tanh_pz_scalar, +) +from .pz_integration import ( + IntegratedPZResult, + PZIntegrationCell, + integrate_over_cell, + integrate_pz_over_domain, + integrate_pz_onejet_squared, + integrate_pz_value_squared, + pz_l2norm_bounds, + pz_sobolev_norm_bounds, +) -__all__ = ["Interval"] +from .pz_norms import ( + build_pz_twojet_norm_diagnostics, + pz_norm_from_integrand, + pz_sum_squares, + pz_twojet_l2_integrand, + pz_twojet_l2_norm, + pz_twojet_w12_integrand, + pz_twojet_w12_norm, + pz_twojet_w22_integrand, + pz_twojet_w22_norm, +) + +from .pinn import load_tanh_mlp_checkpoint, sequential_value_jacobian_laplacian +from .shallow_hybrid import ( + ShallowHybridOneJetResult, + integrate_shallow_hybrid_onejet_squared, + integrate_shallow_hybrid_value_squared, + shallow_scalar_hybrid_onejet_reverse, +) +from .deep_hybrid import ( + DeepHybridOneJetResult, + FactoredPolynomialJacobian, + HybridDerivativeFactor, + deep_scalar_hybrid_onejet_reverse, + integrate_deep_hybrid_onejet_squared, + integrate_deep_hybrid_value_squared, + integrate_hybrid_onejet_squared, + integrate_hybrid_value_squared, + scalar_hybrid_onejet_reverse, +) +from .graph_hilbert import ( + ArithmeticGraph, + DualWitnessCertificate, + GraphNode, + GraphNormCertificate, + HilbertGradientCertificate, + HilbertLayerDiagnostic, + HilbertValueCertificate, + SparseReferenceMomentBackend, + build_factored_jacobian_graph, + build_hilbert_value_certificate, + build_hilbert_gradient_certificate, + certify_hybrid_graph_norms, + neumann_polynomial_witness, +) + +__all__ = [ + "Interval", + "PolynomialZonotope", + "PZOneJet", + "PZTwoJet", + "collect_pz_diagnostics", + "pz_to_latex", + "pz_to_markdown_code", + "twojet_to_latex", + "AffineTanhEnclosure", + "QuadraticTanhEnclosure", + "TanhApproximation", + "affine_tanh_double_prime_enclosure", + "affine_tanh_enclosure", + "affine_tanh_prime_enclosure", + "quadratic_tanh_prime_enclosure", + "compute_tanh_polynomial", + "certify_tanh_residual_subdivision", + "tanh_pz_scalar", + "IntegratedPZResult", + "PZIntegrationCell", + "integrate_over_cell", + "integrate_pz_over_domain", + "integrate_pz_onejet_squared", + "integrate_pz_value_squared", + "pz_l2norm_bounds", + "pz_sobolev_norm_bounds", + "build_pz_twojet_norm_diagnostics", + "pz_norm_from_integrand", + "pz_sum_squares", + "pz_twojet_l2_integrand", + "pz_twojet_l2_norm", + "pz_twojet_w12_integrand", + "pz_twojet_w12_norm", + "pz_twojet_w22_integrand", + "pz_twojet_w22_norm", + "sequential_value_jacobian_laplacian", + "load_tanh_mlp_checkpoint", + "ShallowHybridOneJetResult", + "integrate_shallow_hybrid_onejet_squared", + "integrate_shallow_hybrid_value_squared", + "shallow_scalar_hybrid_onejet_reverse", + "HybridDerivativeFactor", + "FactoredPolynomialJacobian", + "DeepHybridOneJetResult", + "deep_scalar_hybrid_onejet_reverse", + "scalar_hybrid_onejet_reverse", + "integrate_deep_hybrid_onejet_squared", + "integrate_deep_hybrid_value_squared", + "integrate_hybrid_onejet_squared", + "integrate_hybrid_value_squared", + "GraphNode", + "ArithmeticGraph", + "SparseReferenceMomentBackend", + "HilbertLayerDiagnostic", + "HilbertValueCertificate", + "DualWitnessCertificate", + "GraphNormCertificate", + "HilbertGradientCertificate", + "build_factored_jacobian_graph", + "build_hilbert_value_certificate", + "build_hilbert_gradient_certificate", + "neumann_polynomial_witness", + "certify_hybrid_graph_norms", +] try: from .pytorch import ( @@ -12,6 +153,18 @@ enable_interval_eval, interval_forward, interval_forward_refine, + pz_l2norm, + pz_onejet_forward, + pz_sobolev_norm, + pz_value_forward, + pz_twojet_forward, + PZValueTraceRecord, + PZValueTraceResult, + PZOneJetTraceRecord, + PZOneJetTraceResult, + PZReductionConfig, + PZTwoJetTraceRecord, + PZTwoJetTraceResult, ) except ImportError: # pragma: no cover - optional dependency pass @@ -24,5 +177,17 @@ "enable_interval_eval", "interval_forward", "interval_forward_refine", + "pz_l2norm", + "pz_onejet_forward", + "pz_sobolev_norm", + "pz_value_forward", + "pz_twojet_forward", + "PZValueTraceRecord", + "PZValueTraceResult", + "PZOneJetTraceRecord", + "PZOneJetTraceResult", + "PZReductionConfig", + "PZTwoJetTraceRecord", + "PZTwoJetTraceResult", ] ) diff --git a/src/intervalnets/deep_hybrid.py b/src/intervalnets/deep_hybrid.py new file mode 100644 index 0000000..ab72d15 --- /dev/null +++ b/src/intervalnets/deep_hybrid.py @@ -0,0 +1,709 @@ +"""Fast factored hybrid one-jets for deep scalar tanh networks. + +The expanded polynomial support of a deep Jacobian grows combinatorially even +when every activation enclosure is only affine. This module therefore keeps +the same polynomial exactly as an arithmetic circuit: cached affine +preactivations feed affine-or-quadratic derivative factors, and the factors are +contracted in reverse order. No monomial or approximation-noise symbol is +discarded. + +For one hidden layer the public dispatcher deliberately calls the specialized +expanded implementation in :mod:`intervalnets.shallow_hybrid`. The shallow +certificate and its direct integral are therefore recovered exactly. For two +or more hidden layers the final Jacobian remains factored. Its norm routine +integrates a retained affine domain core exactly and encloses the unexpanded +higher-order circuit by a pointwise residual. An independent, certified +operator-norm cap is intersected with that enclosure. This last integration +step is sound but can be wider than fully expanding and canonicalizing every +deep monomial; it is what makes the depth-generic path scalable. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from math import inf, nextafter, sqrt +from time import perf_counter +from typing import Any, Literal + +from .interval import Interval +from .polynomial_zonotope import PolynomialZonotope +from .pz_integration import PZIntegrationCell +from .shallow_hybrid import ( + ShallowHybridOneJetResult, + _affine_domain_coefficients, + _hybrid_activation_coefficients, + integrate_shallow_hybrid_onejet_squared, + shallow_scalar_hybrid_onejet_reverse, +) + +try: # pragma: no cover - optional dependency + import torch + from torch import nn +except ImportError: # pragma: no cover + torch = None + nn = None + + +@dataclass(frozen=True) +class HybridDerivativeFactor: + """One unexpanded componentwise derivative PZ factor. + + With ``xi`` denoting the domain and preceding value-approximation symbols, + and ``eta`` the fresh derivative-approximation symbols, the represented + factor is + + ``center + linear @ xi + quadratic * (preactivation @ xi)**2 + error * eta``. + """ + + preactivation_center: Any + preactivation_coefficients: Any + preactivation_lower: Any + preactivation_upper: Any + value_slopes: Any + value_intercepts: Any + value_approximation_radii: Any + center: Any + linear_coefficients: Any + quadratic_coefficients: Any + approximation_radii: Any + affine_approximation_radii: Any + approximation_degrees: Any + relative_slopes: Any + derivative_lower: Any + derivative_upper: Any + active_value_noise: int + derivative_noise_offset: int + + +@dataclass(frozen=True) +class FactoredPolynomialJacobian: + """Exact arithmetic-circuit representation of a scalar-output Jacobian.""" + + input_weight: Any + hidden_weights: tuple[Any, ...] + output_weight: Any + factors: tuple[HybridDerivativeFactor, ...] + num_domain_noise: int + num_value_noise: int + num_derivative_noise: int + interval_lower: Any + interval_upper: Any + + @property + def num_noise(self) -> int: + return self.num_value_noise + self.num_derivative_noise + + @property + def max_degree(self) -> int: + return sum(int(factor.approximation_degrees.max().item()) for factor in self.factors) + + def evaluate(self, noise: Any) -> Any: + """Evaluate the retained polynomial circuit at one or more noise vectors.""" + + if torch is None: + raise ImportError("PyTorch is required for factored Jacobian evaluation.") + template = self.input_weight + values = torch.as_tensor(noise, dtype=template.dtype, device=template.device) + squeeze = values.ndim == 1 + if squeeze: + values = values.unsqueeze(0) + if values.shape[-1] != self.num_noise: + raise ValueError( + f"Expected {self.num_noise} noise coordinates, got {values.shape[-1]}." + ) + + adjoint = self.output_weight.unsqueeze(0).expand(values.shape[0], -1) + derivative_base = self.num_value_noise + for layer_index in range(len(self.factors) - 1, -1, -1): + factor = self.factors[layer_index] + xi = values[:, : factor.active_value_noise] + affine_argument = xi @ factor.preactivation_coefficients.T + derivative = ( + factor.center.unsqueeze(0) + + xi @ factor.linear_coefficients.T + + factor.quadratic_coefficients.unsqueeze(0) * affine_argument.square() + ) + width = factor.center.numel() + eta = values[ + :, + derivative_base + + factor.derivative_noise_offset : derivative_base + + factor.derivative_noise_offset + + width, + ] + derivative = derivative + factor.approximation_radii.unsqueeze(0) * eta + adjoint = adjoint * derivative + if layer_index: + adjoint = adjoint @ self.hidden_weights[layer_index - 1] + gradient = adjoint @ self.input_weight + return gradient.squeeze(0) if squeeze else gradient + + def interval_enclosure(self) -> Interval: + lower = self.interval_lower.detach().cpu().tolist() + upper = self.interval_upper.detach().cpu().tolist() + return Interval.from_bounds(lower, upper) + + +@dataclass(frozen=True) +class DeepHybridOneJetResult: + """Depth-generic uncompressed factored one-jet certificate.""" + + value: PolynomialZonotope + jacobian: FactoredPolynomialJacobian + factors: tuple[HybridDerivativeFactor, ...] + value_center: Any + value_domain_coefficients: Any + value_error_generators: Any + gradient_affine_center: Any + gradient_affine_domain_coefficients: Any + gradient_pointwise_remainder: Any + gradient_spectral_bound: float + timings: dict[str, float] + + +HybridOneJetResult = ShallowHybridOneJetResult | DeepHybridOneJetResult + + +def _require_torch() -> None: + if torch is None or nn is None: + raise ImportError("PyTorch is required for deep hybrid certification.") + + +def _tanh_scalar_network(module: Any) -> tuple[list[Any], list[Any], Any]: + """Return alternating affine/activation layers for a scalar tanh MLP.""" + + children = list(module.children()) if isinstance(module, nn.Sequential) else [] + if len(children) < 3 or len(children) % 2 != 1: + raise ValueError( + "The fast hybrid path requires alternating Linear/Tanh layers and " + "one final scalar Linear layer." + ) + linears: list[Any] = [] + activations: list[Any] = [] + for index, child in enumerate(children[:-1]): + expected = nn.Linear if index % 2 == 0 else nn.Tanh + if not isinstance(child, expected): + raise ValueError( + "The fast hybrid path requires Linear -> Tanh repetitions " + "followed by one scalar Linear layer." + ) + (linears if index % 2 == 0 else activations).append(child) + output = children[-1] + if not isinstance(output, nn.Linear) or output.out_features != 1: + raise ValueError("The fast hybrid path requires one scalar Linear output.") + if len(linears) != len(activations): + raise ValueError("Every hidden Linear layer must be followed by Tanh.") + previous = linears[0].out_features + for layer in linears[1:]: + if layer.in_features != previous: + raise ValueError("Adjacent hidden layer dimensions do not match.") + previous = layer.out_features + if output.in_features != previous: + raise ValueError("The output layer dimension does not match the final hidden layer.") + return linears, activations, output + + +def _as_torch_pz(module: Any, x: PolynomialZonotope) -> PolynomialZonotope: + parameter = next(module.parameters()) + if isinstance(x.center, torch.Tensor): + return x + return PolynomialZonotope( + torch.as_tensor(x.center, dtype=parameter.dtype, device=parameter.device), + { + exponent: torch.as_tensor( + coefficient, dtype=parameter.dtype, device=parameter.device + ) + for exponent, coefficient in x.terms.items() + }, + num_noise=x.num_noise, + noise_kinds=x.noise_kinds, + ) + + +def _linear_interval_row(lower: Any, upper: Any, weight: Any) -> tuple[Any, Any]: + positive = torch.clamp(weight, min=0.0) + negative = torch.clamp(weight, max=0.0) + return lower @ positive + upper @ negative, upper @ positive + lower @ negative + + +def _multiply_intervals( + left_lower: Any, + left_upper: Any, + right_lower: Any, + right_upper: Any, +) -> tuple[Any, Any]: + candidates = torch.stack( + ( + left_lower * right_lower, + left_lower * right_upper, + left_upper * right_lower, + left_upper * right_upper, + ) + ) + return candidates.amin(dim=0), candidates.amax(dim=0) + + +def _reverse_affine_core_and_interval( + linears: list[Any], + output: Any, + factors: list[HybridDerivativeFactor], + input_dim: int, +) -> tuple[Any, Any, Any, Any, Any]: + """Retain the constant/linear domain core and enclose the exact circuit.""" + + dtype = linears[0].weight.dtype + device = linears[0].weight.device + center = output.weight.detach()[0].to(dtype=dtype, device=device) + domain_linear = torch.zeros( + (center.numel(), input_dim), dtype=dtype, device=device + ) + lower = center.clone() + upper = center.clone() + + for layer_index in range(len(factors) - 1, -1, -1): + factor = factors[layer_index] + lower, upper = _multiply_intervals( + lower, + upper, + factor.derivative_lower, + factor.derivative_upper, + ) + old_center = center + old_linear = domain_linear + derivative_linear = factor.linear_coefficients[:, :input_dim] + center = old_center * factor.center + domain_linear = ( + old_center.unsqueeze(1) * derivative_linear + + factor.center.unsqueeze(1) * old_linear + ) + if layer_index: + weight = linears[layer_index].weight.detach().to(dtype=dtype, device=device) + lower, upper = _linear_interval_row(lower, upper, weight) + center = center @ weight + domain_linear = weight.T @ domain_linear + + input_weight = linears[0].weight.detach().to(dtype=dtype, device=device) + lower, upper = _linear_interval_row(lower, upper, input_weight) + center = center @ input_weight + domain_linear = input_weight.T @ domain_linear + + affine_radius = torch.sum(torch.abs(domain_linear), dim=1) + remainder = torch.maximum( + torch.abs(lower - (center + affine_radius)), + torch.abs(upper - (center - affine_radius)), + ) + remainder = torch.nextafter(remainder, torch.full_like(remainder, torch.inf)) + return center, domain_linear, remainder, lower, upper + + +def _padded_spectral_norm(matrix: Any) -> float: + """Return a posteriori padded upper bound for the spectral norm. + + The raw leading singular value is not used on its own. We reconstruct the + SVD, bound the reconstruction residual by its Frobenius norm, and bound + the possible non-orthogonality of the computed singular vectors through + their Gram residuals. A deliberately conservative floating-point term is + added to the three dense contractions. + """ + + u, singular_values, vh = torch.linalg.svd(matrix, full_matrices=False) + reconstructed = (u * singular_values.unsqueeze(0)) @ vh + residual = float(torch.linalg.vector_norm(matrix - reconstructed).item()) + identity = torch.eye( + singular_values.numel(), dtype=matrix.dtype, device=matrix.device + ) + u_orthogonality = float(torch.linalg.vector_norm(u.T @ u - identity).item()) + v_orthogonality = float(torch.linalg.vector_norm(vh @ vh.T - identity).item()) + u_bound = sqrt(1.0 + u_orthogonality) + v_bound = sqrt(1.0 + v_orthogonality) + leading = float(singular_values[0].item()) if singular_values.numel() else 0.0 + frobenius = float(torch.linalg.vector_norm(matrix).item()) + dimension = max(matrix.shape, default=1) + eps = torch.finfo(matrix.dtype).eps + rounding = 1024.0 * eps * dimension * dimension * max(1.0, frobenius) + return nextafter(u_bound * leading * v_bound + residual + rounding, inf) + + +def _gradient_spectral_bound( + linears: list[Any], output: Any, factors: list[HybridDerivativeFactor] +) -> float: + """Independent global cap using certified per-neuron derivative maxima.""" + + output_weight = output.weight.detach() + bound = _padded_spectral_norm( + output_weight * factors[-1].derivative_upper.unsqueeze(0) + ) + for layer_index in range(len(linears) - 1, 0, -1): + weight = linears[layer_index].weight.detach() + bound *= _padded_spectral_norm( + weight * factors[layer_index - 1].derivative_upper.unsqueeze(0) + ) + bound *= _padded_spectral_norm(linears[0].weight.detach()) + return nextafter(bound, inf) + + +def deep_scalar_hybrid_onejet_reverse( + module: Any, + x: PolynomialZonotope, + *, + derivative_flatness_threshold: float = 0.01, + quadratic_certificate_subdivisions: int = 64, +) -> DeepHybridOneJetResult: + """Build a fast uncompressed factored hybrid one-jet for a deep MLP.""" + + _require_torch() + if derivative_flatness_threshold < 0.0: + raise ValueError("derivative_flatness_threshold must be non-negative.") + linears, _, output = _tanh_scalar_network(module) + if len(linears) < 2: + raise ValueError( + "Use scalar_hybrid_onejet_reverse for the depth-generic dispatcher; " + "deep_scalar_hybrid_onejet_reverse requires at least two hidden layers." + ) + x = _as_torch_pz(module, x) + support, input_coefficients = _affine_domain_coefficients(x) + input_dim = x.shape[0] + if x.num_noise != input_dim or len(support) != input_dim: + raise ValueError( + "The fast deep path requires one affine domain symbol per input coordinate." + ) + if linears[0].in_features != input_dim: + raise ValueError("Network input dimension does not match the input PZ.") + + total_start = perf_counter() + forward_start = perf_counter() + value_center = x.center + value_coefficients = input_coefficients.T + raw_factors: list[dict[str, Any]] = [] + value_widths: list[int] = [] + + for layer in linears: + weight = layer.weight.detach().to(dtype=x.center.dtype, device=x.center.device) + bias = layer.bias.detach().to(dtype=x.center.dtype, device=x.center.device) + z_center = weight @ value_center + bias + z_coefficients = weight @ value_coefficients + z_radius = torch.sum(torch.abs(z_coefficients), dim=1) + lower = torch.nextafter( + z_center - z_radius, torch.full_like(z_center, -torch.inf) + ) + upper = torch.nextafter( + z_center + z_radius, torch.full_like(z_center, torch.inf) + ) + ( + value_slopes, + value_intercepts, + value_radii, + constants, + derivative_linears, + quadratics, + derivative_radii, + affine_radii, + degrees, + relative_slopes, + ) = _hybrid_activation_coefficients( + lower, + upper, + flatness_threshold=derivative_flatness_threshold, + certificate_subdivisions=quadratic_certificate_subdivisions, + ) + derivative_center = ( + constants + + derivative_linears * z_center + + quadratics * z_center.square() + ) + derivative_linear_coefficients = ( + derivative_linears + 2.0 * quadratics * z_center + ).unsqueeze(1) * z_coefficients + endpoint_lower = 1.0 - torch.tanh(lower).square() + endpoint_upper = 1.0 - torch.tanh(upper).square() + derivative_lower = torch.minimum(endpoint_lower, endpoint_upper) + crosses_zero = (lower <= 0.0) & (upper >= 0.0) + derivative_upper = torch.where( + crosses_zero, + torch.ones_like(lower), + torch.maximum(endpoint_lower, endpoint_upper), + ) + derivative_lower = torch.nextafter( + derivative_lower, torch.full_like(derivative_lower, -torch.inf) + ) + derivative_upper = torch.nextafter( + derivative_upper, torch.full_like(derivative_upper, torch.inf) + ) + raw_factors.append( + { + "preactivation_center": z_center, + "preactivation_coefficients": z_coefficients, + "preactivation_lower": lower, + "preactivation_upper": upper, + "value_slopes": value_slopes, + "value_intercepts": value_intercepts, + "value_approximation_radii": value_radii, + "center": derivative_center, + "linear_coefficients": derivative_linear_coefficients, + "quadratic_coefficients": quadratics, + "approximation_radii": derivative_radii, + "affine_approximation_radii": affine_radii, + "approximation_degrees": degrees, + "relative_slopes": relative_slopes, + "derivative_lower": derivative_lower, + "derivative_upper": derivative_upper, + "active_value_noise": value_coefficients.shape[1], + } + ) + value_center = value_intercepts + value_slopes * z_center + value_coefficients = torch.cat( + ( + value_slopes.unsqueeze(1) * z_coefficients, + torch.diag(value_radii), + ), + dim=1, + ) + value_widths.append(layer.out_features) + + weight_out = output.weight.detach()[0].to( + dtype=x.center.dtype, device=x.center.device + ) + bias_out = output.bias.detach()[0].to(dtype=x.center.dtype, device=x.center.device) + output_center = torch.dot(weight_out, value_center) + bias_out + output_coefficients = value_coefficients.T @ weight_out + forward_seconds = perf_counter() - forward_start + + factors: list[HybridDerivativeFactor] = [] + derivative_offset = 0 + for width, raw in zip(value_widths, raw_factors): + factors.append( + HybridDerivativeFactor( + **raw, + derivative_noise_offset=derivative_offset, + ) + ) + derivative_offset += width + + value_noise_count = value_coefficients.shape[1] + derivative_noise_count = sum(value_widths) + total_noise = value_noise_count + derivative_noise_count + noise_kinds = ( + x.noise_kinds + + ("approximation_pointwise",) * (value_noise_count - x.num_noise) + + ("approximation_pointwise",) * derivative_noise_count + ) + padding = (0,) * (total_noise - x.num_noise) + value_terms: dict[tuple[int, ...], Any] = {} + for exponent, coefficient in zip(support, output_coefficients[:input_dim]): + value_terms[exponent + padding] = coefficient.unsqueeze(0) + for index, coefficient in enumerate(output_coefficients[input_dim:]): + exponent = [0] * total_noise + exponent[input_dim + index] = 1 + value_terms[tuple(exponent)] = coefficient.unsqueeze(0) + value = PolynomialZonotope( + output_center.unsqueeze(0), + value_terms, + num_noise=total_noise, + noise_kinds=noise_kinds, + ) + + reverse_start = perf_counter() + gradient_center, gradient_linear, gradient_remainder, j_lower, j_upper = ( + _reverse_affine_core_and_interval(linears, output, factors, input_dim) + ) + spectral_bound = _gradient_spectral_bound(linears, output, factors) + reverse_seconds = perf_counter() - reverse_start + + hidden_weights = tuple( + layer.weight.detach().to(dtype=x.center.dtype, device=x.center.device) + for layer in linears[1:] + ) + factored = FactoredPolynomialJacobian( + input_weight=linears[0].weight.detach().to( + dtype=x.center.dtype, device=x.center.device + ), + hidden_weights=hidden_weights, + output_weight=weight_out, + factors=tuple(factors), + num_domain_noise=input_dim, + num_value_noise=value_noise_count, + num_derivative_noise=derivative_noise_count, + interval_lower=j_lower, + interval_upper=j_upper, + ) + total_seconds = perf_counter() - total_start + return DeepHybridOneJetResult( + value=value, + jacobian=factored, + factors=tuple(factors), + value_center=output_center, + value_domain_coefficients=output_coefficients[:input_dim], + value_error_generators=output_coefficients[input_dim:], + gradient_affine_center=gradient_center, + gradient_affine_domain_coefficients=gradient_linear, + gradient_pointwise_remainder=gradient_remainder, + gradient_spectral_bound=spectral_bound, + timings={ + "forward_cache_and_value": forward_seconds, + "reverse_jacobian_certificate": reverse_seconds, + "onejet_construction": total_seconds, + }, + ) + + +def scalar_hybrid_onejet_reverse( + module: Any, + x: PolynomialZonotope, + **kwargs: Any, +) -> HybridOneJetResult: + """Depth-generic dispatcher that exactly recovers the shallow fast path.""" + + _require_torch() + linears, _, _ = _tanh_scalar_network(module) + if len(linears) == 1: + return shallow_scalar_hybrid_onejet_reverse(module, x, **kwargs) + kwargs.pop("chebyshev_degree", None) + kwargs.pop("residual_subdivisions", None) + return deep_scalar_hybrid_onejet_reverse(module, x, **kwargs) + + +def _deep_value_squared_components(result: DeepHybridOneJetResult) -> tuple[Any, Any]: + center = result.value_center + domain = result.value_domain_coefficients + errors = result.value_error_generators + diagonal = errors.square() + normalized_center = ( + center.square() + + torch.sum(domain.square()) / 3.0 + + 0.5 * torch.sum(diagonal) + ) + center_cross = 2.0 * center * errors + # The 2*alpha_j*eta_i coefficients receive E|alpha_j| = 1/2. + domain_cross = domain[:, None] * errors[None, :] + gram = errors[:, None] * errors[None, :] + rows, columns = torch.triu_indices( + len(errors), len(errors), offset=1, device=errors.device + ) + normalized_radius = ( + torch.sum(torch.abs(center_cross)) + + torch.sum(torch.abs(domain_cross)) + + 0.5 * torch.sum(torch.abs(torch.diagonal(gram))) + + 2.0 * torch.sum(torch.abs(gram[rows, columns])) + ) + return normalized_center, normalized_radius + + +def _deep_interval_from_normalized_bounds( + lower: float, + upper: float, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"], +): + if output not in {"interval", "pz"}: + raise ValueError("output must be either 'interval' or 'pz'.") + if not isinstance(cell.volume, (int, float)): + raise NotImplementedError("The deep direct integral requires a scalar cell volume.") + volume = float(cell.volume) + total = Interval.from_bounds( + nextafter(volume * lower, -inf), + nextafter(volume * upper, inf), + ) + if output == "interval": + return total + return PolynomialZonotope.constant(total.midpoint).add_independent_error( + total.radius, kind="global_symbolic_residual" + ) + + +def integrate_deep_hybrid_value_squared( + result: DeepHybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Integrate the deep affine value PZ with refined pointwise moments.""" + + center, radius = _deep_value_squared_components(result) + lower = max(0.0, float((center - radius).detach().cpu().item())) + upper = max(0.0, float((center + radius).detach().cpu().item())) + return _deep_interval_from_normalized_bounds(lower, upper, cell, output=output) + + +def integrate_deep_hybrid_onejet_squared( + result: DeepHybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Integrate the deep factored ``|Y|^2 + |J|_F^2`` enclosure. + + The affine domain core is integrated using exact uniform-box moments. The + higher-order factored circuit is retained without support reduction and is + enclosed as one pointwise residual for this integration functional. The + Jacobian upper bound is intersected with an independent spectral bound. + """ + + value_center, value_radius = _deep_value_squared_components(result) + gradient_center = result.gradient_affine_center + gradient_linear = result.gradient_affine_domain_coefficients + gradient_remainder = result.gradient_pointwise_remainder + gradient_moment_center = ( + torch.dot(gradient_center, gradient_center) + + torch.sum(gradient_linear.square()) / 3.0 + + 0.5 * torch.sum(gradient_remainder.square()) + ) + # Apply the same coefficientwise functional after collapsing the exact + # factored higher-order circuit to the certified pointwise remainder R. + # Constant*R uses moment 1, alpha_j*R uses E|alpha_j|=1/2, and R**2 is + # one-sided because it is non-negative. + gradient_radius = torch.sum( + 2.0 * torch.abs(gradient_center) * gradient_remainder + + torch.sum(torch.abs(gradient_linear), dim=1) * gradient_remainder + + 0.5 * gradient_remainder.square() + ) + gradient_lower = max( + 0.0, + float((gradient_moment_center - gradient_radius).detach().cpu().item()), + ) + envelope_upper = max( + 0.0, + float((gradient_moment_center + gradient_radius).detach().cpu().item()), + ) + spectral_upper = result.gradient_spectral_bound**2 + gradient_upper = min(envelope_upper, spectral_upper) + value_lower = max( + 0.0, float((value_center - value_radius).detach().cpu().item()) + ) + value_upper = max( + 0.0, float((value_center + value_radius).detach().cpu().item()) + ) + return _deep_interval_from_normalized_bounds( + value_lower + gradient_lower, + value_upper + gradient_upper, + cell, + output=output, + ) + + +def integrate_hybrid_onejet_squared( + result: HybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Depth-generic dispatcher for the optimized hybrid integral.""" + + if isinstance(result, ShallowHybridOneJetResult): + return integrate_shallow_hybrid_onejet_squared(result, cell, output=output) + return integrate_deep_hybrid_onejet_squared(result, cell, output=output) + + +def integrate_hybrid_value_squared( + result: HybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Depth-generic dispatcher for the optimized hybrid L2 integral.""" + + if isinstance(result, ShallowHybridOneJetResult): + from .shallow_hybrid import integrate_shallow_hybrid_value_squared + + return integrate_shallow_hybrid_value_squared(result, cell, output=output) + return integrate_deep_hybrid_value_squared(result, cell, output=output) diff --git a/src/intervalnets/graph_hilbert.py b/src/intervalnets/graph_hilbert.py new file mode 100644 index 0000000..478ea7a --- /dev/null +++ b/src/intervalnets/graph_hilbert.py @@ -0,0 +1,1035 @@ +"""Dependency-preserving graph moments and Hilbert norm certificates. + +The scalable certificate in this module deliberately separates two concerns. +Small arithmetic circuits can be expanded by :class:`SparseReferenceMomentBackend` +to provide an exact regression oracle. Large value circuits are compressed at +activation boundaries to their orthogonal affine projection in +``L2([-1,1]^d)`` plus a certified Hilbert remainder. The discarded part is +never re-labelled as independent pointwise noise. + +For ``W^{1,2}`` lower bounds we use Neumann-compatible polynomial witnesses. +Integration by parts then reduces the uncertain gradient pairing to a +value-only pairing, for which the Hilbert remainder is directly applicable. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from hashlib import sha1 +from math import comb, inf, nextafter, sqrt +from typing import Any, Iterable, Literal + +from .interval import Interval +from .polynomial_zonotope import PolynomialZonotope, box_monomial_moment +from .pz_integration import PZIntegrationCell +from .pz_tanh import compute_tanh_polynomial, quadratic_tanh_prime_enclosure + +try: # pragma: no cover - optional dependencies + import numpy as np +except ImportError: # pragma: no cover + np = None # type: ignore[assignment] + +try: # pragma: no cover - optional dependency + import torch + from torch import nn +except ImportError: # pragma: no cover + torch = None + nn = None + + +@dataclass(frozen=True) +class GraphNode: + """One immutable arithmetic-circuit node.""" + + id: int + operation: str + children: tuple[int, ...] + payload: Any + shape: tuple[int, ...] + degree: int + domain_support: frozenset[int] + residual_support: frozenset[int] + + +@dataclass(frozen=True) +class ArithmeticGraph: + """Hash-consed dependency graph with one designated output node.""" + + nodes: tuple[GraphNode, ...] + output_id: int + num_domain_noise: int + num_noise: int + + def evaluate(self, noise: Any) -> Any: + if torch is None: + raise ImportError("PyTorch is required for arithmetic-graph evaluation.") + values = torch.as_tensor(noise) + squeeze = values.ndim == 1 + if squeeze: + values = values.unsqueeze(0) + if values.shape[-1] != self.num_noise: + raise ValueError( + f"Expected {self.num_noise} noise coordinates, got {values.shape[-1]}." + ) + cache: dict[int, Any] = {} + for node in self.nodes: + children = [cache[index] for index in node.children] + if node.operation == "noise": + value = values + elif node.operation == "slice": + start, stop = node.payload + value = children[0][..., start:stop] + elif node.operation == "constant": + value = node.payload.to(dtype=values.dtype, device=values.device) + elif node.operation == "add": + value = children[0] + children[1] + elif node.operation == "hadamard": + value = children[0] * children[1] + elif node.operation == "square": + value = children[0].square() + elif node.operation == "scale": + value = children[0] * node.payload.to( + dtype=values.dtype, device=values.device + ) + elif node.operation == "linear_map": + matrix = node.payload.to(dtype=values.dtype, device=values.device) + value = children[0] @ matrix.T + else: # pragma: no cover - builder prevents this + raise RuntimeError(f"Unknown graph operation {node.operation!r}.") + cache[node.id] = value + result = cache[self.output_id] + return result.squeeze(0) if squeeze else result + + +def _payload_key(payload: Any) -> Any: + if torch is not None and isinstance(payload, torch.Tensor): + array = payload.detach().cpu().contiguous().numpy() + return (tuple(array.shape), str(array.dtype), sha1(array.tobytes()).digest()) + return payload + + +class _GraphBuilder: + def __init__(self, num_domain_noise: int, num_noise: int): + self.num_domain_noise = int(num_domain_noise) + self.num_noise = int(num_noise) + self.nodes: list[GraphNode] = [] + self.cache: dict[Any, int] = {} + + def node( + self, + operation: str, + children: Iterable[int] = (), + payload: Any = None, + *, + shape: tuple[int, ...], + degree: int, + domain_support: Iterable[int] = (), + residual_support: Iterable[int] = (), + ) -> int: + child_tuple = tuple(children) + key = (operation, child_tuple, _payload_key(payload), shape) + if key in self.cache: + return self.cache[key] + node_id = len(self.nodes) + node = GraphNode( + id=node_id, + operation=operation, + children=child_tuple, + payload=payload, + shape=shape, + degree=int(degree), + domain_support=frozenset(domain_support), + residual_support=frozenset(residual_support), + ) + self.nodes.append(node) + self.cache[key] = node_id + return node_id + + def unary(self, operation: str, child: int, payload: Any = None) -> int: + source = self.nodes[child] + return self.node( + operation, + (child,), + payload, + shape=source.shape, + degree=2 * source.degree if operation == "square" else source.degree, + domain_support=source.domain_support, + residual_support=source.residual_support, + ) + + def binary(self, operation: str, left: int, right: int) -> int: + lhs, rhs = self.nodes[left], self.nodes[right] + if lhs.shape != rhs.shape: + raise ValueError("Binary graph operations require equal shapes.") + return self.node( + operation, + (left, right), + shape=lhs.shape, + degree=(lhs.degree + rhs.degree if operation == "hadamard" else max(lhs.degree, rhs.degree)), + domain_support=lhs.domain_support | rhs.domain_support, + residual_support=lhs.residual_support | rhs.residual_support, + ) + + +def build_factored_jacobian_graph(jacobian: Any) -> ArithmeticGraph: + """Translate ``FactoredPolynomialJacobian`` to an exact immutable graph.""" + + if torch is None: + raise ImportError("PyTorch is required for factored graph construction.") + builder = _GraphBuilder(jacobian.num_domain_noise, jacobian.num_noise) + noise = builder.node( + "noise", + shape=(jacobian.num_noise,), + degree=1, + domain_support=range(jacobian.num_domain_noise), + residual_support=range(jacobian.num_domain_noise, jacobian.num_noise), + ) + + def constant(value: Any) -> int: + tensor = value.detach().clone() + return builder.node( + "constant", payload=tensor, shape=tuple(tensor.shape), degree=0 + ) + + def slice_node(start: int, stop: int) -> int: + residual = range(max(start, jacobian.num_domain_noise), stop) + domain = range(start, min(stop, jacobian.num_domain_noise)) + return builder.node( + "slice", + (noise,), + (start, stop), + shape=(stop - start,), + degree=1, + domain_support=domain, + residual_support=residual, + ) + + def linear(child: int, matrix: Any) -> int: + source = builder.nodes[child] + matrix = matrix.detach().clone() + return builder.node( + "linear_map", + (child,), + matrix, + shape=(matrix.shape[0],), + degree=source.degree, + domain_support=source.domain_support, + residual_support=source.residual_support, + ) + + adjoint = constant(jacobian.output_weight) + derivative_base = jacobian.num_value_noise + for layer_index in range(len(jacobian.factors) - 1, -1, -1): + factor = jacobian.factors[layer_index] + xi = slice_node(0, factor.active_value_noise) + affine_argument = linear(xi, factor.preactivation_coefficients) + derivative = builder.binary( + "add", constant(factor.center), linear(xi, factor.linear_coefficients) + ) + quadratic = builder.unary( + "scale", + builder.unary("square", affine_argument), + factor.quadratic_coefficients.detach().clone(), + ) + derivative = builder.binary("add", derivative, quadratic) + width = factor.center.numel() + eta = slice_node( + derivative_base + factor.derivative_noise_offset, + derivative_base + factor.derivative_noise_offset + width, + ) + derivative = builder.binary( + "add", + derivative, + builder.unary( + "scale", eta, factor.approximation_radii.detach().clone() + ), + ) + adjoint = builder.binary("hadamard", adjoint, derivative) + if layer_index: + adjoint = linear(adjoint, jacobian.hidden_weights[layer_index - 1].T) + output_id = linear(adjoint, jacobian.input_weight.T) + return ArithmeticGraph( + nodes=tuple(builder.nodes), + output_id=output_id, + num_domain_noise=jacobian.num_domain_noise, + num_noise=jacobian.num_noise, + ) + + +def _add_polynomials(left: dict[tuple[int, ...], Any], right: dict[tuple[int, ...], Any]): + result = {key: value.clone() for key, value in left.items()} + for exponent, coefficient in right.items(): + result[exponent] = result.get(exponent, torch.zeros_like(coefficient)) + coefficient + if not torch.count_nonzero(result[exponent]): + result.pop(exponent) + return result + + +class SparseReferenceMomentBackend: + """Exact domain-only sparse expansion used as a small-case oracle.""" + + def __init__(self, graph: ArithmeticGraph): + if torch is None: + raise ImportError("PyTorch is required for sparse reference moments.") + self.graph = graph + self._cache: dict[int, dict[tuple[int, ...], Any]] = {} + self.expanded_term_pairs = 0 + + def expand(self, node_id: int | None = None) -> dict[tuple[int, ...], Any]: + node_id = self.graph.output_id if node_id is None else int(node_id) + if node_id in self._cache: + return self._cache[node_id] + node = self.graph.nodes[node_id] + children = [self.expand(index) for index in node.children] + zero = (0,) * self.graph.num_domain_noise + if node.operation == "noise": + template = next( + item.payload + for item in self.graph.nodes + if item.operation == "constant" + ) + result = {} + for index in range(self.graph.num_domain_noise): + exponent = [0] * self.graph.num_domain_noise + exponent[index] = 1 + coefficient = torch.zeros( + self.graph.num_noise, + dtype=template.dtype, + device=template.device, + ) + coefficient[index] = 1.0 + result[tuple(exponent)] = coefficient + elif node.operation == "slice": + start, stop = node.payload + result = {key: value[start:stop] for key, value in children[0].items()} + elif node.operation == "constant": + result = {zero: node.payload} + elif node.operation == "add": + result = _add_polynomials(children[0], children[1]) + elif node.operation in {"hadamard", "square"}: + left = children[0] + right = children[0] if node.operation == "square" else children[1] + result: dict[tuple[int, ...], Any] = {} + for left_exp, left_coeff in left.items(): + for right_exp, right_coeff in right.items(): + self.expanded_term_pairs += 1 + exponent = tuple(a + b for a, b in zip(left_exp, right_exp)) + coefficient = left_coeff * right_coeff + result[exponent] = result.get( + exponent, torch.zeros_like(coefficient) + ) + coefficient + elif node.operation == "scale": + result = {key: value * node.payload for key, value in children[0].items()} + elif node.operation == "linear_map": + result = {key: value @ node.payload.T for key, value in children[0].items()} + else: # pragma: no cover + raise RuntimeError(f"Unsupported reference operation {node.operation!r}.") + self._cache[node_id] = result + return result + + def sum_squares(self, node_id: int | None = None) -> float: + polynomial = self.expand(node_id) + total = 0.0 + for left_exp, left_coeff in polynomial.items(): + for right_exp, right_coeff in polynomial.items(): + exponent = tuple(a + b for a, b in zip(left_exp, right_exp)) + moment = box_monomial_moment(exponent) / (2.0**len(exponent)) + total += float(torch.dot(left_coeff.reshape(-1), right_coeff.reshape(-1))) * moment + return total + + @property + def diagnostics(self) -> dict[str, int]: + return { + "graph_nodes": len(self.graph.nodes), + "expanded_nodes": len(self._cache), + "output_terms": len(self.expand()), + "expanded_term_pairs": self.expanded_term_pairs, + } + + +@dataclass(frozen=True) +class HilbertLayerDiagnostic: + layer: int + width: int + preactivation_remainder: float + polynomial_projection_remainder: float + uniform_approximation_remainder: float + total_output_remainder: float + maximum_preactivation_width: float + + +@dataclass(frozen=True) +class HilbertValueCertificate: + """Affine value projection plus a certified normalized-L2 remainder.""" + + center: Any + domain_coefficients: Any + remainder: float + polynomial_degree: int + residual_subdivisions: int + layers: tuple[HilbertLayerDiagnostic, ...] + moment_states: int + preactivation_centers: tuple[Any, ...] + preactivation_coefficients: tuple[Any, ...] + preactivation_remainders: tuple[float, ...] + + @property + def nominal_norm(self) -> float: + value = self.center.square() + self.domain_coefficients.square().sum() / 3.0 + return sqrt(max(0.0, float(value.detach().cpu().item()))) + + +@dataclass(frozen=True) +class DualWitnessCertificate: + lower_bound: float + nominal_pairing: float + remainder_penalty: float + witness_norm: float + transformed_witness_norm: float + coefficients: tuple[float, ...] + basis: str + + +@dataclass(frozen=True) +class HilbertGradientCertificate: + """Affine gradient projection plus normalized-L2 vector remainder.""" + + center: Any + domain_coefficients: Any + remainder: float + factor_remainders: tuple[float, ...] + product_projection_remainders: tuple[float, ...] + moment_states: int + + @property + def nominal_norm(self) -> float: + value = torch.dot(self.center, self.center) + self.domain_coefficients.square().sum() / 3.0 + return sqrt(max(0.0, float(value.detach().cpu().item()))) + + +@dataclass(frozen=True) +class GraphNormCertificate: + l2_squared: Interval + w12_squared: Interval + value: HilbertValueCertificate + gradient: HilbertGradientCertificate + w12_witness: DualWitnessCertificate + previous_l2_squared: Interval + previous_w12_squared: Interval + normalized_diagnostics: dict[str, float] + + +def _require_numeric_dependencies() -> None: + if torch is None or nn is None or np is None: + raise ImportError("PyTorch and NumPy are required for Hilbert graph certification.") + + +def _tanh_scalar_linears(module: Any) -> list[Any]: + children = list(module.children()) if isinstance(module, nn.Sequential) else [] + if len(children) < 3 or len(children) % 2 != 1: + raise ValueError("Expected Linear/Tanh repetitions followed by scalar Linear.") + linears: list[Any] = [] + for index, child in enumerate(children[:-1]): + expected = nn.Linear if index % 2 == 0 else nn.Tanh + if not isinstance(child, expected): + raise ValueError("Expected Linear/Tanh repetitions followed by scalar Linear.") + if index % 2 == 0: + linears.append(child) + output = children[-1] + if not isinstance(output, nn.Linear) or output.out_features != 1: + raise ValueError("The Hilbert graph certificate requires scalar output.") + linears.append(output) + return linears + + +def _affine_domain_matrix(domain: PolynomialZonotope) -> Any: + if len(domain.shape) != 1: + raise ValueError("The Hilbert graph certificate requires a flat affine domain.") + columns: list[Any] = [] + for noise_index in range(domain.num_noise): + exponent = [0] * domain.num_noise + exponent[noise_index] = 1 + coefficient = domain.terms.get(tuple(exponent)) + if coefficient is None: + raise ValueError("Every domain coordinate must have one affine generator.") + columns.append( + coefficient + if isinstance(coefficient, torch.Tensor) + else torch.as_tensor(coefficient, dtype=torch.float64) + ) + if len(domain.terms) != len(columns) or any( + kind != "domain" for kind in domain.noise_kinds + ): + raise ValueError("Only affine domain generators are supported.") + return torch.stack(columns, dim=1) + + +def _spectral_norm_upper(matrix: Any) -> float: + u, singular, vh = torch.linalg.svd(matrix, full_matrices=False) + reconstructed = (u * singular.unsqueeze(0)) @ vh + reconstruction_error = float( + torch.linalg.vector_norm(matrix - reconstructed).item() + ) + identity = torch.eye( + singular.numel(), dtype=matrix.dtype, device=matrix.device + ) + u_error = float(torch.linalg.vector_norm(u.T @ u - identity).item()) + v_error = float(torch.linalg.vector_norm(vh @ vh.T - identity).item()) + estimate = float(singular[0].item()) if singular.numel() else 0.0 + frobenius = float(torch.linalg.vector_norm(matrix).item()) + eps = torch.finfo(matrix.dtype).eps + padding = 4096.0 * eps * max(matrix.shape, default=1) ** 2 * max(1.0, frobenius) + return nextafter( + sqrt(1.0 + u_error) + * sqrt(1.0 + v_error) + * estimate + + reconstruction_error + + padding, + inf, + ) + + +def _convolve_moments(left: Any, right: Any) -> Any: + order = left.shape[-1] - 1 + output = np.zeros_like(left) + for degree in range(order + 1): + for right_degree in range(degree + 1): + output[..., degree] += ( + comb(degree, right_degree) + * left[..., degree - right_degree] + * right[..., right_degree] + ) + return output + + +def _affine_raw_and_cross_moments(center: Any, coefficients: Any, order: int): + """Return E[q^k] and E[alpha_j q^k] in extended precision.""" + + c = np.asarray(center.detach().cpu().numpy(), dtype=np.longdouble) + a = np.asarray(coefficients.detach().cpu().numpy(), dtype=np.longdouble) + width, dimension = a.shape + prefix = np.zeros((dimension + 1, width, order + 1), dtype=np.longdouble) + suffix = np.zeros_like(prefix) + for degree in range(order + 1): + prefix[0, :, degree] = c**degree + suffix[dimension, :, 0] = 1.0 + for coordinate in range(dimension): + factor = np.zeros((width, order + 1), dtype=np.longdouble) + for degree in range(0, order + 1, 2): + factor[:, degree] = a[:, coordinate] ** degree / (degree + 1) + prefix[coordinate + 1] = _convolve_moments(prefix[coordinate], factor) + for coordinate in range(dimension - 1, -1, -1): + factor = np.zeros((width, order + 1), dtype=np.longdouble) + for degree in range(0, order + 1, 2): + factor[:, degree] = a[:, coordinate] ** degree / (degree + 1) + suffix[coordinate] = _convolve_moments(factor, suffix[coordinate + 1]) + + cross = np.zeros((width, dimension, order + 1), dtype=np.longdouble) + for coordinate in range(dimension): + leave_one_out = _convolve_moments(prefix[coordinate], suffix[coordinate + 1]) + for degree in range(1, order + 1): + for alpha_power in range(1, degree + 1, 2): + cross[:, coordinate, degree] += ( + comb(degree, alpha_power) + * a[:, coordinate] ** alpha_power + * leave_one_out[:, degree - alpha_power] + / (alpha_power + 2) + ) + return prefix[dimension], cross, (dimension + 1) * width * (order + 1) + + +def _project_polynomials_of_affine_forms( + center: Any, + coefficients: Any, + polynomial_coefficients: Any, +): + degree = polynomial_coefficients.shape[1] - 1 + moments, cross, states = _affine_raw_and_cross_moments( + center, coefficients, 2 * degree + ) + proposal = np.asarray(polynomial_coefficients, dtype=np.longdouble) + mean = np.sum(proposal * moments[:, : degree + 1], axis=1) + affine_cross = np.sum( + proposal[:, :, None] * cross[:, :, : degree + 1].transpose(0, 2, 1), + axis=1, + ) + linear = 3.0 * affine_cross + squared_coefficients = np.zeros( + (len(center), 2 * degree + 1), dtype=np.longdouble + ) + for left_degree in range(degree + 1): + for right_degree in range(degree + 1): + squared_coefficients[:, left_degree + right_degree] += ( + proposal[:, left_degree] * proposal[:, right_degree] + ) + energy = np.sum(squared_coefficients * moments, axis=1) + projection_energy = mean**2 + np.sum(linear**2, axis=1) / 3.0 + scale = np.maximum(1.0, np.maximum(np.abs(energy), np.abs(projection_energy))) + rounding = ( + 16384.0 + * np.finfo(np.longdouble).eps + * (coefficients.shape[1] + 1) + * (2 * degree + 1) ** 2 + * scale + ) + residual_squared_upper = np.maximum(0.0, energy - projection_energy + rounding) + dtype, device = center.dtype, center.device + rounded_mean = np.asarray(mean, dtype=np.float64) + rounded_linear = np.asarray(linear, dtype=np.float64) + conversion_remainder = np.sqrt( + (mean - rounded_mean.astype(np.longdouble)) ** 2 + + np.sum( + (linear - rounded_linear.astype(np.longdouble)) ** 2, axis=1 + ) + / 3.0 + ) + projected_center = torch.as_tensor(rounded_mean, dtype=dtype, device=device) + projected_linear = torch.as_tensor(rounded_linear, dtype=dtype, device=device) + projection_remainder = torch.as_tensor( + np.asarray(np.sqrt(residual_squared_upper) + conversion_remainder, dtype=np.float64), + dtype=dtype, + device=device, + ) + projection_remainder = torch.nextafter( + projection_remainder, torch.full_like(projection_remainder, torch.inf) + ) + return projected_center, projected_linear, projection_remainder, states + + +def build_hilbert_value_certificate( + module: Any, + domain: PolynomialZonotope, + *, + polynomial_degree: int = 5, + residual_subdivisions: int = 2048, +) -> HilbertValueCertificate: + """Compress the exact value graph to affine projection plus L2 remainder.""" + + _require_numeric_dependencies() + if polynomial_degree < 1: + raise ValueError("polynomial_degree must be positive.") + if residual_subdivisions < 1: + raise ValueError("residual_subdivisions must be positive.") + linears = _tanh_scalar_linears(module) + parameter = next(module.parameters()) + center = torch.as_tensor( + domain.center, dtype=parameter.dtype, device=parameter.device + ) + coefficients = _affine_domain_matrix(domain).to( + dtype=parameter.dtype, device=parameter.device + ) + remainder = 0.0 + diagnostics: list[HilbertLayerDiagnostic] = [] + moment_states = 0 + cached_preactivation_centers: list[Any] = [] + cached_preactivation_coefficients: list[Any] = [] + cached_preactivation_remainders: list[float] = [] + for layer_index, layer in enumerate(linears[:-1]): + weight = layer.weight.detach().to(dtype=center.dtype, device=center.device) + bias = layer.bias.detach().to(dtype=center.dtype, device=center.device) + preactivation_center = weight @ center + bias + preactivation_coefficients = weight @ coefficients + preactivation_remainder = _spectral_norm_upper(weight) * remainder + cached_preactivation_centers.append(preactivation_center) + cached_preactivation_coefficients.append(preactivation_coefficients) + cached_preactivation_remainders.append(preactivation_remainder) + radius = torch.sum(torch.abs(preactivation_coefficients), dim=1) + lower = preactivation_center - radius + upper = preactivation_center + radius + approximations = [ + compute_tanh_polynomial( + (float(lo), float(hi)), + degree=polynomial_degree, + subdivisions=residual_subdivisions, + ) + for lo, hi in zip(lower.detach().cpu(), upper.detach().cpu()) + ] + proposal = np.asarray( + [approximation.coeffs for approximation in approximations], + dtype=np.float64, + ) + uniform = torch.as_tensor( + [approximation.delta for approximation in approximations], + dtype=center.dtype, + device=center.device, + ) + center, coefficients, projection, states = _project_polynomials_of_affine_forms( + preactivation_center, preactivation_coefficients, proposal + ) + moment_states += states + local = projection + uniform + compression_remainder = nextafter( + float(torch.linalg.vector_norm(local).detach().cpu().item()), inf + ) + remainder = nextafter(preactivation_remainder + compression_remainder, inf) + diagnostics.append( + HilbertLayerDiagnostic( + layer=layer_index, + width=layer.out_features, + preactivation_remainder=preactivation_remainder, + polynomial_projection_remainder=float( + torch.linalg.vector_norm(projection).detach().cpu().item() + ), + uniform_approximation_remainder=float( + torch.linalg.vector_norm(uniform).detach().cpu().item() + ), + total_output_remainder=remainder, + maximum_preactivation_width=float( + (2.0 * radius.max()).detach().cpu().item() + ), + ) + ) + output = linears[-1] + output_weight = output.weight.detach()[0].to( + dtype=center.dtype, device=center.device + ) + output_bias = output.bias.detach()[0].to(dtype=center.dtype, device=center.device) + final_center = torch.dot(output_weight, center) + output_bias + final_coefficients = output_weight @ coefficients + final_remainder = nextafter( + _spectral_norm_upper(output_weight.unsqueeze(0)) * remainder, inf + ) + return HilbertValueCertificate( + center=final_center, + domain_coefficients=final_coefficients, + remainder=final_remainder, + polynomial_degree=polynomial_degree, + residual_subdivisions=residual_subdivisions, + layers=tuple(diagnostics), + moment_states=moment_states, + preactivation_centers=tuple(cached_preactivation_centers), + preactivation_coefficients=tuple(cached_preactivation_coefficients), + preactivation_remainders=tuple(cached_preactivation_remainders), + ) + + +def _project_affine_hadamard( + left_center: Any, + left_coefficients: Any, + right_center: Any, + right_coefficients: Any, +): + """Orthogonally project componentwise products of affine forms.""" + + center = left_center * right_center + torch.sum( + left_coefficients * right_coefficients, dim=1 + ) / 3.0 + coefficients = ( + left_center.unsqueeze(1) * right_coefficients + + right_center.unsqueeze(1) * left_coefficients + ) + left_norm = torch.sum(left_coefficients.square(), dim=1) + right_norm = torch.sum(right_coefficients.square(), dim=1) + gram = torch.sum(left_coefficients * right_coefficients, dim=1) + coordinate_overlap = torch.sum( + left_coefficients.square() * right_coefficients.square(), dim=1 + ) + fourth = ( + (left_norm * right_norm + 2.0 * gram.square()) / 9.0 + - (2.0 / 15.0) * coordinate_overlap + ) + energy = ( + left_center.square() * right_center.square() + + left_center.square() * right_norm / 3.0 + + right_center.square() * left_norm / 3.0 + + 4.0 * left_center * right_center * gram / 3.0 + + fourth + ) + projection_energy = center.square() + coefficients.square().sum(dim=1) / 3.0 + scale = torch.maximum( + torch.ones_like(energy), torch.maximum(torch.abs(energy), torch.abs(projection_energy)) + ) + eps = torch.finfo(energy.dtype).eps + padding = 16384.0 * eps * (left_coefficients.shape[1] + 1) * scale + remainder = torch.sqrt(torch.clamp(energy - projection_energy + padding, min=0.0)) + remainder = torch.nextafter(remainder, torch.full_like(remainder, torch.inf)) + return center, coefficients, remainder + + +def build_hilbert_gradient_certificate( + module: Any, + value: HilbertValueCertificate, + *, + derivative_certificate_subdivisions: int = 64, +) -> HilbertGradientCertificate: + """Compress the reverse derivative graph to affine projection plus L2 error.""" + + _require_numeric_dependencies() + if derivative_certificate_subdivisions < 1: + raise ValueError("derivative_certificate_subdivisions must be positive.") + linears = _tanh_scalar_linears(module) + derivative_centers: list[Any] = [] + derivative_coefficients: list[Any] = [] + derivative_remainders: list[float] = [] + moment_states = 0 + tanh_prime_lipschitz = nextafter(4.0 / (3.0 * sqrt(3.0)), inf) + for preactivation_center, preactivation_coefficients, preactivation_remainder in zip( + value.preactivation_centers, + value.preactivation_coefficients, + value.preactivation_remainders, + ): + radius = torch.sum(torch.abs(preactivation_coefficients), dim=1) + approximations = [ + quadratic_tanh_prime_enclosure( + (float(lo), float(hi)), + certificate_subdivisions=derivative_certificate_subdivisions, + ) + for lo, hi in zip( + (preactivation_center - radius).detach().cpu(), + (preactivation_center + radius).detach().cpu(), + ) + ] + proposal = np.asarray( + [approximation.coeffs for approximation in approximations], + dtype=np.float64, + ) + uniform = torch.as_tensor( + [approximation.delta for approximation in approximations], + dtype=preactivation_center.dtype, + device=preactivation_center.device, + ) + projected_center, projected_coefficients, projection, states = ( + _project_polynomials_of_affine_forms( + preactivation_center, preactivation_coefficients, proposal + ) + ) + moment_states += states + local = nextafter( + float(torch.linalg.vector_norm(projection + uniform).detach().cpu().item()), + inf, + ) + total = nextafter(tanh_prime_lipschitz * preactivation_remainder + local, inf) + derivative_centers.append(projected_center) + derivative_coefficients.append(projected_coefficients) + derivative_remainders.append(total) + + output = linears[-1] + center = output.weight.detach()[0].to( + dtype=value.center.dtype, device=value.center.device + ) + coefficients = torch.zeros( + (center.numel(), value.domain_coefficients.numel()), + dtype=center.dtype, + device=center.device, + ) + remainder = 0.0 + product_remainders: list[float] = [] + for layer_index in range(len(derivative_centers) - 1, -1, -1): + old_center, old_coefficients = center, coefficients + center, coefficients, projection = _project_affine_hadamard( + old_center, + old_coefficients, + derivative_centers[layer_index], + derivative_coefficients[layer_index], + ) + projection_remainder = nextafter( + float(torch.linalg.vector_norm(projection).detach().cpu().item()), inf + ) + nominal_sup = float( + torch.max( + torch.abs(old_center) + torch.sum(torch.abs(old_coefficients), dim=1) + ).detach().cpu().item() + ) + remainder = nextafter( + remainder + + nominal_sup * derivative_remainders[layer_index] + + projection_remainder, + inf, + ) + product_remainders.append(projection_remainder) + if layer_index: + weight = linears[layer_index].weight.detach().to( + dtype=center.dtype, device=center.device + ) + center = center @ weight + coefficients = weight.T @ coefficients + remainder = nextafter(_spectral_norm_upper(weight) * remainder, inf) + + input_weight = linears[0].weight.detach().to( + dtype=center.dtype, device=center.device + ) + center = center @ input_weight + coefficients = input_weight.T @ coefficients + remainder = nextafter(_spectral_norm_upper(input_weight) * remainder, inf) + return HilbertGradientCertificate( + center=center, + domain_coefficients=coefficients, + remainder=remainder, + factor_remainders=tuple(derivative_remainders), + product_projection_remainders=tuple(reversed(product_remainders)), + moment_states=moment_states, + ) + + +def _axis_aligned_radii(domain: PolynomialZonotope) -> Any: + matrix = _affine_domain_matrix(domain) + if matrix.shape[0] != matrix.shape[1]: + raise NotImplementedError("Neumann witnesses currently require a full box.") + diagonal = torch.diagonal(matrix) + off_diagonal = matrix - torch.diag(diagonal) + if torch.count_nonzero(off_diagonal): + raise NotImplementedError("Neumann witnesses currently require an axis-aligned box.") + if torch.any(diagonal == 0.0): + raise ValueError("Neumann witnesses require positive box radii.") + return torch.abs(diagonal) + + +def neumann_polynomial_witness( + value: HilbertValueCertificate, + domain: PolynomialZonotope, +) -> DualWitnessCertificate: + """Certify an H1 lower bound using cubic Neumann polynomial witnesses.""" + + radii = np.asarray(_axis_aligned_radii(domain).detach().cpu(), dtype=np.float64) + affine = np.asarray(value.domain_coefficients.detach().cpu(), dtype=np.float64) + center = float(value.center.detach().cpu().item()) + dimension = len(radii) + # phi_i = alpha_i^3/3-alpha_i has zero physical normal derivative. + phi_l2 = 1.0 / 63.0 - 2.0 / 15.0 + 1.0 / 3.0 + k = np.empty(dimension + 1, dtype=np.float64) + h = np.empty_like(k) + c = np.empty_like(k) + k[0] = h[0] = 1.0 + c[0] = center + k[1:] = phi_l2 + 8.0 / (15.0 * radii**2) + beta = 1.0 + 2.0 / radii**2 + h[1:] = 1.0 / 63.0 - 2.0 * beta / 15.0 + beta**2 / 3.0 + c[1:] = affine * (1.0 / 15.0 - beta / 3.0) + + candidates: list[Any] = [] + candidates.append(c / k) + for scale in np.logspace(-10.0, 10.0, 161): + candidates.append(c / (k + scale * h)) + for index in range(dimension + 1): + direction = np.zeros_like(c) + direction[index] = 1.0 + candidates.append(direction) + + best = None + for direction in candidates: + witness_norm = sqrt(max(0.0, float(np.dot(k * direction, direction)))) + if witness_norm == 0.0: + continue + transformed = sqrt(max(0.0, float(np.dot(h * direction, direction)))) + pairing = abs(float(np.dot(c, direction))) + penalty = value.remainder * transformed + lower = max(0.0, pairing - penalty) / witness_norm + if best is None or lower > best[0]: + best = (lower, pairing, penalty, witness_norm, transformed, direction) + assert best is not None + lower = nextafter(max(0.0, best[0]), -inf) + return DualWitnessCertificate( + lower_bound=max(0.0, lower), + nominal_pairing=best[1], + remainder_penalty=nextafter(best[2], inf), + witness_norm=nextafter(best[3], inf), + transformed_witness_norm=nextafter(best[4], inf), + coefficients=tuple(float(item) for item in best[5]), + basis="constant_plus_coordinatewise_cubic_neumann", + ) + + +def _scaled_squared_interval(lower: float, upper: float, volume: float) -> Interval: + return Interval.from_bounds( + max(0.0, nextafter(volume * lower * lower, -inf)), + nextafter(volume * upper * upper, inf), + ) + + +def certify_hybrid_graph_norms( + module: Any, + result: Any, + cell: PZIntegrationCell, + *, + polynomial_degree: int = 5, + residual_subdivisions: int = 2048, + derivative_certificate_subdivisions: int = 64, +) -> GraphNormCertificate: + """Return intersected L2/W12 certificates with positive lower mechanisms.""" + + from .deep_hybrid import ( + integrate_hybrid_onejet_squared, + integrate_hybrid_value_squared, + ) + + if not isinstance(cell.volume, (int, float)): + raise NotImplementedError("Graph Hilbert certification requires scalar volume.") + volume = float(cell.volume) + value = build_hilbert_value_certificate( + module, + cell.domain, + polynomial_degree=polynomial_degree, + residual_subdivisions=residual_subdivisions, + ) + gradient = build_hilbert_gradient_certificate( + module, + value, + derivative_certificate_subdivisions=derivative_certificate_subdivisions, + ) + previous_l2 = integrate_hybrid_value_squared(result, cell) + previous_w12 = integrate_hybrid_onejet_squared(result, cell) + nominal = value.nominal_norm + reverse_lower = max(0.0, nominal - value.remainder) + reverse_upper = nominal + value.remainder + old_l2_lower = sqrt(max(0.0, float(previous_l2.lower) / volume)) + old_l2_upper = sqrt(max(0.0, float(previous_l2.upper) / volume)) + l2_lower = max(old_l2_lower, reverse_lower) + l2_upper = min(old_l2_upper, reverse_upper) + if l2_lower > l2_upper: + raise RuntimeError("Independent sound L2 certificates have empty intersection.") + + witness = neumann_polynomial_witness(value, cell.domain) + old_w12_lower = sqrt(max(0.0, float(previous_w12.lower) / volume)) + old_w12_upper = sqrt(max(0.0, float(previous_w12.upper) / volume)) + graph_nominal_w12 = sqrt(nominal * nominal + gradient.nominal_norm**2) + graph_remainder_w12 = sqrt( + value.remainder * value.remainder + gradient.remainder * gradient.remainder + ) + graph_w12_lower = max(0.0, graph_nominal_w12 - graph_remainder_w12) + graph_w12_upper = graph_nominal_w12 + graph_remainder_w12 + w12_lower = max( + old_w12_lower, l2_lower, witness.lower_bound, graph_w12_lower + ) + w12_upper = min(old_w12_upper, graph_w12_upper) + if w12_lower > w12_upper: + raise RuntimeError("Independent sound W12 certificates have empty intersection.") + + l2_interval = _scaled_squared_interval(l2_lower, l2_upper, volume) + w12_interval = _scaled_squared_interval(w12_lower, w12_upper, volume) + # Preserve monotonicity bit-for-bit as well as mathematically. Re-scaling + # a square root can otherwise move an endpoint by one ulp past the old + # interval even though the real-number bounds are identical. + l2_interval = Interval.from_bounds( + max(float(previous_l2.lower), float(l2_interval.lower)), + min(float(previous_l2.upper), float(l2_interval.upper)), + ) + w12_interval = Interval.from_bounds( + max(float(previous_w12.lower), float(w12_interval.lower)), + min(float(previous_w12.upper), float(w12_interval.upper)), + ) + return GraphNormCertificate( + l2_squared=l2_interval, + w12_squared=w12_interval, + value=value, + gradient=gradient, + w12_witness=witness, + previous_l2_squared=previous_l2, + previous_w12_squared=previous_w12, + normalized_diagnostics={ + "value_nominal_norm": nominal, + "value_l2_remainder": value.remainder, + "l2_reverse_triangle_lower": reverse_lower, + "l2_reverse_triangle_upper": reverse_upper, + "w12_dual_lower": witness.lower_bound, + "gradient_nominal_norm": gradient.nominal_norm, + "gradient_l2_remainder": gradient.remainder, + "w12_graph_nominal_norm": graph_nominal_w12, + "w12_graph_remainder": graph_remainder_w12, + "w12_graph_reverse_lower": graph_w12_lower, + "w12_graph_reverse_upper": graph_w12_upper, + "combined_l2_lower": l2_lower, + "combined_l2_upper": l2_upper, + "combined_w12_lower": w12_lower, + "combined_w12_upper": w12_upper, + }, + ) diff --git a/src/intervalnets/pinn.py b/src/intervalnets/pinn.py new file mode 100644 index 0000000..6df3845 --- /dev/null +++ b/src/intervalnets/pinn.py @@ -0,0 +1,163 @@ +"""Efficient differential propagation helpers for PINN training.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +try: + import torch + from torch import nn +except ImportError: # pragma: no cover - PyTorch is an optional dependency + torch = None + nn = None + + +def load_tanh_mlp_checkpoint( + checkpoint_path: str | Path, + *, + map_location: Any = "cpu", +) -> "nn.Sequential": + """Load a saved sequential tanh MLP without training it. + + The architecture is reconstructed from the two-dimensional ``weight`` + tensors in the checkpoint's ``state_dict``. This keeps benchmark + notebooks tied to the exact saved architecture instead of duplicating its + dimensions in several places. + + Checkpoints may either contain a bare state dictionary or a mapping with a + ``state_dict`` entry, as produced by the PINN benchmark notebook. + """ + + if torch is None or nn is None: # pragma: no cover - optional dependency + raise ImportError("PyTorch is required to load a PINN checkpoint.") + + path = Path(checkpoint_path) + if not path.is_file(): + raise FileNotFoundError( + f"PINN checkpoint not found: {path}. Regenerate it explicitly in " + "the training notebook with RETRAIN = True." + ) + + payload = torch.load(path, map_location=map_location, weights_only=True) + state_dict = payload.get("state_dict", payload) if isinstance(payload, dict) else payload + if not isinstance(state_dict, dict): + raise ValueError("Checkpoint must contain a state_dict mapping.") + + linear_layers: list[tuple[int, torch.Tensor, torch.Tensor]] = [] + for key, weight in state_dict.items(): + if not key.endswith(".weight") or not isinstance(weight, torch.Tensor): + continue + prefix = key.removesuffix(".weight") + if not prefix.isdigit() or weight.ndim != 2: + continue + bias = state_dict.get(f"{prefix}.bias") + if not isinstance(bias, torch.Tensor) or bias.shape != (weight.shape[0],): + raise ValueError(f"Missing or incompatible bias for checkpoint layer {prefix}.") + linear_layers.append((int(prefix), weight, bias)) + + linear_layers.sort(key=lambda item: item[0]) + if not linear_layers: + raise ValueError("Checkpoint does not contain any sequential Linear layers.") + for (_, previous_weight, _), (_, weight, _) in zip(linear_layers, linear_layers[1:]): + if weight.shape[1] != previous_weight.shape[0]: + raise ValueError("Checkpoint Linear layer dimensions are incompatible.") + + layers: list[nn.Module] = [] + for position, (_, weight, _) in enumerate(linear_layers): + layers.append( + nn.Linear( + int(weight.shape[1]), + int(weight.shape[0]), + device=weight.device, + dtype=weight.dtype, + ) + ) + if position + 1 < len(linear_layers): + layers.append(nn.Tanh()) + + model = nn.Sequential(*layers) + model.load_state_dict(state_dict) + model.eval() + return model + + +def sequential_value_jacobian_laplacian( + module: Any, + x: "torch.Tensor", +) -> tuple["torch.Tensor", "torch.Tensor", "torch.Tensor"]: + """Evaluate a tanh MLP together with its Jacobian and Laplacian. + + The routine propagates first derivatives and the trace of the Hessian + directly through ``Linear`` and ``Tanh`` layers. It is differentiable + with respect to the network parameters, so the returned Laplacian can be + used in an ordinary PINN residual without performing one second-order + autograd call per input coordinate. + + Parameters + ---------- + module: + A ``torch.nn.Sequential`` tanh MLP containing ``Linear``, ``Tanh``, + and optional ``Identity`` modules. + x: + A batched tensor with shape ``(..., input_dim)``. + + Returns + ------- + value: + Network output with shape ``(..., output_dim)``. + jacobian: + Physical-input Jacobian with shape + ``(..., output_dim, input_dim)``. + laplacian: + Componentwise Hessian trace with shape ``(..., output_dim)``. + """ + + if torch is None or nn is None: # pragma: no cover - optional dependency + raise ImportError("PyTorch is required for PINN differential propagation.") + if not isinstance(module, nn.Sequential): + raise TypeError("module must be a torch.nn.Sequential tanh MLP.") + if not isinstance(x, torch.Tensor) or x.ndim < 1: + raise TypeError("x must be a torch.Tensor with a final input dimension.") + + value = x + jacobian = None + laplacian = None + input_dim = int(x.shape[-1]) + + for layer in module: + if isinstance(layer, nn.Linear): + if int(layer.in_features) != int(value.shape[-1]): + raise ValueError("Linear layer width does not match the propagated value.") + value = layer(value) + if jacobian is None: + if int(layer.in_features) != input_dim: + raise ValueError("The first Linear layer must consume the physical input.") + batch_shape = tuple(x.shape[:-1]) + jacobian = layer.weight.reshape( + (1,) * len(batch_shape) + tuple(layer.weight.shape) + ).expand(batch_shape + tuple(layer.weight.shape)) + laplacian = torch.zeros_like(value) + else: + jacobian = torch.einsum("oi,...id->...od", layer.weight, jacobian) + laplacian = torch.einsum("oi,...i->...o", layer.weight, laplacian) + elif isinstance(layer, nn.Tanh): + if jacobian is None or laplacian is None: + raise ValueError("Tanh cannot precede the first Linear layer.") + activated = torch.tanh(value) + first = 1.0 - activated.square() + second = -2.0 * activated * first + laplacian = second * jacobian.square().sum(dim=-1) + first * laplacian + jacobian = first.unsqueeze(-1) * jacobian + value = activated + elif isinstance(layer, nn.Identity): + continue + else: + raise NotImplementedError( + "PINN differential propagation supports only Sequential, Linear, " + f"Tanh, and Identity modules; got {type(layer).__name__}." + ) + + if jacobian is None or laplacian is None: + raise ValueError("The network must contain at least one Linear layer.") + return value, jacobian, laplacian diff --git a/src/intervalnets/polynomial_zonotope.py b/src/intervalnets/polynomial_zonotope.py new file mode 100644 index 0000000..045f22c --- /dev/null +++ b/src/intervalnets/polynomial_zonotope.py @@ -0,0 +1,836 @@ +from __future__ import annotations + +from contextlib import contextmanager +from dataclasses import dataclass +from math import inf, nextafter, prod +from typing import Any, Iterator, Mapping, Sequence + +from .interval import Interval + +try: # pragma: no cover - optional dependency + import torch +except ImportError: # pragma: no cover + torch = None + +Exponent = tuple[int, ...] + + +_PZ_DIAGNOSTIC_STACK: list[tuple[list[dict[str, Any]], str | None]] = [] + + +@contextmanager +def collect_pz_diagnostics(phase: str | None = None) -> Iterator[list[dict[str, Any]]]: + """Collect opt-in polynomial-zonotope multiplication diagnostics. + + The default behavior is unchanged unless this context manager is active. + While active, every ``PolynomialZonotope * PolynomialZonotope`` operation + appends a record containing input term counts, the raw Cartesian-product + pair count, merged output term count, coefficient shape, and maximum output + monomial degree. ``phase`` is copied into each record so callers can group + diagnostics from different parts of a computation. + """ + + records: list[dict[str, Any]] = [] + _PZ_DIAGNOSTIC_STACK.append((records, phase)) + try: + yield records + finally: + _PZ_DIAGNOSTIC_STACK.pop() + + +def _record_pz_multiplication( + *, + left_term_count: int, + right_term_count: int, + output_terms: Mapping[Exponent, Any], + coefficient_shape: tuple[int, ...], +) -> None: + if not _PZ_DIAGNOSTIC_STACK: + return + max_output_degree = max((sum(exp) for exp in output_terms), default=0) + raw_pair_count = left_term_count * right_term_count + for records, phase in _PZ_DIAGNOSTIC_STACK: + records.append({ + "phase": phase, + "left_term_count": left_term_count, + "right_term_count": right_term_count, + "raw_pair_count": raw_pair_count, + "output_term_count": len(output_terms), + "coefficient_shape": coefficient_shape, + "max_output_degree": max_output_degree, + }) + + +def box_monomial_moment(exponent: tuple[int, ...]) -> float: + """Exact integral of a monomial over the box ``[-1, 1]^d``. + + The returned value is ``integral over [-1,1]^d of x**exponent``. Odd + monomials cancel by symmetry. + """ + + if any(k % 2 for k in exponent): + return 0.0 + return prod(2.0 / (k + 1) for k in exponent) + + +def _is_sequence(value: Any) -> bool: + return isinstance(value, (list, tuple)) + + +def _as_tensor(value: Any, *, dtype=None, device=None): + if torch is None: + raise ImportError("PyTorch is required for tensor polynomial zonotopes.") + if isinstance(value, torch.Tensor): + out = value + if dtype is not None or device is not None: + out = out.to(dtype=dtype or out.dtype, device=device or out.device) + return out + return torch.as_tensor(value, dtype=dtype or torch.get_default_dtype(), device=device) + + +def _to_fallback(value: Any): + if _is_sequence(value): + return tuple(_to_fallback(item) for item in value) + return float(value) + + +def _fallback_shape(value: Any) -> tuple[int, ...]: + if isinstance(value, tuple): + if not value: + return (0,) + return (len(value),) + _fallback_shape(value[0]) + return () + + +def _fallback_map(value: Any, op): + if isinstance(value, tuple): + return tuple(_fallback_map(item, op) for item in value) + return op(value) + + +def _fallback_zip(left: Any, right: Any, op): + if isinstance(left, tuple) and isinstance(right, tuple): + if len(left) != len(right): + raise ValueError("Shape mismatch.") + return tuple(_fallback_zip(l, r, op) for l, r in zip(left, right)) + if isinstance(left, tuple) or isinstance(right, tuple): + raise ValueError("Shape mismatch.") + return op(left, right) + + +def _fallback_get(value: Any, item: Any): + if not isinstance(item, tuple): + return value[item] + out = value + for idx in item: + out = out[idx] + return out + + +def _fallback_linear_contract(matrix: Any, coeff: Any): + rows = tuple(tuple(float(value) for value in row) for row in matrix) + if not isinstance(coeff, tuple): + raise ValueError("linear_map expects coefficients with a leading input axis.") + if any(len(row) != len(coeff) for row in rows): + raise ValueError("Linear map weight/input dimension mismatch.") + outputs = [] + for row in rows: + acc = None + for weight, item in zip(row, coeff): + term = _mul_coeff(item, weight) + acc = term if acc is None else _add_coeff(acc, term) + outputs.append(acc if acc is not None else 0.0) + return tuple(outputs) + + +def _is_zero_coeff(value: Any, *, atol: float = 0.0) -> bool: + """Return whether a coefficient is identically zero. + + By default this performs exact zero detection so certified arithmetic does + not silently discard small nonzero dependencies. A positive ``atol`` may be + supplied by explicit opt-in callers for tolerance-based cleanup. + """ + + if torch is not None and isinstance(value, torch.Tensor): + if atol == 0.0: + return bool(torch.all(value == 0).item()) + return bool(torch.all(torch.abs(value) <= atol).item()) + if isinstance(value, tuple): + return all(_is_zero_coeff(item, atol=atol) for item in value) + return abs(float(value)) <= atol if atol != 0.0 else float(value) == 0.0 + +def _zero_like(value: Any): + if torch is not None and isinstance(value, torch.Tensor): + return torch.zeros_like(value) + return _fallback_map(value, lambda _: 0.0) + + +def _add_coeff(left: Any, right: Any): + if torch is not None and isinstance(left, torch.Tensor): + return left + right + return _fallback_zip(left, right, lambda l, r: l + r) + + +def _mul_coeff(left: Any, right: Any): + if torch is not None and isinstance(left, torch.Tensor): + return left * right + if torch is not None and isinstance(right, torch.Tensor): + return left * right + if isinstance(left, tuple) and not isinstance(right, tuple): + return _fallback_map(left, lambda item: item * float(right)) + if isinstance(right, tuple) and not isinstance(left, tuple): + return _fallback_map(right, lambda item: float(left) * item) + return _fallback_zip(left, right, lambda l, r: l * r) + + +def _abs_coeff(value: Any): + if torch is not None and isinstance(value, torch.Tensor): + return torch.abs(value) + return _fallback_map(value, abs) + + +def _pad_lower(value: Any): + if torch is not None and isinstance(value, torch.Tensor): + return torch.nextafter(value, torch.full_like(value, float("-inf"))) + return _fallback_map(value, lambda item: nextafter(float(item), -inf)) + + +def _pad_upper(value: Any): + if torch is not None and isinstance(value, torch.Tensor): + return torch.nextafter(value, torch.full_like(value, float("inf"))) + return _fallback_map(value, lambda item: nextafter(float(item), inf)) + + +def _canonical_exponent(exponent: tuple[int, ...], num_noise: int) -> Exponent: + if len(exponent) > num_noise: + raise ValueError("Exponent length exceeds num_noise.") + padded = tuple(int(item) for item in exponent) + (0,) * (num_noise - len(exponent)) + if any(item < 0 for item in padded): + raise ValueError("Exponents must be non-negative.") + return padded + +def _canonical_noise_kinds(noise_kinds: tuple[str, ...] | list[str] | None, num_noise: int) -> tuple[str, ...]: + if num_noise < 0: + raise ValueError("num_noise must be non-negative.") + if noise_kinds is None: + return ("unknown",) * num_noise + kinds = tuple(str(kind) for kind in noise_kinds) + if len(kinds) != num_noise: + raise ValueError("noise_kinds length must match num_noise.") + return kinds + + +def _merge_noise_kinds(left: tuple[str, ...], right: tuple[str, ...]) -> tuple[str, ...]: + if len(left) != len(right): + raise ValueError("noise_kinds length mismatch.") + merged = [] + for l_kind, r_kind in zip(left, right): + if l_kind == r_kind: + merged.append(l_kind) + elif l_kind == "unknown": + merged.append(r_kind) + elif r_kind == "unknown": + merged.append(l_kind) + else: + raise ValueError(f"Incompatible noise metadata: {l_kind!r} != {r_kind!r}.") + return tuple(merged) + +def _format_latex_number(value: Any, precision: int) -> str: + number = float(value) + if number == 0.0: + number = 0.0 + return f"{number:.{precision}g}" + + +def _coefficient_scalar(value: Any, index: tuple[int, ...]) -> float: + if torch is not None and isinstance(value, torch.Tensor): + return float(value[index].item() if index else value.item()) + return float(_fallback_get(value, index) if index else value) + + +def _coefficient_indices(shape: tuple[int, ...]) -> list[tuple[int, ...]]: + if not shape: + return [()] + indices: list[tuple[int, ...]] = [] + def rec(prefix: tuple[int, ...], dims: tuple[int, ...]) -> None: + if not dims: + indices.append(prefix) + return + for i in range(dims[0]): + rec(prefix + (i,), dims[1:]) + rec((), shape) + return indices + + +def _latex_noise_symbol(kind: str, position: int, variable_prefix: str) -> str: + if kind == "domain": + base = r"\xi" + elif kind.startswith("approximation"): + base = r"\eta" + else: + base = variable_prefix + return f"{base}_{{{position + 1}}}" + + +def _latex_monomial(exponent: Exponent, noise_kinds: tuple[str, ...], variable_prefix: str) -> str: + factors = [] + for i, power in enumerate(exponent): + if power == 0: + continue + symbol = _latex_noise_symbol(noise_kinds[i], i, variable_prefix) + factors.append(symbol if power == 1 else f"{symbol}^{{{power}}}") + return " ".join(factors) + + +def _latex_entry_label(base_label: str, index: tuple[int, ...]) -> str: + if not index: + return base_label + return f"{base_label}_{{{','.join(str(i) for i in index)}}}" + + +def _format_latex_expression(center: float, terms: list[tuple[Exponent, float]], noise_kinds: tuple[str, ...], *, variable_prefix: str, max_terms: int | None, precision: int) -> str: + pieces = [_format_latex_number(center, precision)] + visible_terms = terms if max_terms is None else terms[:max_terms] + for exponent, coeff in visible_terms: + if coeff == 0.0: + continue + sign = "+" if coeff >= 0 else "-" + magnitude = abs(coeff) + monomial = _latex_monomial(exponent, noise_kinds, variable_prefix) + coeff_text = _format_latex_number(magnitude, precision) + if monomial and coeff_text == "1": + body = monomial + elif monomial: + body = f"{coeff_text} {monomial}" + else: + body = coeff_text + pieces.append(f"{sign} {body}") + omitted = max(0, len(terms) - len(visible_terms)) + if omitted: + pieces.append(f"+ \\cdots\\;({omitted} omitted terms)") + return " ".join(pieces) + + +def pz_to_latex(z: PolynomialZonotope, *, variable_prefix: str = r"\epsilon", max_terms: int | None = None, precision: int = 4) -> str: + """Render a polynomial zonotope as compact LaTeX. + + Scalar coefficients are rendered as one expression. Vector, matrix, and + higher-order tensor coefficients are rendered entrywise in an ``aligned`` + block using zero-based tensor indices. + """ + + if max_terms is not None and max_terms < 0: + raise ValueError("max_terms must be non-negative or None.") + if precision < 1: + raise ValueError("precision must be positive.") + + rows = [] + sorted_terms = sorted(z.terms.items(), key=lambda item: (sum(item[0]), item[0])) + for index in _coefficient_indices(z.shape): + center = _coefficient_scalar(z.center, index) + entry_terms = [(exp, _coefficient_scalar(coeff, index)) for exp, coeff in sorted_terms] + entry_terms = [(exp, coeff) for exp, coeff in entry_terms if coeff != 0.0] + rows.append(f"{_latex_entry_label('Z', index)} &= {_format_latex_expression(center, entry_terms, z.noise_kinds, variable_prefix=variable_prefix, max_terms=max_terms, precision=precision)}") + if len(rows) == 1: + return rows[0].replace("Z &= ", "") + return "\\begin{aligned}\n" + " \\\\\n".join(rows) + "\n\\end{aligned}" + + +def twojet_to_latex(jet: PZTwoJet, *, max_terms: int | None = None, precision: int = 4) -> str: + """Render a polynomial-zonotope two-jet as compact LaTeX sections.""" + + sections = [] + for label, z in (("Y", jet.Y), ("J", jet.J), ("H", jet.H)): + rendered = pz_to_latex(z, max_terms=max_terms, precision=precision) + if z.shape: + rendered = rendered.replace("Z_{", f"{label}_{{") + else: + rendered = f"{label} = {rendered}" + sections.append(rendered) + return "\n\n".join(sections) + + +def pz_to_markdown_code(z: PolynomialZonotope, *, variable_prefix: str = r"\epsilon", max_terms: int | None = None, precision: int = 4) -> str: + """Render a polynomial zonotope in a fenced LaTeX Markdown code block.""" + + return f"```latex\n{pz_to_latex(z, variable_prefix=variable_prefix, max_terms=max_terms, precision=precision)}\n```" + + +@dataclass(frozen=True, init=False) +class PolynomialZonotope: + """Polynomial zonotope with explicit monomial dependencies. + + Represents ``center + sum(terms[alpha] * eps**alpha)`` for + ``eps_i in [-1, 1]``. Coefficients may be torch tensors or the lightweight + tuple/float fallback used by the interval module. + """ + + center: Any + terms: dict[Exponent, Any] + num_noise: int + shape: tuple[int, ...] + dtype: Any + device: Any + noise_kinds: tuple[str, ...] + + def __init__(self, center: Any, terms: Mapping[tuple[int, ...], Any] | None = None, num_noise: int | None = None, noise_kinds: tuple[str, ...] | list[str] | None = None): + use_torch = torch is not None and (isinstance(center, torch.Tensor) or any(isinstance(v, torch.Tensor) for v in (terms or {}).values())) + c = _as_tensor(center) if use_torch else _to_fallback(center) + inferred_noise = max((len(exp) for exp in (terms or {})), default=0) + p = inferred_noise if num_noise is None else int(num_noise) + if p < inferred_noise: + raise ValueError("num_noise is smaller than a supplied exponent length.") + kinds = _canonical_noise_kinds(noise_kinds, p) + clean: dict[Exponent, Any] = {} + for exp, coeff in (terms or {}).items(): + key = _canonical_exponent(tuple(exp), p) + value = _as_tensor(coeff, dtype=c.dtype, device=c.device) if torch is not None and isinstance(c, torch.Tensor) else _to_fallback(coeff) + if (torch is not None and isinstance(c, torch.Tensor) and tuple(value.shape) != tuple(c.shape)) or (not (torch is not None and isinstance(c, torch.Tensor)) and _fallback_shape(value) != _fallback_shape(c)): + raise ValueError("Term coefficient shape must match center shape.") + if key in clean: + value = _add_coeff(clean[key], value) + if _is_zero_coeff(value): + clean.pop(key, None) + else: + clean[key] = value + object.__setattr__(self, "center", c) + object.__setattr__(self, "terms", clean) + object.__setattr__(self, "num_noise", p) + object.__setattr__(self, "shape", tuple(c.shape) if torch is not None and isinstance(c, torch.Tensor) else _fallback_shape(c)) + object.__setattr__(self, "dtype", c.dtype if torch is not None and isinstance(c, torch.Tensor) else float) + object.__setattr__(self, "device", c.device if torch is not None and isinstance(c, torch.Tensor) else None) + object.__setattr__(self, "noise_kinds", kinds) + + @classmethod + def constant(cls, value: Any, num_noise: int = 0, noise_kinds: tuple[str, ...] | list[str] | None = None) -> "PolynomialZonotope": + return cls(value, {}, num_noise=num_noise, noise_kinds=noise_kinds) + + @classmethod + def from_box(cls, lower: Any, upper: Any) -> "PolynomialZonotope": + if torch is not None and (isinstance(lower, torch.Tensor) or isinstance(upper, torch.Tensor)): + lo = _as_tensor(lower) + hi = _as_tensor(upper, dtype=lo.dtype, device=lo.device) + if lo.shape != hi.shape: + raise ValueError("Lower/upper shape mismatch.") + if torch.any(lo > hi): + raise ValueError("Lower bounds must not exceed upper bounds.") + center = (lo + hi) / 2 + radius = (hi - lo) / 2 + p = radius.numel() + terms = {} + for idx in range(p): + coeff = torch.zeros_like(center) + coeff.reshape(-1)[idx] = radius.reshape(-1)[idx] + exp = [0] * p + exp[idx] = 1 + terms[tuple(exp)] = coeff + return cls(center, terms, num_noise=p, noise_kinds=("domain",) * p) + lo = _to_fallback(lower); hi = _to_fallback(upper) + def check(l, h): + if isinstance(l, tuple): + if len(l) != len(h): raise ValueError("Lower/upper shape mismatch.") + for a, b in zip(l, h): check(a, b) + elif l > h: raise ValueError("Lower bounds must not exceed upper bounds.") + check(lo, hi) + center = _fallback_zip(lo, hi, lambda l, h: (l + h) / 2.0) + radius = _fallback_zip(lo, hi, lambda l, h: (h - l) / 2.0) + flat_paths: list[tuple[int, ...]] = [] + def paths(v, prefix=()): + if isinstance(v, tuple): + for i, item in enumerate(v): paths(item, prefix + (i,)) + else: flat_paths.append(prefix) + paths(radius) + def coeff_for(path): + def rec(v, pref=()): + if isinstance(v, tuple): return tuple(rec(item, pref + (i,)) for i, item in enumerate(v)) + return v if pref == path else 0.0 + return rec(radius) + terms = {} + for i, path in enumerate(flat_paths): + exp = [0] * len(flat_paths); exp[i] = 1 + terms[tuple(exp)] = coeff_for(path) + return cls(center, terms, num_noise=len(flat_paths), noise_kinds=("domain",) * len(flat_paths)) + + + def _align(self, other: "PolynomialZonotope"): + p = max(self.num_noise, other.num_noise) + left = self.with_num_noise(p) + right = other.with_num_noise(p) + merged = _merge_noise_kinds(left.noise_kinds, right.noise_kinds) + return left.with_noise_kinds(merged), right.with_noise_kinds(merged) + + def with_noise_kinds(self, noise_kinds: tuple[str, ...] | list[str]) -> "PolynomialZonotope": + kinds = _canonical_noise_kinds(noise_kinds, self.num_noise) + if kinds == self.noise_kinds: return self + return PolynomialZonotope(self.center, self.terms, num_noise=self.num_noise, noise_kinds=kinds) + + def with_num_noise(self, num_noise: int) -> "PolynomialZonotope": + if num_noise < self.num_noise: + raise ValueError("num_noise cannot shrink existing exponents.") + if num_noise == self.num_noise: return self + return PolynomialZonotope(self.center, {exp + (0,) * (num_noise - self.num_noise): c for exp, c in self.terms.items()}, num_noise=num_noise, noise_kinds=self.noise_kinds + ("unknown",) * (num_noise - self.num_noise)) + + def __add__(self, other: Any): + if not isinstance(other, PolynomialZonotope): + return PolynomialZonotope(_add_coeff(self.center, other), self.terms, num_noise=self.num_noise, noise_kinds=self.noise_kinds) + left, right = self._align(other) + if left.shape != right.shape: raise ValueError("Shape mismatch for addition.") + terms = dict(left.terms) + for exp, coeff in right.terms.items(): terms[exp] = _add_coeff(terms[exp], coeff) if exp in terms else coeff + return PolynomialZonotope(_add_coeff(left.center, right.center), terms, num_noise=left.num_noise, noise_kinds=left.noise_kinds) + + __radd__ = __add__ + + def __neg__(self): + return self * -1.0 + + def __sub__(self, other: Any): + return self + (-other if isinstance(other, PolynomialZonotope) else -float(other)) + + def __rsub__(self, other: Any): + return (-self) + other + + def __mul__(self, other: Any): + if not isinstance(other, PolynomialZonotope): + return PolynomialZonotope(_mul_coeff(self.center, other), {e: _mul_coeff(c, other) for e, c in self.terms.items()}, num_noise=self.num_noise, noise_kinds=self.noise_kinds) + left, right = self._align(other) + if left.shape != () and right.shape != () and left.shape != right.shape: + raise ValueError("Polynomial-zonotope multiplication requires at least one scalar coefficient shape or equal shapes.") + terms: dict[Exponent, Any] = {} + def add(exp, coeff): terms.__setitem__(exp, _add_coeff(terms[exp], coeff) if exp in terms else coeff) + for exp, coeff in right.terms.items(): add(exp, _mul_coeff(left.center, coeff)) + for exp, coeff in left.terms.items(): add(exp, _mul_coeff(coeff, right.center)) + for e1, c1 in left.terms.items(): + for e2, c2 in right.terms.items(): add(tuple(a + b for a, b in zip(e1, e2)), _mul_coeff(c1, c2)) + _record_pz_multiplication( + left_term_count=len(left.terms), + right_term_count=len(right.terms), + output_terms=terms, + coefficient_shape=left.shape if left.shape != () else right.shape, + ) + return PolynomialZonotope(_mul_coeff(left.center, right.center), terms, num_noise=left.num_noise, noise_kinds=left.noise_kinds) + + __rmul__ = __mul__ + + + def evaluate_polynomial(self, coeffs: Any) -> "PolynomialZonotope": + """Evaluate a scalar power-basis polynomial on this zonotope. + + ``coeffs`` are in ascending power order: ``c0, c1, ...``. The + implementation uses Horner evaluation and preserves all existing + polynomial dependencies. + """ + + coeff_tuple = tuple(coeffs) + if not coeff_tuple: + raise ValueError("coeffs must not be empty.") + result = PolynomialZonotope.constant(coeff_tuple[-1], num_noise=self.num_noise, noise_kinds=self.noise_kinds) + for coeff in reversed(coeff_tuple[:-1]): + result = result * self + coeff + return result + + def add_independent_error(self, radius: Any, target_shape: tuple[int, ...] = (), *, kind: str = "approximation", metadata: str | None = None) -> "PolynomialZonotope": + """Add a fresh independent error variable with the given radius. + + Existing exponent vectors are extended by one zero entry, while the new + error term receives exponent ``(0, ..., 0, 1)``. For tensor-backed + zonotopes, ``target_shape`` may be supplied to create a coefficient of + that shape; it must match the zonotope shape so the resulting object is + well-formed. + """ + + new_noise = self.num_noise + 1 + terms = {exp + (0,): coeff for exp, coeff in self.terms.items()} + if torch is not None and isinstance(self.center, torch.Tensor): + shape = tuple(target_shape) if target_shape else self.shape + if shape != self.shape: + raise ValueError("target_shape must match this zonotope's coefficient shape.") + coeff = torch.as_tensor(radius, dtype=self.center.dtype, device=self.center.device) + if tuple(coeff.shape) == () and self.shape != (): + coeff = torch.full_like(self.center, float(coeff.item())) + else: + coeff = coeff.to(dtype=self.center.dtype, device=self.center.device) + if tuple(coeff.shape) != self.shape: + coeff = torch.broadcast_to(coeff, self.shape).clone() + else: + if target_shape and tuple(target_shape) != self.shape: + raise ValueError("target_shape must match this zonotope's coefficient shape.") + coeff = _mul_coeff(_zero_like(self.center), 0.0) + coeff = _add_coeff(coeff, _to_fallback(radius)) if self.shape == () else _fallback_map(self.center, lambda _: float(radius)) + terms[(0,) * self.num_noise + (1,)] = coeff + return PolynomialZonotope(self.center, terms, num_noise=new_noise, noise_kinds=self.noise_kinds + (str(metadata) if metadata is not None else str(kind),)) + + def add_independent_errors( + self, + radii: Any, + *, + kind: str = "approximation", + metadata: str | None = None, + ) -> "PolynomialZonotope": + """Add one fresh independent error symbol per nonzero tensor entry. + + This is the batched counterpart of :meth:`add_independent_error`. + ``radii`` must be scalar for a scalar zonotope or broadcastable to the + coefficient shape. Each nonzero flattened entry receives its own + basis-shaped coefficient, so no dependency is introduced between + different output coordinates. + """ + + if torch is None or not isinstance(self.center, torch.Tensor): + if self.shape == (): + return self.add_independent_error( + radii, + kind=kind, + metadata=metadata, + ) + raise NotImplementedError( + "Batched independent errors currently require tensor-backed coefficients." + ) + + radius_tensor = torch.as_tensor( + radii, + dtype=self.center.dtype, + device=self.center.device, + ) + if tuple(radius_tensor.shape) == () and self.shape != (): + radius_tensor = torch.full_like(self.center, float(radius_tensor.item())) + elif tuple(radius_tensor.shape) != self.shape: + radius_tensor = torch.broadcast_to(radius_tensor, self.shape).clone() + + nonzero = torch.nonzero(radius_tensor.reshape(-1) != 0, as_tuple=False).reshape(-1) + error_count = int(nonzero.numel()) + if error_count == 0: + return self + + terms = { + exponent + (0,) * error_count: coefficient + for exponent, coefficient in self.terms.items() + } + for local_index, flat_index in enumerate(nonzero.tolist()): + coefficient = torch.zeros_like(self.center) + coefficient.reshape(-1)[flat_index] = radius_tensor.reshape(-1)[flat_index] + exponent = [0] * (self.num_noise + error_count) + exponent[self.num_noise + local_index] = 1 + terms[tuple(exponent)] = coefficient + + label = str(metadata) if metadata is not None else str(kind) + return PolynomialZonotope( + self.center, + terms, + num_noise=self.num_noise + error_count, + noise_kinds=self.noise_kinds + (label,) * error_count, + ) + + + def integrate_noise(self, noise_indices: Sequence[int]) -> "PolynomialZonotope": + """Integrate selected noise variables coefficient-by-coefficient. + + For each monomial term, variables in ``noise_indices`` are integrated + exactly over ``[-1, 1]`` using :func:`box_monomial_moment`. Variables + not listed are retained, and terms with identical retained exponents are + merged. Odd integrated exponents have zero moment and are dropped. + """ + + indices = tuple(int(index) for index in noise_indices) + if len(set(indices)) != len(indices): + raise ValueError("noise_indices must not contain duplicates.") + if any(index < 0 or index >= self.num_noise for index in indices): + raise ValueError("noise index out of range.") + + integrated = set(indices) + retained_indices = tuple(index for index in range(self.num_noise) if index not in integrated) + retained_kinds = tuple(self.noise_kinds[index] for index in retained_indices) + zero_retained = (0,) * len(retained_indices) + + center = self.center + terms: dict[Exponent, Any] = {} + for exponent, coeff in self.terms.items(): + integrated_exponent = tuple(exponent[index] for index in indices) + moment = box_monomial_moment(integrated_exponent) + if moment == 0.0: + continue + retained_exponent = tuple(exponent[index] for index in retained_indices) + integrated_coeff = _mul_coeff(coeff, moment) + if retained_exponent == zero_retained: + center = _add_coeff(center, integrated_coeff) + else: + terms[retained_exponent] = _add_coeff(terms[retained_exponent], integrated_coeff) if retained_exponent in terms else integrated_coeff + + return PolynomialZonotope(center, terms, num_noise=len(retained_indices), noise_kinds=retained_kinds) + + def integrate_domain_noise(self) -> "PolynomialZonotope": + """Integrate all noise variables labeled ``"domain"``.""" + + return self.integrate_noise([index for index, kind in enumerate(self.noise_kinds) if kind == "domain"]) + + def integrate_domain(self, domain_indices: Sequence[int] | None = None, *, mode: str = "pointwise_interval", volume: float | None = None): + """Integrate domain variables with pointwise-residual-safe semantics. + + This delegates to :func:`intervalnets.pz_integration.integrate_pz_over_domain` + and returns an ``IntegratedPZResult`` that separates the exact retained + polynomial from the scalar/tensor interval radius accumulated from + pointwise approximation residuals. Use ``mode="symbolic"`` only when + approximation variables are intended to represent global symbolic + uncertainties whose moments may be preserved. + """ + + from .pz_integration import integrate_pz_over_domain + + return integrate_pz_over_domain(self, domain_indices, mode=mode, volume=volume) + + def linear_map(self, matrix: Any, bias: Any | None = None) -> "PolynomialZonotope": + """Apply a linear map along the leading coefficient axis. + + For a weight matrix ``A`` with shape ``(m, n)``, coefficients with + shape ``(n,)``, ``(n, d_in)``, or ``(n, d_in, d_in)`` are mapped to + ``(m,)``, ``(m, d_in)``, or ``(m, d_in, d_in)`` by contracting over + the leading/output axis. ``bias`` is added to the center only. + """ + + if torch is not None and isinstance(self.center, torch.Tensor): + weight = _as_tensor(matrix, dtype=self.center.dtype, device=self.center.device) + if weight.ndim != 2: + raise ValueError("linear_map weight must be a 2-D matrix.") + if self.center.ndim < 1 or self.center.shape[0] != weight.shape[1]: + raise ValueError("Linear map weight/input dimension mismatch.") + + def apply(coeff: Any): + return torch.einsum("ij,j...->i...", weight, coeff) + + center = apply(self.center) + if bias is not None: + center = center + _as_tensor(bias, dtype=self.center.dtype, device=self.center.device) + return PolynomialZonotope(center, {exp: apply(coeff) for exp, coeff in self.terms.items()}, num_noise=self.num_noise, noise_kinds=self.noise_kinds) + + mapped_center = _fallback_linear_contract(matrix, self.center) + if bias is not None: + mapped_center = _add_coeff(mapped_center, _to_fallback(bias)) + return PolynomialZonotope( + mapped_center, + {exp: _fallback_linear_contract(matrix, coeff) for exp, coeff in self.terms.items()}, + num_noise=self.num_noise, + noise_kinds=self.noise_kinds, + ) + + def tensor_product(self, other: "PolynomialZonotope") -> "PolynomialZonotope": + if torch is None or not isinstance(self.center, torch.Tensor) or not isinstance(other.center, torch.Tensor): + raise NotImplementedError("tensor_product currently requires torch-backed coefficients.") + left, right = self._align(other) + def outer(a, b): return torch.einsum("...,...->...", a, b) if a.ndim == b.ndim == 0 else torch.outer(a.reshape(-1), b.reshape(-1)).reshape(*a.shape, *b.shape) + terms: dict[Exponent, Any] = {} + def add(exp, coeff): terms.__setitem__(exp, terms[exp] + coeff if exp in terms else coeff) + for exp, coeff in right.terms.items(): add(exp, outer(left.center, coeff)) + for exp, coeff in left.terms.items(): add(exp, outer(coeff, right.center)) + for e1, c1 in left.terms.items(): + for e2, c2 in right.terms.items(): add(tuple(a + b for a, b in zip(e1, e2)), outer(c1, c2)) + return PolynomialZonotope(outer(left.center, right.center), terms, num_noise=left.num_noise, noise_kinds=left.noise_kinds) + + def __getitem__(self, item: Any) -> "PolynomialZonotope": + if torch is not None and isinstance(self.center, torch.Tensor): + return PolynomialZonotope(self.center[item], {e: c[item] for e, c in self.terms.items()}, num_noise=self.num_noise, noise_kinds=self.noise_kinds) + return PolynomialZonotope( + _fallback_get(self.center, item), + {e: _fallback_get(c, item) for e, c in self.terms.items()}, + num_noise=self.num_noise, + noise_kinds=self.noise_kinds, + ) + + @staticmethod + def stack(items: list["PolynomialZonotope"] | tuple["PolynomialZonotope", ...], dim: int = 0) -> "PolynomialZonotope": + if not items: raise ValueError("stack requires at least one item.") + p = max(item.num_noise for item in items) + aligned = [item.with_num_noise(p) for item in items] + merged_kinds = aligned[0].noise_kinds + for item in aligned[1:]: + merged_kinds = _merge_noise_kinds(merged_kinds, item.noise_kinds) + aligned = [item.with_noise_kinds(merged_kinds) for item in aligned] + if torch is None or not isinstance(aligned[0].center, torch.Tensor): + if dim != 0: raise NotImplementedError("fallback stack supports dim=0 only.") + exps = set().union(*(item.terms.keys() for item in aligned)) + return PolynomialZonotope(tuple(item.center for item in aligned), {e: tuple(item.terms.get(e, _zero_like(item.center)) for item in aligned) for e in exps}, num_noise=p, noise_kinds=merged_kinds) + exps = set().union(*(item.terms.keys() for item in aligned)) + return PolynomialZonotope(torch.stack([item.center for item in aligned], dim=dim), {e: torch.stack([item.terms.get(e, torch.zeros_like(item.center)) for item in aligned], dim=dim) for e in exps}, num_noise=p, noise_kinds=merged_kinds) + + def interval_enclosure(self): + radius = _zero_like(self.center) + for coeff in self.terms.values(): radius = _add_coeff(radius, _abs_coeff(coeff)) + lower = _pad_lower(_add_coeff(self.center, _mul_coeff(radius, -1.0))) + upper = _pad_upper(_add_coeff(self.center, radius)) + try: + from .pytorch import IntervalTensor + if torch is not None and isinstance(lower, torch.Tensor): + return IntervalTensor.from_bounds(lower, upper) + except ImportError: # pragma: no cover + pass + return Interval.from_bounds(lower, upper) + + +@dataclass(frozen=True) +class PZOneJet: + """Polynomial-zonotope value/Jacobian one-jet. + + ``J`` is the derivative with respect to the physical input variable. The + scalable neural-network path retains a dependent polynomial core and adds + certified pointwise residual symbols only for explicitly reduced terms. + """ + + Y: PolynomialZonotope + J: PolynomialZonotope + + @classmethod + def from_input(cls, X: PolynomialZonotope, input_dim: int) -> "PZOneJet": + """Initialize the exact one-jet ``(X, I)`` for a flat input PZ.""" + + if torch is None: + raise ImportError("PyTorch is required to initialize PZOneJet constants.") + if input_dim < 0: + raise ValueError("input_dim must be non-negative.") + kwargs = {} + if isinstance(X.center, torch.Tensor): + kwargs = {"dtype": X.center.dtype, "device": X.center.device} + return cls( + Y=X, + J=PolynomialZonotope.constant( + torch.eye(input_dim, **kwargs), + num_noise=X.num_noise, + noise_kinds=X.noise_kinds, + ), + ) + + +@dataclass(frozen=True) +class PZTwoJet: + """Polynomial-zonotope value/Jacobian/Hessian two-jet. + + ``J`` and ``H`` are derivatives with respect to the physical input + variable ``x``, not derivatives with respect to polynomial-zonotope noise + variables. + """ + + Y: PolynomialZonotope + J: PolynomialZonotope + H: PolynomialZonotope + + @classmethod + def from_input(cls, X: PolynomialZonotope, input_dim: int) -> "PZTwoJet": + """Initialize the two-jet for an input polynomial zonotope. + + The value component is the input zonotope itself. The Jacobian is the + constant identity with shape ``(input_dim, input_dim)`` and the Hessian + is the constant zero tensor with shape + ``(input_dim, input_dim, input_dim)``. Both constants use ``X``'s noise + dimension so future propagation keeps dependencies aligned. + """ + + if torch is None: + raise ImportError("PyTorch is required to initialize PZTwoJet constants.") + if input_dim < 0: + raise ValueError("input_dim must be non-negative.") + kwargs = {} + if isinstance(X.center, torch.Tensor): + kwargs = {"dtype": X.center.dtype, "device": X.center.device} + return cls( + Y=X, + J=PolynomialZonotope.constant(torch.eye(input_dim, **kwargs), num_noise=X.num_noise, noise_kinds=X.noise_kinds), + H=PolynomialZonotope.constant(torch.zeros(input_dim, input_dim, input_dim, **kwargs), num_noise=X.num_noise, noise_kinds=X.noise_kinds), + ) diff --git a/src/intervalnets/pytorch.py b/src/intervalnets/pytorch.py index 8a401e3..6765024 100644 --- a/src/intervalnets/pytorch.py +++ b/src/intervalnets/pytorch.py @@ -1,10 +1,280 @@ from __future__ import annotations +from dataclasses import dataclass +import heapq from itertools import product from math import exp, inf, isfinite, log, nextafter, tanh +from time import perf_counter from typing import Any from .interval import Interval +from .polynomial_zonotope import PZOneJet, PZTwoJet, PolynomialZonotope +from .pz_tanh import ( + affine_tanh_double_prime_enclosure, + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + quadratic_tanh_prime_enclosure, +) +from .pz_integration import PZIntegrationCell, pz_l2norm_bounds, pz_sobolev_norm_bounds +from .pz_norms import pz_twojet_l2_norm, pz_twojet_w12_norm, pz_twojet_w22_norm + + +@dataclass(frozen=True) +class PZTwoJetTraceRecord: + """One opt-in trace snapshot from polynomial-zonotope two-jet propagation.""" + + layer_index: int + layer_name: str + layer_type: str + jet: PZTwoJet + summary: dict[str, dict[str, Any]] + + +@dataclass(frozen=True) +class PZTwoJetTraceResult: + """Final two-jet plus per-layer trace snapshots.""" + + final: PZTwoJet + records: list[PZTwoJetTraceRecord] + + +@dataclass(frozen=True) +class PZValueTraceRecord: + """One opt-in trace snapshot from value-only PZ propagation.""" + + layer_index: int + layer_name: str + layer_type: str + value: PolynomialZonotope + summary: dict[str, Any] + + +@dataclass(frozen=True) +class PZValueTraceResult: + """Final value enclosure plus per-layer trace snapshots.""" + + final: PolynomialZonotope + records: list[PZValueTraceRecord] + + +@dataclass(frozen=True) +class PZOneJetTraceRecord: + """Per-layer diagnostics for certified polynomial one-jet propagation.""" + + layer_index: int + layer_name: str + layer_type: str + value: PolynomialZonotope + jacobian: PolynomialZonotope + jacobian_remainder_radius: Any + elapsed_s: float + summary: dict[str, Any] + + +@dataclass(frozen=True) +class PZOneJetTraceResult: + """Final one-jet plus lightweight per-layer diagnostic snapshots.""" + + final: PZOneJet + records: list[PZOneJetTraceRecord] + + +@dataclass(frozen=True) +class _PZOneJetPolynomialState: + """Dependent Jacobian polynomial plus a certified pointwise box remainder.""" + + Y: PolynomialZonotope + J: PolynomialZonotope + jacobian_remainder_radius: Any + tanh_approximation_radii: Any | None = None + tanh_prime_approximation_radii: Any | None = None + tanh_prime_affine_radii: Any | None = None + tanh_prime_polynomial_reduction_radii: Any | None = None + tanh_prime_approximation_degrees: Any | None = None + tanh_prime_relative_slopes: Any | None = None + preactivation_lower: Any | None = None + preactivation_upper: Any | None = None + + +@dataclass(frozen=True) +class PZReductionConfig: + """Certified monomial reduction used by the polynomial one-jet path. + + ``strategy`` may be ``"none"``, ``"topk"``, ``"degree"``, or + ``"pca"``. Every discarded contribution is enclosed by the pointwise + remainder; PCA additionally replaces a bounded candidate set by a few + coefficient-space generators and bounds its orthogonal residual. + """ + + strategy: str = "topk" + max_terms: int = 96 + max_degree: int = 4 + pca_rank: int = 4 + pca_candidates: int = 48 + reduction_variant: str = "A" + generator_budget: int = 0 + derivative_enclosure: str = "affine" + derivative_flatness_threshold: float = 0.01 + quadratic_certificate_subdivisions: int = 64 + quadratic_compression_guard: bool = True + + def __post_init__(self) -> None: + if self.strategy not in {"none", "topk", "degree", "pca"}: + raise ValueError("strategy must be one of: none, topk, degree, pca.") + if self.max_terms < 1: + raise ValueError("max_terms must be positive.") + if self.max_degree < 0 or self.pca_rank < 0 or self.pca_candidates < 0: + raise ValueError("reduction degrees, ranks, and candidate counts must be non-negative.") + if self.reduction_variant.upper() not in {"A", "B", "C"}: + raise ValueError("reduction_variant must be one of: A, B, C.") + if self.generator_budget < 0: + raise ValueError("generator_budget must be non-negative.") + if self.derivative_enclosure not in {"affine", "quadratic_flat"}: + raise ValueError( + "derivative_enclosure must be either 'affine' or 'quadratic_flat'." + ) + if self.derivative_flatness_threshold < 0.0: + raise ValueError("derivative_flatness_threshold must be non-negative.") + if self.quadratic_certificate_subdivisions < 1: + raise ValueError("quadratic_certificate_subdivisions must be positive.") + + +def _pad_nonnegative_radius(radius: torch.Tensor) -> torch.Tensor: + """Round positive radii upward without turning exact zeros into errors.""" + + return torch.where( + radius == 0, + radius, + torch.nextafter(radius, torch.full_like(radius, float("inf"))), + ) + + +def _pz_summary(zonotope: PolynomialZonotope) -> dict[str, Any]: + return { + "shape": zonotope.shape, + "num_noise": zonotope.num_noise, + "noise_kinds": zonotope.noise_kinds, + "term_count": len(zonotope.terms), + "max_degree": max((sum(exp) for exp in zonotope.terms), default=0), + } + + +def _pz_twojet_trace_record(layer_index: int, layer_name: str, layer_type: str, jet: PZTwoJet) -> PZTwoJetTraceRecord: + return PZTwoJetTraceRecord( + layer_index=layer_index, + layer_name=layer_name, + layer_type=layer_type, + jet=jet, + summary={"Y": _pz_summary(jet.Y), "J": _pz_summary(jet.J), "H": _pz_summary(jet.H)}, + ) + + +def _pz_value_trace_record( + layer_index: int, + layer_name: str, + layer_type: str, + value: PolynomialZonotope, +) -> PZValueTraceRecord: + return PZValueTraceRecord( + layer_index=layer_index, + layer_name=layer_name, + layer_type=layer_type, + value=value, + summary=_pz_summary(value), + ) + + +def _pz_onejet_trace_record( + layer_index: int, + layer_name: str, + layer_type: str, + state: _PZOneJetPolynomialState, + elapsed_s: float, + *, + tanh_approximation_radii: Any | None = None, + tanh_prime_approximation_radii: Any | None = None, +) -> PZOneJetTraceRecord: + activation_summary: dict[str, Any] = {} + if tanh_approximation_radii is not None: + radii = tanh_approximation_radii.detach().clone().reshape(-1) + activation_summary.update({ + "tanh_approximation_radii": radii, + "tanh_approximation_radius_min": float(radii.min().item()) + if radii.numel() + else 0.0, + "tanh_approximation_radius_mean": float(radii.mean().item()) + if radii.numel() + else 0.0, + "tanh_approximation_radius_max": float(radii.max().item()) + if radii.numel() + else 0.0, + }) + if tanh_prime_approximation_radii is not None: + radii = tanh_prime_approximation_radii.detach().clone().reshape(-1) + activation_summary.update({ + "tanh_prime_approximation_radii": radii, + "tanh_prime_approximation_radius_min": float(radii.min().item()) + if radii.numel() + else 0.0, + "tanh_prime_approximation_radius_mean": float(radii.mean().item()) + if radii.numel() + else 0.0, + "tanh_prime_approximation_radius_max": float(radii.max().item()) + if radii.numel() + else 0.0, + }) + if state.tanh_prime_affine_radii is not None: + activation_summary["tanh_prime_affine_radii"] = ( + state.tanh_prime_affine_radii.detach().clone().reshape(-1) + ) + if state.tanh_prime_polynomial_reduction_radii is not None: + reduction_radii = ( + state.tanh_prime_polynomial_reduction_radii.detach().clone().reshape(-1) + ) + activation_summary.update({ + "tanh_prime_polynomial_reduction_radii": reduction_radii, + "tanh_prime_polynomial_reduction_radius_mean": float( + reduction_radii.mean().item() + ) if reduction_radii.numel() else 0.0, + "tanh_prime_polynomial_reduction_radius_max": float( + reduction_radii.max().item() + ) if reduction_radii.numel() else 0.0, + }) + if state.tanh_prime_approximation_degrees is not None: + degrees = state.tanh_prime_approximation_degrees.detach().clone().reshape(-1) + activation_summary.update({ + "tanh_prime_approximation_degrees": degrees, + "tanh_prime_quadratic_count": int(torch.count_nonzero(degrees == 2).item()), + }) + if state.tanh_prime_relative_slopes is not None: + activation_summary["tanh_prime_relative_slopes"] = ( + state.tanh_prime_relative_slopes.detach().clone().reshape(-1) + ) + if state.preactivation_lower is not None and state.preactivation_upper is not None: + activation_summary.update({ + "preactivation_lower": state.preactivation_lower.detach().clone().reshape(-1), + "preactivation_upper": state.preactivation_upper.detach().clone().reshape(-1), + }) + return PZOneJetTraceRecord( + layer_index=layer_index, + layer_name=layer_name, + layer_type=layer_type, + value=state.Y, + jacobian=state.J, + jacobian_remainder_radius=state.jacobian_remainder_radius.detach().clone(), + elapsed_s=float(elapsed_s), + summary={ + "Y": _pz_summary(state.Y), + "J": { + **_pz_summary(state.J), + "remainder_max_radius": float(state.jacobian_remainder_radius.max().item()) + if state.jacobian_remainder_radius.numel() else 0.0, + "remainder_mean_radius": float(state.jacobian_remainder_radius.mean().item()) + if state.jacobian_remainder_radius.numel() else 0.0, + }, + **activation_summary, + }, + ) try: import torch @@ -278,6 +548,1200 @@ def _linear_forward(layer, x: IntervalTensor) -> IntervalTensor: return IntervalTensor.from_bounds(tuple(float(value) for value in lower_tensor.tolist()), tuple(float(value) for value in upper_tensor.tolist())) +def _pz_twojet_linear_forward(layer: nn.Linear, jet: PZTwoJet) -> PZTwoJet: + """Propagate a polynomial-zonotope two-jet through ``nn.Linear`` exactly.""" + + _require_torch() + weight = layer.weight.detach() + bias = layer.bias.detach() if layer.bias is not None else None + is_torch_backend = torch is not None and isinstance(jet.Y.center, torch.Tensor) + if is_torch_backend: + weight = weight.to(dtype=jet.Y.center.dtype, device=jet.Y.center.device) + if bias is not None: + bias = bias.to(dtype=jet.Y.center.dtype, device=jet.Y.center.device) + else: + weight = weight.cpu().tolist() + bias = bias.cpu().tolist() if bias is not None else None + + return PZTwoJet( + Y=jet.Y.linear_map(weight, bias), + J=jet.J.linear_map(weight, bias=None), + H=jet.H.linear_map(weight, bias=None), + ) + + +def _pz_value_linear_forward( + layer: nn.Linear, + value: PolynomialZonotope, +) -> PolynomialZonotope: + """Propagate a value-only polynomial zonotope through ``nn.Linear``.""" + + _require_torch() + weight = layer.weight.detach() + bias = layer.bias.detach() if layer.bias is not None else None + if isinstance(value.center, torch.Tensor): + weight = weight.to(dtype=value.center.dtype, device=value.center.device) + if bias is not None: + bias = bias.to(dtype=value.center.dtype, device=value.center.device) + else: + weight = weight.cpu().tolist() + bias = bias.cpu().tolist() if bias is not None else None + return value.linear_map(weight, bias) + + +def _pz_scalar_interval(zonotope: PolynomialZonotope) -> Interval: + """Return the scalar interval enclosure of a scalar polynomial zonotope.""" + + enclosure = zonotope.interval_enclosure() + lower = enclosure.lower + upper = enclosure.upper + if torch is not None and isinstance(lower, torch.Tensor): + if lower.numel() != 1 or upper.numel() != 1: + raise ValueError("Expected a scalar polynomial-zonotope interval enclosure.") + return Interval(float(lower.reshape(()).item()), float(upper.reshape(()).item())) + if isinstance(lower, tuple) or isinstance(upper, tuple): + raise ValueError("Expected a scalar polynomial-zonotope interval enclosure.") + return Interval(float(lower), float(upper)) + + +def _affine_enclosure_pz( + Z_i: PolynomialZonotope, + *, + slope: float, + intercept: float, + radius: float, +) -> PolynomialZonotope: + """Build ``slope * Z_i + intercept + radius * eta`` with pointwise eta.""" + + return (slope * Z_i + intercept).add_independent_error( + radius, kind="approximation_pointwise" + ) + + +def _pz_twojet_tanh_forward(jet: PZTwoJet, chebyshev_degree: int, residual_subdivisions: int) -> PZTwoJet: + """Propagate a polynomial-zonotope two-jet through componentwise ``tanh``. + + For each scalar preactivation ``Z_i``, compute its interval enclosure and + use certified affine-plus-pointwise-residual enclosures for ``tanh``, + ``tanh'``, and ``tanh''``. The resulting scalar enclosures are propagated + by the componentwise two-jet chain rule without silently replacing existing + polynomial dependencies by intervals. + """ + + _require_torch() + if jet.Y.shape == (): + components = 1 + elif len(jet.Y.shape) == 1: + components = jet.Y.shape[0] + else: + raise ValueError("_pz_twojet_tanh_forward expects a scalar or 1-D value zonotope.") + + y_items: list[PolynomialZonotope] = [] + j_items: list[PolynomialZonotope] = [] + h_items: list[PolynomialZonotope] = [] + + current_noise = jet.Y.num_noise + current_noise_kinds = jet.Y.noise_kinds + for i in range(components): + Z_i = jet.Y if jet.Y.shape == () else jet.Y[i] + Z_i = Z_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds) + interval_i = _pz_scalar_interval(Z_i) + + tanh_i = affine_tanh_enclosure(interval_i) + tanh_prime_i = affine_tanh_prime_enclosure(interval_i) + tanh_double_prime_i = affine_tanh_double_prime_enclosure(interval_i) + + Y_i = _affine_enclosure_pz( + Z_i, slope=tanh_i.p, intercept=tanh_i.q, radius=tanh_i.delta + ) + current_noise = Y_i.num_noise + current_noise_kinds = Y_i.noise_kinds + + D1_i = _affine_enclosure_pz( + Z_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds), + slope=tanh_prime_i.p, + intercept=tanh_prime_i.q, + radius=tanh_prime_i.delta, + ) + current_noise = D1_i.num_noise + current_noise_kinds = D1_i.noise_kinds + + D2_i = _affine_enclosure_pz( + Z_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds), + slope=tanh_double_prime_i.p, + intercept=tanh_double_prime_i.q, + radius=tanh_double_prime_i.delta, + ) + current_noise = D2_i.num_noise + current_noise_kinds = D2_i.noise_kinds + + J_i = jet.J if components == 1 and jet.J.shape[:1] != (components,) else jet.J[i, :] + H_i = jet.H if components == 1 and jet.H.shape[:1] != (components,) else jet.H[i, :, :] + J_i = J_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds) + H_i = H_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds) + + y_items.append(Y_i.with_num_noise(current_noise).with_noise_kinds(current_noise_kinds)) + j_items.append(D1_i * J_i) + h_items.append(D2_i * J_i.tensor_product(J_i) + D1_i * H_i) + + if jet.Y.shape == (): + return PZTwoJet(Y=y_items[0], J=j_items[0], H=h_items[0]) + return PZTwoJet( + Y=PolynomialZonotope.stack(y_items, dim=0), + J=PolynomialZonotope.stack(j_items, dim=0), + H=PolynomialZonotope.stack(h_items, dim=0), + ) + + +def _pz_value_tanh_forward( + value: PolynomialZonotope, + chebyshev_degree: int, + residual_subdivisions: int, + *, + return_approximation_radii: bool = False, +) -> PolynomialZonotope | tuple[PolynomialZonotope, Any]: + """Propagate only function values through componentwise ``tanh``. + + The current activation enclosure is affine. All neuron slopes, + intercepts, and certified residual radii are therefore applied in one + tensor operation, followed by one independent residual symbol per neuron. + Traced one-jet propagation may request the exact residual-radius tensor + used for those new symbols alongside the propagated value. + ``chebyshev_degree`` and ``residual_subdivisions`` remain accepted for API + compatibility with the two-jet path. + """ + + del chebyshev_degree, residual_subdivisions + _require_torch() + if value.shape == (): + components = 1 + elif len(value.shape) == 1: + components = value.shape[0] + else: + raise ValueError("_pz_value_tanh_forward expects a scalar or 1-D value zonotope.") + + enclosure = value.interval_enclosure() + lower = enclosure.lower + upper = enclosure.upper + if isinstance(lower, torch.Tensor): + lower_values = lower.reshape(-1).detach().cpu().tolist() + upper_values = upper.reshape(-1).detach().cpu().tolist() + else: + lower_values = [lower] if value.shape == () else list(lower) + upper_values = [upper] if value.shape == () else list(upper) + + approximations = [ + affine_tanh_enclosure(Interval(float(lo), float(hi))) + for lo, hi in zip(lower_values, upper_values) + ] + if len(approximations) != components: + raise RuntimeError("Tanh enclosure component count does not match the PZ shape.") + + if isinstance(value.center, torch.Tensor): + target_shape = value.center.shape + slopes = torch.tensor( + [item.p for item in approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + intercepts = torch.tensor( + [item.q for item in approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + radii = torch.tensor( + [item.delta for item in approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + affine = PolynomialZonotope( + slopes * value.center + intercepts, + { + exponent: slopes * coefficient + for exponent, coefficient in value.terms.items() + }, + num_noise=value.num_noise, + noise_kinds=value.noise_kinds, + ) + result = affine.add_independent_errors( + radii, + kind="approximation_pointwise", + ) + if return_approximation_radii: + return result, radii + return result + + items = [] + for index, approximation in enumerate(approximations): + component = value if value.shape == () else value[index] + items.append( + _affine_enclosure_pz( + component, + slope=approximation.p, + intercept=approximation.q, + radius=approximation.delta, + ) + ) + result = items[0] if value.shape == () else PolynomialZonotope.stack(items, dim=0) + if return_approximation_radii: + radii = [approximation.delta for approximation in approximations] + return result, radii[0] if value.shape == () else tuple(radii) + return result + + +def _pz_value_forward_from_value( + module, + value: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + return_trace: bool = False, +) -> PolynomialZonotope | PZValueTraceResult: + """Propagate a function-value PZ without allocating derivative tensors.""" + + _require_torch() + if reduce: + raise NotImplementedError("PZ value reduction is not implemented yet.") + if isinstance(module, nn.Sequential): + result = value + records = ( + [_pz_value_trace_record(-1, "input", "Input", result)] + if return_trace + else [] + ) + for index, (name, child) in enumerate(module.named_children()): + result = _pz_value_forward_from_value( + child, + result, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=False, + ) + if return_trace: + records.append( + _pz_value_trace_record( + index, + name, + type(child).__name__, + result, + ) + ) + return PZValueTraceResult(final=result, records=records) if return_trace else result + if isinstance(module, nn.Linear): + result = _pz_value_linear_forward(module, value) + elif isinstance(module, nn.Tanh): + result = _pz_value_tanh_forward( + value, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + elif isinstance(module, nn.Identity): + result = value + elif isinstance(module, nn.Flatten): + if len(value.shape) > 1: + raise NotImplementedError( + "PZ value Flatten currently supports already-flat vectors only." + ) + result = value + else: + raise NotImplementedError( + "PZ value forward currently supports nn.Sequential, nn.Linear, " + "nn.Tanh, nn.Identity, and flat-vector nn.Flatten only; got " + f"{type(module).__name__}." + ) + if return_trace: + return PZValueTraceResult( + final=result, + records=[ + _pz_value_trace_record(-1, "input", "Input", value), + _pz_value_trace_record(0, "0", type(module).__name__, result), + ], + ) + return result + + +def _pz_onejet_linear_forward( + layer: nn.Linear, + state: _PZOneJetPolynomialState, +) -> _PZOneJetPolynomialState: + """Propagate the polynomial core and box remainder through a linear layer.""" + + value = _pz_value_linear_forward(layer, state.Y) + weight = layer.weight.detach().to(dtype=state.J.center.dtype, device=state.J.center.device) + jacobian = state.J.linear_map(weight) + radius = torch.abs(weight) @ state.jacobian_remainder_radius + radius = _pad_nonnegative_radius(radius) + return _PZOneJetPolynomialState(value, jacobian, radius) + + +class _CertifiedTermReducer: + """Certified support reducer implementing reference variants A, B, and C. + + Support selection is controlled separately by ``config.strategy``. A + discarded coefficient is symmetrically boxed in variant A. Variants B and + C first use the exact ``[0, 1]`` range of componentwise-even monomials; + variant C additionally retains up to ``generator_budget`` coefficient + directions as fresh pointwise approximation-noise generators. + """ + + def __init__(self, config: PZReductionConfig, shape: tuple[int, ...], template: torch.Tensor): + self.config = config + self.shape = shape + self.kept: list[tuple[float, int, tuple[int, ...], torch.Tensor]] = [] + self.pca: list[tuple[float, int, torch.Tensor]] = [] + self.generators: list[tuple[float, int, torch.Tensor]] = [] + self.radius = torch.zeros(shape, dtype=template.dtype, device=template.device) + self.midpoint = torch.zeros(shape, dtype=template.dtype, device=template.device) + self.counter = 0 + + def _box(self, coefficient: torch.Tensor) -> None: + self.radius = self.radius + torch.abs(coefficient) + + def _discard(self, exponent: tuple[int, ...], coefficient: torch.Tensor) -> None: + variant = self.config.reduction_variant.upper() + even = variant in {"B", "C"} and all(power % 2 == 0 for power in exponent) + if even: + self.midpoint = self.midpoint + 0.5 * coefficient + generator = 0.5 * coefficient + else: + generator = coefficient + + if variant == "C" and self.config.generator_budget: + score = float(torch.linalg.vector_norm(generator).item()) + item = (score, -self.counter, generator) + self.counter += 1 + if len(self.generators) < self.config.generator_budget: + heapq.heappush(self.generators, item) + elif item[:2] > self.generators[0][:2]: + _, _, evicted = heapq.heapreplace(self.generators, item) + self._box(evicted) + else: + self._box(generator) + return + + if variant == "A" and self.config.strategy == "pca": + self._offer_pca(float(torch.linalg.vector_norm(generator).item()), generator) + else: + self._box(generator) + + def _offer_pca(self, score: float, coefficient: torch.Tensor) -> None: + if self.config.strategy != "pca" or self.config.pca_candidates == 0: + self._box(coefficient) + return + item = (score, self.counter, coefficient) + self.counter += 1 + if len(self.pca) < self.config.pca_candidates: + heapq.heappush(self.pca, item) + elif score > self.pca[0][0]: + _, _, evicted = heapq.heapreplace(self.pca, item) + self._box(evicted) + else: + self._box(coefficient) + + def offer(self, exponent: tuple[int, ...], coefficient: torch.Tensor) -> None: + if not bool(torch.any(coefficient != 0).item()): + return + if self.config.strategy == "degree" and sum(exponent) > self.config.max_degree: + self._discard(exponent, coefficient) + return + score = float(torch.linalg.vector_norm(coefficient).item()) + item = (score, self.counter, exponent, coefficient) + self.counter += 1 + if len(self.kept) < self.config.max_terms: + heapq.heappush(self.kept, item) + elif score > self.kept[0][0]: + _, _, evicted_exponent, evicted = heapq.heapreplace(self.kept, item) + self._discard(evicted_exponent, evicted) + else: + self._discard(exponent, coefficient) + + def finish( + self, + center: torch.Tensor, + *, + num_noise: int, + noise_kinds: tuple[str, ...], + ) -> tuple[PolynomialZonotope, torch.Tensor]: + terms: dict[tuple[int, ...], torch.Tensor] = {} + for _, _, exponent, coefficient in self.kept: + terms[exponent] = terms.get(exponent, torch.zeros_like(center)) + coefficient + + center = center + self.midpoint + kinds = noise_kinds + if self.generators: + retained_generators = sorted(self.generators, key=lambda item: (-item[0], -item[1])) + rank = len(retained_generators) + terms = { + old_exp + (0,) * rank: old_coeff for old_exp, old_coeff in terms.items() + } + base_num_noise = num_noise + for index, (_, _, coefficient) in enumerate(retained_generators): + exponent = ( + (0,) * (base_num_noise + index) + + (1,) + + (0,) * (rank - index - 1) + ) + terms[exponent] = coefficient + num_noise += rank + kinds = kinds + ("approximation_pointwise",) * rank + if self.pca and self.config.pca_rank: + generators = torch.stack([item[2].reshape(-1) for item in self.pca], dim=0) + rank = min(self.config.pca_rank, generators.shape[0], generators.shape[1]) + _, _, vh = torch.linalg.svd(generators, full_matrices=False) + directions = vh[:rank] + coordinates = generators @ directions.T + projected_radii = torch.sum(torch.abs(coordinates), dim=0) + residual = generators - coordinates @ directions + self.radius = self.radius + torch.sum(torch.abs(residual), dim=0).reshape(self.shape) + base_num_noise = num_noise + terms = { + old_exp + (0,) * rank: old_coeff for old_exp, old_coeff in terms.items() + } + for index in range(rank): + coefficient = (projected_radii[index] * directions[index]).reshape(self.shape) + exponent = (0,) * (base_num_noise + index) + (1,) + (0,) * (rank - index - 1) + terms[exponent] = coefficient + num_noise += rank + kinds = kinds + ("approximation_pointwise",) * rank + elif self.pca: + for _, _, coefficient in self.pca: + self._box(coefficient) + + radius = _pad_nonnegative_radius(self.radius) + return PolynomialZonotope(center, terms, num_noise=num_noise, noise_kinds=kinds), radius + + +def _rowwise_pz_product( + derivative: PolynomialZonotope, + jacobian: PolynomialZonotope, + config: PZReductionConfig, +) -> tuple[PolynomialZonotope, torch.Tensor]: + """Multiply a vector PZ into Jacobian rows and reduce generators soundly.""" + + derivative, jacobian = derivative._align(jacobian) + center = derivative.center.unsqueeze(1) * jacobian.center + if config.strategy == "none": + terms: dict[tuple[int, ...], torch.Tensor] = {} + def add(exponent, coefficient): + terms[exponent] = terms.get(exponent, torch.zeros_like(center)) + coefficient + for exponent, coefficient in derivative.terms.items(): + add(exponent, coefficient.unsqueeze(1) * jacobian.center) + for exponent, coefficient in jacobian.terms.items(): + add(exponent, derivative.center.unsqueeze(1) * coefficient) + for d_exp, d_coeff in derivative.terms.items(): + for j_exp, j_coeff in jacobian.terms.items(): + add(tuple(a + b for a, b in zip(d_exp, j_exp)), d_coeff.unsqueeze(1) * j_coeff) + return PolynomialZonotope(center, terms, num_noise=derivative.num_noise, noise_kinds=derivative.noise_kinds), torch.zeros_like(center) + + canonical: dict[tuple[int, ...], torch.Tensor] = {} + def add(exponent, coefficient): + canonical[exponent] = canonical.get(exponent, torch.zeros_like(center)) + coefficient + for exponent, coefficient in derivative.terms.items(): + add(exponent, coefficient.unsqueeze(1) * jacobian.center) + for exponent, coefficient in jacobian.terms.items(): + add(exponent, derivative.center.unsqueeze(1) * coefficient) + for d_exp, d_coeff in derivative.terms.items(): + for j_exp, j_coeff in jacobian.terms.items(): + add(tuple(a + b for a, b in zip(d_exp, j_exp)), d_coeff.unsqueeze(1) * j_coeff) + reducer = _CertifiedTermReducer(config, tuple(center.shape), center) + for exponent in sorted(canonical): + reducer.offer(exponent, canonical[exponent]) + return reducer.finish(center, num_noise=derivative.num_noise, noise_kinds=derivative.noise_kinds) + + +def _reduced_quadratic_pz_core( + value: PolynomialZonotope, + constants: torch.Tensor, + linears: torch.Tensor, + quadratics: torch.Tensor, + config: PZReductionConfig, +) -> tuple[PolynomialZonotope, torch.Tensor]: + """Evaluate a componentwise quadratic while streaming through reduction.""" + + if config.strategy == "none": + return linears * value + constants + quadratics * (value * value), torch.zeros_like( + value.center + ) + + center = constants + linears * value.center + quadratics * value.center**2 + canonical: dict[tuple[int, ...], torch.Tensor] = {} + def add(exponent, coefficient): + canonical[exponent] = canonical.get(exponent, torch.zeros_like(center)) + coefficient + linear_factor = linears + 2.0 * quadratics * value.center + items = list(value.terms.items()) + for exponent, coefficient in items: + add(exponent, linear_factor * coefficient) + for left_index, (left_exp, left_coeff) in enumerate(items): + for right_index in range(left_index, len(items)): + right_exp, right_coeff = items[right_index] + factor = 1.0 if left_index == right_index else 2.0 + add( + tuple(a + b for a, b in zip(left_exp, right_exp)), + factor * quadratics * left_coeff * right_coeff, + ) + reducer = _CertifiedTermReducer(config, tuple(center.shape), center) + for exponent in sorted(canonical): + reducer.offer(exponent, canonical[exponent]) + return reducer.finish( + center, + num_noise=value.num_noise, + noise_kinds=value.noise_kinds, + ) + + +def _select_pz_components( + preferred: PolynomialZonotope, + fallback: PolynomialZonotope, + mask: torch.Tensor, +) -> PolynomialZonotope: + """Select vector PZ coefficient components without losing dependencies.""" + + preferred, fallback = preferred._align(fallback) + terms: dict[tuple[int, ...], torch.Tensor] = {} + for exponent in preferred.terms.keys() | fallback.terms.keys(): + preferred_coefficient = preferred.terms.get( + exponent, torch.zeros_like(preferred.center) + ) + fallback_coefficient = fallback.terms.get( + exponent, torch.zeros_like(fallback.center) + ) + terms[exponent] = torch.where( + mask, preferred_coefficient, fallback_coefficient + ) + return PolynomialZonotope( + torch.where(mask, preferred.center, fallback.center), + terms, + num_noise=preferred.num_noise, + noise_kinds=preferred.noise_kinds, + ) + + +def _batched_tanh_derivative_core( + value: PolynomialZonotope, + config: PZReductionConfig, +) -> tuple[ + PolynomialZonotope, + torch.Tensor, + torch.Tensor, + torch.Tensor, + torch.Tensor, + torch.Tensor, + torch.Tensor, + torch.Tensor, + torch.Tensor, +]: + enclosure = value.interval_enclosure() + lower_tensor = torch.as_tensor( + enclosure.lower, dtype=value.center.dtype, device=value.center.device + ).reshape(value.center.shape) + upper_tensor = torch.as_tensor( + enclosure.upper, dtype=value.center.dtype, device=value.center.device + ).reshape(value.center.shape) + lower = lower_tensor.reshape(-1).detach().cpu().tolist() + upper = upper_tensor.reshape(-1).detach().cpu().tolist() + affine_approximations = [ + affine_tanh_prime_enclosure(Interval(float(lo), float(hi))) + for lo, hi in zip(lower, upper) + ] + chosen_coeffs: list[tuple[float, float, float]] = [] + chosen_radii: list[float] = [] + chosen_degrees: list[int] = [] + relative_slopes: list[float] = [] + for lo, hi, affine in zip(lower, upper, affine_approximations): + half_width = (float(hi) - float(lo)) / 2.0 + d_lo = 1.0 - tanh(float(lo)) ** 2 + d_hi = 1.0 - tanh(float(hi)) ** 2 + d_max = 1.0 if float(lo) <= 0.0 <= float(hi) else max(d_lo, d_hi) + interval_radius = (d_max - min(d_lo, d_hi)) / 2.0 + relative_slope = ( + abs(affine.p) * half_width / interval_radius + if interval_radius > 0.0 + else 0.0 + ) + relative_slopes.append(relative_slope) + use_quadratic = ( + config.derivative_enclosure == "quadratic_flat" + and float(lo) <= 0.0 <= float(hi) + and relative_slope <= config.derivative_flatness_threshold + ) + quadratic = ( + quadratic_tanh_prime_enclosure( + Interval(float(lo), float(hi)), + certificate_subdivisions=config.quadratic_certificate_subdivisions, + ) + if use_quadratic + else None + ) + if quadratic is not None and quadratic.delta < affine.delta: + chosen_coeffs.append(quadratic.coeffs) + chosen_radii.append(quadratic.delta) + chosen_degrees.append(2) + else: + chosen_coeffs.append((affine.q, affine.p, 0.0)) + chosen_radii.append(affine.delta) + chosen_degrees.append(1) + + target_shape = value.center.shape + constants = torch.tensor( + [coeffs[0] for coeffs in chosen_coeffs], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + linears = torch.tensor( + [coeffs[1] for coeffs in chosen_coeffs], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + quadratics = torch.tensor( + [coeffs[2] for coeffs in chosen_coeffs], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + proposal_linear_core = PolynomialZonotope( + linears * value.center + constants, + {exponent: linears * coefficient for exponent, coefficient in value.terms.items()}, + num_noise=value.num_noise, + noise_kinds=value.noise_kinds, + ) + if any(degree == 2 for degree in chosen_degrees): + core, quadratic_dropped_radius = _reduced_quadratic_pz_core( + value, + constants, + linears, + quadratics, + config, + ) + else: + core = proposal_linear_core + quadratic_dropped_radius = torch.zeros_like(value.center) + radii = torch.tensor( + chosen_radii, dtype=value.center.dtype, device=value.center.device + ).reshape(target_shape) + affine_radii = torch.tensor( + [item.delta for item in affine_approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + affine_slopes = torch.tensor( + [item.p for item in affine_approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + affine_intercepts = torch.tensor( + [item.q for item in affine_approximations], + dtype=value.center.dtype, + device=value.center.device, + ).reshape(target_shape) + affine_core = PolynomialZonotope( + affine_slopes * value.center + affine_intercepts, + { + exponent: affine_slopes * coefficient + for exponent, coefficient in value.terms.items() + }, + num_noise=value.num_noise, + noise_kinds=value.noise_kinds, + ) + degrees = torch.tensor( + chosen_degrees, dtype=torch.int64, device=value.center.device + ).reshape(target_shape) + if config.quadratic_compression_guard: + keep_quadratic = (degrees == 2) & ( + radii + quadratic_dropped_radius < affine_radii + ) + core = _select_pz_components(core, affine_core, keep_quadratic) + radii = torch.where(keep_quadratic, radii, affine_radii) + quadratic_dropped_radius = torch.where( + keep_quadratic, + quadratic_dropped_radius, + torch.zeros_like(quadratic_dropped_radius), + ) + degrees = torch.where(keep_quadratic, degrees, torch.ones_like(degrees)) + relative_slope_tensor = torch.tensor( + relative_slopes, dtype=value.center.dtype, device=value.center.device + ).reshape(target_shape) + return ( + core, + _pad_nonnegative_radius(radii + quadratic_dropped_radius), + _pad_nonnegative_radius(radii), + lower_tensor, + upper_tensor, + affine_radii, + _pad_nonnegative_radius(quadratic_dropped_radius), + degrees, + relative_slope_tensor, + ) + + +def _pz_onejet_tanh_forward_reduced( + state: _PZOneJetPolynomialState, + chebyshev_degree: int, + residual_subdivisions: int, + config: PZReductionConfig, +) -> _PZOneJetPolynomialState: + ( + derivative, + derivative_radius, + derivative_approximation_radius, + preactivation_lower, + preactivation_upper, + affine_derivative_radius, + derivative_polynomial_reduction_radius, + derivative_degrees, + derivative_relative_slopes, + ) = ( + _batched_tanh_derivative_core(state.Y, config) + ) + value, value_radius = _pz_value_tanh_forward( + state.Y, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + return_approximation_radii=True, + ) + value, jacobian_input = value._align(state.J) + derivative, jacobian_input = derivative._align(jacobian_input) + value = value.with_num_noise(jacobian_input.num_noise).with_noise_kinds(jacobian_input.noise_kinds) + polynomial, dropped_radius = _rowwise_pz_product(derivative, jacobian_input, config) + + d_interval = derivative.interval_enclosure() + j_interval = jacobian_input.interval_enclosure() + d_lower = torch.as_tensor(d_interval.lower, dtype=derivative.center.dtype, device=derivative.center.device) + d_upper = torch.as_tensor(d_interval.upper, dtype=derivative.center.dtype, device=derivative.center.device) + j_lower = torch.as_tensor(j_interval.lower, dtype=jacobian_input.center.dtype, device=jacobian_input.center.device) + j_upper = torch.as_tensor(j_interval.upper, dtype=jacobian_input.center.dtype, device=jacobian_input.center.device) + d_abs = torch.maximum(torch.abs(d_lower), torch.abs(d_upper)).unsqueeze(1) + j_abs = torch.maximum(torch.abs(j_lower), torch.abs(j_upper)) + propagated_radius = ( + d_abs * state.jacobian_remainder_radius + + derivative_radius.unsqueeze(1) * j_abs + + derivative_radius.unsqueeze(1) * state.jacobian_remainder_radius + ) + total_radius = _pad_nonnegative_radius(propagated_radius + dropped_radius) + final_noise = max(value.num_noise, polynomial.num_noise) + kinds = polynomial.with_num_noise(final_noise).noise_kinds + return _PZOneJetPolynomialState( + value.with_num_noise(final_noise).with_noise_kinds(kinds), + polynomial.with_num_noise(final_noise).with_noise_kinds(kinds), + total_radius, + value_radius, + derivative_approximation_radius, + affine_derivative_radius, + derivative_polynomial_reduction_radius, + derivative_degrees, + derivative_relative_slopes, + preactivation_lower, + preactivation_upper, + ) + + +def _pz_onejet_forward_from_state( + module, + state: _PZOneJetPolynomialState, + *, + chebyshev_degree: int, + residual_subdivisions: int, + reduction: PZReductionConfig, +) -> _PZOneJetPolynomialState: + if isinstance(module, nn.Sequential): + result = state + for child in module.children(): + result = _pz_onejet_forward_from_state( + child, + result, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction=reduction, + ) + return result + if isinstance(module, nn.Linear): + return _pz_onejet_linear_forward(module, state) + if isinstance(module, nn.Tanh): + return _pz_onejet_tanh_forward_reduced( + state, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + config=reduction, + ) + if isinstance(module, nn.Identity): + return state + if isinstance(module, nn.Flatten): + if len(state.Y.shape) > 1: + raise NotImplementedError( + "PZ one-jet Flatten currently supports already-flat vectors only." + ) + return state + raise NotImplementedError( + "PZ one-jet forward currently supports nn.Sequential, nn.Linear, " + "nn.Tanh, nn.Identity, and flat-vector nn.Flatten only; got " + f"{type(module).__name__}." + ) + + +def _finalize_pz_onejet(state: _PZOneJetPolynomialState) -> PZOneJet: + """Attach only the accumulated reduction remainder as a pointwise box.""" + + jacobian = state.J.add_independent_errors( + state.jacobian_remainder_radius, + kind="approximation_pointwise", + ) + value = state.Y.with_num_noise(jacobian.num_noise).with_noise_kinds( + jacobian.noise_kinds + ) + return PZOneJet(Y=value, J=jacobian) + + +def pz_onejet_forward( + module, + x: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = True, + reduction_strategy: str = "topk", + max_terms: int = 96, + max_degree: int = 4, + pca_rank: int = 4, + pca_candidates: int = 48, + reduction_variant: str = "A", + generator_budget: int = 0, + derivative_enclosure: str = "affine", + derivative_flatness_threshold: float = 0.01, + quadratic_certificate_subdivisions: int = 64, + quadratic_compression_guard: bool = True, + input_dim: int | None = None, + return_trace: bool = False, +) -> PZOneJet | PZOneJetTraceResult: + """Evaluate a certified polynomial value/Jacobian one-jet. + + Selected Jacobian monomials remain exact. Discarded terms are enclosed by + a separately propagated pointwise remainder, so reduction never silently + becomes interval-only Jacobian propagation. + """ + + _require_torch() + if not isinstance(x, PolynomialZonotope): + raise TypeError("pz_onejet_forward(module, x) requires x to be a PolynomialZonotope.") + if len(x.shape) > 1: + raise NotImplementedError( + "PZ one-jet forward currently supports scalar or flat-vector inputs only." + ) + inferred_dim = 1 if x.shape == () else x.shape[0] + dim = inferred_dim if input_dim is None else int(input_dim) + if dim != inferred_dim: + raise ValueError( + f"input_dim={dim} does not match polynomial-zonotope input dimension {inferred_dim}." + ) + + if not isinstance(x.center, torch.Tensor): + parameter = next(module.parameters(), None) + dtype = ( + parameter.dtype + if parameter is not None and parameter.is_floating_point() + else torch.float64 + ) + device = parameter.device if parameter is not None else None + x = PolynomialZonotope( + torch.as_tensor(x.center, dtype=dtype, device=device), + { + exponent: torch.as_tensor(coefficient, dtype=dtype, device=device) + for exponent, coefficient in x.terms.items() + }, + num_noise=x.num_noise, + noise_kinds=x.noise_kinds, + ) + + strategy = reduction_strategy if reduce else "none" + reduction = PZReductionConfig( + strategy=strategy, + max_terms=max_terms, + max_degree=max_degree, + pca_rank=pca_rank, + pca_candidates=pca_candidates, + reduction_variant=reduction_variant, + generator_budget=generator_budget, + derivative_enclosure=derivative_enclosure, + derivative_flatness_threshold=derivative_flatness_threshold, + quadratic_certificate_subdivisions=quadratic_certificate_subdivisions, + quadratic_compression_guard=quadratic_compression_guard, + ) + identity = torch.eye(dim, dtype=x.center.dtype, device=x.center.device) + initial_jacobian = PolynomialZonotope.constant( + identity, + num_noise=x.num_noise, + noise_kinds=x.noise_kinds, + ) + state = _PZOneJetPolynomialState(x, initial_jacobian, torch.zeros_like(identity)) + records: list[PZOneJetTraceRecord] = [] + if return_trace: + records.append(_pz_onejet_trace_record(-1, "input", "Input", state, 0.0)) + + if return_trace and isinstance(module, nn.Sequential): + result = state + for index, (name, child) in enumerate(module.named_children()): + start = perf_counter() + result = _pz_onejet_forward_from_state( + child, + result, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction=reduction, + ) + records.append( + _pz_onejet_trace_record( + index, + name, + type(child).__name__, + result, + perf_counter() - start, + tanh_approximation_radii=( + result.tanh_approximation_radii + if isinstance(child, nn.Tanh) + else None + ), + tanh_prime_approximation_radii=( + result.tanh_prime_approximation_radii + if isinstance(child, nn.Tanh) + else None + ), + ) + ) + else: + start = perf_counter() + result = _pz_onejet_forward_from_state( + module, + state, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction=reduction, + ) + if return_trace: + records.append( + _pz_onejet_trace_record( + 0, + "0", + type(module).__name__, + result, + perf_counter() - start, + ) + ) + + onejet = _finalize_pz_onejet(result) + return PZOneJetTraceResult(onejet, records) if return_trace else onejet + + +def _pz_twojet_forward_from_jet( + module, + jet: PZTwoJet, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + return_trace: bool = False, +) -> PZTwoJet | PZTwoJetTraceResult: + """Propagate an initialized two-jet through supported PyTorch modules.""" + + _require_torch() + if reduce: + raise NotImplementedError("PZ two-jet reduction is not implemented yet.") + if isinstance(module, nn.Sequential): + result = jet + records = [_pz_twojet_trace_record(-1, "input", "Input", result)] if return_trace else [] + for index, (name, child) in enumerate(module.named_children()): + result = _pz_twojet_forward_from_jet( + child, + result, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=False, + ) + if return_trace: + records.append(_pz_twojet_trace_record(index, name, type(child).__name__, result)) + return PZTwoJetTraceResult(final=result, records=records) if return_trace else result + if isinstance(module, nn.Linear): + result = _pz_twojet_linear_forward(module, jet) + elif isinstance(module, nn.Tanh): + result = _pz_twojet_tanh_forward( + jet, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + elif isinstance(module, nn.Identity): + result = jet + elif isinstance(module, nn.Flatten): + if len(jet.Y.shape) > 1: + raise NotImplementedError("PZ two-jet Flatten currently supports already-flat vectors only.") + result = jet + else: + raise NotImplementedError( + f"PZ two-jet forward currently supports nn.Sequential, nn.Linear, nn.Tanh, nn.Identity, and flat-vector nn.Flatten only; got {type(module).__name__}." + ) + if return_trace: + records = [ + _pz_twojet_trace_record(-1, "input", "Input", jet), + _pz_twojet_trace_record(0, "0", type(module).__name__, result), + ] + return PZTwoJetTraceResult(final=result, records=records) + return result + + +def pz_twojet_forward( + module, + x: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + input_dim: int | None = None, + return_trace: bool = False, +) -> PZTwoJet | PZTwoJetTraceResult: + """Evaluate a supported PyTorch module on a polynomial-zonotope two-jet. + + ``x`` must be a flat scalar/vector polynomial zonotope. The returned + two-jet contains polynomial-zonotope enclosures for the value, Jacobian, + and Hessian with respect to the physical input coordinates. + """ + + _require_torch() + if not isinstance(x, PolynomialZonotope): + raise TypeError("pz_twojet_forward(module, x) requires x to be a PolynomialZonotope.") + if len(x.shape) > 1: + raise NotImplementedError("PZ two-jet forward currently supports scalar or flat-vector inputs only.") + inferred_dim = 1 if x.shape == () else x.shape[0] + dim = inferred_dim if input_dim is None else int(input_dim) + if dim != inferred_dim: + raise ValueError(f"input_dim={dim} does not match polynomial-zonotope input dimension {inferred_dim}.") + jet = PZTwoJet.from_input(x, input_dim=dim) + return _pz_twojet_forward_from_jet( + module, + jet, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=return_trace, + ) + + +def pz_value_forward( + module, + x: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + return_trace: bool = False, +) -> PolynomialZonotope | PZValueTraceResult: + """Evaluate only a network's function-value PZ enclosure. + + Unlike :func:`pz_twojet_forward`, this path never initializes or + propagates Jacobian and Hessian coefficient tensors. It is the intended + forward routine for certified PZ ``L^2`` computation. + """ + + _require_torch() + if not isinstance(x, PolynomialZonotope): + raise TypeError("pz_value_forward(module, x) requires x to be a PolynomialZonotope.") + if len(x.shape) > 1: + raise NotImplementedError( + "PZ value forward currently supports scalar or flat-vector inputs only." + ) + if not isinstance(x.center, torch.Tensor): + parameter = next(module.parameters(), None) + dtype = ( + parameter.dtype + if parameter is not None and parameter.is_floating_point() + else torch.float64 + ) + device = parameter.device if parameter is not None else None + x = PolynomialZonotope( + torch.as_tensor(x.center, dtype=dtype, device=device), + { + exponent: torch.as_tensor( + coefficient, + dtype=dtype, + device=device, + ) + for exponent, coefficient in x.terms.items() + }, + num_noise=x.num_noise, + noise_kinds=x.noise_kinds, + ) + return _pz_value_forward_from_value( + module, + x, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=return_trace, + ) + + +def pz_l2norm( + module, + domain: IntervalTensor, + p: float = 2.0, + *, + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", +) -> Interval: + """Return a value-only PZ enclosure of a module's L2 norm over ``domain``.""" + + _require_torch() + if not isfinite(float(p)) or float(p) != 2.0: + raise NotImplementedError("Polynomial-zonotope norm helpers currently support only p=2.0.") + if not isinstance(domain, IntervalTensor): + raise TypeError("pz_l2norm(module, domain) requires an IntervalTensor domain.") + return pz_l2norm_bounds( + module, + domain, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) + + +def pz_sobolev_norm( + module, + domain: IntervalTensor, + p: float = 2.0, + order: int = 1, + *, + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", +) -> Interval: + """Return a certified PZ enclosure of a module's W^{order,2} norm. + + Order one uses the reduced dependent-polynomial one-jet; order two uses + the dependent polynomial-zonotope two-jet. + """ + + _require_torch() + if not isfinite(float(p)) or float(p) != 2.0: + raise NotImplementedError("Polynomial-zonotope norm helpers currently support only p=2.0.") + if not isinstance(domain, IntervalTensor): + raise TypeError("pz_sobolev_norm(module, domain) requires an IntervalTensor domain.") + if order not in {1, 2}: + raise ValueError("order must be either 1 or 2.") + return pz_sobolev_norm_bounds( + module, + domain, + order=order, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) + def _concretize_affine_bounds( lower_matrix: torch.Tensor, lower_bias: torch.Tensor, @@ -581,6 +2045,7 @@ def _lpnorm_bounds( p: float, iterations: int, theta: float, + enclosure_mode: str = "slope", forward_refine_splits: int = 1, forward_refine_max_cells: int = 256, ) -> Interval: @@ -592,6 +2057,8 @@ def _lpnorm_bounds( raise ValueError("p must be a positive finite real number.") if iterations < 0: raise ValueError("iterations must be non-negative.") + if enclosure_mode not in {"box", "slope"}: + raise ValueError("enclosure_mode must be either 'box' or 'slope'.") _validate_dorfler_theta(theta) if forward_refine_splits < 1: raise ValueError("forward_refine_splits must be at least 1.") @@ -697,6 +2164,27 @@ def _interval_derivative_bounds_tanh(value: Interval) -> Interval: return Interval.from_bounds(lower_out, upper_out) +def _interval_second_derivative_bounds_relu(value: Interval) -> Interval: + _ = value + return Interval.point(0.0) + + +def _interval_second_derivative_bounds_sigmoid(value: Interval) -> Interval: + sigmoid_bounds = _apply_monotone_bounds(IntervalTensor((value.lower,), (value.upper,)), _sigmoid_scalar) + sigma = Interval(sigmoid_bounds.lower[0], sigmoid_bounds.upper[0]) + one = Interval.point(1.0) + two = Interval.point(2.0) + return sigma * (one - sigma) * (one - (two * sigma)) + + +def _interval_second_derivative_bounds_tanh(value: Interval) -> Interval: + tanh_bounds = _apply_monotone_bounds(IntervalTensor((value.lower,), (value.upper,)), tanh) + tanh_interval = Interval(tanh_bounds.lower[0], tanh_bounds.upper[0]) + one = Interval.point(1.0) + two = Interval.point(2.0) + return -(two * tanh_interval * (one - (tanh_interval * tanh_interval))) + + def _matrix_multiply(left: list[list[Interval]], right: list[list[Interval]]) -> list[list[Interval]]: if not left or not right: return [] @@ -736,6 +2224,64 @@ def _matrix_multiply(left: list[list[Interval]], right: list[list[Interval]]) -> ] +def _zero_hessian(output_dim: int, input_dim: int) -> list[list[list[Interval]]]: + return [ + [[Interval.point(0.0) for _ in range(input_dim)] for _ in range(input_dim)] + for _ in range(output_dim) + ] + + +def _outer_product_interval(row_left: list[Interval], row_right: list[Interval]) -> list[list[Interval]]: + size = len(row_left) + if size != len(row_right): + raise ValueError("Rows must have matching lengths for outer-product intervals.") + return [ + [row_left[i] * row_right[j] for j in range(size)] + for i in range(size) + ] + + +def _hessian_compose( + local_jacobian: list[list[Interval]], + local_hessian: list[list[list[Interval]]], + previous_jacobian: list[list[Interval]], + previous_hessian: list[list[list[Interval]]], +) -> tuple[list[list[Interval]], list[list[list[Interval]]]]: + new_jacobian = _matrix_multiply(local_jacobian, previous_jacobian) + if not local_jacobian: + return new_jacobian, [] + + output_dim = len(local_jacobian) + layer_input_dim = len(local_jacobian[0]) + base_input_dim = len(previous_jacobian[0]) if previous_jacobian else 0 + new_hessian = _zero_hessian(output_dim, base_input_dim) + + for out_idx in range(output_dim): + acc = [[Interval.point(0.0) for _ in range(base_input_dim)] for _ in range(base_input_dim)] + + # Chain-rule term: sum_a J_g[k,a] * H_f[a,:,:] + for a in range(layer_input_dim): + coeff = local_jacobian[out_idx][a] + for i in range(base_input_dim): + for j in range(base_input_dim): + acc[i][j] = acc[i][j] + (coeff * previous_hessian[a][i][j]) + + # Curvature term: sum_{a,b} H_g[k,a,b] * J_f[a,:] ⊗ J_f[b,:] + for a in range(layer_input_dim): + for b in range(layer_input_dim): + coeff_h = local_hessian[out_idx][a][b] + if float(coeff_h.lower) == 0.0 and float(coeff_h.upper) == 0.0: + continue + outer = _outer_product_interval(previous_jacobian[a], previous_jacobian[b]) + for i in range(base_input_dim): + for j in range(base_input_dim): + acc[i][j] = acc[i][j] + (coeff_h * outer[i][j]) + + new_hessian[out_idx] = acc + + return new_jacobian, new_hessian + + def _jacobian_for_layer(layer, pre_activation: IntervalTensor) -> list[list[Interval]]: if isinstance(layer, nn.Linear): weight = layer.weight.detach().cpu() @@ -786,6 +2332,46 @@ def _jacobian_for_layer(layer, pre_activation: IntervalTensor) -> list[list[Inte ) +def _hessian_for_layer(layer, pre_activation: IntervalTensor) -> list[list[list[Interval]]]: + if len(pre_activation.shape) != 1: + raise NotImplementedError("Interval Hessians currently support flat vectors only.") + size = len(pre_activation.lower) + if isinstance(layer, nn.Linear): + return _zero_hessian(layer.out_features, size) + if isinstance(layer, nn.Flatten): + return _zero_hessian(size, size) + if isinstance(layer, nn.ReLU): + second_derivatives = [ + _interval_second_derivative_bounds_relu(Interval(pre_activation.lower[idx], pre_activation.upper[idx])) + for idx in range(size) + ] + tensor = _zero_hessian(size, size) + for idx, val in enumerate(second_derivatives): + tensor[idx][idx][idx] = val + return tensor + if isinstance(layer, nn.Sigmoid): + second_derivatives = [ + _interval_second_derivative_bounds_sigmoid(Interval(pre_activation.lower[idx], pre_activation.upper[idx])) + for idx in range(size) + ] + tensor = _zero_hessian(size, size) + for idx, val in enumerate(second_derivatives): + tensor[idx][idx][idx] = val + return tensor + if isinstance(layer, nn.Tanh): + second_derivatives = [ + _interval_second_derivative_bounds_tanh(Interval(pre_activation.lower[idx], pre_activation.upper[idx])) + for idx in range(size) + ] + tensor = _zero_hessian(size, size) + for idx, val in enumerate(second_derivatives): + tensor[idx][idx][idx] = val + return tensor + raise NotImplementedError( + f"Interval Hessian currently supports nn.Linear, nn.ReLU, nn.Sigmoid, nn.Tanh, and nn.Flatten; got {type(layer).__name__}." + ) + + def _sequential_layer_inputs(module: nn.Sequential, domain: IntervalTensor, enclosure_mode: str) -> list[IntervalTensor]: children = list(module.children()) if not children: @@ -821,15 +2407,59 @@ def _eval_jacobian_bounds(model, domain: IntervalTensor, enclosure_mode: str = " return IntervalTensor.from_bounds(lower, upper) +def _eval_hessian_bounds(model, domain: IntervalTensor, enclosure_mode: str = "box") -> IntervalTensor: + if not isinstance(domain, IntervalTensor): + raise TypeError("model.eval_hessian(domain) requires an IntervalTensor domain.") + if len(domain.shape) != 1: + raise NotImplementedError("Interval Hessian evaluation currently supports flat input boxes only.") + if enclosure_mode not in {"box", "slope"}: + raise ValueError("enclosure_mode must be either 'box' or 'slope'.") + + input_dim = len(domain.lower) + if isinstance(model, nn.Sequential): + layer_inputs = _sequential_layer_inputs(model, domain, enclosure_mode=enclosure_mode) + current_jacobian = _identity_jacobian(input_dim) + current_hessian = _zero_hessian(input_dim, input_dim) + for child, pre_activation in zip(model, layer_inputs): + local_jacobian = _jacobian_for_layer(child, pre_activation) + local_hessian = _hessian_for_layer(child, pre_activation) + current_jacobian, current_hessian = _hessian_compose( + local_jacobian, + local_hessian, + current_jacobian, + current_hessian, + ) + else: + local_jacobian = _jacobian_for_layer(model, domain) + local_hessian = _hessian_for_layer(model, domain) + current_hessian = local_hessian + + lower = tuple( + tuple(tuple(entry.lower for entry in row) for row in output_slice) + for output_slice in current_hessian + ) + upper = tuple( + tuple(tuple(entry.upper for entry in row) for row in output_slice) + for output_slice in current_hessian + ) + return IntervalTensor.from_bounds(lower, upper) + + def _sobolev_pointwise_power_bounds( model, box: IntervalTensor, p: float, + order: int = 1, output: IntervalTensor | None = None, jacobian: IntervalTensor | None = None, + hessian: IntervalTensor | None = None, ) -> Interval: + if order not in {1, 2}: + raise ValueError("order must be either 1 or 2.") output = output if output is not None else model.eval(box) jacobian = jacobian if jacobian is not None else model.eval_jacobian(box) + if order == 2: + hessian = hessian if hessian is not None else model.eval_hessian(box) total = Interval.point(0.0) for lower, upper in zip(output.lower, output.upper): @@ -841,6 +2471,13 @@ def _sobolev_pointwise_power_bounds( derivative_component = Interval(entry_lower, entry_upper) total = total + _interval_pow_scalar(_interval_abs_bounds(derivative_component), p) + if order == 2 and hessian is not None: + for out_slice_lower, out_slice_upper in zip(hessian.lower, hessian.upper): + for row_lower, row_upper in zip(out_slice_lower, out_slice_upper): + for entry_lower, entry_upper in zip(row_lower, row_upper): + second_derivative_component = Interval(entry_lower, entry_upper) + total = total + _interval_pow_scalar(_interval_abs_bounds(second_derivative_component), p) + return total @@ -863,25 +2500,137 @@ def _jacobian_is_exact_zero(jacobian: IntervalTensor) -> bool: ) +def _hessian_is_exact_zero(hessian: IntervalTensor) -> bool: + return all( + float(entry_lower) == 0.0 and float(entry_upper) == 0.0 + for out_slice_lower, out_slice_upper in zip(hessian.lower, hessian.upper) + for row_lower, row_upper in zip(out_slice_lower, out_slice_upper) + for entry_lower, entry_upper in zip(row_lower, row_upper) + ) + + def _sobolev_pointwise_power_bounds_refined( model, box: IntervalTensor, p: float, + order: int, forward_refine_splits: int, forward_refine_max_cells: int, ) -> Interval: if forward_refine_splits <= 1: - return _sobolev_pointwise_power_bounds(model, box, p) + return _sobolev_pointwise_power_bounds(model, box, p, order=order) cells = _subdivide_box(box, splits_per_dim=forward_refine_splits, max_cells=forward_refine_max_cells) - return _hull_intervals([_sobolev_pointwise_power_bounds(model, cell, p) for cell in cells]) + return _hull_intervals([_sobolev_pointwise_power_bounds(model, cell, p, order=order) for cell in cells]) + + +def _weighted_sobolev_squared_contribution( + model, + box: IntervalTensor, + *, + order: int = 1, + forward_refine_splits: int = 1, + forward_refine_max_cells: int = 256, +) -> Interval: + """Return the certified local integral enclosure used by interval AdaQuad. + + This private helper intentionally fixes ``p=2``: it is used by the shared + interval/PZ refinement benchmark to score candidate bisections in the same + squared-energy scale in which Dörfler marking is performed. + """ + + pointwise = _sobolev_pointwise_power_bounds_refined( + model, + box, + 2.0, + order, + forward_refine_splits=forward_refine_splits, + forward_refine_max_cells=forward_refine_max_cells, + ) + volume = _box_volume(box) + return Interval.from_bounds( + float(pointwise.lower) * volume, + float(pointwise.upper) * volume, + ) + + +def _lookahead_sobolev_split_dimension( + model, + box: IntervalTensor, + *, + order: int = 1, + forward_refine_splits: int = 1, + forward_refine_max_cells: int = 256, +) -> tuple[int, tuple[IntervalTensor, IntervalTensor], tuple[Interval, Interval], list[dict[str, float | int]]]: + """Choose a box edge by exhaustive certified one-step look-ahead. + + For every coordinate, bisect the box and compute the sum of the two child + contribution widths. The coordinate minimizing that sum is selected. + The candidate table also reports the reduction relative to the parent + width. These scores guide refinement only; every returned enclosure + remains certified independently of which coordinate wins. + """ + + if len(box.shape) != 1 or len(box.lower) == 0: + raise ValueError("Look-ahead edge selection requires a non-empty flat box.") + parent = _weighted_sobolev_squared_contribution( + model, + box, + order=order, + forward_refine_splits=forward_refine_splits, + forward_refine_max_cells=forward_refine_max_cells, + ) + parent_width = float(parent.upper) - float(parent.lower) + candidates: list[ + tuple[ + float, + int, + tuple[IntervalTensor, IntervalTensor], + tuple[Interval, Interval], + dict[str, float | int], + ] + ] = [] + for split_dim in range(len(box.lower)): + children = _split_box(box, split_dim=split_dim) + contributions = tuple( + _weighted_sobolev_squared_contribution( + model, + child, + order=order, + forward_refine_splits=forward_refine_splits, + forward_refine_max_cells=forward_refine_max_cells, + ) + for child in children + ) + child_width = sum( + float(contribution.upper) - float(contribution.lower) + for contribution in contributions + ) + row: dict[str, float | int] = { + "split_dim": split_dim, + "parent_width": parent_width, + "children_width_sum": child_width, + "predicted_width_reduction": parent_width - child_width, + "predicted_relative_reduction": ( + (parent_width - child_width) / parent_width + if parent_width > 0.0 + else 0.0 + ), + } + candidates.append((child_width, split_dim, children, contributions, row)) + + candidates.sort(key=lambda item: (item[0], item[1])) + _, split_dim, children, contributions, _ = candidates[0] + return split_dim, children, contributions, [item[4] for item in candidates] def _sobolev_norm_bounds( model, domain: IntervalTensor, p: float, + order: int, iterations: int, theta: float, + enclosure_mode: str = "slope", forward_refine_splits: int = 1, forward_refine_max_cells: int = 256, ) -> Interval: @@ -891,8 +2640,12 @@ def _sobolev_norm_bounds( raise NotImplementedError("Sobolev integration currently supports flat input boxes only.") if not isfinite(p) or p <= 0.0: raise ValueError("p must be a positive finite real number.") + if order not in {1, 2}: + raise ValueError("order must be either 1 or 2.") if iterations < 0: raise ValueError("iterations must be non-negative.") + if enclosure_mode not in {"box", "slope"}: + raise ValueError("enclosure_mode must be either 'box' or 'slope'.") _validate_dorfler_theta(theta) if forward_refine_splits < 1: raise ValueError("forward_refine_splits must be at least 1.") @@ -907,12 +2660,16 @@ def _sobolev_norm_bounds( model, box, p, + order, forward_refine_splits=forward_refine_splits, forward_refine_max_cells=forward_refine_max_cells, ) output = model.eval(box) jacobian = model.eval_jacobian(box) - if _interval_tensor_is_exact_constant(output) and _jacobian_is_exact_zero(jacobian): + hessian = model.eval_hessian(box) if order == 2 else None + derivative_zero = _jacobian_is_exact_zero(jacobian) + second_derivative_zero = True if hessian is None else _hessian_is_exact_zero(hessian) + if _interval_tensor_is_exact_constant(output) and derivative_zero and second_derivative_zero: # A rigorously constant box has zero Sobolev seminorm contribution, # so further refinement is unnecessary for the derivative part. indicators.append(0.0) @@ -940,6 +2697,7 @@ def _sobolev_norm_bounds( model, box, p, + order, forward_refine_splits=forward_refine_splits, forward_refine_max_cells=forward_refine_max_cells, ) @@ -954,7 +2712,7 @@ def _sobolev_norm_bounds( return _interval_pow_scalar(non_negative, exponent) -def interval_forward(module, x: IntervalTensor, enclosure_mode: str = "box") -> IntervalTensor: +def _interval_forward(module, x: IntervalTensor, enclosure_mode: str = "box") -> IntervalTensor: _require_torch() if enclosure_mode not in {"box", "slope"}: raise ValueError("enclosure_mode must be either 'box' or 'slope'.") @@ -963,7 +2721,7 @@ def interval_forward(module, x: IntervalTensor, enclosure_mode: str = "box") -> return _sequential_linear_relu_relaxation(module, x) result = x for child in module: - result = interval_forward(child, result, enclosure_mode=enclosure_mode) + result = _interval_forward(child, result, enclosure_mode=enclosure_mode) return result if isinstance(module, nn.Flatten): return IntervalTensor(tuple(x.lower), tuple(x.upper)) @@ -984,17 +2742,27 @@ def interval_forward(module, x: IntervalTensor, enclosure_mode: str = "box") -> if isinstance(module, nn.Identity): return IntervalTensor(tuple(x.lower), tuple(x.upper)) if isinstance(module, IntervalAdd): - left = interval_forward(module.left, x, enclosure_mode=enclosure_mode) - right = interval_forward(module.right, x, enclosure_mode=enclosure_mode) + left = _interval_forward(module.left, x, enclosure_mode=enclosure_mode) + right = _interval_forward(module.right, x, enclosure_mode=enclosure_mode) return _interval_add(left, right) if isinstance(module, IntervalCat): - parts = [interval_forward(branch, x, enclosure_mode=enclosure_mode) for branch in module.branches] + parts = [_interval_forward(branch, x, enclosure_mode=enclosure_mode) for branch in module.branches] return _interval_cat(parts, module.dim) raise NotImplementedError( f"Interval forward currently supports nn.Sequential, nn.Flatten, nn.Linear, nn.ReLU, nn.Sigmoid, nn.Tanh, nn.Softplus, nn.LeakyReLU, nn.Softmax, nn.Identity, IntervalAdd, and IntervalCat only; got {type(module).__name__}." ) +def interval_forward( + module, + x: IntervalTensor, + enclosure_mode: str = "box", +) -> IntervalTensor: + if isinstance(x, IntervalTensor): + return _interval_forward(module, x, enclosure_mode=enclosure_mode) + raise TypeError("interval_forward(module, x) requires x to be an IntervalTensor.") + + def interval_forward_refine( module, x: IntervalTensor, @@ -1009,6 +2777,8 @@ def interval_forward_refine( `interval_forward(...)` once on the full input box. """ _require_torch() + if not isinstance(x, IntervalTensor): + raise TypeError("interval_forward_refine(module, x, ...) requires x to be an IntervalTensor.") if len(x.shape) != 1: raise NotImplementedError("interval_forward_refine currently supports flat vectors only.") if splits_per_dim < 1: @@ -1019,7 +2789,7 @@ def interval_forward_refine( hull_upper: tuple[float, ...] | None = None for cell in cells: - cell_out = interval_forward(module, cell, enclosure_mode=enclosure_mode) + cell_out = _interval_forward(module, cell, enclosure_mode=enclosure_mode) lower = tuple(float(v) for v in cell_out.lower) upper = tuple(float(v) for v in cell_out.upper) @@ -1037,13 +2807,15 @@ def interval_forward_refine( _ORIGINAL_EVAL = getattr(nn.Module, "eval", None) if nn is not None else None _PATCHED = False +_ACTIVE_ENCLOSURE_MODE = "slope" def enable_interval_eval(enclosure_mode: str = "slope") -> None: _require_torch() - global _PATCHED + global _PATCHED, _ACTIVE_ENCLOSURE_MODE if enclosure_mode not in {"box", "slope"}: raise ValueError("enclosure_mode must be either 'box' or 'slope'.") + _ACTIVE_ENCLOSURE_MODE = enclosure_mode if _PATCHED: return @@ -1053,7 +2825,7 @@ def eval_with_interval(self, interval: IntervalTensor | None = None): return result if not isinstance(interval, IntervalTensor): raise TypeError("model.eval(interval) requires an IntervalTensor input.") - return interval_forward(self, interval, enclosure_mode=enclosure_mode) + return interval_forward(self, interval, enclosure_mode=_ACTIVE_ENCLOSURE_MODE) def lpnorm_with_interval( self, @@ -1063,38 +2835,218 @@ def lpnorm_with_interval( theta: float = 0.5, forward_refine_splits: int = 1, forward_refine_max_cells: int = 256, + method: str = "interval", + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", ): _ORIGINAL_EVAL(self) + if method == "pz": + return pz_l2norm( + self, + domain, + p=p, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) + if method != "interval": + raise ValueError("method must be either 'interval' or 'pz'.") return _lpnorm_bounds( self, domain, p, iterations, theta, + enclosure_mode=_ACTIVE_ENCLOSURE_MODE, forward_refine_splits=forward_refine_splits, forward_refine_max_cells=forward_refine_max_cells, ) def eval_jacobian_with_interval(self, domain: IntervalTensor): _ORIGINAL_EVAL(self) - return _eval_jacobian_bounds(self, domain, enclosure_mode=enclosure_mode) + return _eval_jacobian_bounds(self, domain, enclosure_mode=_ACTIVE_ENCLOSURE_MODE) + + def eval_hessian_with_interval(self, domain: IntervalTensor): + _ORIGINAL_EVAL(self) + return _eval_hessian_bounds(self, domain, enclosure_mode=_ACTIVE_ENCLOSURE_MODE) + + def eval_pz_twojet_with_interval( + self, + domain: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + return_trace: bool = False, + ): + _ORIGINAL_EVAL(self) + if not isinstance(domain, PolynomialZonotope): + raise TypeError("model.eval_pz_twojet(domain) requires a PolynomialZonotope input.") + return pz_twojet_forward( + self, + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=return_trace, + ) + + def eval_pz_value_with_interval( + self, + domain: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = False, + return_trace: bool = False, + ): + _ORIGINAL_EVAL(self) + if not isinstance(domain, PolynomialZonotope): + raise TypeError( + "model.eval_pz_value(domain) requires a PolynomialZonotope input." + ) + return pz_value_forward( + self, + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + return_trace=return_trace, + ) + + def eval_pz_onejet_with_interval( + self, + domain: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduce: bool = True, + reduction_strategy: str = "topk", + max_terms: int = 96, + max_degree: int = 4, + pca_rank: int = 4, + pca_candidates: int = 48, + reduction_variant: str = "A", + generator_budget: int = 0, + derivative_enclosure: str = "affine", + derivative_flatness_threshold: float = 0.01, + quadratic_certificate_subdivisions: int = 64, + quadratic_compression_guard: bool = True, + return_trace: bool = False, + ): + _ORIGINAL_EVAL(self) + if not isinstance(domain, PolynomialZonotope): + raise TypeError( + "model.eval_pz_onejet(domain) requires a PolynomialZonotope input." + ) + return pz_onejet_forward( + self, + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduce=reduce, + reduction_strategy=reduction_strategy, + max_terms=max_terms, + max_degree=max_degree, + pca_rank=pca_rank, + pca_candidates=pca_candidates, + reduction_variant=reduction_variant, + generator_budget=generator_budget, + derivative_enclosure=derivative_enclosure, + derivative_flatness_threshold=derivative_flatness_threshold, + quadratic_certificate_subdivisions=quadratic_certificate_subdivisions, + quadratic_compression_guard=quadratic_compression_guard, + return_trace=return_trace, + ) + + def pz_l2norm_with_interval( + self, + domain: IntervalTensor, + p: float = 2.0, + *, + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", + ): + _ORIGINAL_EVAL(self) + return pz_l2norm( + self, + domain, + p=p, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) + + def pz_sobolev_norm_with_interval( + self, + domain: IntervalTensor, + p: float = 2.0, + order: int = 1, + *, + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", + ): + _ORIGINAL_EVAL(self) + return pz_sobolev_norm( + self, + domain, + p=p, + order=order, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) def sobolev_norm_with_interval( self, domain: IntervalTensor, p: float, + order: int = 1, iterations: int = 0, theta: float = 0.5, forward_refine_splits: int = 1, forward_refine_max_cells: int = 256, + method: str = "interval", + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: str = "interval", ): _ORIGINAL_EVAL(self) + if method == "pz": + return pz_sobolev_norm( + self, + domain, + p=p, + order=order, + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + output=output, + ) + if method != "interval": + raise ValueError("method must be either 'interval' or 'pz'.") return _sobolev_norm_bounds( self, domain, p, + order, iterations, theta, + enclosure_mode=_ACTIVE_ENCLOSURE_MODE, forward_refine_splits=forward_refine_splits, forward_refine_max_cells=forward_refine_max_cells, ) @@ -1102,5 +3054,11 @@ def sobolev_norm_with_interval( nn.Module.eval = eval_with_interval nn.Module.lpnorm = lpnorm_with_interval nn.Module.eval_jacobian = eval_jacobian_with_interval + nn.Module.eval_hessian = eval_hessian_with_interval + nn.Module.eval_pz_value = eval_pz_value_with_interval + nn.Module.eval_pz_onejet = eval_pz_onejet_with_interval + nn.Module.eval_pz_twojet = eval_pz_twojet_with_interval + nn.Module.pz_l2norm = pz_l2norm_with_interval + nn.Module.pz_sobolev_norm = pz_sobolev_norm_with_interval nn.Module.sobolev_norm = sobolev_norm_with_interval _PATCHED = True diff --git a/src/intervalnets/pz_integration.py b/src/intervalnets/pz_integration.py new file mode 100644 index 0000000..bdeec89 --- /dev/null +++ b/src/intervalnets/pz_integration.py @@ -0,0 +1,1487 @@ +"""Integration helpers for polynomial zonotopes. + +The helpers in this module keep the exact polynomial contribution separate from +interval uncertainty created by pointwise approximation residuals. Pointwise +residual symbols model a fresh adversarial value at each integration point, so +integrating them by monomial moments would be unsound unless the caller opts +into a purely symbolic treatment explicitly. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from itertools import product +from math import inf, isfinite, nextafter, prod, sqrt +from numbers import Real +from operator import index as integer_index +from typing import Any, Literal, Sequence + +from .interval import Interval +from .polynomial_zonotope import ( + Exponent, + PZOneJet, + PZTwoJet, + PolynomialZonotope, + _abs_coeff, + _add_coeff, + _mul_coeff, + _merge_noise_kinds, + _to_fallback, + _zero_like, + box_monomial_moment, + torch, +) + +try: # pragma: no cover - optional acceleration + import numpy as np +except ImportError: # pragma: no cover + np = None # type: ignore[assignment] + +try: # pragma: no cover - optional dependency + from .pytorch import IntervalTensor +except ImportError: # pragma: no cover + IntervalTensor = None # type: ignore[assignment] + +POINTWISE_RESIDUAL_KINDS = frozenset({"pointwise_residual", "approximation_pointwise"}) +SYMBOLIC_APPROXIMATION_KINDS = frozenset({"approximation_symbolic", "global_symbolic_residual"}) + +IntegrationMode = Literal["pointwise_interval", "symbolic"] +IntegrationOutput = Literal["interval", "pz"] + + +@dataclass(frozen=True) +class PZIntegrationCell: + """A parameterized cell used for geometric polynomial-zonotope integration. + + ``domain`` maps reference variables in ``[-1, 1]^n`` to physical + coordinates. ``domain_noise_indices`` identifies exactly those reference + variables. ``jacobian_density`` is the non-negative density multiplying the + reference integral. For affine axis-aligned boxes this is simply the product + of coordinate radii. + """ + + domain: PolynomialZonotope + domain_noise_indices: tuple[int, ...] + jacobian_density: PolynomialZonotope | float + volume: float | Interval + orientation: int | None + source_box: "IntervalTensor | None" + + @classmethod + def from_affine_box(cls, box: Interval | "IntervalTensor") -> "PZIntegrationCell": + """Create the affine cell mapping ``[-1, 1]^n`` to an interval box.""" + + domain = PolynomialZonotope.from_box(box.lower, box.upper) + radii = tuple(_flatten_scalars(box.radius)) + density = float(prod(radii)) + dim = len(radii) + return cls( + domain=domain.with_noise_kinds(("domain",) * domain.num_noise), + domain_noise_indices=tuple(range(dim)), + jacobian_density=density, + volume=float((2.0**dim) * density), + orientation=1 if density >= 0.0 else -1, + source_box=box if IntervalTensor is not None and isinstance(box, IntervalTensor) else None, + ) + + @classmethod + def from_bounds(cls, lower: Any, upper: Any) -> "PZIntegrationCell": + """Create an affine integration cell from lower and upper box bounds.""" + + if IntervalTensor is not None: + return cls.from_affine_box(IntervalTensor.from_bounds(lower, upper)) + return cls.from_affine_box(Interval.from_bounds(lower, upper)) + + @classmethod + def from_fixed_orientation_domain( + cls, + domain: PolynomialZonotope, + domain_noise_indices: Sequence[int], + *, + determinant: PolynomialZonotope | None = None, + orientation: int | None = None, + injectivity_certificate: Any | None = None, + source_box: "IntervalTensor | None" = None, + ) -> "PZIntegrationCell": + """Restricted hook for future non-affine certified PZ domains. + + Non-affine changes of variables require a certified determinant + polynomial plus injectivity and fixed-orientation certificates. Until the + certificate objects are defined by the implementation, this constructor + intentionally refuses uncertified domains. + """ + + indices = tuple(int(index) for index in domain_noise_indices) + if determinant is None or orientation not in (-1, 1) or injectivity_certificate is None: + raise NotImplementedError( + "Non-affine PZ integration requires a certified determinant polynomial " + "and injectivity/fixed-orientation certificates." + ) + if len(set(indices)) != len(indices): + raise ValueError("domain_noise_indices must not contain duplicates.") + if any(index < 0 or index >= domain.num_noise for index in indices): + raise ValueError("domain noise index out of range.") + + density = determinant * float(orientation) + volume = integrate_pz_over_domain(density, indices, mode="pointwise_interval").interval_enclosure() + return cls( + domain=domain, + domain_noise_indices=indices, + jacobian_density=density, + volume=volume, + orientation=orientation, + source_box=source_box, + ) + + +@dataclass(frozen=True) +class IntegratedPZResult: + """Result of integrating a polynomial zonotope over selected variables.""" + + polynomial: PolynomialZonotope + interval_radius: Any + measure: float + metadata: dict[str, Any] + + def interval_enclosure(self): + """Return an interval enclosure of ``polynomial +/- interval_radius``.""" + + base = self.polynomial.interval_enclosure() + lower_radius = _mul_coeff(self.interval_radius, -1.0) + if IntervalTensor is not None and base.__class__ is IntervalTensor: + return base + IntervalTensor.from_bounds(lower_radius, self.interval_radius) + return base + Interval.from_bounds(lower_radius, self.interval_radius) + + def as_polynomial_zonotope(self) -> PolynomialZonotope: + """Return the integral as one scalar/tensor polynomial zonotope. + + Pointwise residuals cannot be integrated as fixed symbolic values. + Their already integrated radius is therefore represented by one fresh + global residual symbol per output entry. This keeps the result in PZ + form until the caller explicitly requests an interval enclosure. + """ + + return self.polynomial.add_independent_errors( + self.interval_radius, + kind="global_symbolic_residual", + ) + + +def _flatten_scalars(value: Any): + data = _to_fallback(value) + if isinstance(data, tuple): + for item in data: + yield from _flatten_scalars(item) + else: + yield float(data) + + +def _default_domain_indices(zonotope: PolynomialZonotope) -> tuple[int, ...]: + return tuple(index for index, kind in enumerate(zonotope.noise_kinds) if kind == "domain") + + +def _is_pointwise_kind(kind: str) -> bool: + return kind in POINTWISE_RESIDUAL_KINDS + + +def box_monomial_absolute_moment(exponents: Sequence[int]) -> float: + """Exact integral of ``abs(alpha**exponents)`` over ``[-1, 1]^d``. + + Unlike :func:`box_monomial_moment`, odd powers do not cancel. Every + exponent must be a non-negative integer. + """ + + normalized: list[int] = [] + for exponent in exponents: + try: + power = integer_index(exponent) + except TypeError as error: + raise ValueError("monomial exponents must be non-negative integers.") from error + if power < 0: + raise ValueError("monomial exponents must be non-negative integers.") + normalized.append(power) + return prod(2.0 / (power + 1.0) for power in normalized) + + +def _pointwise_term_midpoint_radius( + exponent: Exponent, + coefficient: Any, + domain_indices: Sequence[int], + non_domain_indices: Sequence[int], + *, + density: float = 1.0, +) -> tuple[Any, Any]: + """Integrate one canonical pointwise term by absolute moments and parity.""" + + domain_exponent = tuple(exponent[index] for index in domain_indices) + absolute_moment = density * box_monomial_absolute_moment(domain_exponent) + all_uncertainty_even = all(exponent[index] % 2 == 0 for index in non_domain_indices) + if not all_uncertainty_even: + return _zero_like(coefficient), _mul_coeff(_abs_coeff(coefficient), absolute_moment) + + half_moment = 0.5 * absolute_moment + if all(power % 2 == 0 for power in domain_exponent): + return ( + _mul_coeff(coefficient, half_moment), + _mul_coeff(_abs_coeff(coefficient), half_moment), + ) + return _zero_like(coefficient), _mul_coeff(_abs_coeff(coefficient), half_moment) + + +def integrate_pz_over_domain( + zonotope: PolynomialZonotope, + domain_indices: Sequence[int] | None = None, + *, + mode: IntegrationMode = "pointwise_interval", + volume: float | None = None, +) -> IntegratedPZResult: + """Integrate domain variables while preserving pointwise residual semantics. + + ``volume`` is the physical measure represented by the reference box. If + supplied, every reference-box moment is multiplied by the one constant + density ``volume / 2**len(domain_indices)`` exactly once. + """ + + if mode not in ("pointwise_interval", "symbolic"): + raise ValueError("mode must be 'pointwise_interval' or 'symbolic'.") + indices = _default_domain_indices(zonotope) if domain_indices is None else tuple(int(i) for i in domain_indices) + if len(set(indices)) != len(indices): + raise ValueError("domain_indices must not contain duplicates.") + if any(index < 0 or index >= zonotope.num_noise for index in indices): + raise ValueError("domain index out of range.") + + domain_set = set(indices) + retained_indices = tuple(index for index in range(zonotope.num_noise) if index not in domain_set) + retained_kinds = tuple(zonotope.noise_kinds[index] for index in retained_indices) + reference_measure = float(2 ** len(indices)) + measure = float(volume) if volume is not None else reference_measure + if measure < 0.0: + raise ValueError("volume/measure must be non-negative.") + density = measure / reference_measure + + center = _mul_coeff(zonotope.center, measure) + terms: dict[Exponent, Any] = {} + radius = _zero_like(zonotope.center) + pointwise_indices = tuple(index for index, kind in enumerate(zonotope.noise_kinds) if _is_pointwise_kind(kind)) + zero_retained = (0,) * len(retained_indices) + + for exponent, coeff in zonotope.terms.items(): + has_pointwise = any(exponent[index] for index in pointwise_indices) + if mode == "pointwise_interval" and has_pointwise: + midpoint, term_radius = _pointwise_term_midpoint_radius( + exponent, + coeff, + indices, + retained_indices, + density=density, + ) + center = _add_coeff(center, midpoint) + radius = _add_coeff(radius, term_radius) + continue + + moment = box_monomial_moment(tuple(exponent[index] for index in indices)) + if moment == 0.0: + continue + retained_exponent = tuple(exponent[index] for index in retained_indices) + integrated_coeff = _mul_coeff(coeff, density * moment) + if retained_exponent == zero_retained: + center = _add_coeff(center, integrated_coeff) + else: + terms[retained_exponent] = _add_coeff(terms[retained_exponent], integrated_coeff) if retained_exponent in terms else integrated_coeff + + return IntegratedPZResult( + polynomial=PolynomialZonotope(center, terms, num_noise=len(retained_indices), noise_kinds=retained_kinds), + interval_radius=radius, + measure=measure, + metadata={ + "mode": mode, + "domain_indices": indices, + "reference_measure": reference_measure, + "constant_density": density, + "pointwise_residual_indices": pointwise_indices, + "pointwise_residual_kinds": tuple(zonotope.noise_kinds[index] for index in pointwise_indices), + }, + ) + + +def integrate_over_cell(pz_expr: PolynomialZonotope, cell: PZIntegrationCell, *, output: IntegrationOutput = "interval"): + """Integrate a PZ expression geometrically over an integration cell. + + ``output="pz"`` integrates only the cell domain variables and retains all + non-domain approximation variables symbolically. ``output="interval"`` + first performs pointwise-residual-safe domain integration, then encloses the + retained symbolic approximation-noise polynomial and adds accumulated + pointwise residual radii. + """ + + if output not in ("interval", "pz"): + raise ValueError("output must be 'interval' or 'pz'.") + weighted = pz_expr * cell.jacobian_density + if output == "pz": + return integrate_pz_over_domain(weighted, cell.domain_noise_indices, mode="symbolic").polynomial + return integrate_pz_over_domain(weighted, cell.domain_noise_indices, mode="pointwise_interval").interval_enclosure() + + +TwoJetIntegrandKind = Literal["l2", "w12", "w22"] + + +def _scalar_coordinates(zonotope: PolynomialZonotope) -> list[PolynomialZonotope]: + """Flatten a tensor-valued PZ without converting its scalar coefficients.""" + + if zonotope.shape == (): + return [zonotope] + return [zonotope[index] for index in product(*(range(size) for size in zonotope.shape))] + + +def _twojet_weighted_coordinates( + jet: PZTwoJet, integrand_kind: TwoJetIntegrandKind +) -> tuple[list[PolynomialZonotope], list[float]]: + if integrand_kind not in {"l2", "w12", "w22"}: + raise ValueError("integrand_kind must be 'l2', 'w12', or 'w22'.") + + coordinates = _scalar_coordinates(jet.Y) + weights = [1.0] * len(coordinates) + if integrand_kind in {"w12", "w22"}: + jacobian = _scalar_coordinates(jet.J) + coordinates.extend(jacobian) + weights.extend([1.0] * len(jacobian)) + if integrand_kind == "w22": + shape = jet.H.shape + if len(shape) == 3 and shape[1] == shape[2]: + output_indices: tuple[int | None, ...] = tuple(range(shape[0])) + input_dim = shape[1] + elif len(shape) == 2 and shape[0] == shape[1]: + output_indices = (None,) + input_dim = shape[0] + else: + raise ValueError("w22 direct integration requires a square stored Hessian.") + for output in output_indices: + for row in range(input_dim): + for column in range(row, input_dim): + index = (row, column) if output is None else (output, row, column) + coordinates.append(jet.H[index]) + weights.append(1.0 if row == column else 2.0) + return coordinates, weights + + +def _validate_twojet_metadata(jet: PZTwoJet) -> tuple[int, tuple[str, ...]]: + components = (jet.Y, jet.J, jet.H) + num_noise = components[0].num_noise + if any(component.num_noise != num_noise for component in components[1:]): + raise ValueError("Y, J, and H must have identical num_noise metadata.") + noise_kinds = components[0].noise_kinds + for component in components[1:]: + noise_kinds = _merge_noise_kinds(noise_kinds, component.noise_kinds) + return num_noise, noise_kinds + + +def _coefficient_matrix( + coordinates: Sequence[PolynomialZonotope], union_support: Sequence[Exponent] +) -> tuple[Any, Any]: + centers = [coordinate.center for coordinate in coordinates] + matrix = [ + [coordinate.terms.get(exponent, _zero_like(coordinate.center)) for coordinate in coordinates] + for exponent in union_support + ] + if torch is not None and isinstance(centers[0], torch.Tensor): + center_vector = torch.stack(centers) + if matrix: + return center_vector, torch.stack([torch.stack(row) for row in matrix]) + return center_vector, torch.empty((0, len(centers)), dtype=center_vector.dtype, device=center_vector.device) + return centers, matrix + + +def _weighted_dot(left: Sequence[Any], right: Sequence[Any], weights: Sequence[float]): + result = _zero_like(left[0]) + for lhs, rhs, weight in zip(left, right, weights): + result = _add_coeff(result, _mul_coeff(_mul_coeff(lhs, rhs), weight)) + return result + + +def _is_real_scalar_coefficient(value: Any) -> bool: + if isinstance(value, Real): + return True + return bool( + torch is not None + and isinstance(value, torch.Tensor) + and value.numel() == 1 + and not value.is_complex() + ) + + +def _real_scalar_value(value: Any) -> float: + if torch is not None and isinstance(value, torch.Tensor): + return float(value.detach().item()) + return float(value) + + +def _all_real_scalar_coefficients(centers: Sequence[Any], matrix: Sequence[Sequence[Any]]) -> bool: + """Return whether a possibly mixed coefficient table can use NumPy.""" + + return all(_is_real_scalar_coefficient(value) for value in centers) and all( + _is_real_scalar_coefficient(value) for row in matrix for value in row + ) + + +def _integrate_numpy_twojet_square( + *, + support: Sequence[Exponent], + num_noise: int, + center_square: float, + center_cross: Any, + gram: Any, + scale: float, + measure: float, + domain_indices: tuple[int, ...], + retained_indices: tuple[int, ...], + pointwise_indices: tuple[int, ...], + retained_kinds: tuple[str, ...], + output: IntegrationOutput, +): + """Canonicalize and integrate the dense-real contraction in vectorized batches.""" + + support_size = len(support) + support_array = np.asarray(support, dtype=np.int64).reshape(support_size, num_noise) + + # The affine activation-enclosure pipeline keeps the value PZ affine: + # every support exponent is one distinct unit vector. In this important + # case, all canonical squared terms are known a priori and their box + # moments can be contracted directly. Avoid constructing and sorting the + # O(m^2 * num_noise) pair-exponent table, which is especially expensive for + # 100-dimensional inputs with many approximation symbols. + affine_support = bool( + support_size + and num_noise + and np.all((support_array == 0) | (support_array == 1)) + and np.all(support_array.sum(axis=1) == 1) + ) + if affine_support: + support_noise_indices = np.argmax(support_array, axis=1) + if len(np.unique(support_noise_indices)) == support_size: + domain_lookup = np.zeros(num_noise, dtype=bool) + domain_lookup[np.asarray(domain_indices, dtype=np.int64)] = True + pointwise_lookup = np.zeros(num_noise, dtype=bool) + pointwise_lookup[np.asarray(pointwise_indices, dtype=np.int64)] = True + + support_domain = domain_lookup[support_noise_indices] + support_pointwise = pointwise_lookup[support_noise_indices] + support_symbolic = ~(support_domain | support_pointwise) + + diagonal = np.diag(gram) + integrated_center = float( + scale + * measure + * ( + center_square + + diagonal[support_domain].sum() / 3.0 + + 0.5 * diagonal[support_pointwise].sum() + ) + ) + + pointwise_radius = float( + scale + * measure + * ( + np.abs(2.0 * np.asarray(center_cross)[support_pointwise]).sum() + + 0.5 * np.abs(diagonal[support_pointwise]).sum() + ) + ) + symbolic_radius_unscaled = float( + np.abs(2.0 * np.asarray(center_cross)[support_symbolic]).sum() + + np.abs(diagonal[support_symbolic]).sum() + ) + + row_indices, column_indices = np.triu_indices(support_size, k=1) + off_diagonal = 2.0 * gram[row_indices, column_indices] + pair_pointwise = ( + support_pointwise[row_indices] + | support_pointwise[column_indices] + ) + pair_symbolic = ( + support_symbolic[row_indices] + & support_symbolic[column_indices] + ) + pair_has_domain = ( + support_domain[row_indices] + | support_domain[column_indices] + ) + pointwise_pair_moments = np.where( + pair_has_domain[pair_pointwise], + 0.5 * measure, + measure, + ) + pointwise_radius += float( + scale + * np.sum( + np.abs(off_diagonal[pair_pointwise]) + * pointwise_pair_moments + ) + ) + symbolic_radius_unscaled += float( + np.abs(off_diagonal[pair_symbolic]).sum() + ) + + symbolic_radius = float( + scale * measure * symbolic_radius_unscaled + ) + if output == "pz" and not np.any(support_symbolic): + result = IntegratedPZResult( + polynomial=PolynomialZonotope( + integrated_center, + {}, + num_noise=len(retained_indices), + noise_kinds=retained_kinds, + ), + interval_radius=pointwise_radius, + measure=measure, + metadata={ + "mode": "pointwise_interval", + "direct_twojet_squared": True, + "affine_support": True, + }, + ) + return result.as_polynomial_zonotope() + if output == "pz": + # Retained symbolic affine generators require their exact + # exponents, so use the generic canonicalizing path below. + pass + else: + base = Interval.from_bounds( + nextafter(integrated_center - symbolic_radius, -inf), + nextafter(integrated_center + symbolic_radius, inf), + ) + return base + Interval.from_bounds( + -pointwise_radius, + pointwise_radius, + ) + + row_indices, column_indices = np.triu_indices(support_size) + pair_coefficients = gram[row_indices, column_indices].copy() + pair_coefficients[row_indices != column_indices] *= 2.0 + + all_coefficients = np.concatenate( + ( + np.asarray((center_square,), dtype=float), + 2.0 * np.asarray(center_cross, dtype=float), + pair_coefficients, + ) + ) + + # Canonicalize before any absolute value, exactly as required for + # pointwise approximation residuals. Encode exponent rows as collision-free + # mixed-radix int64 keys whenever possible. Pair exponents then correspond + # exactly to adding their keys, and NumPy sorts eight-byte integers instead + # of repeatedly comparing full exponent rows. + maximum_pair_exponents = ( + 2 * support_array.max(axis=0) + if support_size + else np.zeros(num_noise, dtype=np.int64) + ) + bases = maximum_pair_exponents + 1 + strides_list: list[int] = [] + capacity = 1 + for base in bases: + strides_list.append(capacity) + capacity *= int(base) + if capacity <= np.iinfo(np.int64).max: + strides = np.asarray(strides_list, dtype=np.int64) + support_codes = support_array @ strides + pair_codes = support_codes[row_indices] + support_codes[column_indices] + all_codes = np.concatenate( + ( + np.zeros(1, dtype=np.int64), + support_codes, + pair_codes, + ) + ) + canonical_codes, inverse = np.unique(all_codes, return_inverse=True) + canonical_exponents = ( + (canonical_codes[:, np.newaxis] // strides[np.newaxis, :]) + % bases[np.newaxis, :] + ) + else: + # Extremely wide supports may exceed a signed 64-bit mixed-radix key. + # Keep an exact row-based path for those cases. + pair_exponents = support_array[row_indices] + support_array[column_indices] + all_exponents = np.concatenate( + ( + np.zeros((1, num_noise), dtype=np.int64), + support_array, + pair_exponents, + ), + axis=0, + ) + canonical_exponents, inverse = np.unique( + all_exponents, + axis=0, + return_inverse=True, + ) + canonical_coefficients = np.bincount( + inverse, + weights=all_coefficients, + minlength=len(canonical_exponents), + ) + + if pointwise_indices: + pointwise_mask = np.any(canonical_exponents[:, pointwise_indices] != 0, axis=1) + else: + pointwise_mask = np.zeros(len(canonical_exponents), dtype=bool) + pointwise_exponents = canonical_exponents[pointwise_mask] + pointwise_coefficients = canonical_coefficients[pointwise_mask] + if len(pointwise_coefficients): + if domain_indices: + pointwise_domain_exponents = pointwise_exponents[:, domain_indices] + pointwise_absolute_moments = np.prod( + 2.0 / (pointwise_domain_exponents + 1.0), axis=1 + ) + pointwise_domain_even = np.all( + pointwise_domain_exponents % 2 == 0, axis=1 + ) + else: + pointwise_absolute_moments = np.ones(len(pointwise_coefficients), dtype=float) + pointwise_domain_even = np.ones(len(pointwise_coefficients), dtype=bool) + if retained_indices: + pointwise_uncertainty_even = np.all( + pointwise_exponents[:, retained_indices] % 2 == 0, axis=1 + ) + else: + pointwise_uncertainty_even = np.ones(len(pointwise_coefficients), dtype=bool) + pointwise_half_mask = pointwise_uncertainty_even + pointwise_one_sided = pointwise_uncertainty_even & pointwise_domain_even + radius_factors = np.where(pointwise_half_mask, 0.5, 1.0) + radius = float( + scale + * np.sum( + np.abs(pointwise_coefficients) + * pointwise_absolute_moments + * radius_factors + ) + ) + pointwise_center_shift = float( + 0.5 + * scale + * np.sum( + pointwise_coefficients[pointwise_one_sided] + * pointwise_absolute_moments[pointwise_one_sided] + ) + ) + else: + radius = 0.0 + pointwise_center_shift = 0.0 + + exact_exponents = canonical_exponents[~pointwise_mask] + exact_coefficients = canonical_coefficients[~pointwise_mask] + if domain_indices: + domain_exponents = exact_exponents[:, domain_indices] + even_mask = np.all(domain_exponents % 2 == 0, axis=1) + exact_exponents = exact_exponents[even_mask] + exact_coefficients = exact_coefficients[even_mask] + domain_exponents = domain_exponents[even_mask] + moments = np.prod(2.0 / (domain_exponents + 1.0), axis=1) + else: + moments = np.ones(len(exact_coefficients), dtype=float) + integrated_coefficients = scale * moments * exact_coefficients + + canonical_retained = np.empty((0, len(retained_indices)), dtype=np.int64) + if retained_indices: + retained_exponents = exact_exponents[:, retained_indices] + retained_bases = bases[np.asarray(retained_indices)] + retained_strides_list: list[int] = [] + retained_capacity = 1 + for base in retained_bases: + retained_strides_list.append(retained_capacity) + retained_capacity *= int(base) + encoded_retained = retained_capacity <= np.iinfo(np.int64).max + if encoded_retained: + retained_strides = np.asarray(retained_strides_list, dtype=np.int64) + retained_codes = retained_exponents @ retained_strides + canonical_retained_codes, retained_inverse = np.unique( + retained_codes, + return_inverse=True, + ) + retained_group_count = len(canonical_retained_codes) + canonical_retained = ( + (canonical_retained_codes[:, np.newaxis] // retained_strides[np.newaxis, :]) + % retained_bases[np.newaxis, :] + ) + else: + canonical_retained, retained_inverse = np.unique( + retained_exponents, + axis=0, + return_inverse=True, + ) + retained_group_count = len(canonical_retained) + canonical_integrated = np.bincount( + retained_inverse, + weights=integrated_coefficients, + minlength=retained_group_count, + ) + zero_mask = ( + canonical_retained_codes == 0 + if encoded_retained + else np.all(canonical_retained == 0, axis=1) + ) + center = float(canonical_integrated[zero_mask].sum()) + pointwise_center_shift + symbolic_radius = float(np.abs(canonical_integrated[~zero_mask]).sum()) + else: + center = float(integrated_coefficients.sum()) + pointwise_center_shift + symbolic_radius = 0.0 + canonical_retained = np.zeros((1, 0), dtype=np.int64) + canonical_integrated = np.asarray((center,), dtype=float) + + if output == "pz": + terms = { + tuple(int(power) for power in exponent): float(coefficient) + for exponent, coefficient in zip(canonical_retained, canonical_integrated) + if np.any(exponent != 0) and coefficient != 0.0 + } + result = IntegratedPZResult( + polynomial=PolynomialZonotope( + center, + terms, + num_noise=len(retained_indices), + noise_kinds=retained_kinds, + ), + interval_radius=radius, + measure=measure, + metadata={"mode": "pointwise_interval", "direct_twojet_squared": True}, + ) + return result.as_polynomial_zonotope() + + # Match ``IntegratedPZResult.interval_enclosure`` without materializing a + # retained PolynomialZonotope containing tens of thousands of terms. + base = Interval.from_bounds( + nextafter(center - symbolic_radius, -inf), + nextafter(center + symbolic_radius, inf), + ) + return base + Interval.from_bounds(-radius, radius) + + +def integrate_pz_twojet_squared( + jet: PZTwoJet, + cell: PZIntegrationCell, + integrand_kind: TwoJetIntegrandKind, + *, + output: IntegrationOutput = "interval", +): + """Directly integrate a squared two-jet over a supported affine cell. + + Unsupported densities and Hessian layouts deliberately use the explicit + squared-integrand reference pipeline. + """ + + if output not in ("interval", "pz"): + raise ValueError("output must be 'interval' or 'pz'.") + + def explicit_fallback(): + from .pz_norms import ( + pz_twojet_l2_integrand, + pz_twojet_w12_integrand, + pz_twojet_w22_integrand, + ) + + constructor = { + "l2": pz_twojet_l2_integrand, + "w12": pz_twojet_w12_integrand, + "w22": pz_twojet_w22_integrand, + }.get(integrand_kind) + if constructor is None: + raise ValueError("integrand_kind must be 'l2', 'w12', or 'w22'.") + weighted = constructor(jet) * cell.jacobian_density + result = integrate_pz_over_domain( + weighted, + cell.domain_noise_indices, + mode="pointwise_interval", + ) + return result.as_polynomial_zonotope() if output == "pz" else result.interval_enclosure() + + density = cell.jacobian_density + if not isinstance(density, Real) or not isfinite(float(density)) or float(density) < 0.0: + return explicit_fallback() + + num_noise, noise_kinds = _validate_twojet_metadata(jet) + try: + coordinates, weights = _twojet_weighted_coordinates(jet, integrand_kind) + except ValueError as error: + if "Hessian" in str(error): + return explicit_fallback() + raise + if not coordinates: + return explicit_fallback() + + domain_indices = tuple(int(index) for index in cell.domain_noise_indices) + if len(set(domain_indices)) != len(domain_indices) or any(index < 0 or index >= num_noise for index in domain_indices): + raise ValueError("cell domain noise index out of range or duplicated.") + domain_set = set(domain_indices) + retained_indices = tuple(index for index in range(num_noise) if index not in domain_set) + retained_kinds = tuple(noise_kinds[index] for index in retained_indices) + pointwise_indices = tuple(index for index, kind in enumerate(noise_kinds) if kind in POINTWISE_RESIDUAL_KINDS) + measure = float(2 ** len(domain_indices)) + scale = float(density) + + support = sorted(set().union(*(coordinate.terms for coordinate in coordinates))) + centers, matrix = _coefficient_matrix(coordinates, support) + gram = None + if torch is not None and isinstance(centers, torch.Tensor): + weight_vector = torch.tensor(weights, dtype=centers.dtype, device=centers.device) + weighted_matrix = matrix * weight_vector.unsqueeze(0) + center_cross = weighted_matrix @ centers + gram = weighted_matrix @ matrix.T + center_square = torch.dot(centers * weight_vector, centers) + if ( + np is not None + and not centers.is_complex() + and not matrix.is_complex() + ): + return _integrate_numpy_twojet_square( + support=support, + num_noise=num_noise, + center_square=float(center_square.detach().cpu().item()), + center_cross=center_cross.detach().cpu().numpy(), + gram=gram.detach().cpu().numpy(), + scale=scale, + measure=measure, + domain_indices=domain_indices, + retained_indices=retained_indices, + pointwise_indices=pointwise_indices, + retained_kinds=retained_kinds, + output=output, + ) + elif np is not None and _all_real_scalar_coefficients(centers, matrix): + # Affine PZ propagation may produce a mixture of lightweight floats and + # zero-dimensional torch tensors. Normalize them once, then use one + # BLAS contraction instead of millions of Python scalar operations. + center_vector = np.fromiter( + (_real_scalar_value(value) for value in centers), + dtype=float, + count=len(centers), + ) + coefficient_matrix = np.fromiter( + (_real_scalar_value(value) for row in matrix for value in row), + dtype=float, + count=len(matrix) * len(centers), + ).reshape(len(matrix), len(centers)) + weight_vector = np.asarray(weights, dtype=float) + weighted_matrix = coefficient_matrix * weight_vector[np.newaxis, :] + center_cross = weighted_matrix @ center_vector + gram = weighted_matrix @ coefficient_matrix.T + center_square = float(np.dot(center_vector * weight_vector, center_vector)) + return _integrate_numpy_twojet_square( + support=support, + num_noise=num_noise, + center_square=center_square, + center_cross=center_cross, + gram=gram, + scale=scale, + measure=measure, + domain_indices=domain_indices, + retained_indices=retained_indices, + pointwise_indices=pointwise_indices, + retained_kinds=retained_kinds, + output=output, + ) + else: + center_cross = [_weighted_dot(row, centers, weights) for row in matrix] + center_square = _weighted_dot(centers, centers, weights) + + retained: dict[Exponent, Any] = {} + pointwise: dict[Exponent, Any] = {} + zero_retained = (0,) * len(retained_indices) + + def accumulate(target: dict[Exponent, Any], exponent: Exponent, coefficient: Any) -> None: + target[exponent] = _add_coeff(target[exponent], coefficient) if exponent in target else coefficient + + def route(exponent: Exponent, coefficient: Any) -> None: + scaled = _mul_coeff(coefficient, scale) + if any(exponent[index] for index in pointwise_indices): + accumulate(pointwise, exponent, scaled) + return + moment = box_monomial_moment(tuple(exponent[index] for index in domain_indices)) + if moment == 0.0: + return + retained_exponent = tuple(exponent[index] for index in retained_indices) + accumulate(retained, retained_exponent, _mul_coeff(scaled, moment)) + + route((0,) * num_noise, center_square) + for index, exponent in enumerate(support): + center_coefficient = center_cross[index] + if np is not None and isinstance(center_coefficient, np.generic): + center_coefficient = float(center_coefficient) + route(exponent, _mul_coeff(center_coefficient, 2.0)) + for other_index in range(index, len(support)): + pair_exponent = tuple(a + b for a, b in zip(exponent, support[other_index])) + factor = 1.0 if index == other_index else 2.0 + if gram is None: + gram_coefficient = _weighted_dot(matrix[index], matrix[other_index], weights) + else: + gram_coefficient = gram[index][other_index] + if np is not None and isinstance(gram_coefficient, np.generic): + gram_coefficient = float(gram_coefficient) + route(pair_exponent, _mul_coeff(gram_coefficient, factor)) + + center = retained.pop(zero_retained, _zero_like(centers[0])) + radius = _zero_like(center) + for exponent, coefficient in pointwise.items(): + midpoint, term_radius = _pointwise_term_midpoint_radius( + exponent, + coefficient, + domain_indices, + retained_indices, + ) + center = _add_coeff(center, midpoint) + radius = _add_coeff(radius, term_radius) + result = IntegratedPZResult( + polynomial=PolynomialZonotope(center, retained, num_noise=len(retained_indices), noise_kinds=retained_kinds), + interval_radius=radius, + measure=measure, + metadata={"mode": "pointwise_interval", "direct_twojet_squared": True, "integrand_kind": integrand_kind}, + ) + return result.as_polynomial_zonotope() if output == "pz" else result.interval_enclosure() + + +def integrate_pz_value_squared( + value: PolynomialZonotope, + cell: PZIntegrationCell, + *, + output: IntegrationOutput = "interval", +): + """Directly integrate the squared Euclidean norm of a value-only PZ. + + The lightweight zero components adapt the existing weighted-coordinate + contraction without allocating input-dimensional Jacobian or Hessian + tensors. + """ + + zero = PolynomialZonotope.constant( + 0.0, + num_noise=value.num_noise, + noise_kinds=value.noise_kinds, + ) + return integrate_pz_twojet_squared( + PZTwoJet(Y=value, J=zero, H=zero), + cell, + "l2", + output=output, + ) + + +def integrate_pz_onejet_squared( + jet: PZOneJet, + cell: PZIntegrationCell, + *, + output: IntegrationOutput = "interval", +): + """Directly integrate ``|Y|^2 + |J|_F^2`` for a PZ one-jet. + + The fast one-jet forward path returns a pointwise box enclosure for the + Jacobian. Its squared Frobenius range is therefore computed entrywise and + scaled by the physical cell volume, avoiding a quadratic Gram contraction + over the fresh Jacobian residual symbols. Other one-jets retain the + generic direct PZ contraction as a safe fallback. + """ + + density = cell.jacobian_density + volume = cell.volume + y_noise_indices = { + index + for exponent in jet.Y.terms + for index, power in enumerate(exponent) + if power + } + jacobian_is_fresh_pointwise_box = all( + sum(exponent) == 1 + and exponent.index(1) not in y_noise_indices + and jet.J.noise_kinds[exponent.index(1)] in POINTWISE_RESIDUAL_KINDS + for exponent in jet.J.terms + ) + if output not in ("interval", "pz"): + raise ValueError("output must be 'interval' or 'pz'.") + if ( + isinstance(density, Real) + and isinstance(volume, Real) + and float(density) >= 0.0 + and float(volume) >= 0.0 + and jacobian_is_fresh_pointwise_box + ): + value_integral = integrate_pz_value_squared(jet.Y, cell, output=output) + enclosure = jet.J.interval_enclosure() + lower_square_sum = 0.0 + upper_square_sum = 0.0 + for lower, upper in zip( + _flatten_scalars(enclosure.lower), + _flatten_scalars(enclosure.upper), + ): + lo = float(lower) + hi = float(upper) + lower_square_sum += 0.0 if lo <= 0.0 <= hi else min(lo * lo, hi * hi) + upper_square_sum += max(lo * lo, hi * hi) + jacobian_integral = Interval.from_bounds( + nextafter(float(volume) * lower_square_sum, -inf), + nextafter(float(volume) * upper_square_sum, inf), + ) + if output == "pz": + jacobian_pz = PolynomialZonotope.constant( + jacobian_integral.midpoint, + num_noise=value_integral.num_noise, + noise_kinds=value_integral.noise_kinds, + ).add_independent_error( + jacobian_integral.radius, + kind="global_symbolic_residual", + ) + value_integral = value_integral.with_num_noise( + jacobian_pz.num_noise + ).with_noise_kinds(jacobian_pz.noise_kinds) + return value_integral + jacobian_pz + return value_integral + jacobian_integral + + zero = PolynomialZonotope.constant( + 0.0, + num_noise=jet.Y.num_noise, + noise_kinds=jet.Y.noise_kinds, + ) + return integrate_pz_twojet_squared( + PZTwoJet(Y=jet.Y, J=jet.J, H=zero), + cell, + "w12", + output=output, + ) + + +def _require_interval_tensor_domain(domain: Any): + from .pytorch import IntervalTensor as RuntimeIntervalTensor + + if not isinstance(domain, RuntimeIntervalTensor): + raise TypeError("PZ adaptive integration requires an IntervalTensor domain.") + if len(domain.shape) != 1: + raise NotImplementedError("PZ adaptive integration currently supports flat input boxes only.") + return RuntimeIntervalTensor + + +def _require_l2_output(output: str) -> None: + if output not in {"interval", "pz"}: + raise ValueError("output must be either 'interval' or 'pz'.") + + +def _require_adaptive_parameters(iterations: int, theta: float, chebyshev_degree: int, residual_subdivisions: int) -> None: + if iterations < 0: + raise ValueError("iterations must be non-negative.") + if not isfinite(float(theta)) or float(theta) <= 0.0 or float(theta) > 1.0: + raise ValueError("theta must be a finite real number in the interval (0, 1].") + if chebyshev_degree < 0: + raise ValueError("chebyshev_degree must be non-negative.") + if residual_subdivisions < 1: + raise ValueError("residual_subdivisions must be positive.") + + +def _sqrt_interval_nonnegative(value: Interval) -> Interval: + lower = max(0.0, float(value.lower)) + upper = max(0.0, float(value.upper)) + return Interval.from_bounds(nextafter(sqrt(lower), -inf), nextafter(sqrt(upper), inf)) + + +def _interval_width(value: Interval) -> float: + return max(0.0, float(value.upper) - float(value.lower)) + + +def _interval_add(left: Interval, right: Interval) -> Interval: + return left + right + + +def _dorfler_marking(indicators: list[float], theta: float) -> list[int]: + """Return a minimal Dörfler marked set, matching ``pytorch._dorfler_marking``.""" + + if not indicators: + raise ValueError("Indicators must be non-empty for Dörfler marking.") + total = sum(indicators) + if total <= 0.0: + return [max(range(len(indicators)), key=lambda idx: indicators[idx])] + threshold = theta * total + ranked_indices = sorted(range(len(indicators)), key=lambda idx: indicators[idx], reverse=True) + marked: list[int] = [] + accumulated = 0.0 + for idx in ranked_indices: + marked.append(idx) + accumulated += indicators[idx] + if accumulated >= threshold: + break + return marked + + +def _split_box(box: "IntervalTensor", split_dim: int | None = None) -> tuple["IntervalTensor", "IntervalTensor"]: + """Bisect an interval box, matching ``pytorch._split_box`` behavior.""" + + if split_dim is None: + widths = [float(upper - lower) for lower, upper in zip(box.lower, box.upper)] + split_dim = max(range(len(widths)), key=lambda idx: widths[idx]) + midpoint = 0.5 * (box.lower[split_dim] + box.upper[split_dim]) + lower_left = list(box.lower) + upper_left = list(box.upper) + lower_right = list(box.lower) + upper_right = list(box.upper) + upper_left[split_dim] = midpoint + lower_right[split_dim] = midpoint + from .pytorch import IntervalTensor as RuntimeIntervalTensor + + return RuntimeIntervalTensor.from_bounds(lower_left, upper_left), RuntimeIntervalTensor.from_bounds(lower_right, upper_right) + + +def _choose_split_dim_from_jacobian(box: "IntervalTensor", jacobian: Interval | None) -> int: + widths = [float(upper - lower) for lower, upper in zip(box.lower, box.upper)] + if jacobian is None or len(widths) <= 1: + return max(range(len(widths)), key=lambda idx: widths[idx]) + try: + if len(jacobian.shape) != 2: + raise ValueError + output_dim = len(jacobian.lower) + input_dim = len(jacobian.lower[0]) if output_dim > 0 else 0 + scores = [0.0] * input_dim + for row_idx in range(output_dim): + for col_idx in range(input_dim): + lower = float(jacobian.lower[row_idx][col_idx]) + upper = float(jacobian.upper[row_idx][col_idx]) + scores[col_idx] += max(abs(lower), abs(upper)) + weighted = [width * score for width, score in zip(widths, scores)] + if any(score > 0.0 for score in weighted): + return max(range(len(weighted)), key=lambda idx: weighted[idx]) + except Exception: + pass + return max(range(len(widths)), key=lambda idx: widths[idx]) + + +def _eval_pz_twojet(model, domain: PolynomialZonotope, *, chebyshev_degree: int = 5, residual_subdivisions: int = 128): + if hasattr(model, "eval_pz_twojet"): + return model.eval_pz_twojet(domain, chebyshev_degree=chebyshev_degree, residual_subdivisions=residual_subdivisions) + from .pytorch import pz_twojet_forward + + return pz_twojet_forward(model, domain, chebyshev_degree=chebyshev_degree, residual_subdivisions=residual_subdivisions) + + +def _eval_pz_value( + model, + domain: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, +): + if hasattr(model, "eval_pz_value"): + return model.eval_pz_value( + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + from .pytorch import pz_value_forward + + return pz_value_forward( + model, + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + + +def _eval_pz_onejet( + model, + domain: PolynomialZonotope, + *, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduction_strategy: str = "topk", + max_terms: int = 96, + max_degree: int = 4, + pca_rank: int = 4, + pca_candidates: int = 48, +): + if hasattr(model, "eval_pz_onejet"): + return model.eval_pz_onejet( + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction_strategy=reduction_strategy, + max_terms=max_terms, + max_degree=max_degree, + pca_rank=pca_rank, + pca_candidates=pca_candidates, + ) + from .pytorch import pz_onejet_forward + + return pz_onejet_forward( + model, + domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction_strategy=reduction_strategy, + max_terms=max_terms, + max_degree=max_degree, + pca_rank=pca_rank, + pca_candidates=pca_candidates, + ) + + +@dataclass(frozen=True) +class _CachedSquaredContribution: + """Cached adaptive-quadrature data for one active PZ integration cell.""" + + box: "IntervalTensor" + integrated_pz: PolynomialZonotope + jacobian: Any + split_dim: int + + @property + def contribution(self) -> Interval: + """Compatibility view used only for adaptive error indicators.""" + + return self.integrated_pz.interval_enclosure() + + +def _squared_twojet_integrand(jet: Any, integrand_kind: Literal["l2", "w12", "w22"]) -> PolynomialZonotope: + from .pz_norms import pz_twojet_l2_integrand, pz_twojet_w12_integrand, pz_twojet_w22_integrand + + integrands = { + "l2": pz_twojet_l2_integrand, + "w12": pz_twojet_w12_integrand, + "w22": pz_twojet_w22_integrand, + } + try: + return integrands[integrand_kind](jet) + except KeyError as exc: # pragma: no cover - guarded by Literal/internal callers + raise ValueError("integrand_kind must be one of 'l2', 'w12', or 'w22'.") from exc + + +def _evaluate_squared_contribution_cache( + model, + box: "IntervalTensor", + *, + integrand_kind: Literal["l2", "w12", "w22"], + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + reduction_strategy: str = "topk", + max_terms: int = 96, + max_degree: int = 4, + pca_rank: int = 4, + pca_candidates: int = 48, +) -> _CachedSquaredContribution: + """Evaluate and cache all expensive data needed for one active cell. + + The affine PZ integration cell, two-jet enclosure, directly integrated + squared contribution, Jacobian enclosure, and preferred split dimension + are computed exactly once for the cell lifetime. Refinement discards only + marked parent cells and computes fresh cache entries for their children. + """ + + cell = PZIntegrationCell.from_affine_box(box) + if integrand_kind == "l2": + value = _eval_pz_value( + model, + cell.domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + contribution = integrate_pz_value_squared(value, cell, output="pz") + jacobian = None + elif integrand_kind == "w12": + jet = _eval_pz_onejet( + model, + cell.domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + reduction_strategy=reduction_strategy, + max_terms=max_terms, + max_degree=max_degree, + pca_rank=pca_rank, + pca_candidates=pca_candidates, + ) + contribution = integrate_pz_onejet_squared(jet, cell, output="pz") + jacobian = jet.J.interval_enclosure() + else: + jet = _eval_pz_twojet( + model, + cell.domain, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + contribution = integrate_pz_twojet_squared( + jet, + cell, + integrand_kind, + output="pz", + ) + jacobian = jet.J.interval_enclosure() + return _CachedSquaredContribution( + box=box, + integrated_pz=contribution, + jacobian=jacobian, + split_dim=_choose_split_dim_from_jacobian(box, jacobian), + ) + + +def _integrated_squared_contribution( + model, + box: "IntervalTensor", + *, + integrand_kind: Literal["l2", "w12", "w22"], + chebyshev_degree: int, + residual_subdivisions: int, +) -> tuple[Interval, Interval | None]: + cached = _evaluate_squared_contribution_cache( + model, + box, + integrand_kind=integrand_kind, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + return cached.contribution, cached.jacobian + + +def _independent_pz_sum( + contributions: Sequence[PolynomialZonotope], +) -> PolynomialZonotope: + """Sum cell contributions while keeping their noise symbols independent. + + Noise variables from different adaptive cells describe unrelated local + approximation residuals. Positional PZ addition would accidentally + identify them. This helper first removes unused noise coordinates, then + embeds every cell in a disjoint block of variables before adding terms. + """ + + if not contributions: + return PolynomialZonotope.constant(0.0) + center = _zero_like(contributions[0].center) + used_per_cell: list[tuple[int, ...]] = [] + kinds: list[str] = [] + for contribution in contributions: + if contribution.shape != contributions[0].shape: + raise ValueError("Integrated PZ cell contributions must have equal shapes.") + center = _add_coeff(center, contribution.center) + used = tuple( + index + for index in range(contribution.num_noise) + if any(exponent[index] for exponent in contribution.terms) + ) + used_per_cell.append(used) + kinds.extend(contribution.noise_kinds[index] for index in used) + + total_noise = len(kinds) + terms: dict[Exponent, Any] = {} + offset = 0 + for contribution, used in zip(contributions, used_per_cell): + for exponent, coefficient in contribution.terms.items(): + compact = tuple(exponent[index] for index in used) + embedded = (0,) * offset + compact + (0,) * ( + total_noise - offset - len(used) + ) + terms[embedded] = ( + _add_coeff(terms[embedded], coefficient) + if embedded in terms + else coefficient + ) + offset += len(used) + return PolynomialZonotope( + center, + terms, + num_noise=total_noise, + noise_kinds=tuple(kinds), + ) + + +def _pz_adaptive_squared_integral( + model, + domain: "IntervalTensor", + *, + integrand_kind: Literal["l2", "w12", "w22"], + iterations: int, + theta: float, + chebyshev_degree: int, + residual_subdivisions: int, +) -> PolynomialZonotope: + active_cells = [ + _evaluate_squared_contribution_cache( + model, + domain, + integrand_kind=integrand_kind, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + ] + for _ in range(iterations): + indicators = [ + _interval_width(cell.contribution) + for cell in active_cells + ] + marked_indices = set(_dorfler_marking(indicators, theta)) + refined_cells: list[_CachedSquaredContribution] = [] + for idx, cell in enumerate(active_cells): + if idx not in marked_indices: + refined_cells.append(cell) + continue + for child_box in _split_box(cell.box, split_dim=cell.split_dim): + refined_cells.append( + _evaluate_squared_contribution_cache( + model, + child_box, + integrand_kind=integrand_kind, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + ) + active_cells = refined_cells + + return _independent_pz_sum([cell.integrated_pz for cell in active_cells]) + + +def pz_l2norm_bounds( + model, + domain: "IntervalTensor", + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: IntegrationOutput = "interval", +) -> Interval: + """Adaptive value-only PZ enclosure of the L2 norm over an interval domain.""" + + _require_interval_tensor_domain(domain) + _require_l2_output(output) + _require_adaptive_parameters(iterations, theta, chebyshev_degree, residual_subdivisions) + squared = _pz_adaptive_squared_integral( + model, + domain, + integrand_kind="l2", + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + return _sqrt_interval_nonnegative(squared.interval_enclosure()) + + +def pz_sobolev_norm_bounds( + model, + domain: "IntervalTensor", + order: Literal[1, 2] = 1, + iterations: int = 0, + theta: float = 0.5, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, + output: IntegrationOutput = "interval", +) -> Interval: + """Adaptive PZ enclosure of W^{order,2} Sobolev norms. + + Order one uses the scalable PZ one-jet path and order two uses the full + dependent PZ two-jet path. + """ + + _require_interval_tensor_domain(domain) + _require_l2_output(output) + _require_adaptive_parameters(iterations, theta, chebyshev_degree, residual_subdivisions) + if order not in {1, 2}: + raise ValueError("order must be either 1 or 2.") + squared = _pz_adaptive_squared_integral( + model, + domain, + integrand_kind="w12" if order == 1 else "w22", + iterations=iterations, + theta=theta, + chebyshev_degree=chebyshev_degree, + residual_subdivisions=residual_subdivisions, + ) + return _sqrt_interval_nonnegative(squared.interval_enclosure()) diff --git a/src/intervalnets/pz_norms.py b/src/intervalnets/pz_norms.py new file mode 100644 index 0000000..45e2664 --- /dev/null +++ b/src/intervalnets/pz_norms.py @@ -0,0 +1,232 @@ +"""Polynomial-zonotope norm integrands and certified norm intervals.""" + +from __future__ import annotations + +from collections import Counter +from math import inf, isfinite, nextafter, sqrt +from typing import Any, Sequence + +from .interval import Interval +from .polynomial_zonotope import PZTwoJet, PolynomialZonotope, pz_to_latex, pz_to_markdown_code, twojet_to_latex +from .pz_integration import PZIntegrationCell, integrate_over_cell, integrate_pz_over_domain, integrate_pz_twojet_squared + +try: # pragma: no cover - optional dependency + import torch +except ImportError: # pragma: no cover + torch = None + + +def _zero_scalar_like(z: PolynomialZonotope) -> PolynomialZonotope: + if torch is not None and isinstance(z.center, torch.Tensor): + return PolynomialZonotope.constant(torch.zeros((), dtype=z.center.dtype, device=z.center.device), num_noise=z.num_noise, noise_kinds=z.noise_kinds) + return PolynomialZonotope.constant(0.0, num_noise=z.num_noise, noise_kinds=z.noise_kinds) + + +def _fallback_scalar_indices(value: Any, prefix: tuple[int, ...] = ()): # type: ignore[no-untyped-def] + if isinstance(value, tuple): + for idx, item in enumerate(value): + yield from _fallback_scalar_indices(item, prefix + (idx,)) + else: + yield prefix + + +def _nested_get(value: Any, index: tuple[int, ...]): + out = value + for item in index: + out = out[item] + return out + + +def _scalar_entries(z: PolynomialZonotope): + if z.shape == (): + yield z + return + if torch is not None and isinstance(z.center, torch.Tensor): + for flat_idx in range(z.center.numel()): + multi = tuple(int(i) for i in torch.unravel_index(torch.tensor(flat_idx, device=z.center.device), z.center.shape)) + yield z[multi] + return + for index in _fallback_scalar_indices(z.center): + yield PolynomialZonotope(_nested_get(z.center, index), {exp: _nested_get(coeff, index) for exp, coeff in z.terms.items()}, num_noise=z.num_noise, noise_kinds=z.noise_kinds) + + + +def _pz_metadata_summary(z: PolynomialZonotope) -> dict[str, Any]: + """Return lightweight structural metadata for a polynomial zonotope.""" + + return { + "shape": z.shape, + "term_count": len(z.terms), + "max_degree": max((sum(exp) for exp in z.terms), default=0), + "num_noise": z.num_noise, + "noise_kind_counts": dict(Counter(z.noise_kinds)), + } + + +def build_pz_twojet_norm_diagnostics( + jet: PZTwoJet, + *, + include_integrands: bool = True, + render: bool = True, + max_terms: int | None = 12, + precision: int = 4, +) -> dict[str, Any]: + """Build pre-norm diagnostics for a polynomial-zonotope two-jet. + + The returned dictionary exposes the raw ``Y``, ``J``, and ``H`` zonotopes + by reference so callers can inspect the exact final two-jet before norm + integrand construction. Optional integrands are produced with the same + public helpers used by the norm routines, and renderer output is included + when ``render`` is true. The input jet is never modified. + """ + + diagnostics: dict[str, Any] = { + "jet": {"Y": jet.Y, "J": jet.J, "H": jet.H}, + "metadata": { + "Y": _pz_metadata_summary(jet.Y), + "J": _pz_metadata_summary(jet.J), + "H": _pz_metadata_summary(jet.H), + }, + } + diagnostics["metadata"]["total_term_count"] = sum( + diagnostics["metadata"][label]["term_count"] for label in ("Y", "J", "H") + ) + diagnostics["metadata"]["max_degree"] = max( + diagnostics["metadata"][label]["max_degree"] for label in ("Y", "J", "H") + ) + diagnostics["metadata"]["shapes"] = { + label: diagnostics["metadata"][label]["shape"] for label in ("Y", "J", "H") + } + diagnostics["metadata"]["noise_kind_counts"] = dict( + Counter(kind for label in ("Y", "J", "H") for kind in getattr(jet, label).noise_kinds) + ) + + if render: + diagnostics["rendered"] = { + "latex": { + "twojet": twojet_to_latex(jet, max_terms=max_terms, precision=precision), + "Y": pz_to_latex(jet.Y, max_terms=max_terms, precision=precision), + "J": pz_to_latex(jet.J, max_terms=max_terms, precision=precision), + "H": pz_to_latex(jet.H, max_terms=max_terms, precision=precision), + }, + "markdown": { + "Y": pz_to_markdown_code(jet.Y, max_terms=max_terms, precision=precision), + "J": pz_to_markdown_code(jet.J, max_terms=max_terms, precision=precision), + "H": pz_to_markdown_code(jet.H, max_terms=max_terms, precision=precision), + }, + } + + if include_integrands: + diagnostics["integrands"] = { + "l2_integrand": pz_twojet_l2_integrand(jet), + "w12_integrand": pz_twojet_w12_integrand(jet), + "w22_integrand": pz_twojet_w22_integrand(jet), + } + + return diagnostics + +def pz_sum_squares(z: PolynomialZonotope) -> PolynomialZonotope: + """Return the algebraic sum of squares of every scalar entry in ``z``.""" + + total = _zero_scalar_like(z) + for entry in _scalar_entries(z): + total = total + entry * entry + return total + + +def pz_symmetric_hessian_sum_squares(hessian: PolynomialZonotope) -> PolynomialZonotope: + """Return the dense Hessian square sum using Hessian symmetry. + + The dense two-jet Hessian convention stores one full symmetric matrix per + output with shape ``(output_dim, input_dim, input_dim)``. For that shape, + off-diagonal entries occur twice in the full Frobenius sum, so accumulate + only the upper-triangular entries and double the off-diagonal squares. + Scalar-output Hessians may also be stored as ``(input_dim, input_dim)``; + that lower-rank convention is handled analogously. + """ + + if len(hessian.shape) == 3 and hessian.shape[1] == hessian.shape[2]: + output_dim, input_dim, _ = hessian.shape + output_indices: range | tuple[None, ...] = range(output_dim) + elif len(hessian.shape) == 2 and hessian.shape[0] == hessian.shape[1]: + input_dim = hessian.shape[0] + output_indices = (None,) + else: + return pz_sum_squares(hessian) + + total = _zero_scalar_like(hessian) + for out in output_indices: + for a in range(input_dim): + diagonal = hessian[a, a] if out is None else hessian[out, a, a] + total = total + diagonal * diagonal + for b in range(a + 1, input_dim): + off_diagonal = hessian[a, b] if out is None else hessian[out, a, b] + total = total + 2.0 * off_diagonal * off_diagonal + return total + + +def pz_twojet_l2_integrand(jet: PZTwoJet) -> PolynomialZonotope: + """Squared L2 integrand ``sum_i Y_i^2`` for a two-jet.""" + + return pz_sum_squares(jet.Y) + + +def pz_twojet_w12_integrand(jet: PZTwoJet) -> PolynomialZonotope: + """Squared W^{1,2} integrand ``sum_i Y_i^2 + sum_ij J_ij^2``.""" + + return pz_twojet_l2_integrand(jet) + pz_sum_squares(jet.J) + + +def pz_twojet_w22_integrand(jet: PZTwoJet) -> PolynomialZonotope: + """Squared W^{2,2} integrand including value, Jacobian, and Hessian.""" + + return pz_twojet_w12_integrand(jet) + pz_symmetric_hessian_sum_squares(jet.H) + + +def _require_p2(p: float) -> None: + if not isfinite(float(p)) or float(p) != 2.0: + raise NotImplementedError("Polynomial-zonotope norm helpers currently support only p=2.0.") + + +def _sqrt_interval_nonnegative(value: Interval) -> Interval: + lower = max(0.0, float(value.lower)) + upper = max(0.0, float(value.upper)) + return Interval.from_bounds(nextafter(sqrt(lower), -inf), nextafter(sqrt(upper), inf)) + + +def pz_norm_from_integrand( + integrand: PolynomialZonotope, + cell: PZIntegrationCell | None = None, + *, + domain_indices: Sequence[int] | None = None, + p: float = 2.0, +) -> Interval: + """Integrate a squared PZ integrand exactly over domain variables and sqrt.""" + + _require_p2(p) + if cell is not None: + integral = integrate_over_cell(integrand, cell, output="interval") + else: + integral = integrate_pz_over_domain(integrand, domain_indices, mode="pointwise_interval").interval_enclosure() + return _sqrt_interval_nonnegative(integral) + + +def pz_twojet_l2_norm(jet: PZTwoJet, cell: PZIntegrationCell | None = None, *, p: float = 2.0) -> Interval: + _require_p2(p) + if cell is not None: + return _sqrt_interval_nonnegative(integrate_pz_twojet_squared(jet, cell, "l2")) + return pz_norm_from_integrand(pz_twojet_l2_integrand(jet), p=p) + + +def pz_twojet_w12_norm(jet: PZTwoJet, cell: PZIntegrationCell | None = None, *, p: float = 2.0) -> Interval: + _require_p2(p) + if cell is not None: + return _sqrt_interval_nonnegative(integrate_pz_twojet_squared(jet, cell, "w12")) + return pz_norm_from_integrand(pz_twojet_w12_integrand(jet), p=p) + + +def pz_twojet_w22_norm(jet: PZTwoJet, cell: PZIntegrationCell | None = None, *, p: float = 2.0) -> Interval: + _require_p2(p) + if cell is not None: + return _sqrt_interval_nonnegative(integrate_pz_twojet_squared(jet, cell, "w22")) + return pz_norm_from_integrand(pz_twojet_w22_integrand(jet), p=p) diff --git a/src/intervalnets/pz_tanh.py b/src/intervalnets/pz_tanh.py new file mode 100644 index 0000000..4abcbb6 --- /dev/null +++ b/src/intervalnets/pz_tanh.py @@ -0,0 +1,716 @@ +"""Certified polynomial approximation scaffolding for ``tanh`` on scalar intervals. + +The polynomial fit produced here is only a numerical proposal. It is never +used as proof: callers must rely on ``certify_tanh_residual_subdivision`` (or a +future root-isolation/Remez certificate backend) for the rigorous residual +``delta``. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from math import acos, atanh, cos, inf, nextafter, pi, sqrt, tanh +from typing import Any, Mapping, Sequence +from warnings import warn + +from .interval import Interval + + +@dataclass(frozen=True) +class TanhApproximation: + """Certified scalar polynomial enclosure for ``tanh`` on an interval. + + Attributes: + coeffs: Power-basis coefficients in ascending order, i.e. + ``p(x) = coeffs[0] + coeffs[1] * x + ...``. + lower: Lower endpoint of the scalar domain interval. + upper: Upper endpoint of the scalar domain interval. + delta: Certified non-negative residual satisfying + ``tanh(x) - p(x) in [-delta, delta]`` for every ``x`` in the + interval. + degree: Configured Chebyshev approximation degree. + metadata: Certification/proposal details, including the proposal method + and subdivision certificate settings. + """ + + coeffs: tuple[float, ...] + lower: float + upper: float + delta: float + degree: int + metadata: Mapping[str, Any] = field(default_factory=dict) + + +@dataclass(frozen=True) +class AffineTanhEnclosure: + """Scalar affine-plus-error enclosure for a tanh jet on an interval. + + The returned parameters certify ``function(x) in p*x + q + + delta*[-1, 1]`` for every scalar ``x`` in ``[lower, upper]``. + ``metadata`` records the finite stationary candidates and the final + floating-point safety inflation used for outward rounding. + """ + + p: float + q: float + delta: float + lower: float + upper: float + function: str + metadata: Mapping[str, Any] = field(default_factory=dict) + + +@dataclass(frozen=True) +class QuadraticTanhEnclosure: + """Certified quadratic-plus-error enclosure for a tanh jet. + + ``coeffs`` are power-basis coefficients ``(c, b, a)`` satisfying + ``function(x) in c + b*x + a*x**2 + delta*[-1, 1]`` throughout the + scalar interval. The normalized coefficients are also recorded in + ``metadata`` so callers can evaluate the parabola stably around the + interval midpoint. + """ + + coeffs: tuple[float, float, float] + delta: float + lower: float + upper: float + function: str + metadata: Mapping[str, Any] = field(default_factory=dict) + + +def _tanh_value_from_t(j: int, t: float) -> float: + if j == 0: + return t + if j == 1: + return 1.0 - t * t + if j == 2: + return -2.0 * t + 2.0 * t * t * t + raise ValueError("j must be 0, 1, or 2.") + + +def _clamp(value: float, lower: float, upper: float) -> float: + return min(max(value, lower), upper) + + +def _append_unique( + values: list[float], value: float, *, abs_tol: float = 1e-14 +) -> None: + if not any(abs(value - existing) <= abs_tol for existing in values): + values.append(value) + + +def _append_if_in_t_interval( + values: list[float], t: float, ta: float, tb: float +) -> None: + tlo = min(ta, tb) + thi = max(ta, tb) + tol = 8.0 * 2.220446049250313e-16 * max(1.0, abs(tlo), abs(thi), abs(t)) + if -1.0 < t < 1.0 and tlo - tol <= t <= thi + tol: + _append_unique(values, _clamp(t, -1.0 + 5e-324, 1.0 - 2.220446049250313e-16)) + + +def _stationary_t_candidates( + j: int, p: float, ta: float, tb: float +) -> tuple[float, ...]: + candidates: list[float] = [] + if j == 0: + d = max(0.0, 1.0 - p) + s = sqrt(d) + _append_if_in_t_interval(candidates, s, ta, tb) + _append_if_in_t_interval(candidates, -s, ta, tb) + elif j == 1: + z = _clamp((3.0 * sqrt(3.0) / 4.0) * p, -1.0, 1.0) + theta = acos(z) / 3.0 + for k in range(3): + t = (2.0 / sqrt(3.0)) * cos(theta - 2.0 * pi * k / 3.0) + _append_if_in_t_interval(candidates, t, ta, tb) + elif j == 2: + d = max(0.0, 1.0 - 1.5 * p) + s = sqrt(d) + for u in ((2.0 + s) / 3.0, (2.0 - s) / 3.0): + if 0.0 <= u < 1.0: + v = sqrt(u) + _append_if_in_t_interval(candidates, v, ta, tb) + _append_if_in_t_interval(candidates, -v, ta, tb) + else: + raise ValueError("j must be 0, 1, or 2.") + return tuple(candidates) + + +def _affine_tanh_jet_enclosure( + interval: Interval | Sequence[float], j: int, function_name: str +) -> AffineTanhEnclosure: + lower, upper = _scalar_interval_bounds(interval) + if lower == upper: + value = _tanh_value_from_t(j, tanh(lower)) + return AffineTanhEnclosure( + p=0.0, + q=value, + delta=0.0, + lower=lower, + upper=upper, + function=function_name, + metadata={ + "method": "point-interval", + "x_candidates": (lower,), + "t_candidates": (tanh(lower),), + "residuals": (value,), + "outward_rounding": "exact point interval; no inflation required", + }, + ) + + ta = tanh(lower) + tb = tanh(upper) + fa = _tanh_value_from_t(j, ta) + fb = _tanh_value_from_t(j, tb) + p = (fb - fa) / (upper - lower) + t_candidates = _stationary_t_candidates(j, p, ta, tb) + + x_candidates = [lower, upper] + for t in t_candidates: + x = atanh(t) + if lower < x < upper: + _append_unique(x_candidates, x) + + residuals = [] + candidate_records = [] + for x in x_candidates: + t = tanh(x) + value = _tanh_value_from_t(j, t) + residual = value - p * x + residuals.append(residual) + candidate_records.append({"x": x, "t": t, "residual": residual}) + + rmin = min(residuals) + rmax = max(residuals) + q = (rmax + rmin) / 2.0 + delta = (rmax - rmin) / 2.0 + + # The formulas above identify the exact real residual extrema. Inflate the + # final symmetric radius to account for ordinary floating-point evaluation + # of candidates, residuals, and midpoint/radius arithmetic. + rounding_inflation = ( + nextafter(max(abs(q), abs(delta), abs(p), abs(rmin), abs(rmax), 1.0), inf) + * 32.0 + * 2.220446049250313e-16 + ) + delta = nextafter(delta + rounding_inflation, inf) + + return AffineTanhEnclosure( + p=p, + q=q, + delta=delta, + lower=lower, + upper=upper, + function=function_name, + metadata={ + "method": "finite-stationary-candidates", + "candidate_t_values": t_candidates, + "x_candidates": tuple(x_candidates), + "candidate_records": tuple(candidate_records), + "residual_min": rmin, + "residual_max": rmax, + "outward_rounding": "final nextafter(delta + 32*eps*scale, +inf) safety inflation", + "rounding_inflation": rounding_inflation, + }, + ) + + +def affine_tanh_enclosure(interval: Interval | Sequence[float]) -> AffineTanhEnclosure: + """Return a certified scalar affine enclosure for ``tanh`` on ``interval``.""" + + return _affine_tanh_jet_enclosure(interval, 0, "tanh") + + +def affine_tanh_prime_enclosure( + interval: Interval | Sequence[float], +) -> AffineTanhEnclosure: + """Return a certified scalar affine enclosure for ``tanh'`` on ``interval``.""" + + return _affine_tanh_jet_enclosure(interval, 1, "tanh_prime") + + +def affine_tanh_double_prime_enclosure( + interval: Interval | Sequence[float], +) -> AffineTanhEnclosure: + """Return a certified scalar affine enclosure for ``tanh''`` on ``interval``.""" + + return _affine_tanh_jet_enclosure(interval, 2, "tanh_double_prime") + + +def _tanh_prime_third_derivative_from_t(t: float) -> float: + """Return the third derivative of ``tanh'`` as a polynomial in tanh(x).""" + + return 16.0 * t - 40.0 * t**3 + 24.0 * t**5 + + +def _tanh_prime_first_derivative_from_t(t: float) -> float: + return -2.0 * t + 2.0 * t**3 + + +def _tanh_prime_second_derivative_from_t(t: float) -> float: + return -2.0 + 8.0 * t**2 - 6.0 * t**4 + + +def _quadratic_residual_taylor_certificate( + lower: float, + upper: float, + coeffs: tuple[float, float, float], + subdivisions: int, +) -> tuple[float, float, tuple[Mapping[str, float], ...]]: + """Bound the residual range by a fixed, non-adaptive Taylor-form pass.""" + + if subdivisions < 1: + raise ValueError("subdivisions must be positive.") + c, b, a = coeffs + step = (upper - lower) / subdivisions + residual_lower = inf + residual_upper = -inf + records: list[Mapping[str, float]] = [] + critical_t = (0.0, sqrt(2.0 / 3.0), -sqrt(2.0 / 3.0)) + for index in range(subdivisions): + bin_lower = lower + index * step + bin_upper = upper if index + 1 == subdivisions else lower + (index + 1) * step + center = (bin_lower + bin_upper) / 2.0 + radius = (bin_upper - bin_lower) / 2.0 + t_center = tanh(center) + function_value = 1.0 - t_center * t_center + polynomial_value = c + b * center + a * center * center + residual_center = function_value - polynomial_value + residual_slope = _tanh_prime_first_derivative_from_t(t_center) - ( + b + 2.0 * a * center + ) + + ta, tb = tanh(bin_lower), tanh(bin_upper) + second_candidates = [ + _tanh_prime_second_derivative_from_t(ta) - 2.0 * a, + _tanh_prime_second_derivative_from_t(tb) - 2.0 * a, + ] + for t in critical_t: + if ta <= t <= tb: + second_candidates.append( + _tanh_prime_second_derivative_from_t(t) - 2.0 * a + ) + second_lower = min(second_candidates) + second_upper = max(second_candidates) + linear_radius = abs(residual_slope) * radius + quadratic_scale = 0.5 * radius * radius + local_lower = ( + residual_center + - linear_radius + + min(0.0, quadratic_scale * second_lower) + ) + local_upper = ( + residual_center + + linear_radius + + max(0.0, quadratic_scale * second_upper) + ) + residual_lower = min(residual_lower, local_lower) + residual_upper = max(residual_upper, local_upper) + records.append({ + "lower": bin_lower, + "upper": bin_upper, + "residual_lower": local_lower, + "residual_upper": local_upper, + }) + return residual_lower, residual_upper, tuple(records) + + +def quadratic_tanh_prime_enclosure( + interval: Interval | Sequence[float], + *, + certificate_subdivisions: int = 64, +) -> QuadraticTanhEnclosure: + """Return a cheap certified three-point quadratic enclosure for ``tanh'``. + + The proposal interpolates ``tanh'`` at the lower endpoint, midpoint, and + upper endpoint. Its certificate is the classical interpolation remainder + + ``|f(x)-q(x)| <= sup_I |f'''| |(x-l)(x-m)(x-u)| / 3!``. + + For equally spaced nodes, the second factor has the exact maximum + ``2*h**3/(3*sqrt(3))``, where ``h=(u-l)/2``. Moreover + ``f'''(x)=16*t-40*t**3+24*t**5`` with ``t=tanh(x)``; its extrema are found + from the endpoints and the four closed-form roots of + ``15*t**4-15*t**2+2=0``. A fixed, non-adaptive Taylor-form pass then + certifies and recenters the residual of the floating-point parabola. No + fitting iteration, optimization, root search, or adaptive refinement is + used. + """ + + lower, upper = _scalar_interval_bounds(interval) + midpoint = (lower + upper) / 2.0 + if lower == upper: + value = 1.0 - tanh(lower) ** 2 + return QuadraticTanhEnclosure( + coeffs=(value, 0.0, 0.0), + delta=0.0, + lower=lower, + upper=upper, + function="tanh_prime", + metadata={ + "method": "point-interval", + "midpoint": midpoint, + "half_width": 0.0, + "normalized_coeffs": (value, 0.0, 0.0), + "third_derivative_candidates": (tanh(lower),), + "third_derivative_sup": abs( + _tanh_prime_third_derivative_from_t(tanh(lower)) + ), + "rounding_inflation": 0.0, + }, + ) + + half_width = (upper - lower) / 2.0 + values = tuple(1.0 - tanh(x) ** 2 for x in (lower, midpoint, upper)) + f_lower, f_midpoint, f_upper = values + + # q(x) = alpha*s**2 + beta*s + gamma, s=(x-midpoint)/half_width. + alpha = (f_lower - 2.0 * f_midpoint + f_upper) / 2.0 + beta = (f_upper - f_lower) / 2.0 + gamma = f_midpoint + a = alpha / (half_width * half_width) + b = beta / half_width - 2.0 * a * midpoint + c = gamma - beta * midpoint / half_width + a * midpoint * midpoint + + ta, tb = tanh(lower), tanh(upper) + t_candidates = [ta, tb] + root_disc = sqrt(105.0) + for squared in ((15.0 - root_disc) / 30.0, (15.0 + root_disc) / 30.0): + root = sqrt(squared) + _append_if_in_t_interval(t_candidates, root, ta, tb) + _append_if_in_t_interval(t_candidates, -root, ta, tb) + third_derivative_sup = max( + abs(_tanh_prime_third_derivative_from_t(t)) for t in t_candidates + ) + global_interpolation_radius = ( + third_derivative_sup * half_width**3 / (9.0 * sqrt(3.0)) + ) + + residual_lower, residual_upper, certificate_records = ( + _quadratic_residual_taylor_certificate( + lower, + upper, + (c, b, a), + certificate_subdivisions, + ) + ) + residual_shift = (residual_upper + residual_lower) / 2.0 + c += residual_shift + certificate_radius = (residual_upper - residual_lower) / 2.0 + + # Account for ordinary floating-point construction/evaluation of the + # normalized interpolant, consistently with the affine enclosure backend. + scale = max( + 1.0, + abs(alpha) + abs(beta) + abs(gamma), + abs(global_interpolation_radius), + abs(certificate_radius), + third_derivative_sup, + ) + rounding_inflation = nextafter(scale, inf) * 128.0 * 2.220446049250313e-16 + delta = nextafter(certificate_radius + rounding_inflation, inf) + + return QuadraticTanhEnclosure( + coeffs=(c, b, a), + delta=delta, + lower=lower, + upper=upper, + function="tanh_prime", + metadata={ + "method": "endpoint-midpoint-interpolation-remainder", + "nodes": (lower, midpoint, upper), + "values": values, + "midpoint": midpoint, + "half_width": half_width, + "normalized_coeffs": (gamma + residual_shift, beta, alpha), + "third_derivative_candidates": tuple(t_candidates), + "third_derivative_sup": third_derivative_sup, + "global_interpolation_radius": global_interpolation_radius, + "certificate_subdivisions": certificate_subdivisions, + "certificate_residual_lower": residual_lower, + "certificate_residual_upper": residual_upper, + "certificate_records": certificate_records, + "residual_shift": residual_shift, + "rounding_inflation": rounding_inflation, + "outward_rounding": "final nextafter(delta + 128*eps*scale, +inf) safety inflation", + }, + ) + + +def _solve_dense_system(matrix: list[list[float]], rhs: list[float]) -> list[float]: + """Solve a small dense linear system by Gaussian elimination.""" + + n = len(rhs) + aug = [row[:] + [value] for row, value in zip(matrix, rhs)] + for col in range(n): + pivot = max(range(col, n), key=lambda row: abs(aug[row][col])) + if aug[pivot][col] == 0.0: + raise ValueError("Singular interpolation system for tanh proposal.") + aug[col], aug[pivot] = aug[pivot], aug[col] + scale = aug[col][col] + aug[col] = [value / scale for value in aug[col]] + for row in range(n): + if row == col: + continue + factor = aug[row][col] + if factor: + aug[row] = [ + value - factor * pivot_value + for value, pivot_value in zip(aug[row], aug[col]) + ] + return [aug[row][-1] for row in range(n)] + + +def _chebyshev_interpolation_power_coeffs( + lower: float, upper: float, degree: int +) -> tuple[float, ...]: + """Return a Chebyshev-node interpolation proposal in power basis.""" + + from math import cos, pi + + if degree == 0: + return (tanh((lower + upper) / 2.0),) + midpoint = (lower + upper) / 2.0 + half_width = (upper - lower) / 2.0 + nodes = [ + midpoint + half_width * cos((2 * k + 1) * pi / (2 * (degree + 1))) + for k in range(degree + 1) + ] + vandermonde = [[node**power for power in range(degree + 1)] for node in nodes] + values = [tanh(node) for node in nodes] + return tuple(_solve_dense_system(vandermonde, values)) + + +def _scalar_interval_bounds( + interval: Interval | Sequence[float], +) -> tuple[float, float]: + if isinstance(interval, Interval): + lower, upper = interval.lower, interval.upper + else: + if len(interval) != 2: + raise ValueError("interval must contain exactly two endpoints.") + lower, upper = interval + if isinstance(lower, tuple) or isinstance(upper, tuple): + raise ValueError("tanh approximation currently expects a scalar interval.") + lower_f = float(lower) + upper_f = float(upper) + if lower_f > upper_f: + raise ValueError("interval lower endpoint must not exceed upper endpoint.") + return lower_f, upper_f + + +def _poly_interval(coeffs: Sequence[float], interval: Interval) -> Interval: + result = Interval.point(0.0) + for coeff in reversed(tuple(float(c) for c in coeffs)): + result = result * interval + coeff + return result + + +def _tanh_interval(interval: Interval) -> Interval: + lower, upper = _scalar_interval_bounds(interval) + return Interval(nextafter(tanh(lower), -inf), nextafter(tanh(upper), inf)) + + +def compute_tanh_polynomial( + interval: Interval | Sequence[float], + degree: int | None = None, + *, + chebyshev_degree: int | None = None, + remez_degree: int | None = None, + subdivisions: int = 64, +) -> TanhApproximation: + """Compute and certify a polynomial approximation to ``tanh``. + + The current first-pass proposal uses a Chebyshev least-squares/interpolatory + fit converted to the power basis (a stable near-minimax-style starting + point, not a proof). The public option is ``chebyshev_degree`` to reflect + that implemented proposal method. ``remez_degree`` is accepted only as a + temporary deprecated alias and, when used, is recorded in metadata as + ``legacy_remez_degree``. Regardless of how the proposal is produced, the + returned ``delta`` is always obtained from + ``certify_tanh_residual_subdivision``. + """ + + candidates = [ + ("degree", degree), + ("chebyshev_degree", chebyshev_degree), + ("remez_degree", remez_degree), + ] + configured = [(name, int(value)) for name, value in candidates if value is not None] + if not configured: + raise TypeError("Either degree or chebyshev_degree must be supplied.") + if len({value for _, value in configured}) != 1: + raise ValueError( + "degree, chebyshev_degree, and remez_degree must agree when supplied together." + ) + configured_degree = configured[0][1] + used_legacy_remez = remez_degree is not None + if used_legacy_remez: + warn( + "remez_degree is deprecated; use chebyshev_degree because the implemented tanh proposal uses Chebyshev.fit.", + DeprecationWarning, + stacklevel=2, + ) + if configured_degree < 0: + raise ValueError("degree must be non-negative.") + lower, upper = _scalar_interval_bounds(interval) + + if lower == upper: + coeffs = (tanh(lower),) + (0.0,) * configured_degree + proposal = "constant-point" + else: + try: + import numpy as np + from numpy.polynomial import Chebyshev, Polynomial + except ImportError: + # Chebyshev-node interpolation is a stable numerical proposal even + # when NumPy is unavailable. Certification below still provides + # the proof rather than trusting this fit. + coeffs = _chebyshev_interpolation_power_coeffs( + lower, upper, configured_degree + ) + proposal = "chebyshev-interpolation-proposal" + else: + xs = np.linspace(lower, upper, max(2 * (configured_degree + 1), 32)) + cheb = Chebyshev.fit( + xs, np.tanh(xs), deg=configured_degree, domain=[lower, upper] + ) + power: Polynomial = cheb.convert(kind=Polynomial) + coeff_arr = np.asarray(power.coef, dtype=float) + if coeff_arr.size < configured_degree + 1: + coeff_arr = np.pad( + coeff_arr, (0, configured_degree + 1 - coeff_arr.size) + ) + coeffs = tuple(float(c) for c in coeff_arr[: configured_degree + 1]) + proposal = "chebyshev-fit-proposal" + + delta, cert_meta = certify_tanh_residual_subdivision( + (lower, upper), coeffs, subdivisions=subdivisions + ) + metadata = { + "chebyshev_degree": configured_degree, + "proposal": proposal, + "residual_certification": cert_meta, + "proof_note": "Numerical fit is not a proof; delta is certified by interval subdivision.", + } + if used_legacy_remez: + metadata["legacy_remez_degree"] = configured_degree + return TanhApproximation( + coeffs=tuple(float(c) for c in coeffs), + lower=lower, + upper=upper, + delta=delta, + degree=configured_degree, + metadata=metadata, + ) + + +def certify_tanh_residual_subdivision( + interval: Interval | Sequence[float], + coeffs: Sequence[float], + subdivisions: int = 64, +) -> tuple[float, dict[str, Any]]: + """Certify a conservative residual bound for ``tanh(x) - p(x)``. + + This routine subdivides the scalar domain, evaluates ``tanh(I) - p(I)`` + using the repository's outward-rounded interval arithmetic, and returns the + upward-rounded maximum absolute interval residual. It is a rigorous, + conservative fallback for the first implementation pass, not the final + root-isolation implementation envisioned by the blueprint. + """ + + lower, upper = _scalar_interval_bounds(interval) + if subdivisions <= 0: + raise ValueError("subdivisions must be positive.") + if not coeffs: + raise ValueError("coeffs must not be empty.") + + width = (upper - lower) / subdivisions + max_abs = 0.0 + worst_index = 0 + for idx in range(subdivisions): + sub_lower = lower + idx * width + sub_upper = upper if idx == subdivisions - 1 else lower + (idx + 1) * width + sub_interval = Interval(nextafter(sub_lower, -inf), nextafter(sub_upper, inf)) + residual = _tanh_interval(sub_interval) - _poly_interval(coeffs, sub_interval) + rlo, rhi = _scalar_interval_bounds(residual) + local = max(abs(rlo), abs(rhi)) + if local > max_abs: + max_abs = local + worst_index = idx + + delta = nextafter(max_abs, inf) + return delta, { + "method": "outward-rounded-subdivision", + "subdivisions": subdivisions, + "worst_subdivision": worst_index, + "interval": (lower, upper), + "note": "Rigorous conservative fallback; not final root-isolation certification.", + } + + +def _scalar_interval_from_enclosure(enclosure: Any) -> Interval: + """Return a scalar ``Interval`` from a scalar PZ interval enclosure.""" + + lower = enclosure.lower + upper = enclosure.upper + try: + import torch + except ImportError: # pragma: no cover + torch = None + if torch is not None and isinstance(lower, torch.Tensor): + if lower.numel() != 1 or upper.numel() != 1: + raise ValueError("tanh_pz_scalar expects a scalar polynomial zonotope.") + return Interval( + float(lower.reshape(()).item()), float(upper.reshape(()).item()) + ) + if isinstance(lower, tuple) or isinstance(upper, tuple): + raise ValueError("tanh_pz_scalar expects a scalar polynomial zonotope.") + return Interval(float(lower), float(upper)) + + +def tanh_pz_scalar( + Z_i: Any, + chebyshev_degree: int | None = None, + residual_subdivisions: int | None = None, + *, + remez_degree: int | None = None, +): + """Enclose ``tanh(Z_i)`` for a scalar polynomial zonotope. + + The returned zonotope is ``p_i(Z_i) + Delta_i * eta_i`` where ``p_i`` is a + numerically proposed polynomial and ``Delta_i`` is certified by subdivision + interval arithmetic. ``remez_degree`` is accepted only as a temporary + deprecated alias for ``chebyshev_degree``. + """ + + from .polynomial_zonotope import PolynomialZonotope + + if not isinstance(Z_i, PolynomialZonotope): + raise TypeError("Z_i must be a PolynomialZonotope.") + if Z_i.shape != (): + raise ValueError("tanh_pz_scalar expects a scalar polynomial zonotope.") + if chebyshev_degree is None and remez_degree is None: + raise TypeError("chebyshev_degree must be supplied.") + if ( + chebyshev_degree is not None + and remez_degree is not None + and int(chebyshev_degree) != int(remez_degree) + ): + raise ValueError( + "chebyshev_degree and remez_degree must agree when both are supplied." + ) + if residual_subdivisions is None: + raise TypeError("residual_subdivisions must be supplied.") + interval = _scalar_interval_from_enclosure(Z_i.interval_enclosure()) + approx = compute_tanh_polynomial( + interval, + chebyshev_degree=chebyshev_degree, + remez_degree=remez_degree, + subdivisions=residual_subdivisions, + ) + return Z_i.evaluate_polynomial(approx.coeffs).add_independent_error( + approx.delta, kind="approximation_pointwise" + ) diff --git a/src/intervalnets/shallow_hybrid.py b/src/intervalnets/shallow_hybrid.py new file mode 100644 index 0000000..21d4e0a --- /dev/null +++ b/src/intervalnets/shallow_hybrid.py @@ -0,0 +1,528 @@ +"""Fast uncompressed hybrid one-jets for shallow scalar tanh networks. + +This module specializes the architecture ``Linear -> Tanh -> Linear`` with a +scalar output. It contracts the activation-derivative enclosure with the +output layer before expanding the physical-input Jacobian (reverse mode), +retains every domain monomial and every per-neuron approximation-noise +generator, and integrates the resulting squared Jacobian without constructing +its explicit polynomial square. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from math import inf, nextafter, tanh +from time import perf_counter +from typing import Any, Literal + +from .interval import Interval +from .polynomial_zonotope import PZOneJet, PolynomialZonotope +from .pz_integration import PZIntegrationCell +from .pz_tanh import ( + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + quadratic_tanh_prime_enclosure, +) + +try: # pragma: no cover - optional dependency + import torch + from torch import nn +except ImportError: # pragma: no cover + torch = None + nn = None + + +@dataclass(frozen=True) +class ShallowHybridOneJetResult: + """Uncompressed reverse one-jet and its factored integration data.""" + + final: PZOneJet + preactivation_lower: Any + preactivation_upper: Any + derivative_approximation_radii: Any + affine_derivative_approximation_radii: Any + derivative_degrees: Any + derivative_relative_slopes: Any + preactivation_center: Any + preactivation_coefficients: Any + value_approximation_radii: Any + value_center: Any + value_domain_coefficients: Any + value_error_generators: Any + domain_center: Any + domain_coefficients: Any + derivative_error_generators: Any + quadratic_vectors: Any + quadratic_slopes: Any + timings: dict[str, float] + + +def _require_torch() -> None: + if torch is None or nn is None: + raise ImportError("PyTorch is required for shallow hybrid certification.") + + +def _shallow_layers(module: Any): + children = list(module.children()) if isinstance(module, nn.Sequential) else [] + if ( + len(children) != 3 + or not isinstance(children[0], nn.Linear) + or not isinstance(children[1], nn.Tanh) + or not isinstance(children[2], nn.Linear) + or children[2].out_features != 1 + or children[0].out_features != children[2].in_features + ): + raise ValueError( + "The reverse shallow hybrid path requires Linear -> Tanh -> " + "Linear with one scalar output." + ) + return children[0], children[2] + + +def _affine_domain_coefficients(x: PolynomialZonotope) -> tuple[list[tuple[int, ...]], Any]: + if len(x.shape) != 1: + raise ValueError("The shallow reverse path requires a flat input PZ.") + def active_noise_index(exponent: tuple[int, ...]) -> int: + active = [index for index, power in enumerate(exponent) if power] + if len(active) != 1 or exponent[active[0]] != 1: + raise ValueError("The shallow reverse path requires an affine input PZ.") + if x.noise_kinds[active[0]] != "domain": + raise ValueError("Every active input symbol must be a domain noise symbol.") + return active[0] + + # The dense coefficient columns and external noise vectors both follow + # noise-index order. Lexicographic exponent order reverses the standard + # basis for boxes (e.g. (0,1) precedes (1,0)) and breaks exact evaluation. + support = sorted(x.terms, key=active_noise_index) + if not support: + return support, torch.empty((0, x.shape[0]), dtype=x.center.dtype, device=x.center.device) + return support, torch.stack([x.terms[exponent] for exponent in support]) + + +def _hybrid_activation_coefficients( + lower: Any, + upper: Any, + *, + flatness_threshold: float, + certificate_subdivisions: int, +) -> tuple[Any, ...]: + value_slopes: list[float] = [] + value_intercepts: list[float] = [] + value_radii: list[float] = [] + constants: list[float] = [] + linears: list[float] = [] + quadratics: list[float] = [] + radii: list[float] = [] + affine_radii: list[float] = [] + degrees: list[int] = [] + relative_slopes: list[float] = [] + for lo, hi in zip(lower.detach().cpu().tolist(), upper.detach().cpu().tolist()): + interval = Interval(float(lo), float(hi)) + value_affine = affine_tanh_enclosure(interval) + affine = affine_tanh_prime_enclosure(interval) + half_width = 0.5 * (float(hi) - float(lo)) + endpoint_lower = 1.0 - tanh(float(lo)) ** 2 + endpoint_upper = 1.0 - tanh(float(hi)) ** 2 + maximum = 1.0 if lo <= 0.0 <= hi else max(endpoint_lower, endpoint_upper) + interval_radius = 0.5 * (maximum - min(endpoint_lower, endpoint_upper)) + relative_slope = ( + abs(affine.p) * half_width / interval_radius + if interval_radius > 0.0 + else 0.0 + ) + use_quadratic = lo <= 0.0 <= hi and relative_slope <= flatness_threshold + quadratic = ( + quadratic_tanh_prime_enclosure( + Interval(float(lo), float(hi)), + certificate_subdivisions=certificate_subdivisions, + ) + if use_quadratic + else None + ) + if quadratic is not None and quadratic.delta < affine.delta: + constant, linear, quadratic_coefficient = quadratic.coeffs + radius = quadratic.delta + degree = 2 + else: + constant, linear, quadratic_coefficient = affine.q, affine.p, 0.0 + radius = affine.delta + degree = 1 + constants.append(constant) + linears.append(linear) + quadratics.append(quadratic_coefficient) + radii.append(radius) + affine_radii.append(affine.delta) + degrees.append(degree) + relative_slopes.append(relative_slope) + value_slopes.append(value_affine.p) + value_intercepts.append(value_affine.q) + value_radii.append(value_affine.delta) + + options = {"dtype": lower.dtype, "device": lower.device} + return ( + torch.tensor(value_slopes, **options), + torch.tensor(value_intercepts, **options), + torch.tensor(value_radii, **options), + torch.tensor(constants, **options), + torch.tensor(linears, **options), + torch.tensor(quadratics, **options), + torch.tensor(radii, **options), + torch.tensor(affine_radii, **options), + torch.tensor(degrees, dtype=torch.int64, device=lower.device), + torch.tensor(relative_slopes, **options), + ) + + +def shallow_scalar_hybrid_onejet_reverse( + module: Any, + x: PolynomialZonotope, + *, + derivative_flatness_threshold: float = 0.01, + quadratic_certificate_subdivisions: int = 64, + chebyshev_degree: int = 5, + residual_subdivisions: int = 128, +) -> ShallowHybridOneJetResult: + """Build an uncompressed hybrid one-jet by scalar-output reverse mode. + + No monomial support reduction or coefficient boxing is performed. The + derivative approximation error of each hidden neuron remains one shared + pointwise approximation-noise generator across all input derivatives. + """ + + _require_torch() + if derivative_flatness_threshold < 0.0: + raise ValueError("derivative_flatness_threshold must be non-negative.") + first, output = _shallow_layers(module) + parameter = first.weight + if not isinstance(x.center, torch.Tensor): + x = PolynomialZonotope( + torch.as_tensor(x.center, dtype=parameter.dtype, device=parameter.device), + { + exponent: torch.as_tensor( + coefficient, dtype=parameter.dtype, device=parameter.device + ) + for exponent, coefficient in x.terms.items() + }, + num_noise=x.num_noise, + noise_kinds=x.noise_kinds, + ) + support, input_coefficients = _affine_domain_coefficients(x) + input_dim = x.shape[0] + if first.in_features != input_dim: + raise ValueError("Network input dimension does not match the input PZ.") + if len(support) != input_dim: + raise ValueError( + "The shallow reverse path currently requires one affine domain " + "generator per physical input coordinate." + ) + + # The value and reverse-derivative paths use the same dense hidden + # preactivation representation and interval hull. Keep it once rather + # than invoking the generic value forward and rebuilding W*x+b below. + del chebyshev_degree, residual_subdivisions + preparation_start = perf_counter() + weight_in = first.weight.detach().to(dtype=x.center.dtype, device=x.center.device) + bias_in = first.bias.detach().to(dtype=x.center.dtype, device=x.center.device) + weight_out = output.weight.detach()[0].to(dtype=x.center.dtype, device=x.center.device) + bias_out = output.bias.detach()[0].to(dtype=x.center.dtype, device=x.center.device) + + z_center = weight_in @ x.center + bias_in + # One row per hidden neuron and one column per retained domain monomial. + z_coefficients = weight_in @ input_coefficients.T + z_radius = torch.sum(torch.abs(z_coefficients), dim=1) + lower = torch.nextafter(z_center - z_radius, torch.full_like(z_center, -torch.inf)) + upper = torch.nextafter(z_center + z_radius, torch.full_like(z_center, torch.inf)) + preparation_seconds = perf_counter() - preparation_start + + certification_start = perf_counter() + ( + value_slopes, + value_intercepts, + value_radii, + constants, + linears, + quadratics, + radii, + affine_radii, + degrees, + relative_slopes, + ) = _hybrid_activation_coefficients( + lower, + upper, + flatness_threshold=derivative_flatness_threshold, + certificate_subdivisions=quadratic_certificate_subdivisions, + ) + certification_seconds = perf_counter() - certification_start + + value_start = perf_counter() + hidden_value_center = value_intercepts + value_slopes * z_center + value_center = torch.dot(weight_out, hidden_value_center) + bias_out + value_domain_coefficients = z_coefficients.T @ (weight_out * value_slopes) + value_error_generators = weight_out * value_radii + value_seconds = perf_counter() - value_start + + construction_start = perf_counter() + derivative_center = constants + linears * z_center + quadratics * z_center.square() + derivative_linear = (linears + 2.0 * quadratics * z_center).unsqueeze(1) * z_coefficients + reverse_vectors = weight_out.unsqueeze(1) * weight_in + gradient_center = derivative_center @ reverse_vectors + gradient_linear = derivative_linear.T @ reverse_vectors + + pair_rows, pair_columns = torch.triu_indices( + len(support), len(support), device=x.center.device + ) + pair_factor = torch.where( + pair_rows == pair_columns, + torch.ones_like(pair_rows, dtype=x.center.dtype), + torch.full_like(pair_rows, 2.0, dtype=x.center.dtype), + ) + selected = torch.nonzero(quadratics != 0.0, as_tuple=False).reshape(-1) + if selected.numel(): + quadratic_slopes = z_coefficients[selected] + quadratic_vectors = quadratics[selected].unsqueeze(1) * reverse_vectors[selected] + pair_hidden_coefficients = ( + quadratic_slopes[:, pair_rows] + * quadratic_slopes[:, pair_columns] + * pair_factor.unsqueeze(0) + ) + gradient_quadratic = ( + (pair_hidden_coefficients * quadratics[selected].unsqueeze(1)).T + @ reverse_vectors[selected] + ) + else: + quadratic_slopes = z_coefficients.new_empty((0, len(support))) + quadratic_vectors = z_coefficients.new_empty((0, input_dim)) + gradient_quadratic = z_coefficients.new_zeros((len(pair_rows), input_dim)) + + derivative_error_generators = (weight_out * radii).unsqueeze(1) * weight_in + domain_coefficients = torch.cat((gradient_linear, gradient_quadratic), dim=0) + + total_noise = x.num_noise + 2 * first.out_features + noise_kinds = x.noise_kinds + ("approximation_pointwise",) * (2 * first.out_features) + value_terms: dict[tuple[int, ...], Any] = {} + terms: dict[tuple[int, ...], Any] = {} + padding = (0,) * (total_noise - x.num_noise) + for exponent, value_coefficient, gradient_coefficient in zip( + support, value_domain_coefficients, gradient_linear + ): + value_terms[exponent + padding] = value_coefficient.unsqueeze(0) + terms[exponent + padding] = gradient_coefficient + for row, column, coefficient in zip(pair_rows.tolist(), pair_columns.tolist(), gradient_quadratic): + exponent = tuple(a + b for a, b in zip(support[row], support[column])) + terms[exponent + padding] = coefficient + for neuron, coefficient in enumerate(value_error_generators): + exponent = [0] * total_noise + exponent[x.num_noise + neuron] = 1 + value_terms[tuple(exponent)] = coefficient.unsqueeze(0) + for neuron, coefficient in enumerate(derivative_error_generators): + exponent = [0] * total_noise + exponent[x.num_noise + first.out_features + neuron] = 1 + terms[tuple(exponent)] = coefficient + + value = PolynomialZonotope( + value_center.unsqueeze(0), + value_terms, + num_noise=total_noise, + noise_kinds=noise_kinds, + ) + jacobian = PolynomialZonotope( + gradient_center.unsqueeze(0), + {exponent: coefficient.unsqueeze(0) for exponent, coefficient in terms.items()}, + num_noise=total_noise, + noise_kinds=noise_kinds, + ) + construction_seconds = perf_counter() - construction_start + return ShallowHybridOneJetResult( + final=PZOneJet(Y=value, J=jacobian), + preactivation_lower=lower, + preactivation_upper=upper, + derivative_approximation_radii=radii, + affine_derivative_approximation_radii=affine_radii, + derivative_degrees=degrees, + derivative_relative_slopes=relative_slopes, + preactivation_center=z_center, + preactivation_coefficients=z_coefficients, + value_approximation_radii=value_radii, + value_center=value_center, + value_domain_coefficients=value_domain_coefficients, + value_error_generators=value_error_generators, + domain_center=gradient_center, + domain_coefficients=domain_coefficients, + derivative_error_generators=derivative_error_generators, + quadratic_vectors=quadratic_vectors, + quadratic_slopes=quadratic_slopes, + timings={ + "preactivation_preparation": preparation_seconds, + "activation_certification": certification_seconds, + "value_construction": value_seconds, + "reverse_jacobian_construction": construction_seconds, + "onejet_construction": ( + preparation_seconds + + certification_seconds + + value_seconds + + construction_seconds + ), + }, + ) + + +def _value_squared_components(result: ShallowHybridOneJetResult) -> tuple[Any, Any]: + """Return the refined normalized center/radius for the scalar value square.""" + + center = result.value_center + linear = result.value_domain_coefficients + errors = result.value_error_generators + # eta_i**2 is pointwise but non-negative, hence it contributes the + # one-sided interval [0, e_i**2]. Store this as midpoint plus radius. + error_diagonal = errors.square() + normalized_center = ( + center.square() + + torch.sum(linear.square()) / 3.0 + + 0.5 * torch.sum(error_diagonal) + ) + + center_error_cross = 2.0 * center * errors + # E|alpha_j| = 1/2, so the coefficient 2*l_j*e_i has radius |l_j*e_i|. + domain_error_cross = linear[:, None] * errors[None, :] + error_gram = errors[:, None] * errors[None, :] + off_rows, off_columns = torch.triu_indices( + len(errors), len(errors), offset=1, device=errors.device + ) + normalized_radius = ( + torch.sum(torch.abs(center_error_cross)) + + torch.sum(torch.abs(domain_error_cross)) + + 0.5 * torch.sum(torch.abs(torch.diagonal(error_gram))) + + 2.0 * torch.sum(torch.abs(error_gram[off_rows, off_columns])) + ) + return normalized_center, normalized_radius + + +def _gradient_squared_components( + result: ShallowHybridOneJetResult, +) -> tuple[Any, Any]: + """Return normalized center/radius for the squared gradient.""" + + center = result.domain_center + coefficients = result.domain_coefficients + errors = result.derivative_error_generators + input_dim = center.numel() + linear = coefficients[:input_dim] + quadratic_vectors = result.quadratic_vectors + slopes = result.quadratic_slopes + + normalized_center = torch.dot(center, center) + torch.sum(linear.square()) / 3.0 + if len(quadratic_vectors): + slope_norms = torch.sum(slopes.square(), dim=1) + normalized_center = normalized_center + (2.0 / 3.0) * torch.sum( + (quadratic_vectors @ center) * slope_norms + ) + slope_gram = slopes @ slopes.T + squared_coordinate_overlap = slopes.square() @ slopes.square().T + fourth_moments = ( + (slope_norms[:, None] * slope_norms[None, :] + 2.0 * slope_gram.square()) / 9.0 + - (2.0 / 15.0) * squared_coordinate_overlap + ) + normalized_center = normalized_center + torch.sum( + (quadratic_vectors @ quadratic_vectors.T) * fourth_moments + ) + + # Canonicalize each alpha^beta eta_i coefficient by one dense contraction + # before taking absolute values. The rows are constant, linear, then the + # upper-triangular quadratic domain monomials. Their normalized absolute + # moments are 1, 1/2, 1/3 (diagonal), and 1/4 (off diagonal). + domain_with_center = torch.cat((center.unsqueeze(0), coefficients), dim=0) + domain_error_cross = 2.0 * (domain_with_center @ errors.T) + pair_rows, pair_columns = torch.triu_indices( + input_dim, input_dim, device=center.device + ) + quadratic_absolute_moments = torch.where( + pair_rows == pair_columns, + torch.full_like(pair_rows, 1.0 / 3.0, dtype=center.dtype), + torch.full_like(pair_rows, 1.0 / 4.0, dtype=center.dtype), + ) + absolute_moments = torch.cat( + ( + torch.ones(1, dtype=center.dtype, device=center.device), + torch.full( + (input_dim,), 0.5, dtype=center.dtype, device=center.device + ), + quadratic_absolute_moments, + ) + ) + error_gram = errors @ errors.T + diagonal = torch.diagonal(error_gram) + off_rows, off_columns = torch.triu_indices( + len(errors), len(errors), offset=1, device=errors.device + ) + normalized_radius = ( + torch.sum(torch.abs(domain_error_cross) * absolute_moments.unsqueeze(1)) + + 0.5 * torch.sum(torch.abs(diagonal)) + + 2.0 * torch.sum(torch.abs(error_gram[off_rows, off_columns])) + ) + normalized_center = normalized_center + 0.5 * torch.sum(diagonal) + return normalized_center, normalized_radius + + +def _scaled_interval_from_components( + center: Any, + radius: Any, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"], +): + if output not in {"interval", "pz"}: + raise ValueError("output must be either 'interval' or 'pz'.") + if not isinstance(cell.volume, (int, float)): + raise NotImplementedError("The shallow direct integral requires a scalar cell volume.") + integrated_center = float(cell.volume) * float(center.detach().cpu().item()) + integrated_radius = float(cell.volume) * float(radius.detach().cpu().item()) + total = Interval.from_bounds( + nextafter(integrated_center - integrated_radius, -inf), + nextafter(integrated_center + integrated_radius, inf), + ) + if output == "interval": + return total + return PolynomialZonotope.constant(total.midpoint).add_independent_error( + total.radius, kind="global_symbolic_residual" + ) + + +def integrate_shallow_hybrid_value_squared( + result: ShallowHybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Direct refined pointwise integral of ``|Y|^2``.""" + + center, radius = _value_squared_components(result) + return _scaled_interval_from_components(center, radius, cell, output=output) + + +def integrate_shallow_hybrid_onejet_squared( + result: ShallowHybridOneJetResult, + cell: PZIntegrationCell, + *, + output: Literal["interval", "pz"] = "interval", +): + """Direct symbolic integral of ``|Y|^2 + |J|_F^2``. + + The domain-only quadratic Jacobian core is integrated using exact second + and fourth moments of independent ``U[-1,1]`` domain symbols. Terms that + contain derivative approximation noise are canonicalized in batched Gram + products and use the established pointwise-noise integration semantics. + """ + + value_center, value_radius = _value_squared_components(result) + gradient_center, gradient_radius = _gradient_squared_components(result) + + # Value and derivative approximation errors use disjoint symbol blocks. + # Their canonical pointwise coefficients therefore cannot collide, while + # all domain-only contributions can be summed before the single final + # intervalization. This is the joint W12 specialization of the direct + # integrated-square algorithm. + normalized_center = value_center + gradient_center + normalized_radius = value_radius + gradient_radius + return _scaled_interval_from_components( + normalized_center, normalized_radius, cell, output=output + ) diff --git a/tests/test_deep_hybrid.py b/tests/test_deep_hybrid.py new file mode 100644 index 0000000..058f374 --- /dev/null +++ b/tests/test_deep_hybrid.py @@ -0,0 +1,174 @@ +from __future__ import annotations + +import pytest + +torch = pytest.importorskip("torch") +from torch import nn + +from intervalnets import ( + DeepHybridOneJetResult, + IntervalTensor, + PZIntegrationCell, + PolynomialZonotope, + integrate_deep_hybrid_onejet_squared, + integrate_deep_hybrid_value_squared, + integrate_hybrid_onejet_squared, + integrate_hybrid_value_squared, + integrate_pz_value_squared, + scalar_hybrid_onejet_reverse, + shallow_scalar_hybrid_onejet_reverse, +) + + +def _deep_model() -> nn.Sequential: + model = nn.Sequential( + nn.Linear(2, 3), + nn.Tanh(), + nn.Linear(3, 2), + nn.Tanh(), + nn.Linear(2, 1), + ).double() + with torch.no_grad(): + model[0].weight.copy_( + torch.tensor([[1.1, -0.3], [0.5, 0.8], [-0.9, 0.4]]) + ) + model[0].bias.copy_(torch.tensor([0.0, 0.1, -0.15])) + model[2].weight.copy_( + torch.tensor([[0.7, -0.4, 0.5], [-0.2, 0.9, 0.6]]) + ) + model[2].bias.copy_(torch.tensor([0.05, -0.08])) + model[4].weight.copy_(torch.tensor([[0.8, -0.55]])) + model[4].bias.copy_(torch.tensor([0.12])) + return model + + +def _realizing_noises(result: DeepHybridOneJetResult, samples: torch.Tensor): + value_noises = [samples / 0.35] + derivative_noises = [] + for factor in result.factors: + xi = torch.cat(value_noises, dim=1) + preactivation = ( + factor.preactivation_center.unsqueeze(0) + + xi @ factor.preactivation_coefficients.T + ) + exact_value = torch.tanh(preactivation) + value_core = ( + factor.value_intercepts.unsqueeze(0) + + factor.value_slopes.unsqueeze(0) * preactivation + ) + value_eta = torch.where( + factor.value_approximation_radii.unsqueeze(0) == 0.0, + torch.zeros_like(exact_value), + (exact_value - value_core) + / factor.value_approximation_radii.unsqueeze(0), + ) + affine_argument = xi @ factor.preactivation_coefficients.T + derivative_core = ( + factor.center.unsqueeze(0) + + xi @ factor.linear_coefficients.T + + factor.quadratic_coefficients.unsqueeze(0) + * affine_argument.square() + ) + exact_derivative = 1.0 - exact_value.square() + derivative_eta = torch.where( + factor.approximation_radii.unsqueeze(0) == 0.0, + torch.zeros_like(exact_derivative), + (exact_derivative - derivative_core) + / factor.approximation_radii.unsqueeze(0), + ) + value_noises.append(value_eta) + derivative_noises.append(derivative_eta) + return torch.cat((*value_noises, *derivative_noises), dim=1) + + +def test_deep_factored_hybrid_retains_all_factors_and_is_sound() -> None: + model = _deep_model() + lower = torch.full((2,), -0.35, dtype=torch.float64) + upper = torch.full((2,), 0.35, dtype=torch.float64) + domain = PolynomialZonotope.from_box(lower, upper) + result = scalar_hybrid_onejet_reverse( + model, domain, derivative_flatness_threshold=1.0 + ) + assert isinstance(result, DeepHybridOneJetResult) + assert len(result.factors) == 2 + assert result.jacobian.num_noise == 2 + 2 * (3 + 2) + assert sum(factor.center.numel() for factor in result.factors) == 5 + + generator = torch.Generator().manual_seed(31) + samples = lower + (upper - lower) * torch.rand((2048, 2), generator=generator) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + noises = _realizing_noises(result, samples.detach()) + assert torch.max(torch.abs(noises)) <= 1.0 + 5e-12 + factored_gradients = result.jacobian.evaluate(noises) + assert torch.allclose(factored_gradients, gradients, rtol=2e-11, atol=2e-11) + + enclosure = result.jacobian.interval_enclosure() + enclosure_lower = torch.as_tensor(enclosure.lower).reshape(1, 2) + enclosure_upper = torch.as_tensor(enclosure.upper).reshape(1, 2) + assert torch.all(gradients >= enclosure_lower) + assert torch.all(gradients <= enclosure_upper) + + +def test_deep_factored_integral_encloses_sampled_w12() -> None: + model = _deep_model() + box = IntervalTensor.from_bounds([-0.35, -0.35], [0.35, 0.35]) + cell = PZIntegrationCell.from_affine_box(box) + result = scalar_hybrid_onejet_reverse( + model, cell.domain, derivative_flatness_threshold=1.0 + ) + integral = integrate_deep_hybrid_onejet_squared(result, cell) + + generator = torch.Generator().manual_seed(37) + lower = torch.tensor(box.lower, dtype=torch.float64) + upper = torch.tensor(box.upper, dtype=torch.float64) + samples = lower + (upper - lower) * torch.rand((12000, 2), generator=generator) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + sampled = float( + (values.square().squeeze(1) + gradients.square().sum(1)).mean().detach() + ) + normalized_lower = float(integral.lower) / float(cell.volume) + normalized_upper = float(integral.upper) / float(cell.volume) + assert normalized_lower <= sampled <= normalized_upper + assert result.gradient_spectral_bound > 0.0 + + value_integral = integrate_deep_hybrid_value_squared(result, cell) + explicit_value_integral = integrate_pz_value_squared(result.value, cell) + assert float(value_integral.lower) == pytest.approx( + float(explicit_value_integral.lower), rel=2e-12, abs=2e-12 + ) + assert float(value_integral.upper) == pytest.approx( + float(explicit_value_integral.upper), rel=2e-12, abs=2e-12 + ) + + +def test_depth_generic_dispatch_recovers_shallow_implementation() -> None: + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + torch.manual_seed(41) + domain = PolynomialZonotope.from_box( + torch.full((2,), -0.2, dtype=torch.float64), + torch.full((2,), 0.2, dtype=torch.float64), + ) + direct = shallow_scalar_hybrid_onejet_reverse( + model, domain, derivative_flatness_threshold=1.0 + ) + dispatched = scalar_hybrid_onejet_reverse( + model, domain, derivative_flatness_threshold=1.0 + ) + assert type(dispatched) is type(direct) + assert torch.equal(dispatched.final.Y.center, direct.final.Y.center) + assert torch.equal(dispatched.final.J.center, direct.final.J.center) + cell = PZIntegrationCell.from_affine_box( + IntervalTensor.from_bounds([-0.2, -0.2], [0.2, 0.2]) + ) + expected = integrate_hybrid_onejet_squared(direct, cell) + actual = integrate_hybrid_onejet_squared(dispatched, cell) + assert float(actual.lower) == float(expected.lower) + assert float(actual.upper) == float(expected.upper) + expected_l2 = integrate_hybrid_value_squared(direct, cell) + actual_l2 = integrate_hybrid_value_squared(dispatched, cell) + assert float(actual_l2.lower) == float(expected_l2.lower) + assert float(actual_l2.upper) == float(expected_l2.upper) diff --git a/tests/test_graph_hilbert.py b/tests/test_graph_hilbert.py new file mode 100644 index 0000000..c67016d --- /dev/null +++ b/tests/test_graph_hilbert.py @@ -0,0 +1,179 @@ +from __future__ import annotations + +from math import sqrt + +import pytest + +torch = pytest.importorskip("torch") +from torch import nn + +from intervalnets import ( + DeepHybridOneJetResult, + HilbertValueCertificate, + IntervalTensor, + PZIntegrationCell, + PolynomialZonotope, + SparseReferenceMomentBackend, + build_factored_jacobian_graph, + build_hilbert_value_certificate, + certify_hybrid_graph_norms, + integrate_hybrid_onejet_squared, + integrate_hybrid_value_squared, + neumann_polynomial_witness, + scalar_hybrid_onejet_reverse, +) +from intervalnets.pz_integration import integrate_pz_value_squared + + +def _deep_model() -> nn.Sequential: + model = nn.Sequential( + nn.Linear(2, 3), + nn.Tanh(), + nn.Linear(3, 2), + nn.Tanh(), + nn.Linear(2, 1), + ).double() + with torch.no_grad(): + model[0].weight.copy_( + torch.tensor([[0.6, -0.2], [0.3, 0.5], [-0.4, 0.25]]) + ) + model[0].bias.copy_(torch.tensor([0.05, -0.1, 0.15])) + model[2].weight.copy_( + torch.tensor([[0.5, -0.25, 0.3], [-0.15, 0.4, 0.35]]) + ) + model[2].bias.copy_(torch.tensor([0.02, -0.04])) + model[4].weight.copy_(torch.tensor([[0.7, -0.45]])) + model[4].bias.copy_(torch.tensor([0.3])) + return model + + +def test_factored_graph_evaluation_preserves_shared_noise_exactly() -> None: + model = _deep_model() + domain = PolynomialZonotope.from_box( + torch.full((2,), -0.2, dtype=torch.float64), + torch.full((2,), 0.2, dtype=torch.float64), + ) + result = scalar_hybrid_onejet_reverse(model, domain) + assert isinstance(result, DeepHybridOneJetResult) + graph = build_factored_jacobian_graph(result.jacobian) + noise = -1.0 + 2.0 * torch.rand( + (128, result.jacobian.num_noise), + generator=torch.Generator().manual_seed(101), + dtype=torch.float64, + ) + assert torch.allclose( + graph.evaluate(noise), result.jacobian.evaluate(noise), rtol=2e-13, atol=2e-13 + ) + # Reusing the exact same noise vector must be deterministic; no graph node + # may silently manufacture an independent copy of a residual symbol. + assert torch.equal(graph.evaluate(noise), graph.evaluate(noise)) + + +def test_sparse_reference_moment_matches_explicit_pz_integral() -> None: + model = _deep_model() + box = IntervalTensor.from_bounds([-0.15, -0.1], [0.15, 0.1]) + cell = PZIntegrationCell.from_affine_box(box) + result = scalar_hybrid_onejet_reverse(model, cell.domain) + assert isinstance(result, DeepHybridOneJetResult) + graph = build_factored_jacobian_graph(result.jacobian) + backend = SparseReferenceMomentBackend(graph) + polynomial = dict(backend.expand()) + zero = (0,) * graph.num_domain_noise + center = polynomial.pop(zero) + pz = PolynomialZonotope( + center, + polynomial, + num_noise=graph.num_domain_noise, + noise_kinds=("domain",) * graph.num_domain_noise, + ) + explicit = integrate_pz_value_squared(pz, cell) + expected = backend.sum_squares() * float(cell.volume) + assert float(explicit.lower) == pytest.approx(expected, rel=5e-12, abs=5e-12) + assert float(explicit.upper) == pytest.approx(expected, rel=5e-12, abs=5e-12) + assert backend.diagnostics["output_terms"] > 1 + + +def test_hilbert_value_compression_contains_sampled_l2_and_is_positive() -> None: + model = _deep_model() + box = IntervalTensor.from_bounds([-0.1, -0.1], [0.1, 0.1]) + cell = PZIntegrationCell.from_affine_box(box) + certificate = build_hilbert_value_certificate( + model, cell.domain, polynomial_degree=5, residual_subdivisions=256 + ) + assert certificate.remainder < certificate.nominal_norm + lower = certificate.nominal_norm - certificate.remainder + upper = certificate.nominal_norm + certificate.remainder + samples = -0.1 + 0.2 * torch.rand( + (20000, 2), generator=torch.Generator().manual_seed(103), dtype=torch.float64 + ) + sampled = sqrt(float(model(samples).square().mean())) + assert 0.0 < lower <= sampled <= upper + assert certificate.moment_states > 0 + + +def test_larger_residual_budget_tightens_hilbert_compression() -> None: + model = _deep_model() + cell = PZIntegrationCell.from_affine_box( + IntervalTensor.from_bounds([-0.1, -0.1], [0.1, 0.1]) + ) + coarse = build_hilbert_value_certificate( + model, cell.domain, polynomial_degree=5, residual_subdivisions=64 + ) + fine = build_hilbert_value_certificate( + model, cell.domain, polynomial_degree=5, residual_subdivisions=256 + ) + assert fine.remainder <= coarse.remainder + assert fine.nominal_norm == pytest.approx(coarse.nominal_norm, rel=2e-14) + + +def test_neumann_witness_is_certified_for_affine_function() -> None: + domain = PolynomialZonotope.from_box( + torch.tensor([-0.1, -0.2], dtype=torch.float64), + torch.tensor([0.1, 0.2], dtype=torch.float64), + ) + value = HilbertValueCertificate( + center=torch.tensor(0.4, dtype=torch.float64), + domain_coefficients=torch.tensor([0.2, -0.1], dtype=torch.float64), + remainder=0.0, + polynomial_degree=1, + residual_subdivisions=1, + layers=(), + moment_states=0, + preactivation_centers=(), + preactivation_coefficients=(), + preactivation_remainders=(), + ) + witness = neumann_polynomial_witness(value, domain) + exact = sqrt( + 0.4**2 + + (0.2**2 + (-0.1) ** 2) / 3.0 + + (0.2 / 0.1) ** 2 + + (-0.1 / 0.2) ** 2 + ) + assert 0.0 < witness.lower_bound <= exact + + +def test_combined_graph_certificate_intersects_old_bounds_and_scales() -> None: + model = _deep_model() + box = IntervalTensor.from_bounds([-0.1, -0.1], [0.1, 0.1]) + cell = PZIntegrationCell.from_affine_box(box) + result = scalar_hybrid_onejet_reverse(model, cell.domain) + old_l2 = integrate_hybrid_value_squared(result, cell) + old_w12 = integrate_hybrid_onejet_squared(result, cell) + certificate = certify_hybrid_graph_norms( + model, + result, + cell, + polynomial_degree=5, + residual_subdivisions=256, + derivative_certificate_subdivisions=32, + ) + assert float(certificate.l2_squared.lower) >= float(old_l2.lower) + assert float(certificate.l2_squared.upper) <= float(old_l2.upper) + assert float(certificate.w12_squared.lower) >= float(old_w12.lower) + assert float(certificate.w12_squared.upper) <= float(old_w12.upper) + volume = float(cell.volume) + normalized_l2_lower = sqrt(float(certificate.l2_squared.lower) / volume) + raw_l2_lower = sqrt(float(certificate.l2_squared.lower)) + assert raw_l2_lower == pytest.approx(sqrt(volume) * normalized_l2_lower) + assert normalized_l2_lower > 0.0 diff --git a/tests/test_pinn.py b/tests/test_pinn.py new file mode 100644 index 0000000..f180c6d --- /dev/null +++ b/tests/test_pinn.py @@ -0,0 +1,87 @@ +import pytest + +torch = pytest.importorskip("torch") + +from intervalnets import load_tanh_mlp_checkpoint, sequential_value_jacobian_laplacian + + +def test_load_tanh_mlp_checkpoint_reconstructs_saved_network(tmp_path): + torch.manual_seed(13) + expected = torch.nn.Sequential( + torch.nn.Linear(4, 6), + torch.nn.Tanh(), + torch.nn.Linear(6, 3), + torch.nn.Tanh(), + torch.nn.Linear(3, 1), + ).double() + checkpoint = tmp_path / "pinn.pt" + torch.save({"state_dict": expected.state_dict(), "seed": 13}, checkpoint) + + loaded = load_tanh_mlp_checkpoint(checkpoint) + x = torch.randn(7, 4, dtype=torch.float64) + + assert isinstance(loaded, torch.nn.Sequential) + assert [type(layer) for layer in loaded] == [ + torch.nn.Linear, + torch.nn.Tanh, + torch.nn.Linear, + torch.nn.Tanh, + torch.nn.Linear, + ] + assert not loaded.training + assert torch.equal(loaded(x), expected(x)) + + +def test_load_tanh_mlp_checkpoint_requires_explicit_regeneration(tmp_path): + with pytest.raises(FileNotFoundError, match="RETRAIN = True"): + load_tanh_mlp_checkpoint(tmp_path / "missing.pt") + + +def test_sequential_value_jacobian_laplacian_matches_autograd(): + torch.manual_seed(17) + model = torch.nn.Sequential( + torch.nn.Linear(3, 5), + torch.nn.Tanh(), + torch.nn.Linear(5, 4), + torch.nn.Tanh(), + torch.nn.Linear(4, 2), + ).double() + x = torch.randn(4, 3, dtype=torch.float64, requires_grad=True) + + value, jacobian, laplacian = sequential_value_jacobian_laplacian(model, x) + + expected_jacobian = [] + expected_laplacian = [] + direct = model(x) + for output in range(direct.shape[-1]): + gradient = torch.autograd.grad( + direct[:, output].sum(), x, create_graph=True, retain_graph=True + )[0] + trace = torch.zeros(x.shape[0], dtype=x.dtype) + for coordinate in range(x.shape[-1]): + second = torch.autograd.grad( + gradient[:, coordinate].sum(), + x, + retain_graph=True, + )[0][:, coordinate] + trace = trace + second + expected_jacobian.append(gradient) + expected_laplacian.append(trace) + + expected_jacobian = torch.stack(expected_jacobian, dim=1) + expected_laplacian = torch.stack(expected_laplacian, dim=1) + assert torch.allclose(value, direct, rtol=1e-12, atol=1e-12) + assert torch.allclose(jacobian, expected_jacobian, rtol=1e-11, atol=1e-12) + assert torch.allclose(laplacian, expected_laplacian, rtol=1e-10, atol=1e-12) + + +def test_laplacian_remains_differentiable_with_respect_to_parameters(): + torch.manual_seed(19) + model = torch.nn.Sequential( + torch.nn.Linear(2, 3), torch.nn.Tanh(), torch.nn.Linear(3, 1) + ).double() + x = torch.randn(6, 2, dtype=torch.float64) + value, _, laplacian = sequential_value_jacobian_laplacian(model, x) + (value.square().mean() + laplacian.square().mean()).backward() + assert all(parameter.grad is not None for parameter in model.parameters()) + assert all(torch.isfinite(parameter.grad).all() for parameter in model.parameters()) diff --git a/tests/test_polynomial_zonotope.py b/tests/test_polynomial_zonotope.py new file mode 100644 index 0000000..251ef1f --- /dev/null +++ b/tests/test_polynomial_zonotope.py @@ -0,0 +1,524 @@ +import pytest + +from intervalnets import PZTwoJet, PolynomialZonotope, collect_pz_diagnostics + +try: + import torch +except ImportError: # pragma: no cover + torch = None + + +def test_from_box_interval_enclosure_fallback(): + z = PolynomialZonotope.from_box((-1.0, 2.0), (3.0, 4.0)) + assert z.num_noise == 2 + assert z.shape == (2,) + enclosure = z.interval_enclosure() + assert enclosure.lower[0] <= -1.0 <= enclosure.upper[0] + assert enclosure.lower[1] <= 2.0 <= enclosure.upper[1] + assert enclosure.lower[0] <= 3.0 <= enclosure.upper[0] + assert enclosure.lower[1] <= 4.0 <= enclosure.upper[1] + + +def test_addition_merges_equal_exponents(): + z1 = PolynomialZonotope(1.0, {(1,): 2.0}, num_noise=1) + z2 = PolynomialZonotope(3.0, {(1,): 4.0}, num_noise=1) + out = z1 + z2 + assert out.center == 4.0 + assert out.terms[(1,)] == 6.0 + + +def test_constructor_sums_duplicate_canonical_exponents(): + z = PolynomialZonotope(0.0, {(1,): 2.0, (1, 0): 3.0}, num_noise=2) + + assert z.terms == {(1, 0): 5.0} + + +def test_constructor_prunes_cancelled_monomials_after_duplicate_merge(): + z = PolynomialZonotope(0.0, {(1,): 2.0, (1, 0): -2.0, (0, 1): 4.0}, num_noise=2) + + assert z.terms == {(0, 1): 4.0} + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_constructor_prunes_all_zero_torch_tensor_coefficients(): + z = PolynomialZonotope( + torch.zeros(2, dtype=torch.float64), + { + (1,): torch.zeros(2, dtype=torch.float64), + (2,): torch.tensor([0.0, 1.0], dtype=torch.float64), + }, + num_noise=1, + ) + + assert (1,) not in z.terms + assert torch.allclose(z.terms[(2,)], torch.tensor([0.0, 1.0], dtype=torch.float64)) + + +def test_constructor_keeps_exact_nonzero_coefficients_without_tolerance_pruning(): + z = PolynomialZonotope(0.0, {(1,): 1e-300}, num_noise=1) + + assert z.terms == {(1,): 1e-300} + + +def test_operations_prune_terms_that_cancel_to_zero(): + z = PolynomialZonotope(0.0, {(1,): 2.0}, num_noise=1) + neg_z = PolynomialZonotope(0.0, {(1,): -2.0}, num_noise=1) + vector = PolynomialZonotope((0.0,), {(1,): (2.0,)}, num_noise=1) + + assert (z + neg_z).terms == {} + assert (z * 0.0).terms == {} + assert vector.linear_map(((0.0,),)).terms == {} + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_stack_and_tensor_product_prune_zero_terms_via_constructor(): + zero = PolynomialZonotope(torch.tensor(0.0), {(1,): torch.tensor(0.0)}, num_noise=1) + nonzero = PolynomialZonotope(torch.tensor(1.0), {(1,): torch.tensor(2.0)}, num_noise=1) + + stacked = PolynomialZonotope.stack((zero, zero)) + product = zero.tensor_product(nonzero) + + assert stacked.terms == {} + assert product.terms == {} + + +def test_scalar_polynomial_multiplication_convolves_exponents(): + z = PolynomialZonotope(1.0, {(1,): 2.0}, num_noise=1) + out = z * z + assert out.center == 1.0 + assert out.terms[(1,)] == 4.0 + assert out.terms[(2,)] == 4.0 + + +def test_fallback_tuple_backed_matrix_getitem_indexes_recursively(): + z = PolynomialZonotope( + ((1.0, 2.0), (3.0, 4.0)), + {(1,): ((0.1, 0.2), (0.3, 0.4))}, + num_noise=1, + ) + + out = z[0, 1] + + assert out.shape == () + assert out.center == 2.0 + assert out.terms == {(1,): 0.2} + assert out.num_noise == z.num_noise + assert out.noise_kinds == z.noise_kinds + + +def test_collect_pz_diagnostics_records_polynomial_multiplication_complexity(): + left = PolynomialZonotope(1.0, {(1,): 2.0, (2,): 3.0}, num_noise=1) + right = PolynomialZonotope(3.0, {(1,): 4.0}, num_noise=1) + + _ = left * right + + with collect_pz_diagnostics("unit") as records: + _ = left * right + + assert records == [ + { + "phase": "unit", + "left_term_count": 2, + "right_term_count": 1, + "raw_pair_count": 2, + "output_term_count": 3, + "coefficient_shape": (), + "max_output_degree": 3, + } + ] + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_torch_scalar_times_vector_and_stack_and_tensor_product(): + scalar = PolynomialZonotope(torch.tensor(2.0), {(1,): torch.tensor(3.0)}, num_noise=1) + vector = PolynomialZonotope.constant(torch.tensor([1.0, -1.0]), num_noise=1) + product = scalar * vector + assert product.shape == (2,) + assert torch.allclose(product.center, torch.tensor([2.0, -2.0])) + assert torch.allclose(product.terms[(1,)], torch.tensor([3.0, -3.0])) + + stacked = PolynomialZonotope.stack((product[0], product[1])) + assert stacked.shape == (2,) + assert torch.allclose(stacked.center, product.center) + + outer = vector.tensor_product(vector) + assert outer.shape == (2, 2) + assert torch.allclose(outer.center, torch.tensor([[1.0, -1.0], [-1.0, 1.0]])) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_torch_box_enclosure_contains_corners(): + lower = torch.tensor([-2.0, 1.0]) + upper = torch.tensor([4.0, 5.0]) + z = PolynomialZonotope.from_box(lower, upper) + interval = z.interval_enclosure() + lo, hi = interval.to_torch(dtype=torch.float64) + assert torch.all(lo <= lower.double()) + assert torch.all(hi >= upper.double()) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_twojet_from_input_initializes_physical_input_derivatives(): + lower = torch.tensor([-1.0, 2.0], dtype=torch.float64) + upper = torch.tensor([3.0, 4.0], dtype=torch.float64) + X = PolynomialZonotope.from_box(lower, upper) + + jet = PZTwoJet.from_input(X, input_dim=2) + + assert jet.Y is X + assert jet.J.num_noise == X.num_noise + assert jet.H.num_noise == X.num_noise + assert jet.J.shape == (2, 2) + assert jet.H.shape == (2, 2, 2) + assert jet.J.terms == {} + assert jet.H.terms == {} + assert torch.allclose(jet.J.center, torch.eye(2, dtype=torch.float64)) + assert torch.allclose(jet.H.center, torch.zeros(2, 2, 2, dtype=torch.float64)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_linear_map_contracts_value_jacobian_and_hessian_shapes(): + center = torch.tensor([1.0, -2.0], dtype=torch.float64) + coeff = torch.tensor([0.5, 1.5], dtype=torch.float64) + z = PolynomialZonotope(center, {(1,): coeff}, num_noise=1) + weight = torch.tensor([[2.0, -1.0], [0.0, 3.0], [1.0, 1.0]], dtype=torch.float64) + bias = torch.tensor([0.25, -0.5, 1.0], dtype=torch.float64) + + out = z.linear_map(weight, bias) + + assert out.shape == (3,) + assert torch.allclose(out.center, weight.matmul(center) + bias) + assert torch.allclose(out.terms[(1,)], weight.matmul(coeff)) + + jac = PolynomialZonotope.constant(torch.arange(6, dtype=torch.float64).reshape(2, 3), num_noise=1) + jac_out = jac.linear_map(weight, bias=None) + assert jac_out.shape == (3, 3) + assert torch.allclose(jac_out.center, torch.einsum("ij,jk->ik", weight, jac.center)) + + hess = PolynomialZonotope.constant(torch.arange(18, dtype=torch.float64).reshape(2, 3, 3), num_noise=1) + hess_out = hess.linear_map(weight, bias=None) + assert hess_out.shape == (3, 3, 3) + assert torch.allclose(hess_out.center, torch.einsum("ij,jkl->ikl", weight, hess.center)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_twojet_linear_forward_matches_layer_affine_map(): + from torch import nn + from intervalnets.pytorch import _pz_twojet_linear_forward + + layer = nn.Linear(2, 3, dtype=torch.float64) + with torch.no_grad(): + layer.weight.copy_(torch.tensor([[1.0, 2.0], [-1.0, 0.5], [3.0, -2.0]], dtype=torch.float64)) + layer.bias.copy_(torch.tensor([0.1, -0.2, 0.3], dtype=torch.float64)) + + X = PolynomialZonotope.from_box(torch.tensor([-1.0, 0.0], dtype=torch.float64), torch.tensor([1.0, 2.0], dtype=torch.float64)) + jet = PZTwoJet.from_input(X, input_dim=2) + out = _pz_twojet_linear_forward(layer, jet) + + assert out.Y.shape == (3,) + assert out.J.shape == (3, 2) + assert out.H.shape == (3, 2, 2) + assert torch.allclose(out.Y.center, layer.weight.detach().matmul(X.center) + layer.bias.detach()) + assert torch.allclose(out.J.center, layer.weight.detach()) + assert torch.allclose(out.H.center, torch.zeros(3, 2, 2, dtype=torch.float64)) + for exp, coeff in X.terms.items(): + assert torch.allclose(out.Y.terms[exp], layer.weight.detach().matmul(coeff)) + + +def test_evaluate_polynomial_uses_power_basis_and_horner_dependencies(): + z = PolynomialZonotope(1.0, {(1,): 2.0}, num_noise=1) + out = z.evaluate_polynomial((3.0, 4.0, 5.0)) + expected = 3.0 + 4.0 * z + 5.0 * z * z + assert out.center == expected.center + assert out.terms == expected.terms + + +def test_add_independent_error_extends_existing_exponents(): + z = PolynomialZonotope(1.0, {(1, 2): 3.0}, num_noise=2) + out = z.add_independent_error(0.25) + assert out.num_noise == 3 + assert out.center == 1.0 + assert out.terms[(1, 2, 0)] == 3.0 + assert out.terms[(0, 0, 1)] == 0.25 + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_add_independent_errors_uses_one_symbol_per_tensor_entry(): + value = PolynomialZonotope.constant( + torch.zeros(2, dtype=torch.float64), + num_noise=1, + noise_kinds=("domain",), + ) + + out = value.add_independent_errors( + torch.tensor([0.1, 0.2], dtype=torch.float64), + kind="approximation_pointwise", + ) + + assert out.num_noise == 3 + assert out.noise_kinds == ( + "domain", + "approximation_pointwise", + "approximation_pointwise", + ) + assert torch.equal( + out.terms[(0, 1, 0)], + torch.tensor([0.1, 0.0], dtype=torch.float64), + ) + assert torch.equal( + out.terms[(0, 0, 1)], + torch.tensor([0.0, 0.2], dtype=torch.float64), + ) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_stack_aligns_sliced_scalars_and_merges_exponents(): + z1 = PolynomialZonotope(torch.tensor([1.0, 2.0], dtype=torch.float64), {(1,): torch.tensor([0.5, 1.5], dtype=torch.float64)}, num_noise=1) + z2 = PolynomialZonotope(torch.tensor(3.0, dtype=torch.float64), {(0, 1): torch.tensor(2.0, dtype=torch.float64)}, num_noise=2) + stacked = PolynomialZonotope.stack((z1[1], z2)) + assert stacked.num_noise == 2 + assert stacked.shape == (2,) + assert torch.allclose(stacked.center, torch.tensor([2.0, 3.0], dtype=torch.float64)) + assert torch.allclose(stacked.terms[(1, 0)], torch.tensor([1.5, 0.0], dtype=torch.float64)) + assert torch.allclose(stacked.terms[(0, 1)], torch.tensor([0.0, 2.0], dtype=torch.float64)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_tanh_pz_scalar_adds_certified_fresh_noise_and_encloses_samples(): + from intervalnets.pz_tanh import tanh_pz_scalar + + z = PolynomialZonotope.from_box(torch.tensor(-0.5, dtype=torch.float64), torch.tensor(0.75, dtype=torch.float64)) + out = tanh_pz_scalar(z, chebyshev_degree=5, residual_subdivisions=64) + assert out.shape == () + assert out.num_noise == z.num_noise + 1 + assert any(exp[-1] == 1 for exp in out.terms) + enclosure = out.interval_enclosure() + lo, hi = enclosure.to_torch(dtype=torch.float64) + for value in torch.linspace(-0.5, 0.75, steps=9, dtype=torch.float64): + expected = torch.tanh(value) + assert lo <= expected <= hi + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_twojet_tanh_forward_preserves_shapes_and_encloses_autograd_samples(): + from intervalnets.pytorch import _pz_twojet_tanh_forward + + lower = torch.tensor([-0.4, 0.2], dtype=torch.float64) + upper = torch.tensor([0.6, 0.8], dtype=torch.float64) + X = PolynomialZonotope.from_box(lower, upper) + jet = PZTwoJet.from_input(X, input_dim=2) + + out = _pz_twojet_tanh_forward(jet, chebyshev_degree=5, residual_subdivisions=64) + + assert out.Y.shape == (2,) + assert out.J.shape == (2, 2) + assert out.H.shape == (2, 2, 2) + # Each scalar tanh two-jet component introduces three fresh pointwise + # residual variables: one each for tanh, tanh', and tanh''. + assert out.Y.num_noise == X.num_noise + 3 * 2 + assert out.J.num_noise == out.Y.num_noise + assert out.H.num_noise == out.Y.num_noise + assert out.Y.noise_kinds.count("approximation_pointwise") == 6 + assert out.J.noise_kinds.count("approximation_pointwise") == 6 + assert out.H.noise_kinds.count("approximation_pointwise") == 6 + + y_lo, y_hi = out.Y.interval_enclosure().to_torch(dtype=torch.float64) + j_lo, j_hi = out.J.interval_enclosure().to_torch(dtype=torch.float64) + h_lo, h_hi = out.H.interval_enclosure().to_torch(dtype=torch.float64) + + for x0 in torch.linspace(float(lower[0]), float(upper[0]), steps=5, dtype=torch.float64): + for x1 in torch.linspace(float(lower[1]), float(upper[1]), steps=5, dtype=torch.float64): + point = torch.stack((x0, x1)).requires_grad_(True) + value = torch.tanh(point) + rows = [] + hessians = [] + for i in range(2): + grad = torch.autograd.grad(value[i], point, create_graph=True, retain_graph=True)[0] + rows.append(grad) + h_rows = [] + for j in range(2): + h_rows.append(torch.autograd.grad(grad[j], point, retain_graph=True)[0]) + hessians.append(torch.stack(h_rows)) + jac = torch.stack(rows) + hess = torch.stack(hessians) + assert torch.all(y_lo <= value.detach()) and torch.all(value.detach() <= y_hi) + assert torch.all(j_lo <= jac.detach()) and torch.all(jac.detach() <= j_hi) + assert torch.all(h_lo <= hess.detach()) and torch.all(hess.detach() <= h_hi) + + +def test_noise_metadata_defaults_and_validation(): + z = PolynomialZonotope(0.0, {(1, 0): 2.0}, num_noise=2) + assert z.noise_kinds == ("unknown", "unknown") + with pytest.raises(ValueError, match="noise_kinds length"): + PolynomialZonotope(0.0, {(1,): 2.0}, num_noise=1, noise_kinds=("domain", "extra")) + + +def test_domain_noise_stays_first_when_approximation_error_is_appended_fallback(): + z = PolynomialZonotope.from_box((-1.0, 2.0), (3.0, 4.0)) + out = z.add_independent_error(0.25) + assert out.noise_kinds == ("domain", "domain", "approximation") + assert out.terms[(1, 0, 0)] == z.terms[(1, 0)] + assert out.terms[(0, 1, 0)] == z.terms[(0, 1)] + assert out.terms[(0, 0, 1)] == (0.25, 0.25) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_tanh_residual_noise_is_appended_and_labeled_after_domain_noise(): + from intervalnets.pz_tanh import tanh_pz_scalar + + z = PolynomialZonotope.from_box(torch.tensor(-0.5, dtype=torch.float64), torch.tensor(0.75, dtype=torch.float64)) + out = tanh_pz_scalar(z, chebyshev_degree=5, residual_subdivisions=64) + + assert z.noise_kinds == ("domain",) + assert out.noise_kinds == ("domain", "approximation_pointwise") + assert any(exp == (0, 1) for exp in out.terms) + + +def test_box_monomial_moment_even_and_odd_exponents(): + from intervalnets.polynomial_zonotope import box_monomial_moment + + assert box_monomial_moment((2, 0)) == pytest.approx(4.0 / 3.0) + assert box_monomial_moment((2, 4)) == pytest.approx(4.0 / 15.0) + assert box_monomial_moment((1, 2)) == 0.0 + + +def test_integrate_noise_scalar_cancels_odd_and_merges_even_terms_fallback(): + z = PolynomialZonotope( + 1.0, + { + (2, 0): 3.0, + (0, 1): 5.0, + (2, 1): 7.0, + (1, 0): 11.0, + }, + num_noise=2, + noise_kinds=("domain", "approximation"), + ) + + out = z.integrate_noise([0]) + + assert out.num_noise == 1 + assert out.noise_kinds == ("approximation",) + assert out.center == pytest.approx(3.0) # 1 + 3 * int_{-1}^1 x^2 dx + assert out.terms == {(1,): pytest.approx(44.0 / 3.0)} # 5*2 + 7*(2/3) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_integrate_noise_vector_matrix_and_tensor_coefficients_preserve_metadata_dtype_device(): + dtype = torch.float64 + vector = PolynomialZonotope( + torch.tensor([1.0, 2.0], dtype=dtype), + { + (2, 0): torch.tensor([3.0, 6.0], dtype=dtype), + (0, 1): torch.tensor([5.0, 7.0], dtype=dtype), + (2, 1): torch.tensor([9.0, 12.0], dtype=dtype), + (1, 1): torch.tensor([100.0, 200.0], dtype=dtype), + }, + num_noise=2, + noise_kinds=("domain", "approximation"), + ) + vector_out = vector.integrate_domain_noise() + + assert vector_out.shape == (2,) + assert vector_out.dtype == dtype + assert vector_out.device == vector.center.device + assert vector_out.noise_kinds == ("approximation",) + assert torch.allclose(vector_out.center, torch.tensor([3.0, 6.0], dtype=dtype)) + assert torch.allclose(vector_out.terms[(1,)], torch.tensor([16.0, 22.0], dtype=dtype)) + assert (0,) not in vector_out.terms + + matrix_coeff = torch.arange(4, dtype=dtype).reshape(2, 2) + matrix = PolynomialZonotope( + torch.ones(2, 2, dtype=dtype), + {(0, 2): matrix_coeff, (1, 0): torch.full((2, 2), 99.0, dtype=dtype)}, + num_noise=2, + noise_kinds=("approximation", "domain"), + ) + matrix_out = matrix.integrate_domain_noise() + assert matrix_out.shape == (2, 2) + assert matrix_out.noise_kinds == ("approximation",) + assert torch.allclose(matrix_out.center, torch.ones(2, 2, dtype=dtype) + matrix_coeff * (2.0 / 3.0)) + assert torch.allclose(matrix_out.terms[(1,)], torch.full((2, 2), 198.0, dtype=dtype)) + + tensor_coeff = torch.arange(24, dtype=dtype).reshape(2, 3, 4) + tensor = PolynomialZonotope( + torch.zeros(2, 3, 4, dtype=dtype), + {(0, 2, 1): tensor_coeff, (0, 0, 1): torch.ones(2, 3, 4, dtype=dtype)}, + num_noise=3, + noise_kinds=("approximation", "domain", "approximation"), + ) + tensor_out = tensor.integrate_domain_noise() + assert tensor_out.shape == (2, 3, 4) + assert tensor_out.noise_kinds == ("approximation", "approximation") + assert torch.allclose(tensor_out.terms[(0, 1)], tensor_coeff * (2.0 / 3.0) + torch.ones(2, 3, 4, dtype=dtype) * 2.0) + + +def test_pz_to_latex_scalar_terms_render_variable_powers(): + from intervalnets import pz_to_latex + + z = PolynomialZonotope( + 1.0, + {(2, 0): 3.0, (0, 1): -1.0}, + num_noise=2, + noise_kinds=("domain", "unknown"), + ) + + rendered = pz_to_latex(z) + + assert "1" in rendered + assert r"3 \xi_{1}^{2}" in rendered + assert r"- \epsilon_{2}" in rendered + + +def test_pz_to_latex_vector_and_tensor_entries_include_indices_fallback(): + from intervalnets import pz_to_latex, twojet_to_latex + + vector = PolynomialZonotope((1.0, 2.0), {(1,): (0.5, 1.5)}, num_noise=1) + rendered_vector = pz_to_latex(vector) + assert "Z_{0} &= 1" in rendered_vector + assert "Z_{1} &= 2" in rendered_vector + + y = PolynomialZonotope((1.0, 2.0), {}, num_noise=1) + j = PolynomialZonotope(((1.0, 0.0), (0.0, 1.0)), {(1,): ((2.0, 0.0), (0.0, 3.0))}, num_noise=1) + h = PolynomialZonotope( + (((0.0, 0.0), (0.0, 0.0)), ((0.0, 0.0), (0.0, 0.0))), + {(1,): (((4.0, 0.0), (0.0, 0.0)), ((0.0, 0.0), (0.0, 5.0)))}, + num_noise=1, + ) + rendered_jet = twojet_to_latex(PZTwoJet(y, j, h)) + assert "Y_{0}" in rendered_jet + assert "J_{0,0}" in rendered_jet + assert "H_{1,1,1}" in rendered_jet + + +def test_pz_to_latex_distinguishes_domain_approximation_and_unknown_noise(): + from intervalnets import pz_to_latex + + z = PolynomialZonotope( + 0.0, + {(1, 0, 0): 1.0, (0, 1, 0): 2.0, (0, 0, 1): 3.0}, + num_noise=3, + noise_kinds=("domain", "approximation_pointwise", "unknown"), + ) + + rendered = pz_to_latex(z) + + assert r"\xi_{1}" in rendered + assert r"\eta_{2}" in rendered + assert r"\epsilon_{3}" in rendered + + +def test_pz_to_latex_truncation_reports_omitted_terms(): + from intervalnets import pz_to_latex, pz_to_markdown_code + + z = PolynomialZonotope( + 0.0, + {(1, 0, 0): 1.0, (0, 1, 0): 2.0, (0, 0, 1): 3.0}, + num_noise=3, + ) + + rendered = pz_to_latex(z, max_terms=1) + + assert "omitted terms" in rendered + assert "2 omitted terms" in rendered + assert pz_to_markdown_code(z, max_terms=1).startswith("```latex\n") diff --git a/tests/test_pytorch.py b/tests/test_pytorch.py index 3f7c401..44c4211 100644 --- a/tests/test_pytorch.py +++ b/tests/test_pytorch.py @@ -13,7 +13,58 @@ interval_forward, interval_forward_refine, ) -from intervalnets.pytorch import _eval_jacobian_bounds, _interval_pow_scalar +from intervalnets.pytorch import ( + PZReductionConfig, + _CertifiedTermReducer, + _eval_hessian_bounds, + _eval_jacobian_bounds, + _interval_pow_scalar, + _lookahead_sobolev_split_dimension, +) + + +def _finish_reference_reducer(variant: str, generator_budget: int = 0): + center = torch.zeros(2, dtype=torch.float64) + reducer = _CertifiedTermReducer( + PZReductionConfig( + strategy="topk", + max_terms=1, + reduction_variant=variant, + generator_budget=generator_budget, + ), + tuple(center.shape), + center, + ) + reducer.offer((1,), torch.tensor([10.0, 0.0], dtype=torch.float64)) + reducer.offer((2,), torch.tensor([2.0, -4.0], dtype=torch.float64)) + return reducer.finish(center, num_noise=1, noise_kinds=("domain",)) + + +def test_reference_reduction_variants_use_even_monomial_range() -> None: + reduced_a, radius_a = _finish_reference_reducer("A") + reduced_b, radius_b = _finish_reference_reducer("B") + reduced_c0, radius_c0 = _finish_reference_reducer("C", generator_budget=0) + + assert torch.equal(reduced_a.center, torch.tensor([0.0, 0.0], dtype=torch.float64)) + assert torch.allclose(radius_a, torch.tensor([2.0, 4.0], dtype=torch.float64)) + assert torch.equal(reduced_b.center, torch.tensor([1.0, -2.0], dtype=torch.float64)) + assert torch.allclose(radius_b, torch.tensor([1.0, 2.0], dtype=torch.float64)) + assert torch.equal(reduced_c0.center, reduced_b.center) + assert torch.equal(radius_c0, radius_b) + + +def test_reference_variant_c_retains_fresh_correlated_generator() -> None: + reduced, radius = _finish_reference_reducer("C", generator_budget=1) + + assert torch.equal(reduced.center, torch.tensor([1.0, -2.0], dtype=torch.float64)) + assert torch.equal(radius, torch.zeros(2, dtype=torch.float64)) + assert reduced.num_noise == 2 + assert reduced.noise_kinds == ("domain", "approximation_pointwise") + assert any( + exponent == (0, 1) + and torch.equal(coefficient, torch.tensor([1.0, -2.0], dtype=torch.float64)) + for exponent, coefficient in reduced.terms.items() + ) def test_relu_negative_interval_rounds_outward_to_zero() -> None: @@ -99,6 +150,24 @@ def test_slope_enclosure_tightens_relu_dependency_example() -> None: assert slope_bounds.upper[0] <= 1.0 + 1e-6 +def test_lookahead_sobolev_split_selects_the_influential_coordinate() -> None: + enable_interval_eval() + model = nn.Linear(2, 1, bias=False).to(dtype=torch.float64) + with torch.no_grad(): + model.weight.copy_(torch.tensor([[1.0, 0.0]], dtype=torch.float64)) + box = IntervalTensor.from_bounds([0.0, -1.0], [2.0, 1.0]) + + split_dim, children, contributions, candidates = ( + _lookahead_sobolev_split_dimension(model, box) + ) + + assert split_dim == 0 + assert len(children) == len(contributions) == 2 + assert [row["split_dim"] for row in candidates] == [0, 1] + assert candidates[0]["children_width_sum"] < candidates[1]["children_width_sum"] + assert candidates[0]["predicted_width_reduction"] > 0.0 + + def test_slope_enclosure_remains_valid_on_sampled_points() -> None: torch.manual_seed(11) model = nn.Sequential(nn.Linear(2, 6), nn.ReLU(), nn.Linear(6, 1)) @@ -218,24 +287,6 @@ def test_enable_interval_eval_defaults_to_slope_mode() -> None: assert result.upper[0] <= 1.0 + 1e-6 -def test_linear_interval_matches_expected_affine_bounds() -> None: - layer = nn.Linear(2, 1) - with torch.no_grad(): - layer.weight.copy_(torch.tensor([[2.0, -3.0]])) - layer.bias.copy_(torch.tensor([0.5])) - - interval = IntervalTensor.from_bounds([1.0, 2.0], [1.5, 2.5]) - output = interval_forward(layer, interval) - - candidates = [ - 2.0 * x1 - 3.0 * x2 + 0.5 - for x1 in [1.0, 1.5] - for x2 in [2.0, 2.5] - ] - assert output.lower[0] <= min(candidates) - assert output.upper[0] >= max(candidates) - - def test_zero_network_contains_zero_with_rounding_margin() -> None: layer = nn.Linear(3, 2) with torch.no_grad(): @@ -280,7 +331,6 @@ def test_eval_overload_runs_interval_propagation() -> None: assert result.upper[0] >= 1.125 - def test_softmax_bounds_match_closed_form_in_two_dimensions() -> None: softmax = nn.Softmax(dim=-1) interval = IntervalTensor.from_bounds([-1.0, 0.25], [0.5, 1.75]) @@ -694,6 +744,52 @@ def test_eval_jacobian_dead_relu_path_stays_exact_zero() -> None: assert jacobian.upper[0][0] == 0.0 +def test_eval_hessian_linear_layer_is_exact_zero_tensor() -> None: + enable_interval_eval() + layer = nn.Linear(2, 1) + with torch.no_grad(): + layer.weight.copy_(torch.tensor([[1.5, -0.5]])) + layer.bias.copy_(torch.tensor([0.2])) + + domain = IntervalTensor.from_bounds([-1.0, -2.0], [0.5, 3.0]) + hessian = layer.eval_hessian(domain) + + assert hessian.lower[0][0][0] == 0.0 + assert hessian.upper[0][0][0] == 0.0 + assert hessian.lower[0][0][1] == 0.0 + assert hessian.upper[0][0][1] == 0.0 + assert hessian.lower[0][1][0] == 0.0 + assert hessian.upper[0][1][0] == 0.0 + assert hessian.lower[0][1][1] == 0.0 + assert hessian.upper[0][1][1] == 0.0 + + +def test_eval_hessian_tanh_network_encloses_corner_second_derivatives() -> None: + enable_interval_eval() + model = nn.Sequential(nn.Linear(2, 1, bias=False), nn.Tanh()) + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[1.25, -0.75]])) + + domain = IntervalTensor.from_bounds([-0.4, -0.2], [0.5, 0.6]) + hessian = _eval_hessian_bounds(model, domain, enclosure_mode="slope") + + # For z = w·x and y=tanh(z), Hessian(y) = tanh''(z) * (w ⊗ w) + weights = model[0].weight.detach().to(torch.float64)[0] + for x0 in (domain.lower[0], domain.upper[0]): + for x1 in (domain.lower[1], domain.upper[1]): + z = float(weights[0]) * x0 + float(weights[1]) * x1 + tanh_z = math.tanh(z) + tanh_second = -2.0 * tanh_z * (1.0 - tanh_z * tanh_z) + expected_00 = tanh_second * float(weights[0]) * float(weights[0]) + expected_01 = tanh_second * float(weights[0]) * float(weights[1]) + expected_11 = tanh_second * float(weights[1]) * float(weights[1]) + + assert hessian.lower[0][0][0] <= expected_00 <= hessian.upper[0][0][0] + assert hessian.lower[0][0][1] <= expected_01 <= hessian.upper[0][0][1] + assert hessian.lower[0][1][0] <= expected_01 <= hessian.upper[0][1][0] + assert hessian.lower[0][1][1] <= expected_11 <= hessian.upper[0][1][1] + + def test_sobolev_norm_constant_network_matches_closed_form() -> None: enable_interval_eval() model = nn.Sequential(nn.Linear(1, 1)) @@ -709,6 +805,36 @@ def test_sobolev_norm_constant_network_matches_closed_form() -> None: assert (bounds.upper - bounds.lower) < 1e-10 +def test_sobolev_norm_order_one_matches_default_behavior() -> None: + enable_interval_eval() + model = nn.Sequential(nn.Linear(1, 1)) + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[0.75]])) + model[0].bias.copy_(torch.tensor([0.1])) + + domain = IntervalTensor.from_bounds([-1.0], [1.0]) + default_order = model.sobolev_norm(domain, p=2.0, iterations=3) + order_one = model.sobolev_norm(domain, p=2.0, order=1, iterations=3) + + assert order_one.lower <= default_order.upper + assert default_order.lower <= order_one.upper + + +def test_sobolev_norm_order_two_is_at_least_order_one_for_tanh_model() -> None: + enable_interval_eval() + model = nn.Sequential(nn.Linear(1, 1), nn.Tanh()) + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[1.2]])) + model[0].bias.copy_(torch.tensor([0.0])) + + domain = IntervalTensor.from_bounds([-0.7], [0.9]) + w12 = model.sobolev_norm(domain, p=2.0, order=1, iterations=4) + w22 = model.sobolev_norm(domain, p=2.0, order=2, iterations=4) + + assert w22.lower >= w12.lower + assert w22.upper >= w12.upper + + def test_sobolev_norm_refinement_tightens_interval() -> None: enable_interval_eval() model = nn.Sequential(nn.Linear(1, 6), nn.ReLU(), nn.Linear(6, 1)) @@ -848,6 +974,8 @@ def test_sobolev_norm_rejects_invalid_parameters() -> None: _ = model.sobolev_norm(domain, p=float("inf"), iterations=0) with pytest.raises(ValueError): _ = model.sobolev_norm(domain, p=2.0, iterations=-1) + with pytest.raises(ValueError): + _ = model.sobolev_norm(domain, p=2.0, order=3, iterations=0) with pytest.raises(ValueError): _ = model.sobolev_norm(domain, p=2.0, iterations=0, forward_refine_splits=0) @@ -873,6 +1001,13 @@ def test_eval_jacobian_requires_interval_tensor_domain() -> None: _ = model.eval_jacobian([0.0, 1.0]) +def test_eval_hessian_requires_interval_tensor_domain() -> None: + enable_interval_eval() + model = nn.Sequential(nn.Linear(1, 1), nn.Tanh()) + with pytest.raises(TypeError): + _ = model.eval_hessian([0.0, 1.0]) + + def test_softmax_jacobian_encloses_autograd_corner_gradients() -> None: enable_interval_eval() softmax = nn.Softmax(dim=-1) @@ -937,3 +1072,390 @@ def test_tanh_jacobian_encloses_autograd_corner_gradients() -> None: for col in range(2): exact = float(grad[col].item()) assert jacobian.lower[row][col] <= exact <= jacobian.upper[row][col] + + +def test_pz_twojet_forward_sequential_linear_tanh_identity_returns_twojet() -> None: + from intervalnets import PZTwoJet, PolynomialZonotope, pz_twojet_forward + + torch.manual_seed(0) + model = nn.Sequential(nn.Identity(), nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)) + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, 0.1], dtype=torch.float64), + torch.tensor([0.3, 0.4], dtype=torch.float64), + ) + + out = pz_twojet_forward(model.double(), domain, residual_subdivisions=32) + + assert isinstance(out, PZTwoJet) + assert out.Y.shape == (1,) + assert out.J.shape == (1, 2) + assert out.H.shape == (1, 2, 2) + + +def test_pz_value_forward_matches_twojet_value_enclosure() -> None: + from intervalnets import PolynomialZonotope, pz_twojet_forward, pz_value_forward + + torch.manual_seed(0) + model = nn.Sequential( + nn.Identity(), + nn.Linear(2, 3), + nn.Tanh(), + nn.Linear(3, 1), + ).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, 0.1], dtype=torch.float64), + torch.tensor([0.3, 0.4], dtype=torch.float64), + ) + + value = pz_value_forward(model, domain) + twojet_value = pz_twojet_forward(model, domain).Y + value_interval = value.interval_enclosure() + twojet_interval = twojet_value.interval_enclosure() + + assert value.shape == (1,) + assert len(value.terms) == len(twojet_value.terms) + assert value.num_noise == 2 + 3 + assert torch.allclose( + torch.tensor(value_interval.lower), + torch.tensor(twojet_interval.lower), + ) + assert torch.allclose( + torch.tensor(value_interval.upper), + torch.tensor(twojet_interval.upper), + ) + assert value.num_noise < twojet_value.num_noise + + +def test_pz_value_forward_trace_reports_each_layer() -> None: + from intervalnets import PZValueTraceResult, PolynomialZonotope, pz_value_forward + + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-1.0, -0.5], dtype=torch.float64), + torch.tensor([1.0, 0.5], dtype=torch.float64), + ) + + traced = pz_value_forward(model, domain, return_trace=True) + + assert isinstance(traced, PZValueTraceResult) + assert traced.final.shape == (1,) + assert [record.layer_type for record in traced.records] == [ + "Input", + "Linear", + "Tanh", + "Linear", + ] + assert traced.records[-1].summary["term_count"] == len(traced.final.terms) + + +def test_enable_interval_eval_adds_eval_pz_twojet_method() -> None: + from intervalnets import PZTwoJet, PolynomialZonotope + + enable_interval_eval() + layer = nn.Linear(2, 1).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-1.0, -0.5], dtype=torch.float64), + torch.tensor([1.0, 0.5], dtype=torch.float64), + ) + + out = layer.eval_pz_twojet(domain) + + assert isinstance(out, PZTwoJet) + assert out.Y.shape == (1,) + assert out.J.shape == (1, 2) + assert out.H.shape == (1, 2, 2) + + +def test_enable_interval_eval_adds_eval_pz_value_method() -> None: + from intervalnets import PolynomialZonotope + + enable_interval_eval() + layer = nn.Linear(2, 1).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-1.0, -0.5], dtype=torch.float64), + torch.tensor([1.0, 0.5], dtype=torch.float64), + ) + + out = layer.eval_pz_value(domain) + + assert out.shape == (1,) + assert out.num_noise == 2 + + +def test_pz_onejet_forward_encloses_sampled_jacobians() -> None: + from intervalnets import PZOneJet, PolynomialZonotope, pz_onejet_forward + + torch.manual_seed(17) + model = nn.Sequential( + nn.Linear(3, 5), + nn.Tanh(), + nn.Linear(5, 4), + nn.Tanh(), + nn.Linear(4, 1), + ).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, -0.1, 0.0], dtype=torch.float64), + torch.tensor([0.3, 0.2, 0.4], dtype=torch.float64), + ) + + onejet = pz_onejet_forward(model, domain) + enclosure = onejet.J.interval_enclosure() + + assert isinstance(onejet, PZOneJet) + assert onejet.Y.shape == (1,) + assert onejet.J.shape == (1, 3) + assert onejet.Y.num_noise == onejet.J.num_noise + for _ in range(32): + point = torch.empty(3, dtype=torch.float64).uniform_(-1.0, 1.0) + point = 0.5 * ( + torch.tensor(domain.interval_enclosure().lower) + + torch.tensor(domain.interval_enclosure().upper) + ) + 0.5 * ( + torch.tensor(domain.interval_enclosure().upper) + - torch.tensor(domain.interval_enclosure().lower) + ) * point + point.requires_grad_(True) + gradient = torch.autograd.grad(model(point).sum(), point)[0] + for column in range(3): + assert enclosure.lower[0][column] <= float(gradient[column]) + assert float(gradient[column]) <= enclosure.upper[0][column] + + +def test_pz_onejet_trace_is_lightweight_and_reports_layer_timings() -> None: + from intervalnets import PZOneJetTraceResult, PolynomialZonotope, pz_onejet_forward + + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, -0.1], dtype=torch.float64), + torch.tensor([0.3, 0.2], dtype=torch.float64), + ) + + traced = pz_onejet_forward(model, domain, return_trace=True) + + assert isinstance(traced, PZOneJetTraceResult) + assert [record.layer_type for record in traced.records] == [ + "Input", + "Linear", + "Tanh", + "Linear", + ] + assert all(record.elapsed_s >= 0.0 for record in traced.records) + assert traced.records[-1].summary["J"]["shape"] == (1, 2) + activation = traced.records[2].summary + preactivation_lower = activation["preactivation_lower"] + preactivation_upper = activation["preactivation_upper"] + expected_preactivation = traced.records[1].value.interval_enclosure() + assert torch.equal( + preactivation_lower, + torch.as_tensor(expected_preactivation.lower, dtype=preactivation_lower.dtype), + ) + assert torch.equal( + preactivation_upper, + torch.as_tensor(expected_preactivation.upper, dtype=preactivation_upper.dtype), + ) + assert bool(torch.all(preactivation_lower <= preactivation_upper)) + value_radii = activation["tanh_approximation_radii"] + assert tuple(value_radii.shape) == (3,) + assert bool(torch.all(value_radii >= 0.0)) + assert activation["tanh_approximation_radius_min"] == pytest.approx( + float(value_radii.min()) + ) + assert activation["tanh_approximation_radius_mean"] == pytest.approx( + float(value_radii.mean()) + ) + assert activation["tanh_approximation_radius_max"] == pytest.approx( + float(value_radii.max()) + ) + activation_value = traced.records[2].value + for neuron, radius in enumerate(value_radii): + exponent = tuple( + 1 if index == domain.num_noise + neuron else 0 + for index in range(activation_value.num_noise) + ) + coefficient = activation_value.terms[exponent] + assert coefficient[neuron] == pytest.approx(float(radius)) + assert torch.count_nonzero(coefficient).item() == 1 + radii = activation["tanh_prime_approximation_radii"] + assert tuple(radii.shape) == (3,) + assert bool(torch.all(radii >= 0.0)) + assert activation["tanh_prime_approximation_radius_min"] == pytest.approx( + float(radii.min()) + ) + assert activation["tanh_prime_approximation_radius_mean"] == pytest.approx( + float(radii.mean()) + ) + assert activation["tanh_prime_approximation_radius_max"] == pytest.approx( + float(radii.max()) + ) + assert "tanh_approximation_radii" not in traced.records[1].summary + assert "tanh_prime_approximation_radii" not in traced.records[1].summary + assert "preactivation_lower" not in traced.records[1].summary + assert "preactivation_upper" not in traced.records[1].summary + + +def test_enable_interval_eval_adds_eval_pz_onejet_method() -> None: + from intervalnets import PZOneJet, PolynomialZonotope + + enable_interval_eval() + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, -0.1], dtype=torch.float64), + torch.tensor([0.3, 0.2], dtype=torch.float64), + ) + + out = model.eval_pz_onejet(domain) + + assert isinstance(out, PZOneJet) + assert out.Y.shape == (1,) + assert out.J.shape == (1, 2) + + +@pytest.mark.parametrize("strategy", ["topk", "degree", "pca"]) +def test_polynomial_onejet_reductions_enclose_sampled_jacobians(strategy) -> None: + from intervalnets import PolynomialZonotope, pz_onejet_forward + + torch.manual_seed(2718) + model = nn.Sequential( + nn.Linear(3, 6), nn.Tanh(), nn.Linear(6, 5), nn.Tanh(), nn.Linear(5, 1) + ).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.25, -0.15, -0.1], dtype=torch.float64), + torch.tensor([0.2, 0.3, 0.25], dtype=torch.float64), + ) + jet = pz_onejet_forward( + model, + domain, + reduction_strategy=strategy, + max_terms=8, + max_degree=2, + pca_rank=2, + pca_candidates=8, + ) + enclosure = jet.J.interval_enclosure() + + assert any( + any(power and jet.J.noise_kinds[index] == "domain" for index, power in enumerate(exponent)) + for exponent in jet.J.terms + ), "the retained Jacobian must still be a domain-dependent polynomial" + for _ in range(32): + point = torch.tensor( + [ + torch.empty((), dtype=torch.float64).uniform_(lo, hi).item() + for lo, hi in zip(domain.interval_enclosure().lower, domain.interval_enclosure().upper) + ], + dtype=torch.float64, + requires_grad=True, + ) + gradient = torch.autograd.grad(model(point).sum(), point)[0] + for column in range(3): + assert enclosure.lower[0][column] <= float(gradient[column]) + assert float(gradient[column]) <= enclosure.upper[0][column] + + +def test_quadratic_flat_onejet_switches_and_encloses_sampled_jacobians() -> None: + from intervalnets import PolynomialZonotope, pz_onejet_forward + + torch.manual_seed(31415) + model = nn.Sequential( + nn.Linear(2, 4), nn.Tanh(), nn.Linear(4, 1) + ).double() + with torch.no_grad(): + model[0].bias.zero_() + domain = PolynomialZonotope.from_box( + torch.tensor([-1.0, -1.0], dtype=torch.float64), + torch.tensor([1.0, 1.0], dtype=torch.float64), + ) + traced = pz_onejet_forward( + model, + domain, + reduction_strategy="topk", + max_terms=32, + derivative_enclosure="quadratic_flat", + derivative_flatness_threshold=1.0, + quadratic_compression_guard=False, + return_trace=True, + ) + activation = traced.records[2].summary + degrees = activation["tanh_prime_approximation_degrees"] + assert bool(torch.all(degrees == 2)) + assert activation["tanh_prime_quadratic_count"] == 4 + assert bool( + torch.all( + activation["tanh_prime_approximation_radii"] + < activation["tanh_prime_affine_radii"] + ) + ) + + enclosure = traced.final.J.interval_enclosure() + for _ in range(64): + point = torch.empty(2, dtype=torch.float64).uniform_(-1.0, 1.0) + point.requires_grad_(True) + gradient = torch.autograd.grad(model(point).sum(), point)[0] + for column in range(2): + assert enclosure.lower[0][column] <= float(gradient[column]) + assert float(gradient[column]) <= enclosure.upper[0][column] + + +@pytest.mark.parametrize(("variant", "generator_budget"), [("B", 0), ("C", 2)]) +def test_parity_aware_onejet_reductions_enclose_sampled_jacobians( + variant, generator_budget +) -> None: + from intervalnets import PolynomialZonotope, pz_onejet_forward + + torch.manual_seed(1618) + model = nn.Sequential( + nn.Linear(2, 4), nn.Tanh(), nn.Linear(4, 3), nn.Tanh(), nn.Linear(3, 1) + ).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.4, -0.3], dtype=torch.float64), + torch.tensor([0.4, 0.3], dtype=torch.float64), + ) + jet = pz_onejet_forward( + model, + domain, + reduction_strategy="topk", + max_terms=6, + reduction_variant=variant, + generator_budget=generator_budget, + derivative_enclosure="quadratic_flat", + derivative_flatness_threshold=1.0, + quadratic_compression_guard=False, + ) + enclosure = jet.J.interval_enclosure() + for _ in range(32): + point = torch.tensor( + [ + torch.empty((), dtype=torch.float64).uniform_(lo, hi).item() + for lo, hi in zip( + domain.interval_enclosure().lower, + domain.interval_enclosure().upper, + ) + ], + dtype=torch.float64, + requires_grad=True, + ) + gradient = torch.autograd.grad(model(point).sum(), point)[0] + for column in range(2): + assert enclosure.lower[0][column] <= float(gradient[column]) + assert float(gradient[column]) <= enclosure.upper[0][column] + + +def test_unreduced_onejet_preserves_polynomial_chain_rule_terms() -> None: + from intervalnets import PolynomialZonotope, pz_onejet_forward + + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, -0.1], dtype=torch.float64), + torch.tensor([0.2, 0.1], dtype=torch.float64), + ) + jet = pz_onejet_forward(model, domain, reduce=False) + assert jet.J.terms + assert any(sum(exponent) >= 1 for exponent in jet.J.terms) + + +def test_eval_pz_twojet_rejects_non_polynomial_zonotope_input() -> None: + enable_interval_eval() + layer = nn.Identity() + + with pytest.raises(TypeError, match="PolynomialZonotope"): + layer.eval_pz_twojet(torch.zeros(1)) diff --git a/tests/test_pz_integration.py b/tests/test_pz_integration.py new file mode 100644 index 0000000..8cc1def --- /dev/null +++ b/tests/test_pz_integration.py @@ -0,0 +1,706 @@ +from dataclasses import replace +from math import tanh + +import pytest + +from intervalnets import PZOneJet, PZTwoJet, PolynomialZonotope +from intervalnets.pz_integration import ( + IntegratedPZResult, + POINTWISE_RESIDUAL_KINDS, + PZIntegrationCell, + box_monomial_absolute_moment, + integrate_over_cell, + integrate_pz_over_domain, + integrate_pz_onejet_squared, + integrate_pz_twojet_squared, +) +from intervalnets.pz_tanh import ( + affine_tanh_double_prime_enclosure, + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + tanh_pz_scalar, +) + +try: + import torch +except ImportError: # pragma: no cover + torch = None + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_onejet_square_uses_exact_pointwise_jacobian_box_range(): + cell = PZIntegrationCell.from_bounds([-1.0], [1.0]) + value = PolynomialZonotope.constant( + torch.tensor([1.0], dtype=torch.float64), + num_noise=1, + noise_kinds=("domain",), + ) + jacobian = PolynomialZonotope.constant( + torch.tensor([[2.0]], dtype=torch.float64), + num_noise=1, + noise_kinds=("domain",), + ).add_independent_errors( + torch.tensor([[0.5]], dtype=torch.float64), + kind="approximation_pointwise", + ) + value = value.with_num_noise(jacobian.num_noise).with_noise_kinds( + jacobian.noise_kinds + ) + + result = integrate_pz_onejet_squared(PZOneJet(value, jacobian), cell) + + assert result.lower == pytest.approx(6.5) + assert result.upper == pytest.approx(14.5) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_onejet_square_can_remain_a_pz_until_final_intervalization(): + cell = PZIntegrationCell.from_bounds([-1.0], [1.0]) + value = PolynomialZonotope( + torch.tensor([1.0], dtype=torch.float64), + {(1, 0): torch.tensor([0.25], dtype=torch.float64)}, + num_noise=2, + noise_kinds=("domain", "approximation_pointwise"), + ) + jacobian = PolynomialZonotope.constant( + torch.tensor([[2.0]], dtype=torch.float64), + num_noise=2, + noise_kinds=value.noise_kinds, + ).add_independent_errors( + torch.tensor([[0.5]], dtype=torch.float64), + kind="approximation_pointwise", + ) + value = value.with_num_noise(jacobian.num_noise).with_noise_kinds( + jacobian.noise_kinds + ) + jet = PZOneJet(value, jacobian) + + integrated = integrate_pz_onejet_squared(jet, cell, output="pz") + direct_interval = integrate_pz_onejet_squared(jet, cell, output="interval") + final_interval = integrated.interval_enclosure() + + assert isinstance(integrated, PolynomialZonotope) + assert "domain" not in integrated.noise_kinds + assert integrated.noise_kinds[-1] == "global_symbolic_residual" + _assert_interval_close(final_interval, direct_interval) + + +def test_integrated_result_can_be_reencoded_as_a_polynomial_zonotope(): + polynomial = PolynomialZonotope( + 2.0, + {(1,): 0.25}, + num_noise=1, + noise_kinds=("approximation_symbolic",), + ) + result = IntegratedPZResult( + polynomial=polynomial, + interval_radius=0.5, + measure=2.0, + metadata={}, + ) + + integrated = result.as_polynomial_zonotope() + + assert integrated.noise_kinds == ( + "approximation_symbolic", + "global_symbolic_residual", + ) + _assert_interval_close(integrated.interval_enclosure(), result.interval_enclosure()) + + +def test_adaptive_cell_pz_sum_keeps_local_noise_symbols_independent(): + from intervalnets.pz_integration import _independent_pz_sum + + positive = PolynomialZonotope( + 0.0, + {(1,): 1.0}, + num_noise=1, + noise_kinds=("global_symbolic_residual",), + ) + negative = PolynomialZonotope( + 0.0, + {(1,): -1.0}, + num_noise=1, + noise_kinds=("global_symbolic_residual",), + ) + + total = _independent_pz_sum((positive, negative)) + enclosure = total.interval_enclosure() + + assert total.num_noise == 2 + assert enclosure.lower == pytest.approx(-2.0) + assert enclosure.upper == pytest.approx(2.0) + + +def _assert_interval_close(left, right): + assert float(left.lower) == pytest.approx(float(right.lower), rel=1e-12, abs=1e-12) + assert float(left.upper) == pytest.approx(float(right.upper), rel=1e-12, abs=1e-12) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +@pytest.mark.parametrize("kind", ["l2", "w12", "w22"]) +def test_direct_twojet_square_matches_explicit_with_unequal_tensor_supports(kind): + from intervalnets.pz_norms import pz_twojet_l2_integrand, pz_twojet_w12_integrand, pz_twojet_w22_integrand + + torch.manual_seed(19) + kinds = ("domain", "approximation_symbolic", "approximation_pointwise") + y = PolynomialZonotope(torch.randn(2, dtype=torch.float64), {(1, 0, 0): torch.randn(2, dtype=torch.float64)}, num_noise=3, noise_kinds=kinds) + j = PolynomialZonotope(torch.randn(2, 2, dtype=torch.float64), {(0, 1, 0): torch.randn(2, 2, dtype=torch.float64), (1, 0, 1): torch.randn(2, 2, dtype=torch.float64)}, num_noise=3, noise_kinds=kinds) + h = PolynomialZonotope(torch.randn(2, 2, 2, dtype=torch.float64), {(2, 0, 0): torch.randn(2, 2, 2, dtype=torch.float64), (0, 0, 1): torch.randn(2, 2, 2, dtype=torch.float64)}, num_noise=3, noise_kinds=kinds) + jet = PZTwoJet(y, j, h) + cell = PZIntegrationCell.from_bounds((-2.0,), (2.0,)) + constructors = {"l2": pz_twojet_l2_integrand, "w12": pz_twojet_w12_integrand, "w22": pz_twojet_w22_integrand} + + direct = integrate_pz_twojet_squared(jet, cell, kind) + explicit = integrate_over_cell(constructors[kind](jet), cell, output="interval") + + _assert_interval_close(direct, explicit) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_twojet_square_canonicalizes_pointwise_cancellation_and_keeps_odd_domain_terms(): + kinds = ("domain", "approximation_pointwise") + zero_j = PolynomialZonotope.constant(torch.zeros((2, 1), dtype=torch.float64), num_noise=2, noise_kinds=kinds) + zero_h = PolynomialZonotope.constant(torch.zeros((2, 1, 1), dtype=torch.float64), num_noise=2, noise_kinds=kinds) + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + + cancelling_y = PolynomialZonotope(torch.tensor([1.0, 1.0], dtype=torch.float64), {(0, 1): torch.tensor([1.0, -1.0], dtype=torch.float64)}, num_noise=2, noise_kinds=kinds) + cancelling = integrate_pz_twojet_squared(PZTwoJet(cancelling_y, zero_j, zero_h), cell, "l2") + assert float(cancelling.lower) == pytest.approx(4.0) + assert float(cancelling.upper) == pytest.approx(8.0) + + odd_y = PolynomialZonotope(torch.tensor([0.0], dtype=torch.float64), {(1, 0): torch.tensor([1.0], dtype=torch.float64), (0, 1): torch.tensor([1.0], dtype=torch.float64)}, num_noise=2, noise_kinds=kinds) + odd_jet = PZTwoJet(odd_y, zero_j[:1], zero_h[:1]) + direct = integrate_pz_twojet_squared(odd_jet, cell, "l2") + from intervalnets.pz_norms import pz_twojet_l2_integrand + explicit = integrate_over_cell(pz_twojet_l2_integrand(odd_jet), cell, output="interval") + _assert_interval_close(direct, explicit) + # The odd alpha*eta term does not vanish: it receives the absolute domain + # moment int |alpha| = 1, while eta**2 is reduced one-sided to [0, 2]. + assert float(direct.lower) == pytest.approx(-4.0 / 3.0) + assert float(direct.upper) == pytest.approx(14.0 / 3.0) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_twojet_square_falls_back_for_polynomial_density(): + kinds = ("domain",) + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + cell = replace(cell, jacobian_density=PolynomialZonotope.constant(1.0, num_noise=1, noise_kinds=kinds)) + jet = PZTwoJet( + PolynomialZonotope(torch.tensor([1.0], dtype=torch.float64), {(1,): torch.tensor([0.5], dtype=torch.float64)}, num_noise=1, noise_kinds=kinds), + PolynomialZonotope.constant(torch.zeros((1, 1), dtype=torch.float64), num_noise=1, noise_kinds=kinds), + PolynomialZonotope.constant(torch.zeros((1, 1, 1), dtype=torch.float64), num_noise=1, noise_kinds=kinds), + ) + from intervalnets.pz_norms import pz_twojet_l2_integrand + + direct = integrate_pz_twojet_squared(jet, cell, "l2") + explicit = integrate_over_cell(pz_twojet_l2_integrand(jet), cell, output="interval") + _assert_interval_close(direct, explicit) + + +def test_direct_twojet_square_vectorizes_float_coefficients(monkeypatch): + import intervalnets.pz_integration as pz_integration + + if pz_integration.np is None: + pytest.skip("NumPy not installed") + kinds = ("domain", "approximation_pointwise") + y = PolynomialZonotope( + (1.0, -0.5), + { + (1, 0): (0.25, 0.75), + (0, 1): (-0.1, 0.2), + }, + num_noise=2, + noise_kinds=kinds, + ) + zero_j = PolynomialZonotope.constant(((0.0,), (0.0,)), num_noise=2, noise_kinds=kinds) + zero_h = PolynomialZonotope.constant((((0.0,),), ((0.0,),)), num_noise=2, noise_kinds=kinds) + jet = PZTwoJet(y, zero_j, zero_h) + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + + from intervalnets.pz_norms import pz_twojet_l2_integrand + + explicit = integrate_over_cell(pz_twojet_l2_integrand(jet), cell, output="interval") + + def fail_python_dot(*args, **kwargs): + raise AssertionError("float coefficients should use the vectorized NumPy path") + + monkeypatch.setattr(pz_integration, "_weighted_dot", fail_python_dot) + direct = integrate_pz_twojet_squared(jet, cell, "l2") + + _assert_interval_close(direct, explicit) + + +def test_direct_twojet_square_python_path_matches_improved_explicit_rule(monkeypatch): + import intervalnets.pz_integration as pz_integration + from intervalnets.pz_norms import pz_twojet_l2_integrand + + kinds = ("domain", "approximation_pointwise") + y = PolynomialZonotope( + (0.0,), + {(1, 0): (1.0,), (0, 1): (1.0,)}, + num_noise=2, + noise_kinds=kinds, + ) + zero_j = PolynomialZonotope.constant(((0.0,),), num_noise=2, noise_kinds=kinds) + zero_h = PolynomialZonotope.constant((((0.0,),),), num_noise=2, noise_kinds=kinds) + jet = PZTwoJet(y, zero_j, zero_h) + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + explicit = integrate_over_cell(pz_twojet_l2_integrand(jet), cell, output="interval") + + monkeypatch.setattr(pz_integration, "np", None) + direct = integrate_pz_twojet_squared(jet, cell, "l2") + + _assert_interval_close(direct, explicit) + assert float(direct.lower) == pytest.approx(-4.0 / 3.0) + assert float(direct.upper) == pytest.approx(14.0 / 3.0) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_value_squared_affine_fast_path_matches_explicit_reference(): + from intervalnets.pz_integration import integrate_pz_value_squared + from intervalnets.pz_norms import pz_sum_squares + + cell = PZIntegrationCell.from_bounds([-1.0, -0.5], [1.0, 0.5]) + value = PolynomialZonotope( + torch.tensor([0.2, -0.1], dtype=torch.float64), + { + (1, 0, 0, 0): torch.tensor([0.4, -0.2], dtype=torch.float64), + (0, 1, 0, 0): torch.tensor([0.3, 0.1], dtype=torch.float64), + (0, 0, 1, 0): torch.tensor([0.05, -0.07], dtype=torch.float64), + (0, 0, 0, 1): torch.tensor([-0.02, 0.08], dtype=torch.float64), + }, + num_noise=4, + noise_kinds=( + "domain", + "domain", + "approximation_pointwise", + "approximation_symbolic", + ), + ) + + direct = integrate_pz_value_squared(value, cell) + explicit = integrate_over_cell(pz_sum_squares(value), cell, output="interval") + + _assert_interval_close(direct, explicit) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_adaptive_squared_contribution_avoids_explicit_integrand(monkeypatch): + import intervalnets.pz_integration as pz_integration + from intervalnets import IntervalTensor, enable_interval_eval + + enable_interval_eval() + model = torch.nn.Sequential( + torch.nn.Linear(1, 2, dtype=torch.float64), + torch.nn.Tanh(), + torch.nn.Linear(2, 1, dtype=torch.float64), + ) + domain = IntervalTensor.from_bounds([-0.5], [0.5]) + + def fail_explicit_integrand(*args, **kwargs): + raise AssertionError("adaptive affine cells must use direct squared integration") + + def fail_python_dot(*args, **kwargs): + raise AssertionError("adaptive scalar coefficients should use a vectorized path") + + monkeypatch.setattr(pz_integration, "_squared_twojet_integrand", fail_explicit_integrand) + monkeypatch.setattr(pz_integration, "_weighted_dot", fail_python_dot) + cached = pz_integration._evaluate_squared_contribution_cache( + model, + domain, + integrand_kind="w22", + chebyshev_degree=3, + residual_subdivisions=16, + ) + + assert cached.contribution.lower <= cached.contribution.upper + + +def test_integrating_pointwise_residual_adds_radius_not_symbolic_moment(): + z = PolynomialZonotope( + 1.0, + { + (1, 0): 3.0, # odd domain monomial integrates to zero + (2, 0): 6.0, # exact contribution: 6 * int alpha^2 = 4 + (0, 1): 0.25, # pointwise residual contribution: 0.25 * volume 2 + }, + num_noise=2, + noise_kinds=("domain", "pointwise_residual"), + ) + + result = integrate_pz_over_domain(z) + + assert isinstance(result, IntegratedPZResult) + assert result.measure == 2.0 + assert result.polynomial.num_noise == 1 + assert result.polynomial.noise_kinds == ("pointwise_residual",) + assert result.polynomial.center == pytest.approx(6.0) # 2 * center + 6 * 2/3 + assert result.polynomial.terms == {} + assert result.interval_radius == pytest.approx(0.5) + + +def test_box_monomial_absolute_moment_keeps_odd_powers(): + assert box_monomial_absolute_moment((1,)) == pytest.approx(1.0) + assert box_monomial_absolute_moment((1, 2)) == pytest.approx(2.0 / 3.0) + assert box_monomial_absolute_moment(()) == pytest.approx(1.0) + with pytest.raises(ValueError): + box_monomial_absolute_moment((-1,)) + + +def test_pointwise_mixed_term_uses_absolute_moment_and_even_power_is_one_sided(): + z = PolynomialZonotope( + 0.0, + { + (1, 1): 2.0, + (0, 2): 3.0, + }, + num_noise=2, + noise_kinds=("domain", "approximation_pointwise"), + ) + + result = integrate_pz_over_domain(z) + interval = result.interval_enclosure() + + # 2 alpha eta contributes [-2, 2], using int |alpha| = 1 rather than + # either the signed moment zero or the full reference measure two. + # 3 eta**2 contributes [0, 6] through midpoint 3 and radius 3. + assert result.polynomial.center == pytest.approx(3.0) + assert result.interval_radius == pytest.approx(5.0) + assert interval.lower == pytest.approx(-2.0) + assert interval.upper == pytest.approx(8.0) + + +def test_even_pointwise_power_with_odd_domain_power_uses_half_absolute_moment(): + z = PolynomialZonotope( + 0.0, + {(1, 2): 4.0}, + num_noise=2, + noise_kinds=("domain", "pointwise_residual"), + ) + + result = integrate_pz_over_domain(z) + + assert result.polynomial.center == pytest.approx(0.0) + assert result.interval_radius == pytest.approx(2.0) + + +def test_global_symbolic_noise_keeps_signed_moment_semantics(): + z = PolynomialZonotope( + 0.0, + {(1, 1): 2.0, (2, 1): 3.0}, + num_noise=2, + noise_kinds=("domain", "approximation_symbolic"), + ) + + result = integrate_pz_over_domain(z) + + assert result.interval_radius == pytest.approx(0.0) + assert set(result.polynomial.terms) == {(1,)} + assert result.polynomial.terms[(1,)] == pytest.approx(2.0) + + +def test_pointwise_parity_refinement_is_conservative_with_global_symbols(): + kinds = ("domain", "approximation_symbolic", "approximation_pointwise") + even = PolynomialZonotope( + 0.0, + {(0, 2, 2): 4.0}, + num_noise=3, + noise_kinds=kinds, + ) + ambiguous = PolynomialZonotope( + 0.0, + {(0, 1, 2): 4.0}, + num_noise=3, + noise_kinds=kinds, + ) + + even_interval = integrate_pz_over_domain(even).interval_enclosure() + ambiguous_interval = integrate_pz_over_domain(ambiguous).interval_enclosure() + + assert float(even_interval.lower) == pytest.approx(0.0) + assert float(even_interval.upper) == pytest.approx(8.0) + assert float(ambiguous_interval.lower) == pytest.approx(-8.0) + assert float(ambiguous_interval.upper) == pytest.approx(8.0) + + +def test_geometric_volume_scales_pointwise_residual_radius(): + z = PolynomialZonotope( + 0.0, + {(0, 1): 2.0}, + num_noise=2, + noise_kinds=("domain", "approximation_pointwise"), + ) + + result = integrate_pz_over_domain(z, volume=7.5) + + assert result.polynomial.center == 0.0 + assert result.polynomial.terms == {} + assert result.interval_radius == pytest.approx(15.0) + assert result.measure == 7.5 + + +def test_optional_volume_scales_absolute_and_signed_moments_by_one_density(): + z = PolynomialZonotope( + 1.0, + {(2, 0): 3.0, (1, 1): 2.0}, + num_noise=2, + noise_kinds=("domain", "approximation_pointwise"), + ) + + result = integrate_pz_over_domain(z, volume=6.0) + + # Constant density is 6 / 2 = 3. The exact alpha**2 term contributes 6, + # and 2 alpha eta has radius 2 * 3 * int|alpha| = 6. + assert result.polynomial.center == pytest.approx(12.0) + assert result.interval_radius == pytest.approx(6.0) + + +def test_affine_cell_applies_jacobian_density_once_to_absolute_moment(): + cell = PZIntegrationCell.from_bounds((-2.0,), (2.0,)) + mixed = PolynomialZonotope( + 0.0, + {(1, 1): 1.0}, + num_noise=2, + noise_kinds=("domain", "approximation_pointwise"), + ) + + result = integrate_over_cell(mixed, cell, output="interval") + + # J_X = 2 and int_{-1}^1 |alpha| d alpha = 1. The physical volume four + # must not be applied again after the absolute moment. + assert float(result.lower) == pytest.approx(-2.0) + assert float(result.upper) == pytest.approx(2.0) + + +def test_symbolic_mode_keeps_residual_symbol_and_integrates_by_moments_only_when_requested(): + z = PolynomialZonotope( + 0.0, + {(0, 1): 0.25}, + num_noise=2, + noise_kinds=("domain", "pointwise_residual"), + ) + + result = integrate_pz_over_domain(z, mode="symbolic") + + assert result.interval_radius == 0.0 + assert result.polynomial.center == 0.0 + assert result.polynomial.terms[(1,)] == pytest.approx(0.5) + assert result.polynomial.noise_kinds == ("pointwise_residual",) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_tanh_pz_scalar_marks_default_residual_as_pointwise(): + z = PolynomialZonotope(torch.tensor(0.1, dtype=torch.float64), {(1,): torch.tensor(0.05, dtype=torch.float64)}, num_noise=1, noise_kinds=("domain",)) + + out = tanh_pz_scalar(z, chebyshev_degree=3, residual_subdivisions=16) + + assert out.noise_kinds[-1] == "approximation_pointwise" + + +def test_affine_tanh_residuals_integrate_as_pointwise_interval_radius(): + from intervalnets.pytorch import _affine_enclosure_pz + + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + x = cell.domain[0] + helpers = ( + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + affine_tanh_double_prime_enclosure, + ) + + for helper in helpers: + enclosure = helper((-1.0, 1.0)) + residual = _affine_enclosure_pz( + x, slope=enclosure.p, intercept=enclosure.q, radius=enclosure.delta + ) - (enclosure.p * x + enclosure.q) + + assert residual.noise_kinds[-1] in POINTWISE_RESIDUAL_KINDS + + result = integrate_over_cell(x * residual, cell, output="interval") + + # The odd domain factor is not discarded through its signed moment. + # Pointwise residual handling integrates its absolute moment instead: + # int_{-1}^1 |alpha| d alpha = 1. + assert result.lower < 0.0 + assert result.upper > 0.0 + assert max(abs(float(result.lower)), abs(float(result.upper))) == pytest.approx( + enclosure.delta + ) + + +def test_affine_tanh_squared_pointwise_regression_matches_direct_path(): + from intervalnets.pz_integration import integrate_pz_value_squared + from intervalnets.pz_norms import pz_sum_squares + from intervalnets.pytorch import _affine_enclosure_pz + + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + x = cell.domain[0] + enclosure = affine_tanh_enclosure((-1.0, 1.0)) + value = _affine_enclosure_pz( + x, + slope=enclosure.p, + intercept=enclosure.q, + radius=enclosure.delta, + ) + + explicit = integrate_over_cell(pz_sum_squares(value), cell, output="interval") + direct = integrate_pz_value_squared(value, cell) + p = enclosure.p + rho = enclosure.delta + expected_lower = 2.0 * p * p / 3.0 - 2.0 * p * rho + expected_upper = 2.0 * p * p / 3.0 + 2.0 * p * rho + 2.0 * rho * rho + + _assert_interval_close(direct, explicit) + assert float(explicit.lower) == pytest.approx(expected_lower) + assert float(explicit.upper) == pytest.approx(expected_upper) + true_integral = 2.0 * (1.0 - tanh(1.0)) + assert float(explicit.lower) < true_integral + assert float(explicit.upper) > true_integral + +def test_affine_cell_integrates_one_dimensional_polynomial_exactly(): + cell = PZIntegrationCell.from_bounds((1.0,), (3.0,)) + x = cell.domain[0] + expr = x * x + + result = integrate_over_cell(expr, cell, output="pz") + + assert cell.domain.noise_kinds == ("domain",) + assert cell.domain.center == (2.0,) + assert cell.domain.terms[(1,)] == (1.0,) + assert cell.jacobian_density == pytest.approx(1.0) + assert cell.volume == pytest.approx(2.0) + assert result.num_noise == 0 + assert result.center == pytest.approx(26.0 / 3.0) + assert result.terms == {} + + +def test_affine_cell_integrates_two_dimensional_polynomial_exactly(): + cell = PZIntegrationCell.from_bounds((1.0, -2.0), (3.0, 4.0)) + x = cell.domain[0] + y = cell.domain[1] + expr = x + 2.0 * y * y + + result = integrate_over_cell(expr, cell, output="pz") + + # integral over [1,3]x[-2,4] of x + 2 y^2 dxdy + expected = 24.0 + 96.0 + assert cell.jacobian_density == pytest.approx(3.0) + assert cell.volume == pytest.approx(12.0) + assert result.center == pytest.approx(expected) + assert result.terms == {} + + +def test_integrate_over_cell_interval_combines_symbolic_and_pointwise_residuals(): + cell = PZIntegrationCell.from_bounds((0.0,), (2.0,)) + x = cell.domain[0] + expr = x + PolynomialZonotope( + 0.0, + {(0, 1): 0.5, (0, 0, 1): 0.25}, + num_noise=3, + noise_kinds=("domain", "approximation_symbolic", "approximation_pointwise"), + ) + + result = integrate_over_cell(expr, cell, output="interval") + + assert result.lower == pytest.approx(0.5) + assert result.upper == pytest.approx(3.5) + + +def test_non_affine_fixed_orientation_hook_requires_certificates(): + cell = PZIntegrationCell.from_bounds((0.0,), (1.0,)) + + with pytest.raises(NotImplementedError): + PZIntegrationCell.from_fixed_orientation_domain(cell.domain, cell.domain_noise_indices) + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_sum_squares_vector_matrix_tensor_entries(): + from intervalnets.pz_norms import pz_sum_squares + + z = PolynomialZonotope( + torch.tensor([[1.0, 2.0], [3.0, 4.0]], dtype=torch.float64), + {(1,): torch.ones(2, 2, dtype=torch.float64)}, + num_noise=1, + noise_kinds=("domain",), + ) + + out = pz_sum_squares(z) + + assert out.shape == () + assert torch.allclose(out.center, torch.tensor(30.0, dtype=torch.float64)) + assert torch.allclose(out.terms[(1,)], torch.tensor(20.0, dtype=torch.float64)) + assert torch.allclose(out.terms[(2,)], torch.tensor(4.0, dtype=torch.float64)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_twojet_norms_constant_and_affine_match_closed_forms(): + from intervalnets import IntervalTensor, enable_interval_eval + + enable_interval_eval() + const_model = torch.nn.Linear(1, 2, dtype=torch.float64) + with torch.no_grad(): + const_model.weight.zero_() + const_model.bias.copy_(torch.tensor([3.0, -4.0], dtype=torch.float64)) + domain = IntervalTensor.from_bounds([0.0], [2.0]) + + const_norm = const_model.pz_l2norm(domain, p=2.0) + + assert const_norm.lower == pytest.approx((50.0) ** 0.5) + assert const_norm.upper == pytest.approx((50.0) ** 0.5) + + affine_model = torch.nn.Linear(1, 1, dtype=torch.float64) + with torch.no_grad(): + affine_model.weight.fill_(1.0) + affine_model.bias.zero_() + affine_domain = IntervalTensor.from_bounds([-1.0], [1.0]) + + l2 = affine_model.pz_l2norm(affine_domain) + w12 = affine_model.pz_sobolev_norm(affine_domain, order=1) + w22 = affine_model.pz_sobolev_norm(affine_domain, order=2) + + assert l2.lower == pytest.approx((2.0 / 3.0) ** 0.5) + assert l2.upper == pytest.approx((2.0 / 3.0) ** 0.5) + assert w12.lower == pytest.approx((8.0 / 3.0) ** 0.5) + assert w12.upper == pytest.approx((8.0 / 3.0) ** 0.5) + assert w22.lower == pytest.approx(w12.lower) + assert w22.upper == pytest.approx(w12.upper) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_norm_public_methods_reject_non_l2_p(): + from intervalnets import IntervalTensor, enable_interval_eval + + enable_interval_eval() + model = torch.nn.Linear(1, 1, dtype=torch.float64) + domain = IntervalTensor.from_bounds([-1.0], [1.0]) + + with pytest.raises(NotImplementedError, match="p=2.0"): + model.pz_l2norm(domain, p=1.0) + with pytest.raises(NotImplementedError, match="p=2.0"): + model.pz_sobolev_norm(domain, p=3.0, order=1) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_tanh_w22_norm_contains_dense_autograd_quadrature(): + from intervalnets import IntervalTensor, enable_interval_eval + + enable_interval_eval() + model = torch.nn.Sequential(torch.nn.Linear(1, 1, dtype=torch.float64), torch.nn.Tanh()).double() + with torch.no_grad(): + model[0].weight.fill_(0.7) + model[0].bias.fill_(0.1) + domain = IntervalTensor.from_bounds([-0.5], [0.5]) + + bounds = model.pz_sobolev_norm(domain, order=2, chebyshev_degree=5, residual_subdivisions=96) + xs = torch.linspace(-0.5, 0.5, steps=401, dtype=torch.float64) + values = [] + for x_value in xs: + x = x_value.reshape(1).clone().detach().requires_grad_(True) + y = model(x)[0] + grad = torch.autograd.grad(y, x, create_graph=True)[0][0] + hess = torch.autograd.grad(grad, x)[0][0] + values.append((y.detach() ** 2 + grad.detach() ** 2 + hess.detach() ** 2).reshape(())) + dense_integral = torch.trapezoid(torch.stack(values), xs).item() + dense_norm = dense_integral ** 0.5 + + assert bounds.lower <= dense_norm <= bounds.upper diff --git a/tests/test_pz_norms.py b/tests/test_pz_norms.py new file mode 100644 index 0000000..eec7b3b --- /dev/null +++ b/tests/test_pz_norms.py @@ -0,0 +1,388 @@ +import pytest + +from intervalnets import IntervalTensor, PZTwoJet, PolynomialZonotope, enable_interval_eval +from intervalnets.pz_integration import PZIntegrationCell, integrate_over_cell +from intervalnets.pz_norms import build_pz_twojet_norm_diagnostics, pz_norm_from_integrand, pz_sum_squares, pz_symmetric_hessian_sum_squares, pz_twojet_l2_integrand, pz_twojet_l2_norm, pz_twojet_w12_integrand, pz_twojet_w12_norm, pz_twojet_w22_integrand, pz_twojet_w22_norm + +try: + import torch + from torch import nn +except ImportError: # pragma: no cover + torch = None + nn = None + +pytestmark = pytest.mark.skipif(torch is None, reason="PyTorch not installed") + + +def _assert_same_pz(left: PolynomialZonotope, right: PolynomialZonotope): + assert left.num_noise == right.num_noise + assert left.noise_kinds == right.noise_kinds + assert left.shape == right.shape + if torch is not None and isinstance(left.center, torch.Tensor): + assert torch.allclose(left.center, right.center) + assert set(left.terms) == set(right.terms) + for exponent in left.terms: + assert torch.allclose(left.terms[exponent], right.terms[exponent]) + else: + assert left.center == pytest.approx(right.center) + assert left.terms == pytest.approx(right.terms) + + + +def test_pz_twojet_norm_diagnostics_do_not_mutate_input_jet(): + y = PolynomialZonotope(torch.tensor([1.0, -0.5], dtype=torch.float64), {(1, 0): torch.tensor([0.25, 0.1], dtype=torch.float64)}, num_noise=2, noise_kinds=("domain", "approximation")) + j = PolynomialZonotope(torch.tensor([[1.0], [-1.0]], dtype=torch.float64), {(0, 1): torch.tensor([[0.1], [0.2]], dtype=torch.float64)}, num_noise=2, noise_kinds=("domain", "approximation")) + h = PolynomialZonotope.constant(torch.zeros(2, 1, 1, dtype=torch.float64), num_noise=2, noise_kinds=("domain", "approximation")) + jet = PZTwoJet(y, j, h) + + before = (jet.Y, jet.J, jet.H) + diagnostics = build_pz_twojet_norm_diagnostics(jet, max_terms=2) + + assert (jet.Y, jet.J, jet.H) == before + assert diagnostics["jet"] == {"Y": jet.Y, "J": jet.J, "H": jet.H} + assert diagnostics["metadata"]["shapes"] == {"Y": (2,), "J": (2, 1), "H": (2, 1, 1)} + assert diagnostics["metadata"]["noise_kind_counts"] == {"domain": 3, "approximation": 3} + assert "twojet" in diagnostics["rendered"]["latex"] + assert diagnostics["rendered"]["markdown"]["Y"].startswith("```latex\n") + + +def test_pz_twojet_norm_diagnostics_integrands_match_norm_helpers(): + y = PolynomialZonotope(torch.tensor([0.5], dtype=torch.float64), {(1,): torch.tensor([0.2], dtype=torch.float64)}, num_noise=1, noise_kinds=("domain",)) + j = PolynomialZonotope(torch.tensor([[1.0, -2.0]], dtype=torch.float64), {(1,): torch.tensor([[0.1, -0.3]], dtype=torch.float64)}, num_noise=1, noise_kinds=("domain",)) + h = PolynomialZonotope(torch.tensor([[[1.0, 0.25], [0.25, -0.5]]], dtype=torch.float64), {(1,): torch.tensor([[[0.1, -0.2], [-0.2, 0.3]]], dtype=torch.float64)}, num_noise=1, noise_kinds=("domain",)) + jet = PZTwoJet(y, j, h) + + diagnostics = build_pz_twojet_norm_diagnostics(jet, render=False) + + assert "rendered" not in diagnostics + _assert_same_pz(diagnostics["integrands"]["l2_integrand"], pz_twojet_l2_integrand(jet)) + _assert_same_pz(diagnostics["integrands"]["w12_integrand"], pz_twojet_w12_integrand(jet)) + _assert_same_pz(diagnostics["integrands"]["w22_integrand"], pz_twojet_w22_integrand(jet)) + +def test_pz_symmetric_hessian_sum_squares_matches_dense_full_sum_for_symmetric_hessian(): + center = torch.tensor( + [ + [[1.0, 2.0, -0.5], [2.0, -1.0, 0.75], [-0.5, 0.75, 1.5]], + [[-0.25, 1.25, 0.5], [1.25, 0.5, -1.5], [0.5, -1.5, 2.0]], + ], + dtype=torch.float64, + ) + coeff = torch.tensor( + [ + [[0.2, -0.1, 0.3], [-0.1, 0.4, -0.2], [0.3, -0.2, 0.1]], + [[-0.3, 0.2, 0.15], [0.2, -0.05, 0.35], [0.15, 0.35, -0.25]], + ], + dtype=torch.float64, + ) + hessian = PolynomialZonotope(center, {(1,): coeff}, num_noise=1, noise_kinds=("domain",)) + + optimized = pz_symmetric_hessian_sum_squares(hessian) + dense = pz_sum_squares(hessian) + + _assert_same_pz(optimized, dense) + + +def test_pz_symmetric_hessian_sum_squares_matches_dense_full_sum_for_scalar_output_hessian(): + center = torch.tensor( + [[1.0, 2.0, -0.5], [2.0, -1.0, 0.75], [-0.5, 0.75, 1.5]], + dtype=torch.float64, + ) + coeff = torch.tensor( + [[0.2, -0.1, 0.3], [-0.1, 0.4, -0.2], [0.3, -0.2, 0.1]], + dtype=torch.float64, + ) + hessian = PolynomialZonotope(center, {(1,): coeff}, num_noise=1, noise_kinds=("domain",)) + + optimized = pz_symmetric_hessian_sum_squares(hessian) + dense = pz_sum_squares(hessian) + + _assert_same_pz(optimized, dense) + + +def test_pz_twojet_w22_integrand_uses_symmetric_hessian_accumulation_equivalent_to_dense_sum(): + y = PolynomialZonotope.constant(torch.tensor([0.5], dtype=torch.float64), num_noise=1, noise_kinds=("domain",)) + j = PolynomialZonotope.constant(torch.tensor([[1.0, -2.0]], dtype=torch.float64), num_noise=1, noise_kinds=("domain",)) + h_center = torch.tensor([[[1.0, 0.25], [0.25, -0.5]]], dtype=torch.float64) + h_coeff = torch.tensor([[[0.1, -0.2], [-0.2, 0.3]]], dtype=torch.float64) + h = PolynomialZonotope(h_center, {(1,): h_coeff}, num_noise=1, noise_kinds=("domain",)) + jet = PZTwoJet(y, j, h) + + optimized = pz_twojet_w22_integrand(jet) + dense = pz_sum_squares(y) + pz_sum_squares(j) + pz_sum_squares(h) + + _assert_same_pz(optimized, dense) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +@pytest.mark.parametrize( + ("norm", "integrand"), + [(pz_twojet_l2_norm, pz_twojet_l2_integrand), (pz_twojet_w12_norm, pz_twojet_w12_integrand), (pz_twojet_w22_norm, pz_twojet_w22_integrand)], +) +@pytest.mark.parametrize("constant", [0.0, 2.5]) +def test_public_twojet_norm_direct_path_matches_explicit_for_zero_and_constant_jets(norm, integrand, constant): + kinds = ("domain",) + jet = PZTwoJet( + PolynomialZonotope.constant(torch.tensor([constant], dtype=torch.float64), num_noise=1, noise_kinds=kinds), + PolynomialZonotope.constant(torch.zeros((1, 2), dtype=torch.float64), num_noise=1, noise_kinds=kinds), + PolynomialZonotope.constant(torch.zeros((1, 2, 2), dtype=torch.float64), num_noise=1, noise_kinds=kinds), + ) + cell = PZIntegrationCell.from_bounds((-1.0,), (1.0,)) + + direct = norm(jet, cell) + explicit = pz_norm_from_integrand(integrand(jet), cell) + + assert float(direct.lower) == pytest.approx(float(explicit.lower), abs=1e-12) + assert float(direct.upper) == pytest.approx(float(explicit.upper), abs=1e-12) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_direct_w22_uses_authoritative_upper_hessian_and_weight_two(): + kinds = ("domain",) + zero_y = PolynomialZonotope.constant(torch.zeros(1, dtype=torch.float64), num_noise=1, noise_kinds=kinds) + zero_j = PolynomialZonotope.constant(torch.zeros((1, 2), dtype=torch.float64), num_noise=1, noise_kinds=kinds) + # The deliberately different lower entry must be ignored. + h = PolynomialZonotope.constant(torch.tensor([[[0.0, 3.0], [100.0, 0.0]]], dtype=torch.float64), num_noise=1, noise_kinds=kinds) + jet = PZTwoJet(zero_y, zero_j, h) + result = pz_twojet_w22_norm(jet, PZIntegrationCell.from_bounds((-1.0,), (1.0,))) + + expected = (2.0 * 2.0 * 3.0**2) ** 0.5 + assert float(result.lower) == pytest.approx(expected) + assert float(result.upper) == pytest.approx(expected) + + +def _small_tanh_model(input_dim=1, hidden_dim=2, output_dim=1): + model = nn.Sequential( + nn.Linear(input_dim, hidden_dim, dtype=torch.float64), + nn.Tanh(), + nn.Linear(hidden_dim, output_dim, dtype=torch.float64), + ).double() + with torch.no_grad(): + model[0].weight.copy_(torch.linspace(-0.4, 0.5, steps=hidden_dim * input_dim, dtype=torch.float64).reshape(hidden_dim, input_dim)) + model[0].bias.copy_(torch.linspace(-0.1, 0.15, steps=hidden_dim, dtype=torch.float64)) + model[2].weight.copy_(torch.linspace(0.25, -0.35, steps=output_dim * hidden_dim, dtype=torch.float64).reshape(output_dim, hidden_dim)) + model[2].bias.copy_(torch.linspace(0.05, 0.1, steps=output_dim, dtype=torch.float64)) + return model + + +def test_pz_l2norm_zero_network_returns_zero_interval(): + enable_interval_eval() + model = nn.Sequential(nn.Linear(2, 2, dtype=torch.float64), nn.Tanh(), nn.Linear(2, 1, dtype=torch.float64)).double() + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[0.2, -0.1], [0.05, 0.15]], dtype=torch.float64)) + model[0].bias.copy_(torch.tensor([0.1, -0.2], dtype=torch.float64)) + model[2].weight.zero_() + model[2].bias.zero_() + + domain = IntervalTensor.from_bounds([-1.0, -2.0], [3.0, 4.0]) + bounds = model.lpnorm(domain, p=2.0, method="pz", iterations=1, chebyshev_degree=3, residual_subdivisions=16) + + assert bounds.lower <= 0.0 <= bounds.upper + assert bounds.upper < 1e-10 + + +def test_pz_l2norm_constant_network_matches_exact_value(): + enable_interval_eval() + model = nn.Sequential(nn.Linear(1, 1, dtype=torch.float64), nn.Tanh(), nn.Linear(1, 1, dtype=torch.float64)).double() + with torch.no_grad(): + model[0].weight.fill_(0.2) + model[0].bias.fill_(0.1) + model[2].weight.zero_() + model[2].bias.fill_(3.0) + + domain = IntervalTensor.from_bounds([0.0], [2.0]) + bounds = model.lpnorm(domain, p=2.0, method="pz", iterations=1, chebyshev_degree=3, residual_subdivisions=16) + exact = 18.0**0.5 + + assert bounds.lower <= exact <= bounds.upper + assert bounds.upper - bounds.lower < 1e-10 + + +def test_pz_l2norm_refinement_tightens_interval(): + enable_interval_eval() + model = _small_tanh_model() + domain = IntervalTensor.from_bounds([-1.0], [1.0]) + + coarse = model.lpnorm(domain, p=2.0, method="pz", iterations=0, chebyshev_degree=3, residual_subdivisions=16) + refined = model.lpnorm(domain, p=2.0, method="pz", iterations=2, chebyshev_degree=3, residual_subdivisions=16) + + assert refined.lower >= coarse.lower + assert refined.upper <= coarse.upper + assert refined.upper - refined.lower <= coarse.upper - coarse.lower + + +def test_pz_l2norm_accepts_dorfler_theta_parameter(): + enable_interval_eval() + model = _small_tanh_model() + domain = IntervalTensor.from_bounds([-1.0], [1.0]) + + bounds = model.lpnorm(domain, p=2.0, method="pz", iterations=2, theta=0.5, chebyshev_degree=3, residual_subdivisions=16) + + assert bounds.lower <= bounds.upper + + +def test_pz_norms_validate_adaptive_and_forward_parameters(): + enable_interval_eval() + model = _small_tanh_model() + domain = IntervalTensor.from_bounds([0.0], [1.0]) + + with pytest.raises(ValueError): + model.lpnorm(domain, p=2.0, method="pz", iterations=-1, chebyshev_degree=3, residual_subdivisions=16) + with pytest.raises(ValueError): + model.lpnorm(domain, p=2.0, method="pz", iterations=1, theta=0.0, chebyshev_degree=3, residual_subdivisions=16) + with pytest.raises(ValueError): + model.lpnorm(domain, p=2.0, method="pz", iterations=0, chebyshev_degree=-1, residual_subdivisions=16) + with pytest.raises(ValueError): + model.lpnorm(domain, p=2.0, method="pz", iterations=0, chebyshev_degree=3, residual_subdivisions=0) + with pytest.raises(ValueError): + model.lpnorm(domain, p=2.0, iterations=0, forward_refine_splits=0) + + +def test_pz_l2norm_contains_monte_carlo_estimate(): + enable_interval_eval() + torch.manual_seed(7) + model = _small_tanh_model(input_dim=2, hidden_dim=2, output_dim=1) + domain = IntervalTensor.from_bounds([-1.0, -0.5], [1.0, 1.5]) + + bounds = model.lpnorm(domain, p=2.0, method="pz", iterations=1, chebyshev_degree=3, residual_subdivisions=24) + + sample_count = 2048 + with torch.no_grad(): + samples = torch.rand(sample_count, 2, dtype=torch.float64) + samples[:, 0] = 2.0 * samples[:, 0] - 1.0 + samples[:, 1] = 2.0 * samples[:, 1] - 0.5 + values = model(samples).squeeze(-1) + estimate = float((4.0 * torch.mean(values.abs() ** 2.0)).sqrt().item()) + + assert bounds.lower <= estimate <= bounds.upper + + +def test_pz_l2norm_uses_value_only_forward(monkeypatch): + enable_interval_eval() + model = _small_tanh_model(input_dim=2, hidden_dim=3, output_dim=1) + domain = IntervalTensor.from_bounds([-1.0, -0.5], [1.0, 0.5]) + + def fail_if_called(*args, **kwargs): + raise AssertionError("L2 computation must not construct a two-jet") + + monkeypatch.setattr( + "intervalnets.pz_integration._eval_pz_twojet", + fail_if_called, + ) + + bounds = model.pz_l2norm( + domain, + iterations=1, + chebyshev_degree=3, + residual_subdivisions=16, + ) + + assert bounds.lower <= bounds.upper + + +def test_pz_w12_order_one_sobolev_behavior(): + enable_interval_eval() + model = _small_tanh_model() + domain = IntervalTensor.from_bounds([-0.7], [0.9]) + + default_order = model.pz_sobolev_norm(domain, iterations=1, chebyshev_degree=3, residual_subdivisions=16) + order_one = model.pz_sobolev_norm(domain, order=1, iterations=1, chebyshev_degree=3, residual_subdivisions=16) + + assert order_one.lower <= default_order.upper + assert default_order.lower <= order_one.upper + + +def test_pz_w12_uses_onejet_without_constructing_hessian(monkeypatch): + enable_interval_eval() + model = _small_tanh_model(input_dim=2, hidden_dim=3, output_dim=1) + domain = IntervalTensor.from_bounds([-0.2, -0.1], [0.3, 0.2]) + + def fail_if_called(*args, **kwargs): + raise AssertionError("W12 computation must not construct a two-jet") + + monkeypatch.setattr( + "intervalnets.pz_integration._eval_pz_twojet", + fail_if_called, + ) + + bounds = model.pz_sobolev_norm( + domain, + order=1, + iterations=1, + chebyshev_degree=3, + residual_subdivisions=16, + ) + + assert bounds.lower <= bounds.upper + + +def test_pz_w22_order_two_sobolev_behavior(): + enable_interval_eval() + model = _small_tanh_model() + domain = IntervalTensor.from_bounds([-0.7], [0.9]) + + w12 = model.pz_sobolev_norm(domain, order=1, iterations=1, chebyshev_degree=3, residual_subdivisions=16) + w22 = model.pz_sobolev_norm(domain, order=2, iterations=1, chebyshev_degree=3, residual_subdivisions=16) + + # The reduced polynomial one-jet and full two-jet paths use different + # certified remainders, so their interval bounds need not be nested. Soundness + # and ||f||_{W12} <= ||f||_{W22} only require this cross-bound relation. + assert w22.upper >= w12.lower + + +def test_pz_domain_integration_of_odd_monomials_gives_zero(): + cell = PZIntegrationCell.from_bounds([-1.0], [1.0]) + x = cell.domain[0] + + result = integrate_over_cell(x + x * x * x, cell, output="pz") + + assert result.num_noise == 0 + assert result.center == pytest.approx(0.0) + assert result.terms == {} + + +def test_pz_approximation_noise_remains_after_domain_integration(): + cell = PZIntegrationCell.from_bounds([-1.0], [1.0]) + expr = PolynomialZonotope( + 1.0, + {(1, 0): 2.0, (0, 1): 0.25}, + num_noise=2, + noise_kinds=("domain", "approximation_symbolic"), + ) + + result = integrate_over_cell(expr, cell, output="pz") + + assert result.noise_kinds == ("approximation_symbolic",) + assert result.center == pytest.approx(2.0) + assert result.terms[(1,)] == pytest.approx(0.5) + + +def test_pz_output_keeps_only_non_domain_noise_after_integrating_network_integrand(): + enable_interval_eval() + model = _small_tanh_model() + cell = PZIntegrationCell.from_bounds([-0.5], [0.5]) + jet = model.eval_pz_twojet(cell.domain, chebyshev_degree=3, residual_subdivisions=16) + integrand = pz_twojet_l2_integrand(jet) + + result = integrate_over_cell(integrand, cell, output="pz") + + assert "domain" not in result.noise_kinds + assert result.num_noise == len(result.noise_kinds) + assert result.num_noise > 0 + + +def test_pz_interval_output_intervalizes_retained_approximation_noise_after_integration(): + cell = PZIntegrationCell.from_bounds([-1.0], [1.0]) + expr = PolynomialZonotope( + 1.0, + {(0, 1): 0.25}, + num_noise=2, + noise_kinds=("domain", "approximation_symbolic"), + ) + + symbolic = integrate_over_cell(expr, cell, output="pz") + interval = integrate_over_cell(expr, cell, output="interval") + + assert symbolic.noise_kinds == ("approximation_symbolic",) + assert symbolic.terms[(1,)] == pytest.approx(0.5) + assert interval.lower == pytest.approx(1.5) + assert interval.upper == pytest.approx(2.5) diff --git a/tests/test_pz_tanh.py b/tests/test_pz_tanh.py new file mode 100644 index 0000000..61f74c4 --- /dev/null +++ b/tests/test_pz_tanh.py @@ -0,0 +1,162 @@ +import math + +import pytest + +from intervalnets import Interval +from intervalnets.pz_tanh import ( + AffineTanhEnclosure, + TanhApproximation, + affine_tanh_double_prime_enclosure, + affine_tanh_enclosure, + affine_tanh_prime_enclosure, + quadratic_tanh_prime_enclosure, + certify_tanh_residual_subdivision, + compute_tanh_polynomial, +) + + +def _poly(coeffs, x): + acc = 0.0 + for coeff in reversed(coeffs): + acc = acc * x + coeff + return acc + + +def test_compute_tanh_polynomial_returns_certified_metadata(): + approx = compute_tanh_polynomial( + Interval(-1.0, 1.0), chebyshev_degree=5, subdivisions=32 + ) + + assert isinstance(approx, TanhApproximation) + assert approx.degree == 5 + assert len(approx.coeffs) == 6 + assert approx.lower == -1.0 + assert approx.upper == 1.0 + assert approx.delta >= 0.0 + assert approx.metadata["chebyshev_degree"] == 5 + assert "not a proof" in approx.metadata["proof_note"] + assert ( + approx.metadata["residual_certification"]["method"] + == "outward-rounded-subdivision" + ) + + +def test_compute_tanh_polynomial_accepts_deprecated_remez_alias(): + with pytest.warns(DeprecationWarning, match="remez_degree is deprecated"): + approx = compute_tanh_polynomial( + Interval(-1.0, 1.0), remez_degree=5, subdivisions=32 + ) + + assert approx.metadata["chebyshev_degree"] == 5 + assert approx.metadata["legacy_remez_degree"] == 5 + assert "remez_degree" not in approx.metadata + + +def test_subdivision_certificate_bounds_sampled_residuals(): + coeffs = (0.0, 1.0) # p(x)=x is intentionally crude away from zero. + delta, metadata = certify_tanh_residual_subdivision( + (-1.0, 1.0), coeffs, subdivisions=64 + ) + + assert delta > 0.0 + assert metadata["subdivisions"] == 64 + assert "not final root-isolation" in metadata["note"] + for idx in range(41): + x = -1.0 + idx / 20.0 + assert abs(math.tanh(x) - _poly(coeffs, x)) <= delta + + +def test_compute_tanh_polynomial_validates_proposal_with_certificate(): + approx = compute_tanh_polynomial((-2.0, 0.5), degree=3, subdivisions=80) + delta, _ = certify_tanh_residual_subdivision( + (approx.lower, approx.upper), approx.coeffs, subdivisions=80 + ) + + assert approx.delta == delta + for x in (-2.0, -1.25, -0.1, 0.5): + assert abs(math.tanh(x) - _poly(approx.coeffs, x)) <= approx.delta + + +def _tanh_prime(x): + t = math.tanh(x) + return 1.0 - t * t + + +def _tanh_double_prime(x): + t = math.tanh(x) + return -2.0 * t + 2.0 * t * t * t + + +@pytest.mark.parametrize( + ("helper", "func", "name", "interval"), + [ + (affine_tanh_enclosure, math.tanh, "tanh", (-2.0, 1.25)), + (affine_tanh_prime_enclosure, _tanh_prime, "tanh_prime", (-1.5, 1.75)), + ( + affine_tanh_double_prime_enclosure, + _tanh_double_prime, + "tanh_double_prime", + (-2.0, 2.0), + ), + ], +) +def test_affine_tanh_enclosures_bound_sampled_values(helper, func, name, interval): + enclosure = helper(interval) + + assert isinstance(enclosure, AffineTanhEnclosure) + assert enclosure.lower == interval[0] + assert enclosure.upper == interval[1] + assert enclosure.function == name + assert enclosure.delta >= 0.0 + assert enclosure.metadata["method"] == "finite-stationary-candidates" + assert "outward_rounding" in enclosure.metadata + assert len(enclosure.metadata["x_candidates"]) >= 2 + for idx in range(101): + x = interval[0] + (interval[1] - interval[0]) * idx / 100.0 + assert abs(func(x) - (enclosure.p * x + enclosure.q)) <= enclosure.delta + + +@pytest.mark.parametrize( + ("helper", "func", "name"), + [ + (affine_tanh_enclosure, math.tanh, "tanh"), + (affine_tanh_prime_enclosure, _tanh_prime, "tanh_prime"), + (affine_tanh_double_prime_enclosure, _tanh_double_prime, "tanh_double_prime"), + ], +) +def test_affine_tanh_enclosures_handle_point_intervals(helper, func, name): + enclosure = helper(Interval(0.25, 0.25)) + + assert enclosure.p == 0.0 + assert enclosure.q == func(0.25) + assert enclosure.delta == 0.0 + assert enclosure.lower == 0.25 + assert enclosure.upper == 0.25 + assert enclosure.function == name + assert enclosure.metadata["method"] == "point-interval" + assert enclosure.metadata["x_candidates"] == (0.25,) + + +@pytest.mark.parametrize( + "interval", + [(-2.0, 2.0), (-1.576, 1.613), (-1.0, 1.0), (-0.35, 0.8), (0.2, 1.4)], +) +def test_quadratic_tanh_prime_enclosure_bounds_sampled_values(interval) -> None: + lower, upper = interval + enclosure = quadratic_tanh_prime_enclosure(Interval(lower, upper)) + c, b, a = enclosure.coeffs + + assert enclosure.delta >= 0.0 + assert enclosure.metadata["certificate_subdivisions"] == 64 + for index in range(2001): + x = lower + (upper - lower) * index / 2000.0 + residual = _tanh_prime(x) - (c + b * x + a * x * x) + assert abs(residual) <= enclosure.delta + + +def test_quadratic_tanh_prime_enclosure_is_tighter_on_symmetric_bump() -> None: + interval = Interval(-1.6, 1.6) + affine = affine_tanh_prime_enclosure(interval) + quadratic = quadratic_tanh_prime_enclosure(interval) + + assert quadratic.delta < 0.4 * affine.delta diff --git a/tests/test_pz_twojet.py b/tests/test_pz_twojet.py new file mode 100644 index 0000000..91306d0 --- /dev/null +++ b/tests/test_pz_twojet.py @@ -0,0 +1,256 @@ +import math + +import pytest + +from intervalnets import PolynomialZonotope, enable_interval_eval, pz_twojet_forward +from intervalnets.pz_tanh import certify_tanh_residual_subdivision, compute_tanh_polynomial + +try: + import torch + from torch import nn +except ImportError: # pragma: no cover + torch = None + nn = None + + +def _poly(coeffs, x): + acc = 0.0 + for coeff in reversed(coeffs): + acc = acc * x + float(coeff) + return acc + + +def _eval_pz(z: PolynomialZonotope, eps): + if torch is None: + raise ImportError("PyTorch is required for this test helper.") + value = z.center.clone() if isinstance(z.center, torch.Tensor) else torch.tensor(z.center, dtype=torch.float64) + eps = torch.as_tensor(eps, dtype=value.dtype, device=value.device) + for exp, coeff in z.terms.items(): + monomial = torch.ones((), dtype=value.dtype, device=value.device) + for idx, power in enumerate(exp): + if power: + monomial = monomial * eps[idx].pow(power) + value = value + coeff * monomial + return value + + +def _assert_contains(interval, sample, atol=1e-10): + lo, hi = interval.to_torch(dtype=torch.float64) + sample = sample.detach().to(dtype=torch.float64) + assert torch.all(sample >= lo - atol), f"sample below lower bound: {sample} < {lo}" + assert torch.all(sample <= hi + atol), f"sample above upper bound: {sample} > {hi}" + + +def test_arithmetic_merges_equal_exponents_and_scales_fallback_coefficients(): + z1 = PolynomialZonotope(1.0, {(1,): 2.0}, num_noise=1) + z2 = PolynomialZonotope(3.0, {(1,): -5.0}, num_noise=1) + + summed = z1 + z2 + scaled = -2.5 * z1 + product = z1 * z2 + + assert summed.center == 4.0 + assert summed.terms[(1,)] == -3.0 + assert scaled.center == -2.5 + assert scaled.terms[(1,)] == -5.0 + assert product.center == 3.0 + assert product.terms[(1,)] == 1.0 + assert product.terms[(2,)] == -10.0 + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_arithmetic_scalar_times_vector_matrix_tensor_pz_and_outer_products(): + scalar = PolynomialZonotope(torch.tensor(2.0, dtype=torch.float64), {(1,): torch.tensor(-0.5, dtype=torch.float64)}, num_noise=1) + vector = PolynomialZonotope.constant(torch.tensor([1.0, -3.0], dtype=torch.float64), num_noise=1) + matrix = PolynomialZonotope.constant(torch.arange(1.0, 5.0, dtype=torch.float64).reshape(2, 2), num_noise=1) + tensor = PolynomialZonotope.constant(torch.arange(1.0, 9.0, dtype=torch.float64).reshape(2, 2, 2), num_noise=1) + + vector_product = scalar * vector + matrix_product = scalar * matrix + tensor_product = scalar * tensor + + assert vector_product.shape == (2,) + assert matrix_product.shape == (2, 2) + assert tensor_product.shape == (2, 2, 2) + assert torch.allclose(vector_product.center, torch.tensor([2.0, -6.0], dtype=torch.float64)) + assert torch.allclose(vector_product.terms[(1,)], torch.tensor([-0.5, 1.5], dtype=torch.float64)) + assert torch.allclose(matrix_product.center, 2.0 * matrix.center) + assert torch.allclose(matrix_product.terms[(1,)], -0.5 * matrix.center) + assert torch.allclose(tensor_product.center, 2.0 * tensor.center) + assert torch.allclose(tensor_product.terms[(1,)], -0.5 * tensor.center) + + outer = vector.tensor_product(vector) + assert outer.shape == (2, 2) + assert torch.allclose(outer.center, torch.tensor([[1.0, -3.0], [-3.0, 9.0]], dtype=torch.float64)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_interval_enclosure_contains_random_noise_samples(): + torch.manual_seed(0) + z = PolynomialZonotope( + torch.tensor([0.5, -1.0], dtype=torch.float64), + { + (1, 0, 0): torch.tensor([0.25, -0.5], dtype=torch.float64), + (0, 2, 0): torch.tensor([-0.1, 0.2], dtype=torch.float64), + (1, 0, 1): torch.tensor([0.05, 0.15], dtype=torch.float64), + }, + num_noise=3, + ) + enclosure = z.interval_enclosure() + for _ in range(128): + eps = 2.0 * torch.rand(3, dtype=torch.float64) - 1.0 + _assert_contains(enclosure, _eval_pz(z, eps)) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_shape_correctness_for_pz_twojet_network_outputs(): + d, m = 3, 2 + model = nn.Sequential(nn.Linear(d, 4, dtype=torch.float64), nn.Tanh(), nn.Linear(4, m, dtype=torch.float64)).double() + domain = PolynomialZonotope.from_box(torch.full((d,), -0.2, dtype=torch.float64), torch.full((d,), 0.3, dtype=torch.float64)) + + out = pz_twojet_forward(model, domain, chebyshev_degree=5, residual_subdivisions=64) + + assert out.Y.shape == (m,) + assert out.J.shape == (m, d) + assert out.H.shape == (m, d, d) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_affine_only_network_matches_exact_affine_value_jacobian_and_zero_hessian(): + d, h, m = 2, 3, 2 + model = nn.Sequential(nn.Linear(d, h, dtype=torch.float64), nn.Linear(h, m, dtype=torch.float64)).double() + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[1.0, -2.0], [0.5, 3.0], [-1.5, 0.25]], dtype=torch.float64)) + model[0].bias.copy_(torch.tensor([0.1, -0.2, 0.3], dtype=torch.float64)) + model[1].weight.copy_(torch.tensor([[2.0, -1.0, 0.5], [-0.25, 1.5, -2.0]], dtype=torch.float64)) + model[1].bias.copy_(torch.tensor([-0.4, 0.7], dtype=torch.float64)) + domain = PolynomialZonotope.from_box(torch.tensor([-1.0, 0.25], dtype=torch.float64), torch.tensor([0.5, 1.25], dtype=torch.float64)) + + out = pz_twojet_forward(model, domain) + expected_weight = model[1].weight.detach().matmul(model[0].weight.detach()) + expected_bias = model[1].weight.detach().matmul(model[0].bias.detach()) + model[1].bias.detach() + + assert torch.allclose(out.Y.center, expected_weight.matmul(domain.center) + expected_bias) + for exp, coeff in domain.terms.items(): + assert torch.allclose(out.Y.terms[exp], expected_weight.matmul(coeff)) + assert torch.allclose(out.J.center, expected_weight) + assert out.J.terms == {} + assert torch.equal(out.H.center, torch.zeros(m, d, d, dtype=torch.float64)) + assert out.H.terms == {} + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_tanh_residual_certificate_bounds_sampled_residuals_by_delta(): + approx = compute_tanh_polynomial((-1.25, 0.75), degree=5, subdivisions=96) + delta, metadata = certify_tanh_residual_subdivision((approx.lower, approx.upper), approx.coeffs, subdivisions=96) + + assert approx.delta == delta + assert metadata["method"] == "outward-rounded-subdivision" + for x in torch.linspace(approx.lower, approx.upper, steps=101, dtype=torch.float64): + residual = math.tanh(float(x)) - _poly(approx.coeffs, float(x)) + assert abs(residual) <= approx.delta + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_small_tanh_network_pz_twojet_encloses_autograd_samples(): + enable_interval_eval() + d, m = 2, 1 + model = nn.Sequential(nn.Linear(d, 2, dtype=torch.float64), nn.Tanh(), nn.Linear(2, m, dtype=torch.float64)).double() + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[0.4, -0.2], [0.1, 0.3]], dtype=torch.float64)) + model[0].bias.copy_(torch.tensor([0.05, -0.1], dtype=torch.float64)) + model[2].weight.copy_(torch.tensor([[0.5, -0.3]], dtype=torch.float64)) + model[2].bias.copy_(torch.tensor([0.02], dtype=torch.float64)) + lower = torch.tensor([-0.4, -0.2], dtype=torch.float64) + upper = torch.tensor([0.5, 0.3], dtype=torch.float64) + domain = PolynomialZonotope.from_box(lower, upper) + + out = model.eval_pz_twojet(domain, chebyshev_degree=5, residual_subdivisions=64) + y_interval = out.Y.interval_enclosure() + j_interval = out.J.interval_enclosure() + h_interval = out.H.interval_enclosure() + + samples = [lower, upper, (lower + upper) / 2] + samples.extend(lower + (upper - lower) * torch.tensor(pair, dtype=torch.float64) for pair in ((0.2, 0.8), (0.7, 0.1), (0.9, 0.6))) + for sample in samples: + x = sample.clone().detach().requires_grad_(True) + y = model(x) + jac_rows = [] + hessians = [] + for i in range(m): + grad = torch.autograd.grad(y[i], x, create_graph=True, retain_graph=True)[0] + jac_rows.append(grad) + hess_rows = [] + for j in range(d): + hess_rows.append(torch.autograd.grad(grad[j], x, retain_graph=True)[0]) + hessians.append(torch.stack(hess_rows)) + jac = torch.stack(jac_rows) + hess = torch.stack(hessians) + + _assert_contains(y_interval, y) + _assert_contains(j_interval, jac) + _assert_contains(h_interval, hess) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_pz_twojet_trace_is_opt_in_and_records_sequential_children(): + from intervalnets import PZTwoJet, PZTwoJetTraceResult + + model = nn.Sequential(nn.Identity(), nn.Linear(2, 3, dtype=torch.float64), nn.Tanh(), nn.Linear(3, 1, dtype=torch.float64)).double() + domain = PolynomialZonotope.from_box( + torch.tensor([-0.2, 0.1], dtype=torch.float64), + torch.tensor([0.3, 0.4], dtype=torch.float64), + ) + + untraced = pz_twojet_forward(model, domain, residual_subdivisions=32) + traced = pz_twojet_forward(model, domain, residual_subdivisions=32, return_trace=True) + + assert isinstance(untraced, PZTwoJet) + assert isinstance(traced, PZTwoJetTraceResult) + assert len(traced.records) == 1 + len(model) + assert traced.records[0].layer_index == -1 + assert traced.records[0].layer_name == "input" + assert traced.records[0].layer_type == "Input" + assert [record.layer_type for record in traced.records[1:]] == ["Identity", "Linear", "Tanh", "Linear"] + + for record in traced.records: + assert set(record.summary) == {"Y", "J", "H"} + for component_name in ("Y", "J", "H"): + component = getattr(record.jet, component_name) + summary = record.summary[component_name] + assert summary["shape"] == component.shape + assert summary["num_noise"] == component.num_noise + assert summary["noise_kinds"] == component.noise_kinds + assert summary["term_count"] == len(component.terms) + assert summary["max_degree"] == max((sum(exp) for exp in component.terms), default=0) + + assert traced.final.Y.shape == untraced.Y.shape + assert traced.final.J.shape == untraced.J.shape + assert traced.final.H.shape == untraced.H.shape + assert traced.final.Y.num_noise == untraced.Y.num_noise + assert traced.final.J.num_noise == untraced.J.num_noise + assert traced.final.H.num_noise == untraced.H.num_noise + for traced_interval, untraced_interval in ( + (traced.final.Y.interval_enclosure(), untraced.Y.interval_enclosure()), + (traced.final.J.interval_enclosure(), untraced.J.interval_enclosure()), + (traced.final.H.interval_enclosure(), untraced.H.interval_enclosure()), + ): + traced_lower, traced_upper = traced_interval.to_torch(dtype=torch.float64) + untraced_lower, untraced_upper = untraced_interval.to_torch(dtype=torch.float64) + assert torch.allclose(traced_lower, untraced_lower) + assert torch.allclose(traced_upper, untraced_upper) + + +@pytest.mark.skipif(torch is None, reason="PyTorch not installed") +def test_eval_pz_twojet_return_trace_uses_monkey_patched_method(): + from intervalnets import PZTwoJetTraceResult + + enable_interval_eval() + model = nn.Sequential(nn.Linear(1, 2, dtype=torch.float64), nn.Tanh()).double() + domain = PolynomialZonotope.from_box(torch.tensor([-0.1], dtype=torch.float64), torch.tensor([0.2], dtype=torch.float64)) + + traced = model.eval_pz_twojet(domain, residual_subdivisions=32, return_trace=True) + + assert isinstance(traced, PZTwoJetTraceResult) + assert len(traced.records) == 1 + len(model) + assert traced.records[-1].jet is traced.final diff --git a/tests/test_shallow_hybrid.py b/tests/test_shallow_hybrid.py new file mode 100644 index 0000000..8cbf37c --- /dev/null +++ b/tests/test_shallow_hybrid.py @@ -0,0 +1,139 @@ +from __future__ import annotations + +from math import sqrt + +import pytest + +torch = pytest.importorskip("torch") +from torch import nn + +from intervalnets import ( + IntervalTensor, + PZIntegrationCell, + PolynomialZonotope, + integrate_pz_onejet_squared, + integrate_pz_value_squared, + integrate_shallow_hybrid_onejet_squared, + integrate_shallow_hybrid_value_squared, + shallow_scalar_hybrid_onejet_reverse, +) +from intervalnets.pytorch import pz_value_forward + + +def _model() -> nn.Sequential: + model = nn.Sequential(nn.Linear(2, 3), nn.Tanh(), nn.Linear(3, 1)).double() + with torch.no_grad(): + model[0].weight.copy_(torch.tensor([[1.2, -0.4], [0.7, 0.9], [-1.1, 0.3]])) + model[0].bias.copy_(torch.tensor([0.0, 0.15, -0.05])) + model[2].weight.copy_(torch.tensor([[0.8, -0.6, 0.5]])) + model[2].bias.copy_(torch.tensor([0.1])) + return model + + +def test_shallow_reverse_hybrid_is_uncompressed_and_sound() -> None: + model = _model() + lower = torch.tensor([-0.6, -0.5], dtype=torch.float64) + upper = torch.tensor([0.6, 0.5], dtype=torch.float64) + domain = PolynomialZonotope.from_box(lower, upper) + result = shallow_scalar_hybrid_onejet_reverse( + model, + domain, + derivative_flatness_threshold=1.0, + ) + + quadratic_count = int(torch.count_nonzero(result.derivative_degrees == 2)) + expected_domain_terms = 2 + (3 if quadratic_count else 0) + assert len(result.domain_coefficients) == expected_domain_terms + assert len(result.derivative_error_generators) == 3 + assert len(result.final.J.terms) == expected_domain_terms + 3 + + samples = lower + (upper - lower) * torch.rand((4096, 2), generator=torch.Generator().manual_seed(7)) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + enclosure = result.final.J.interval_enclosure() + enclosure_lower = torch.as_tensor(enclosure.lower).reshape(1, 2) + enclosure_upper = torch.as_tensor(enclosure.upper).reshape(1, 2) + assert torch.all(gradients >= enclosure_lower) + assert torch.all(gradients <= enclosure_upper) + + +def test_shallow_reverse_reuses_preactivation_for_value_and_derivative() -> None: + model = _model() + lower = torch.tensor([-0.6, -0.5], dtype=torch.float64) + upper = torch.tensor([0.6, 0.5], dtype=torch.float64) + domain = PolynomialZonotope.from_box(lower, upper) + result = shallow_scalar_hybrid_onejet_reverse(model, domain) + + support = sorted( + domain.terms, + key=lambda exponent: next(i for i, power in enumerate(exponent) if power), + ) + input_coefficients = torch.stack([domain.terms[exponent] for exponent in support]) + expected_center = model[0].weight @ domain.center + model[0].bias + expected_coefficients = model[0].weight @ input_coefficients.T + expected_radius = torch.sum(torch.abs(expected_coefficients), dim=1) + assert torch.allclose(result.preactivation_center, expected_center) + assert torch.allclose(result.preactivation_coefficients, expected_coefficients) + assert torch.all(result.preactivation_lower <= expected_center - expected_radius) + assert torch.all(result.preactivation_upper >= expected_center + expected_radius) + + # The specialized value construction uses that same prepared data and + # reproduces the generic affine-tanh value enclosure. + generic = pz_value_forward(model, domain).interval_enclosure() + specialized = result.final.Y.interval_enclosure() + assert torch.allclose( + torch.as_tensor(specialized.lower), torch.as_tensor(generic.lower) + ) + assert torch.allclose( + torch.as_tensor(specialized.upper), torch.as_tensor(generic.upper) + ) + assert set(result.timings) >= { + "preactivation_preparation", + "activation_certification", + "value_construction", + "reverse_jacobian_construction", + "onejet_construction", + } + + +def test_shallow_direct_integral_matches_generic_uncompressed_reference() -> None: + model = _model() + box = IntervalTensor.from_bounds([-0.25, -0.2], [0.25, 0.2]) + cell = PZIntegrationCell.from_affine_box(box) + result = shallow_scalar_hybrid_onejet_reverse( + model, + cell.domain, + derivative_flatness_threshold=1.0, + ) + + specialized = integrate_shallow_hybrid_onejet_squared(result, cell) + reference = integrate_pz_onejet_squared(result.final, cell) + assert float(specialized.lower) == pytest.approx(float(reference.lower), rel=2e-12, abs=2e-12) + assert float(specialized.upper) == pytest.approx(float(reference.upper), rel=2e-12, abs=2e-12) + + # The exact sampled norm must be enclosed as a coarse additional check. + generator = torch.Generator().manual_seed(11) + samples = torch.tensor(box.lower, dtype=torch.float64) + ( + torch.tensor(box.upper, dtype=torch.float64) + - torch.tensor(box.lower, dtype=torch.float64) + ) * torch.rand((10000, 2), generator=generator) + samples.requires_grad_(True) + values = model(samples) + gradients = torch.autograd.grad(values.sum(), samples)[0] + sampled_squared = float( + (values.square().squeeze(1) + gradients.square().sum(1)).mean().detach() + ) + volume = float(cell.volume) + assert float(specialized.lower) / volume <= sampled_squared + assert sampled_squared <= float(specialized.upper) / volume + assert sqrt(max(0.0, float(specialized.upper) / volume)) > 0.0 + + specialized_l2 = integrate_shallow_hybrid_value_squared(result, cell) + reference_l2 = integrate_pz_value_squared(result.final.Y, cell) + assert float(specialized_l2.lower) == pytest.approx( + float(reference_l2.lower), rel=2e-12, abs=2e-12 + ) + assert float(specialized_l2.upper) == pytest.approx( + float(reference_l2.upper), rel=2e-12, abs=2e-12 + )