Repository navigation
[LiveFIFO] Globally optimal FIFO-sizing search strategies (bottom-up exact, branch-and-bound) - #298
Draft
fpjentzsch wants to merge 20 commits into
Draft
fpjentzsch wants to merge 20 commits into
fpjentzsch wants to merge 20 commits into
Conversation
Add two search methods next to the per-FIFO binary searches of the live FIFO sizing driver, selected by the new search_strategy kwarg of experiment_fifosizing (fifo_search_order is a deprecated alias for binary_<order>): - bottom_up_exact: best-first search from per-FIFO lower bounds upwards that only enlarges FIFOs observed blocking their producer (Stuijk et al. 2008); the first feasible configuration is cost-optimal. - top_down_bnb: branch-and-bound over boxes of depth vectors seeded with a binary search result; anytime, exact when run to completion. Both report a proven lower bound and the optimality gap and never return an infeasible configuration (the seed on budget exhaustion). The searches live in the oracle-agnostic driver module fifo_search.py; the hardware oracle, the capability probe and the reports are in live_fifo.py. Virtual FIFO hardware: the gauge records the peak occupancy of every run (READ_FILL now returns it in bounded mode as well) and run-sticky stall flags in bits [7:4] of READ_STALL, which stay valid in deadlocks; the controller issues READ_STALL as a 7-byte packet and returns the status in the low byte like READ_FILL. A new testbench with the controller in the loop checks both. Driver tests cover the strategies against brute force on random monotone instances and the full experiment on an emulated ring; a conftest stand-in for pynq lets them run off the board. The default CI matrix adds a budgeted top_down_bnb run, fifosizing_exact.json runs both exact searches.
…d seed variants of the exact searches The report_name kwarg of experiment_fifosizing selects the report sub-directory so that the two top_down_bnb seed variants do not overwrite each other. fifosizing_exact.json is removed, the default matrix covers all strategies.
The barrier waited for every gauge's detached input consumption as well as its output production. The first gauge consumes from the instrumentation source, which is not running in the detached phase, so it reported dirty forever and the controller's re-issue loop blocked the AXI read. The periods are recorded on the output side and are final once the production is done, so the barrier now only awaits the output side (plus partially transferred frames). The driver uses the barrier instead of a fixed 5 s sleep; the controller testbench replays the driver's sequence including its accelerator reset before every run.
…O sizing run scripts/compare_fifo_search_strategies.py reads the per-strategy fifo_sizing_report.json files of a Measurement job and prints one Markdown table per stop condition with the resulting FIFO size, number of hardware evaluations and search duration of every strategy.
…he strategy comparison
… strategy comparison
Every pop recomputed the cost of each child over all FIFOs, which made the Python side of a pop O(num_fifos * |dependency set|): about 3 ms per pop on a 142-FIFO model, comparable to the hardware evaluation itself. A child differs from its parent in one FIFO, so its cost is the parent's cost plus that FIFO's cost difference, rounded well below one bit so that equal-cost configurations keep the order of a direct computation.
…f sleeping LiveFIFOOracle.evaluate observes a candidate configuration by polling the new steady-state registers of the instrumentation wrapper (window maxima of interval and latency, idle cycles, age of the oldest pending frame) instead of sleeping for the fixed iteration runtime. Definitive negatives end an evaluation immediately (idle_timeout, pending_timeout, interval_violation, latency_violation); a positive verdict needs steady_state_frames completed frames with the window maxima within the thresholds (horizon_reached). The horizon is the old observation time in frames, derived from the paced latency and interval, at least 4 frames; the old runtime times cap_factor is a logged safety cap that falls back to the single-read decision. The measured interval and latency are the window maxima. EvalResult records the deciding rule, the frames and the evaluation time; they appear in all_iterations and the search summary counts the decisions. Bitfiles without the registers (detected from the HWH) keep the fixed runtime. The driver tests emulate the registers (frames completing over polls, deadlocking and slowly degrading configurations, the cap) and the fallback for old hardware.
Six read-only AXI-lite registers for steady-state detection in live FIFO sizing, appended after run_frames so that every existing register keeps its address and semantics: lat_win_min/max and int_win_min/max (extremes over the sliding window of avg_n frames, recomputed by a sequential one-entry-per-cycle scan after every completed frame, never in the cycle of a window update so that the window buffers see at most two accesses per cycle and II=1 is kept), idle_cycles (cycles since the last completed frame) and oldest_pending_age (cycles since the start of the oldest frame in flight, 0 when none; the head of the timestamp queue is held in a register because hls::stream cannot be peeked). The interval of the first frame after a cfg[0] rising edge is measured against the previous run or the reset and is excluded from the interval window extremes. The timestamp queue is now a static stream; a local stream loses its content between calls in C simulation. Vitis HLS 2024.2 keeps II=1 (depth 3) for the instrument process; the core costs about 1300 LUTs and 800 FFs more. The simulation testbench template is updated to the current register map (it still addressed a years-old layout), reads the new registers, checks their invariants and stops after a configurable number of frames.
Add finn.util.fifo_cost: the LUTs, flip-flops, RAMB18 tiles and URAM288s the consolidated RTL FIFO takes for a depth, width and ram_style (the fitted resource_models coefficients for LUT/BRAM/URAM at the style fifo.sv's RAM_STYLE_EFF ladder resolves, the RTL's register inventory for FFs), the fifo.sv geometry (implemented capacity, lo/hi memory spaces, pipeline and output queue), the depths at which the cost vector changes (the candidate ladder of a depth search, verified against a full scan), a documented scalarization in LUT equivalents of the target device's capacity, the monotone cost envelope a search that may only deepen FIFOs should minimise, and a JSON export of the ladder. MakePYNQDriver writes the per-FIFO tables and the device weights into the live FIFO sizing driver's settings.json (fifo_cost_tables); the driver's PrimitiveCost looks them up (envelope, realization, breakpoints), the Lattice accepts per-FIFO candidate lists, and experiment_fifosizing selects the cost model with cost_model="bits" (default) or "primitives", exporting the realized depths and reporting the primitive vector of the final configuration in both modes. The module docstring maps the TEG branch's fifo_depth_search functions onto the new API.
The steady-state registers cost about 1500 LUTs and 700 FFs on the instrumentation wrapper, which the CI benchmark comparison counts in the (top) row of post_synth_resources.json with a 1 % tolerance. They are now gated in the template by INSTR_STEADY_STATE: with the new DataflowBuildConfig option instrumentation_steady_state (default False) off, the generated C++ is the original wrapper (same register map, same code, Vitis HLS estimate identical: 2178 LUT / 2221 FF for the core); only the static timestamp stream (C simulation fix) is kept in both variants. ZynqBuild enables the registers automatically for live FIFO sizing builds, which need them for the steady-state detection of the driver. The enabled variant keeps the window update at function level where it is exclusive with the min/max scan, which keeps II=1 (nesting it like the original code lost the II to carried dependences on the scanner state).
…al_search # Conflicts: # driver/finn_plus_driver/overlays/live_fifo.py # driver/tests/test_live_fifo_experiment.py
ci/cfg/live_fifosizing_costcmp.yml builds every DUT of live_fifosizing.yml once per experiment config: fifosizing_cost_bits.json and fifosizing_cost_primitives.json run the same branch-and-bound search (seeded by the largest-first binary search, stop condition both) minimising the storage in bits or the FPGA primitive cost, and each result gets its own follow-up build. collect.py ignores experiments_config when matching reference experiments, since it only selects what is measured. scripts/compare_fifo_cost_models.py joins the search reports with the follow-up builds' FIFO depths, synthesized FIFO resources and measured interval into one table per DUT and cost model.
…low-up build The follow-up configuration copied every parameter of the measured run, including the new experiments_config of the cost-function comparison, so the follow-up measurement tried to run experiment_fifosizing on a build with real FIFOs and a normal driver (pipeline 158970). The follow-up uses the default experiment config again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two search strategies to live FIFO sizing that can find globally cost-optimal FIFO depth configurations, next to the existing per-FIFO binary searches, so that the binary-search results can be compared against a proven optimum.
What is in the PR
Virtual FIFO hardware (
finn-rtllib/fifo_virtual)fifo_gauge.sv:READ_FILLnow returns the peak occupancy of a run in every mode (before: the capacity register in bounded mode), andREAD_STALLreturns run-sticky stall flags in bits [7:4] next to the per-frame flags, so a deadlocked run still reports which FIFOs blocked their producer. Both are cleared by everyRUN_*command.fifo_controller.sv:READ_STALLis issued as a 7-byte packet likeREAD_FILLand returns the status in the low byte (the previous 4-byte packet returned the wrong slot; the driver never used it).BARRIER_CLEANfix: the barrier waited for the detached input consumption of every gauge, but the first gauge consumes from the instrumentation source, which is not running in that phase, so it reported dirty forever and the controller's re-issue loop blocked the AXI read. The barrier now only awaits the detached output production (the periods are final once that is done). The driver uses the barrier instead of a fixed 5 s sleep.sim/fifo_controller_tb.sv+fifo_controller.tcl: controller in the loop, driven over AXI-lite exactly like the driver (detached run with idle source, barrier, paced run, bounded run at the paced peaks reproducing them without a producer stall, unbounded capacity value, deadlock with sticky flags). Both testbenches pass in xsim 2024.2.Driver (
driver/finn_plus_driver/overlays)fifo_search.py(new, pure Python): oracle protocol, feasibility predicate (same thresholds as before), cost model, depth lattice, evaluation cache/log, the binary searches re-expressed on the oracle, the lower-bound phase,bottom_up_exact,top_down_bnb, and the strategy registry.live_fifo.py:LiveFIFOOracle(one hardware evaluation = the old inner loop; gauge reads only when a strategy asks for them),search_strategykwarg ofexperiment_fifosizing(fifo_search_orderstays as a deprecated alias forbinary_<order>),report_namefor report sub-directories, a capability probe for the gauge extensions, and asearchblock infifo_sizing_report.jsonwith evaluations, duration,optimal_proven, lower bound, gap and the full evaluation log.driver/tests/test_fifo_search.py(both exact searches against brute force on 200 random monotone instances, budgets, pareto mode, unreliable flags) andtest_live_fifo_experiment.py(the whole experiment on an emulated ring);conftest.pystubspynqso they run off the board too.CI
ci/experiments/fifosizing_default.json:search_strategykwargs, latency-only binary runs dropped,bottom_up_exactand twotop_down_bnbseed variants added (600 s budget each) forintervalandboth.collect.pystill picks the cheapest result for the follow-up build.scripts/compare_fifo_search_strategies.py: builds the comparison tables below from a Measurement job's reports.The two search methods, informally
Both rely on monotonicity: making any FIFO deeper never hurts, so if a configuration works, everything above it works, and if one fails, everything below it fails.
bottom_up_exact(Algorithm 1 of Stuijk, Geilen, Basten, Throughput-Buffering Trade-Off Exploration for Cyclo-Static and Synchronous Dataflow Graphs, IEEE Trans. Computers 57(10), 2008, https://doi.org/10.1109/TC.2008.58, with the hardware as oracle). Keep a to-do list of depth vectors sorted by cost, starting at the per-FIFO lower bounds (each FIFO at its minimum with all others unbounded). Pop the cheapest, measure it. If it works, it is optimal, because everything cheaper was already popped and found not to work. If it does not work, ask the gauges which FIFOs blocked their producer: any FIFO that never blocked could have been infinitely large without changing a single event, so only the blocked ones can help. Push one copy per blocked FIFO with that FIFO one step deeper, repeat. Exact, but it has to measure every configuration cheaper than the optimum that the expansions reach; with a paced source the backpressure flags every FIFO upstream of the bottleneck, so on larger DUTs that is far too many points (see results).top_down_bnb(generic branch-and-bound over boxes of depth vectors; not from the paper). Start from a feasible incumbent (a binary-search result or the paced occupancies) and the box between the lower bounds and an upper bound. For each box: if its cheapest corner cannot beat the incumbent, discard it; measure its most expensive corner, and if that fails, discard the whole box; otherwise that corner is a candidate, and the gauges' peak occupancies from the same run give an even cheaper configuration with the identical execution for free. Then split the box along the FIFO with the largest cost range and continue with the "shrink" half first. Anytime: stopping early returns the incumbent plus a bound on what could still be gained; running to completion proves optimality within the box. With only the shrink halves in a fixed FIFO order it degenerates into the per-FIFO binary search.Intermediate results (pipeline 154299,
live_fifosizing.yml, stop_condition = both)Cells: total FIFO size in kB (hardware evaluations, search duration). Cheapest per DUT in bold. ✓ optimality proven, ⏱ 600 s budget exhausted. UB = size of the paced-mode peak occupancies (start configuration), LB = cost of the per-FIFO lower bounds.
Reading: the small DUTs are solved exactly by every method (the lower-bound corner is itself feasible). The B&B improves on the best binary order on every larger DUT, by 1 % to 6 % within the budget and by 11 % on cnv-w2a2, where it ran to completion and proved the optimum.
bottom_up_exactexplodes on everything larger than the small DUTs (tens of thousands of evaluations without reaching a feasible point) and then returns its start configuration, as its theory predicts for a paced pipeline with per-element lattice steps. The two B&B seeds give the same result because the seed only sets the initial incumbent, not the traversal.One open observation from the follow-up builds: with the B&B configuration of the radioml transformer (12 of 78 FIFOs differ from the binary result by one or two elements) the real build reaches 1203 instead of 1174 cycles, although the virtual FIFOs measured the target exactly. All other follow-up builds hit their targets. This looks like the virtual-to-real FIFO gap at the boundary rather than a search bug and needs a closer look before the exact results on transformers are trusted.
Notes for review
bottom_up_exactrefuses (returns its seed) unlessallow_unreliable_flagsis set.cd driver && python -m pytest tests.Later additions on this branch
Steady-state detection (#296). With the new build option
instrumentation_steady_state(enabled automatically for live FIFO sizing builds, off otherwise so that the resource numbers of plain instrumented builds do not change; about 1500 LUTs / 700 FFs when on) the instrumentation wrapper exposes the window minimum/maximum of interval and latency, the cycles since the last completed frame and the age of the oldest frame in flight. The driver'sLiveFIFOOraclethen polls instead of sleeping: an evaluation ends immediately on a definitive negative (idle time or pending age above the threshold, window maximum above the threshold, instrumentation error) and otherwise onceNframes have completed with the window maxima within the thresholds, whereN = max(4, ceil(max(4 * paced_latency, 1e5) / paced_interval))frames mirrors the old observation time (205 frames for kws, 8 for mobilenetv1). The olditeration_runtimetimescap_factor(4) remains as a wall-clock cap that falls back to the old single-read verdict. Bitfiles without the registers keep the old behaviour. The report'ssearchblock lists which rule decided each evaluation (decisions). Verified with a cycle-level C simulation harness,csynth(II=1 kept, register map of the off variant identical to before) and an RTL simulation of the synthesized IP; not yet on a board.FPGA primitive cost model (#278).
src/finn/util/fifo_cost.pymirrors the selection ladder and geometry offifo.sv(throughresource_models._resolve/_fifo_cost, so it agrees withStreamingFIFO.get_fifo_cost()by construction) and returns per-FIFO vectors (LUT, FF, RAMB18, URAM), the depths at which the cost changes (cost_breakpoints, verified against a brute-force scan), the undominated steps of the non-monotone ladder, arealize_depthreplacing "round up to a full BRAM block", and a scalarization with device-capacity-relative weights (device_weights(fpgapart, board)).MakePYNQDriverexports per-FIFO cost tables into the driver'ssettings.json, andexperiment_fifosizing(cost_model="primitives")searches over the breakpoint lattice with that cost (default stays"bits"; both modes report the primitive vector of the final configuration when tables exist). The module docstring contains the mapping ontofifo_depth_search.pyof the TEG branch so that the simulation-based minimization can switch to it. Deviations from the old simulation heuristics: narrow (<5 bit) FIFOs stay LUTRAM at any depth, URAM above 2028 is priced, the LUTRAM/BRAM boundary is 257 with power-of-two steps, shift registers only up to 33 (64 for <12 bit), BRAM lo/hi sub-steps.Cost-function comparison benchmark.
ci/cfg/live_fifosizing_costcmp.ymlbuilds every DUT oflive_fifosizing.ymlonce per experiment config (fifosizing_cost_bits.json,fifosizing_cost_primitives.json): the same branch-and-bound search minimising bits or primitives, each with its own follow-up build.scripts/compare_fifo_cost_models.pytabulates predicted vs. built FIFO cost in both metrics plus the follow-up interval.collect.pyignoresexperiments_configwhen matching reference experiments.Optional improvements
A cost function based on FPGA primitive counts instead of kBDone on the branch (see "Later additions"):finn.util.fifo_costand theprimitivescost model of the driver (Align the distributed FIFO-sizing cost heuristics with the consolidated fifo.sv (after #259) #278). Still open: switching the CI matrix tocost_model: primitivesand consolidating the per-primitive vector into one scalar (currently device-capacity-relative LUT equivalents).Rework of the steady-state trackingDone on the branch (see "Later additions"): early negative verdicts and a frame-based horizon in the instrumentation wrapper and the driver (FIFO sizing: steady-state detection relies on observation-horizon heuristics (live_fifo and distributed_sim) #296). Still open: a board run to see the decision mix, and the long-horizon confirmation of the final configuration.bottom_up_exactis already fixed on the branch: child costs are computed incrementally, 3 to 4x faster per pop on 142 FIFOs.)bottom_up_exactwith a binary-search result so that a budgeted run returns something useful.