Skip to content

Running an experiment

The Run step is a live dashboard. Once you start an experiment, it shows progress as it goes, so you always know how far along it is and how it is doing.

The Run step after a finished run: the completion card, progress cards, the best-so-far chart, and what moved the needle.

Four cards along the top track Points Tried (Design Points on a fixed design), Runs Completed, Elapsed, and Estimated Remaining, with a progress bar underneath showing how much of the run budget is used. Because the search knows its budget, the estimate is honest about how much is left.

The chart plots each configuration as it finishes, in the order it ran, against your primary response. A line traces the best result found so far, so you can watch it climb and then level off as the search settles. Feasible points are filled; points that broke a limit show as hollow rings and are excluded from the ranking.

Beside the chart, a panel ranks the factors by how much moving each one across its tested range changed the result, longest bar first. It fills in when the run completes, once the whole picture is in: while the run is still going, the panel says the answer is on its way. The same chart appears on Results as the main effects.

Below the chart, Live Leaderboard ranks the best points so far, six at a time. Its header says how many it shows, such as “Top 6 of 6 points, updated a moment ago”. Each row gives the rank, the value of each factor, the primary response and a trend chip: New for a point that has just entered the board, Up n or Down n for a point that has moved up or down the board by n places, and Steady for one that has not moved. The point that just finished is highlighted and carries a “Point n” badge, and the line under the table says where it entered the board. If that point broke a limit, it is pinned unranked with a red Infeasible chip.

A row that reads “Baseline, today’s settings”, marked Vs today, is your model as it stands, so you can see what the points are measured against. It needs at least two replications; with one, a note under the table says why the row is missing. Points still running sit below the board, marked Running.

The Run step of PM Optimizer for the experiment Queue size and who goes first, scrolled to the Live Leaderboard and All Points. The leaderboard header reads Top 6 of 6 points, updated a moment ago, and its columns are Rank, Treatment Line: Capacity, Treatment Line: Queue Discipline, Urgent: Completed Count and Trend. The first row is Baseline, today's settings, marked Vs today. Below it come six ranked points with the chips Steady, New with Point 6 on the highlighted row at rank 2, and Down 1, and a line saying the highlighted row is the point that just finished and entered at rank 2. Under it, All Points lists Baseline and the points P1 and P2 with Status Done, Reps 2 / 2 and Why Explored.
Figure 1. The Run step with the Live Leaderboard above All Points: the Baseline row marked Vs today and the point that just finished highlighted.

All Points lists every design point with its status: #, Point, Status, Reps and the primary response. Status reads Queued, Running, Done, Failed or Cut short. A Smart Optimizer run adds a Why column that reads Explored for a point’s first pass, Leader, refined for a leader that earned extra replications, and “Cut short” with the reason for a point the search stopped spending runs on.

By default, each configuration is tested against the same random conditions, replication for replication. This paired comparison means the differences you see come from the settings you changed, not from one configuration happening to get an easier run. It makes the comparison much sharper at no cost to you.

Stop ends the run. The points gathered so far stay on screen for you to read, but a stopped run is not a finished experiment: it is not saved with the model, and the Results tab stays locked. Run the experiment again when you want a result you can rank and keep. A Smart Optimizer run also stops on its own when better results stop coming.

At completion, the top of the dashboard becomes a summary card: how many points were evaluated, how long it took, and the best value found, with one button, View Results. The window then opens the Results step for you, where the ranked table and the export live.

If the app closes in the middle of a run, nothing gathered is lost silently: the next time you open PM Optimizer, it offers the stranded results, to export as a spreadsheet or to discard.