Consider an illustrative measurement: “Trip creation took two minutes.” It sounds useful.
It is not enough to improve the system.
An AI itinerary is produced by a sequence of different operations. A model interprets the request. Another stage shapes the route. Day planning expands the route into daily intent. Place providers return candidates and metadata. Activities are scheduled, the result is reviewed, repairs may run, and the graph is finally persisted.
One total duration tells us that the traveler waited. It does not tell us why.
Measure the stages before optimizing them
A multi-stage workflow needs one trace identity and a duration for each meaningful boundary.
For trip creation, useful stage families include:
- Intake and brief compilation
- Clarification and traveler wait time
- Route construction
- Day planning
- Place research and metadata enrichment
- Activity and meal scheduling
- Quality validation and bounded repair
- Persistence and browser handoff
The exact implementation can change while these categories remain understandable. A trace should record both the total wall-clock duration and the contribution of each stage.
Without that breakdown, the team may optimize database writes while the critical path is waiting on sequential place research. Or it may blame the model when the browser is polling a trip that was already saved.
Wall time is not the sum of every call
Some work runs in parallel.
If four independent place searches each take one second and execute concurrently, they contribute roughly one second to wall time, not four. Their combined provider duration is still useful for cost and load analysis, but adding every call together would overstate the traveler's wait.
A good trace distinguishes:
- Wall-clock stage time: what delayed completion.
- Aggregate call time: how much provider or model work occurred.
- Concurrency: which calls overlapped.
- Queue time: how long work waited before execution.
- Traveler time: time spent waiting for clarification, which should not count as compute latency.
This is the difference between understanding the critical path and merely counting activity.
Compare like trips with like trips
A short city break and a multi-country itinerary do not have the same workload.
Latency analysis should retain bounded complexity descriptors such as day count, route-segment count, selected-place count, planning strategy, and whether repair occurred. It does not need the destination names, authored prompt, trip title, or traveler identity.
These descriptors make comparison fairer. A slower week may contain more long trips rather than a regression. A place-led plan may spend less time interpreting destinations and more time clustering a large candidate set. A repaired trip should not be compared blindly with one accepted on the first pass.
Broad cohorts are useful. Tiny, highly specific slices can become both statistically noisy and privacy-sensitive.
Percentiles show the experience averages hide
An average can look healthy while a meaningful group of travelers waits much longer.
The median describes the ordinary case. The 90th and 95th percentiles reveal the slow tail. Maximum latency can help investigate incidents but is too sensitive to one anomaly to serve as the main product measure.
Each percentile needs a sample size and a defined time window. A p95 calculated from a handful of trips is not a stable service claim. Failed attempts should be reported separately rather than disappearing from the latency chart, since a fast failure is not a performance success.
We also need to distinguish model latency, provider latency, persistence latency, and end-to-end latency. They answer different operational questions.
Cost follows a different shape
The slowest stage is not always the most expensive one.
A model call can consume significant tokens while returning quickly. A provider request can wait on network latency while costing little. Repeated quality passes may add both cost and time. Photo requests may not block trip creation at all but still matter to total product spend.
Stage telemetry should therefore capture bounded model-call counts, token estimates, provider-call counts, cache disposition, and repair attempts alongside duration. Cost attribution can then follow the workflow that initiated the work without retaining the user's text.
This makes tradeoffs visible. A faster model is not an improvement if it causes more repair calls. A broader place search is not free merely because it runs concurrently.
Observability should not become a second copy of the trip
Performance analysis does not require raw prompts or complete itineraries.
A sanitized trace can retain:
- Workflow and stage names
- Version and model-purpose identifiers
- Start, finish, and outcome
- Aggregate token and call counts
- Bounded complexity measures
- Stable failure and fallback codes
- Whether the result was complete, partial, or blocked
It should exclude private conversation text, traveler details, booking information, full provider responses, and user-facing itinerary content. The trace explains system behavior rather than recreating customer data in an analytics table.
Retention and access should match that purpose. Operational detail useful for a current investigation need not live forever.
Optimize from the critical path outward
Once stage evidence exists, improvements become more disciplined.
If route and day planning are sequential but independent work can begin earlier, change the dependency graph. If place research dominates, reduce unnecessary detail calls, deduplicate shared lookups, and bound concurrency. If model input grows with unrelated context, tighten the packet. If persistence is slow, inspect transaction shape and indexes. If the trip is complete but the browser keeps waiting, fix the handoff contract.
Every optimization should preserve correctness. Removing provider detail that the ranker actually uses can make the system faster and the trip worse. Skipping review can improve latency while increasing unusable results. Performance work needs quality and cost measurements beside speed.
Review weekly and monthly for different reasons
A weekly review is useful for catching sudden changes in model routing, provider behavior, code revisions, or failure mix. It keeps the likely cause close to the observed shift.
A monthly review provides enough volume to examine percentile movement, trip-complexity mix, cost per successful result, repair frequency, and whether improvements held across different workflows.
Neither cadence should turn one number into a verdict. The useful questions are: which stage changed, for which class of work, with what effect on success and quality?
The durable lesson
End-to-end latency matters because it is what the traveler experiences. Stage latency matters because it is what engineers can change.
A useful performance system connects the two without collecting the trip itself. It follows the critical path, distinguishes parallel work from waiting, compares similar workloads, and measures cost and quality beside speed.
Only then does “two minutes” become the beginning of an investigation rather than the end of one.
Loading comments...