An engineering team at an AI-native startup runs on a new playbook. Modern tooling lets 3 to 5 engineers ship what demanded 15 back in 2020, and the bottleneck moved from writing code to evaluating models and paying inference bills. The right lean structure from day one prevents six months of rework.
Seniors only until product-market fit
Hire 3 to 5 senior engineers and leave the junior layer for after traction. Product shape changes every week in the early phase; seniority absorbs a pivot without rebuilding foundations. Big teams multiply communication channels, and communication is the scarce resource in the first months.
Two roles that outrank another full-stack hire
The evaluation engineer owns eval suites and quality gates; without one, AI features regress in silence and customers find out first. The data engineer builds corpus pipelines: ingestion, cleaning, and versioning for everything feeding models. Both roles sustain product quality in ways another generalist cannot.
Inference costs belong in the architecture doc on day one
Token spend per user, model routing, and caching strategy belong in the original architecture document. Cost control retrofitted afterwards burns six months of refactoring. Put targets on paper: inference cost per active user, gross margin per plan, spend ceiling per request.
Start on APIs before thinking about GPUs
APIs from Anthropic, OpenAI, and Google cover 90% of use cases in the beginning. Move to fine-tuning or self-hosting when unit economics prove the need: high volume, API costs squeezing margin, or contractual privacy requirements. Premature infrastructure kills startups before traction, with the GPU bill as the root cause.
Eval-driven development
Assemble a golden dataset in week one and treat it as a contract: every prompt, model, or pipeline change runs the suite before merge. A regression gate in CI catches degradation human review misses. An AI feature without an eval suite is a bet charged to the customer's credit card.
What investors ask now
Demos impress for five minutes; eval methodology and margin structure carry due diligence. Bring defensible answers: suite coverage, captured regression rate, inference cost per user, margin trajectory per plan. Series B conversations already treat these numbers as knockout criteria.
Rituals that protect quality
Hold an eval review with the whole team every week, run incident postmortems covering model failures the way code failures get covered, and route prompt changes through review like any code change. Prompts are code: version them, test them, audit them. Teams with these rituals catch regression in hours, ahead of the first customer ticket.
