Tag: Engineering Governance

Not every infrastructure pull request deserves the same review path. A tag change in a development account and a network-policy change in production should not create identical reviewer load. When every change is treated as high risk, reviewers stop trusting the signal. 

In IaC review, I have seen reviewers spend too much attention on low-risk changes while subtle production changes move through with weak context. Risk scoring is useful when it redirects human judgment instead of pretending to replace it. 

Risk-based review gives platform teams a more useful pattern. The system scores an IaC change using evidence from the diff, environment, resource type, dependency criticality, recent incidents, ownership and rollout plan. The score does not replace reviewers. It decides how much review the change deserves. 

What the Score Should Consider 

A good score is boring and explainable. It should include blast radius, production exposure, stateful resource impact, identity or network changes, rollback difficulty, missing evidence and whether the affected service has recent incidents. The goal is not machine judgment. The goal is consistent triage. 

risk_inputs:
  environment: production
  resource_type: iam_policy
  blast_radius: account-wide
  rollback: manual
  owner_present: true
  recent_incidents: 1
  missing_evidence: false 

The output should be equally clear: Fast path, owner review, platform review, security review, staged rollout or block until evidence is complete. 

Why Pull Requests are the Right Surface 

The pull request already has context: Author, diff, reviewers, checks, target branch and deployment environment. That makes it the best place to explain risk while the change is still cheap to adjust. A weekly governance report may be useful for leaders, but it is too late for the engineer trying to merge safely. 

The comment should not dump raw policy output. It should say which resources changed, which risk factors mattered, what review path was selected and what would lower the risk. 

Implementation Sketch 

def risk(change):
    score = 0
    if change.environment == “production”:
        score += 25
    if change.resource in {“iam_policy”, “security_group”, “route_table”}:
        score += 25
    if not change.owner:
        score += 20
    if change.rollback == “manual”:
        score += 15
    return min(score, 100) 

This example is intentionally simple. Production systems should use tested rules, not mysterious weights. If a score changes, reviewers should know which input changed. 

Avoiding Reviewer Fatigue 

The biggest risk is over-escalation. If every production change goes to a central team, the system becomes a bottleneck. Use thresholds carefully. A production tag change with complete evidence may need only normal owner review. A production identity change with a missing rollback deserves more attention. 

Track false positives, false negatives, review latency, override rate and repeated high-risk patterns. If a category repeatedly scores high because the platform lacks a safer workflow, that is roadmap input. 

Evidence and Replay 

Each scored decision should be stored with policy version, inputs, score, selected path, reviewer and final outcome. This matters during incidents. If a change later causes a problem, the team can reconstruct why the review path seemed reasonable at the time. 

The evidence record should not be treated as compliance paperwork. It is operational memory for the delivery system. 

Rollout Plan 

Start in advisory mode for a month. Compare the score with human judgment. Look for missing context and confusing explanations. Then enforce one narrow case, such as production changes with missing owner or missing rollback. Expand only after the system earns trust. 

Risk-based review is strongest when it reduces noise. The platform should make safe changes faster and risky changes clearer, not simply add another required check. 

Calibrating the Score 

Risk scoring fails when teams cannot see how the number was produced. Keep the first model simple and publish the scoring table. If identity changes add 25 points and missing rollback adds 15, say that. Reviewers do not need a mysterious model; they need a consistent way to decide where to spend attention. 

A monthly calibration review should compare score, reviewer decision, deployment outcome and incident follow-up. If low-score changes repeatedly cause issues, the model is missing a signal. If high-score changes routinely pass without concern, the model may be too sensitive. 

Example Workflow Output 

Risk score: 72 / 100
Reason: production IAM policy change, account-wide scope, manual rollback
Decision: platform owner review required
Remediation: add rollback plan or reduce policy scope
Evidence: policy-v4, commit 7a91c2, service owner payments-platform 

This output offers the engineer something to act on. It also gives reviewers a shared language. The discussion becomes about specific risk factors rather than general discomfort. 

Anti-Patterns 

Avoid scores with too many hidden inputs. Avoid global thresholds that ignore environment. Avoid blocking changes without explaining remediation. Avoid treating the model as finished. Infrastructure platforms change constantly, and the review model should evolve with them. 

The best risk-based review systems become quieter over time because the platform learns which changes are routine and which patterns deserve deeper attention. 

Keeping it Practical 

The review model should be easy to explain to a new engineer. If a team cannot describe why a change was routed to security review, the scoring system is too opaque. Good risk scoring gives engineers a shared vocabulary: Production exposure, blast radius, rollback difficulty, ownership and missing evidence. 

Final Check 

Before requiring the score, replay it against the last month of infrastructure changes. Ask whether the model would have escalated the changes that engineers actually worried about.