The 6 Dimensions To Score Before Recommending an AI Engagement

A portfolio with multiple AI initiatives, several running past their original deadline, only a few producing measurable value, and some already obsolete before they reached production. That pattern is common enough that most enterprise AI leaders recognize it immediately.

The question is not whether the portfolio has problems. The question is which initiatives deserve continued investment, which need redesign, which should be consolidated, and which should stop.

That decision is harder than it sounds. Each initiative has a sponsor, a team, sunk cost, and a narrative. Without a structured evaluation surface, portfolio triage defaults to politics, recency, or whoever argues loudest.

This six-dimension scorecard is a practical way to decide whether an AI engagement should move forward.

The Scorecard Is A Decision Surface, Not A Grade

The six dimensions are not a universal score. They are a surface for making a specific decision: should this initiative be funded, redesigned, consolidated, or stopped?

Different organizations weight each dimension differently. A regulated financial institution will weight governance overhead more heavily than a startup. A company with strong data infrastructure may treat data readiness as a near-automatic green. A team with mature deterministic systems may need stronger evidence before approving agentic complexity.

The scorecard works because it forces six explicit conversations that most portfolio reviews skip.

Dimension 1: Problem Clarity

If the stakeholder cannot state the problem in one sentence, the initiative is not ready for an engagement recommendation.

This is not a test of whether the problem is simple. Complex problems can still be clearly stated. “We need to reduce false positives in our fraud detection pipeline without increasing review latency” is complex and clear. “We want to use AI to improve operations” is neither.

The follow-up question that reveals the most: “What would success look like by the next review window?” If the answer is vague or changes depending on who you ask, the problem is not defined well enough to evaluate architecture, team, or investment.

Dimension 2: Data Readiness

Data readiness is not “we have data.” It is: we have labeled, versioned, accessible data with a known owner and a documented refresh cadence.

Most enterprise AI initiatives stall not because the model is wrong but because the data pipeline is not ready for production workloads. The team builds on exported spreadsheets, one-time database snapshots, or data that requires manual assembly before each run.

The diagnostic questions:

• Who maintains this data?
• When was it last validated?
• What happens when the source schema changes?
• Is there a pipeline, or is there a person?

If the answer to the last question is a person, the initiative is carrying infrastructure debt that will surface under production load.

Dimension 3: Complexity Fit

This is the dimension most often misjudged.

Agentic architecture is attractive because it sounds flexible, adaptive, and future-proof. But it adds real cost: more state management, more evaluation burden, more latency, more failure modes, and more governance surface.

The honest question is: does this problem require agentic architecture, or is a deterministic workflow sufficient?

Many enterprise AI initiatives that are framed as “agentic” are actually rule-based classification, structured extraction, or deterministic routing problems. Adding agent-level complexity to those problems creates cost without proportional value.

The scorecard does not penalize agentic architecture. It asks the team to prove the need before the engagement assumes it.

Dimension 4: Governance Overhead

Before an AI system runs in production, someone must be able to answer:

• Who reviews outputs before they reach a user or downstream system?
• What logging exists?
• What triggers a rollback?
• What is the escalation path for unexpected behavior?

If none of those answers exist, the initiative is not ready for production advisory. It needs governance design first.

This dimension becomes more important as the initiative’s blast radius grows. A system that drafts internal summaries has lower governance overhead than a system that updates customer records, triggers financial transactions, or generates external communications.

The follow-up that reveals the most: “What would you do if the system produced a confidently wrong output on a high-stakes decision?” If the team does not have a concrete answer, the governance layer is not designed.

Dimension 5: Integration Surface

Every system the AI initiative must touch is a potential failure point. The question is not just how many integrations exist, but whether each one is documented, maintained, and understood by someone on the team.

Undocumented APIs, manual data transfers, and tribal-knowledge integrations are common sources of unexpected delay in enterprise AI engagements. The initiative looks architecturally ready until the team discovers that a critical data source requires a long procurement process, or that the target system’s API was last updated by someone who left the company.

The diagnostic question: “What is the most fragile integration in this system?” If the team cannot name it, the integration surface has not been assessed.

Dimension 6: Production Trajectory

An initiative without a production timeline is an experiment. That is fine, but it should not be evaluated as an engagement candidate.

The scorecard asks for a production trajectory: how long until production, what are the exit criteria for each stage gate, and what would cause the team to stop.

The last question is the most important. If there are no kill criteria, the initiative will continue consuming resources past the point where the expected value justifies the investment. That is not a technology problem. It is a portfolio governance problem.

# Illustrative policy thresholds; replace with organization-specific gates.
readiness_assessment:
  initiative: "customer support escalation routing"
  dimensions:
    problem_clarity:
      signal: green
      statement: "Reduce misrouted escalations by classifying intent before handoff"
      owner: "VP Customer Operations"
    data_readiness:
      signal: yellow
      note: "Historical escalation data exists but labeling is incomplete"
      follow_up: "Who will label the validation set and by when?"
    complexity_fit:
      signal: green
      note: "Classification, not agentic — deterministic routing with model-assisted intent detection"
    governance_overhead:
      signal: yellow
      note: "Review gate exists for manual escalation but not for automated routing"
      follow_up: "What happens when automated routing is wrong on a high-value customer?"
    integration_surface:
      signal: red
      note: "CRM integration is undocumented and maintained by one engineer"
      follow_up: "What is the fallback if the CRM integration fails during production?"
    production_trajectory:
      signal: green
      note: "Bounded path with stage gates at classification accuracy, routing accuracy, and reviewer trust"
      kill_criteria: "Classification accuracy remains below the approved threshold at the review gate"
  recommendation: "Proceed with bounded scope. Address CRM integration documentation and governance gate design before expanding routing autonomy."

The scorecard structure is straightforward enough to formalize. A portfolio workflow can pull dimension signals from structured intake forms or discovery notes and model each initiative as a typed object with explicit signal values and remediation paths. The model enforces completeness: an initiative cannot reach the recommendation stage without all six dimensions assessed.

from __future__ import annotations

from enum import Enum
from typing import Optional

from pydantic import BaseModel, Field, model_validator


class Signal(str, Enum):
    GREEN = "green"
    YELLOW = "yellow"
    RED = "red"


class DimensionScore(BaseModel):
    signal: Signal
    note: str = Field(..., min_length=10)
    follow_up: Optional[str] = None
    owner: Optional[str] = None


class EngagementReadinessScore(BaseModel):
    initiative: str
    problem_clarity: DimensionScore
    data_readiness: DimensionScore
    complexity_fit: DimensionScore
    governance_overhead: DimensionScore
    integration_surface: DimensionScore
    production_trajectory: DimensionScore

    @model_validator(mode="after")
    def validate_kill_criteria(self) -> "EngagementReadinessScore":
        pt = self.production_trajectory
        if pt.signal in (Signal.GREEN, Signal.YELLOW):
            if "kill" not in pt.note.lower() and not pt.follow_up:
                raise ValueError(
                    "Production trajectory scored green/yellow must specify "
                    "kill criteria in note or follow_up."
                )
        return self

    def recommendation(self) -> str:
        signals = [
            self.problem_clarity.signal,
            self.data_readiness.signal,
            self.complexity_fit.signal,
            self.governance_overhead.signal,
            self.integration_surface.signal,
            self.production_trajectory.signal,
        ]
        red_count = signals.count(Signal.RED)
        green_count = signals.count(Signal.GREEN)

        if red_count >= 3:
            return "STOP"
        if self.problem_clarity.signal == Signal.RED:
            return "STOP"
        if self.production_trajectory.signal == Signal.RED and red_count >= 2:
            return "REDESIGN"
        if green_count >= 5:
            return "FUND"
        return "REDESIGN"

The model_validator encodes a useful review rule: any initiative scored green or yellow on production trajectory must have explicit kill criteria. Without that check, “green” on trajectory becomes a rubber stamp. The recommendation logic is intentionally opinionated: problem clarity is not recoverable with more data, so a red signal there routes directly to stop regardless of the other five dimensions.

What The Scorecard Produces

The scorecard does not produce a number. It produces a recommendation:

• Fund: all six dimensions are green or yellow with clear remediation paths.
• Redesign: the initiative has merit but the current approach has structural gaps that will not close under the existing plan.
• Consolidate: two or more initiatives are solving overlapping problems with separate teams and separate infrastructure.
• Stop: the initiative cannot articulate the problem, does not have the data foundation, or has no credible production trajectory.

The hardest recommendation is “stop.” But it is also the most valuable, because it frees resources for the initiatives that are actually ready.

For the discovery conversation that typically precedes this scorecard, see Enterprise AI Use-Case Intake System. For teams that have already completed the scorecard and need a structured portfolio-level review, see What an Enterprise Agentic Portfolio Review Should Produce in 30 Days.

Using The Scorecard Across A Portfolio

The scorecard becomes most useful when applied across multiple initiatives in the same review cycle.

Most enterprise AI portfolios contain a mix of well-defined projects with clear production paths, promising experiments that need more data or governance work before they are engagement-ready, legacy initiatives that persist because no one has formally recommended stopping them, and overlapping efforts that could be consolidated.

The six dimensions give the review team a common language for comparing initiatives that would otherwise be evaluated on incompatible criteria.

• Score each initiative independently before comparing across the portfolio.
• Weight dimensions based on organizational context, not a universal formula.
• Use red signals as diagnostic inputs, not automatic disqualifiers.
• Require kill criteria for every initiative that receives continued funding.
• Reassess at each stage gate, not just at the initial review.

FAQ

The Decision Rule

Do not fund an AI engagement because the prototype is impressive. Fund it when the problem is clear, the data has an owner, the complexity is justified, the governance layer is named, the integrations are understood, and the path to production includes kill criteria.