Shift-Left vs Shift-Right Testing in the AI Era: Where Does QA Really Begin… and End?

For years, we have heard:

“Shift Left — test early, find bugs early.”

And it makes perfect sense.

But AI is changing the equation.

Imagine an AI-powered customer-support assistant. Before production, your QA team validates 500 scenarios.

498 pass. 2 fail.

The release goes live.

A few days later, customers start reporting that the assistant is giving inconsistent answers to the same question.

Nothing changed in the UI.

No API contract was broken.

All regression tests are green.

So what happened?

The problem wasn’t necessarily a pre-production defect.

It was a production behavior problem.

And this is why I believe the future isn’t:

Shift Left OR Shift Right

It’s:

Shift Left + Shift Right = Continuous Quality

---

What should QA own before production?

Shift Left isn’t simply about writing automation earlier.

It’s about preventing predictable risk before users encounter it.

QA should challenge:

Requirements

→ Are they complete, unambiguous, and testable?

Risk

→ What happens if this feature fails?

Test strategy

→ What should be tested at unit, API, component, integration, and UI levels?

Automation

→ Are we automating the right scenarios or simply increasing test count?

Non-functional quality

→ What about performance, security, accessibility, resilience, and observability?

AI-generated code and tests

→ Did AI generate useful coverage—or just generate more tests?

Here’s an important distinction:

100 AI-generated test cases ≠ 100 meaningful test cases.

QA’s role becomes increasingly important because someone still needs to ask:

“What did we miss?”

---

Now comes the interesting part: AI changes Shift Right

Consider an AI-powered loan assistant.

Before production, QA validates:

“Customer with a $80,000 salary and credit score 750 should receive the appropriate eligibility response.”

Everything passes.

But after release, customers start asking questions like:

“My salary is $80K, but $20K of it is freelance income. Am I eligible?”

The model starts producing inconsistent responses because this combination wasn’t represented well in the evaluation dataset.

Traditional regression testing might never detect it.

But production monitoring can.

This is where Shift Right becomes a quality feedback loop.

Testers can help define:

What should we monitor?
Response accuracy, latency, error rates, failed journeys.

What should we continuously validate?
Critical user journeys, API contracts, AI responses, business rules.

What should trigger investigation?
Error spikes, unusual responses, model-quality degradation, increased retries, user complaints.

What should go back into testing?
Real production scenarios that exposed previously unknown risks.

And this creates an incredibly powerful cycle:

Production behavior → New risk → New test → Automation → Regression → Production monitoring

That’s Continuous Quality.

---

The biggest mindset change

Traditional testing often asks:

“Does the application work according to the requirements?”

AI-enabled systems require additional questions:

“Does it continue to behave correctly when the inputs, context, data, model, or user behavior changes?”

That’s a very different testing problem.

For deterministic software:

Input → Code → Expected Output

For AI-enabled systems:

Input + Context + Data + Model → Output

And the output may not always be deterministic.

Therefore, QA needs to think beyond:

Pass / Fail

and increasingly consider:

Accuracy → Consistency → Robustness → Safety → Drift → User impact

---

So what should QA actually own?

Not production alone.

Not pre-production alone.

And definitely not “all testing.”

QA should help engineer the quality feedback loop.

Before production

Prevent

Requirements
↓
Risk analysis
↓
Test strategy
↓
Automation
↓
Security / Performance / Accessibility
↓
AI evaluation

After production

Detect + Learn

Observability
↓
Real-user behavior
↓
Production failures
↓
AI/model behavior
↓
New risks
↓
New tests
↓
Continuous improvement

---

One simple example

Imagine your regression suite has:

10,000 tests

All 10,000 are green.

But production monitoring shows:

Checkout abandonment up 18%

That should immediately trigger a QA question:

“If all tests passed, what quality signal did our tests fail to represent?”

Maybe the issue was:

• A real-world payment combination
• Slow third-party response
• A browser/device-specific issue
• An unexpected user journey
• A production data condition
• An AI-generated recommendation influencing user behavior

This is why test execution results alone cannot be our definition of quality.

---

The future QA mindset

I think the evolution looks something like this:

Traditional QA
“Find defects.”

↓

Shift-Left QA
“Prevent defects earlier.”

↓

Continuous Quality Engineering
“Prevent, detect, learn, and continuously improve.”

↓

AI-era QA
“Validate not only what the system is designed to do—but how it behaves as the system, data, models, and users evolve.”

The most valuable QA engineer may not be the person who has written the most automated tests.

It may be the person who can answer:

“What can go wrong, how will we detect it, and how will we turn what we learn into better tests?”

Where does your team draw the line between Shift-Left and Shift-Right?

And more importantly: who owns quality signals from production—the QA team, developers, SRE, product, or everyone together? Share your take in the comments.

Bookmark