Shopify AI Automation for Fraud-Risk Order Reviews
Turn Shopify fraud-risk flags into a reviewable order decision: compare the risk assessment with customer history, preserve the evidence, and route ambiguous orders to Gorgias before fulfillment.
Shopify AI automation can put a governed process between a completed fraud-risk result and fulfillment, instead of leaving that moment to a Slack message or a shared spreadsheet. Shopify's own fraud guidance says risk analysis can finish after an order is placed, so a reliable workflow listens for that result, not just the order itself. This guide shows the actual build: a Major app that holds the evidence, an agent that only recommends, and a Gorgias ticket for the cases that need a person.
TL;DR: what you'll build. A Shopify fraud-risk order review app on Major that watches for orders with a medium or high risk result, pulls the customer's order history as context, and has an agent draft a hold/release recommendation with its reasoning attached. Ambiguous cases open a Gorgias ticket for a person; low-risk cases get logged, not auto-released. The Ecommerce Risk & Fulfillment Manager keeps every hold, release, and cancel action in their own hands, with a full evidence trail behind each one.
Why Shopify fraud-risk review becomes an operations bottleneck
Shopify's own guidance on protecting orders from fraud assigns a risk level to each order and expects the merchant to review flagged ones before fulfillment. That review is manual work today.
The Ecommerce Risk & Fulfillment Manager, or whoever covers that shift, has to open each flagged order, check the customer's history, decide whether the signal is a false positive, and then either release the order or hold it. During a sale spike this queue backs up fast.
The real bottleneck is not the fraud score itself. Shopify's analysis usually runs quickly but can complete after the order is already in the system, so a workflow built only on "order created" fires too early and misses the result entirely.
The delay that actually hurts the business is the gap between the risk result becoming available and a person acting on it. A legitimate order sits on hold longer than it needs to, and a risky one can slip through if nobody checks in time.
This is a cross-industry operations problem, not specific to one vertical. It shows up anywhere a Shopify store processes orders at volume, and it pairs naturally with the broader pattern covered in AI workflows for ecommerce and in the governed automation patterns on the governed workflow automation hub.
The fix is not asking an AI model to decide who gets shipped. It's building a durable record of the evidence, a clear escalation path, and a reviewer who can act on both in one place.
The Shopify and Gorgias systems involved
Shopify: orders, risk, and customer history
The object at the center of this workflow is the Shopify Admin GraphQL Order object. Its risk field returns an order risk summary, which is the current, non-deprecated way to read fraud signals; the older risk level and risks fields on the order are deprecated and should not be used in a new build.
The order risk summary exposes two things: a list of risk assessments and a recommendation. Each OrderRiskAssessment carries a risk level (an enum of HIGH, MEDIUM, LOW, NONE, or PENDING) and a set of facts describing why. The PENDING value matters operationally: it means Shopify has not finished analyzing the order yet, so a workflow has to tolerate a result that isn't ready on the first check.
To find flagged orders in bulk, the orders query supports a search syntax with filter keys including risk_level, fraud_protection_level, and chargeback_status, plus cursor-based pagination through a page size argument and a next-page cursor. That's the mechanism a scheduled check uses to pull the current batch of medium- and high-risk orders.
Reading order data requires scopes such as read_orders, and by default Shopify only exposes orders from the past 60 days; access to older orders needs a separate request and the read_all_orders scope, per the access scopes documentation. The Customer object supplies order history context, but names, emails, phone numbers, and addresses are classed as protected customer data requiring their own approval level under Shopify's protected customer data requirements. A custom app built for one store generally gets this access without the public-app review process, but a public or listed app does not.
One honest gotcha: Shopify's API rate limits page describes GraphQL's cost-based throttling and a REST request bucket, but doesn't publish one universal numeric ceiling for every plan. Build with backoff rather than a hardcoded request budget.
Gorgias: the review ticket
Gorgias is the ecommerce helpdesk where the human reviewer actually works the ambiguous case. Creating a ticket is a POST to the tickets endpoint documented at Create ticket: it requires a messages array with at least one entry, and each message needs a channel and a from_agent flag. The response returns a ticket object with an id, status, subject, customer, and a URI you can store for cross-reference.
Checking on existing tickets uses List tickets, a GET request that supports filtering by customer ID and ticket IDs, with cursor-based pagination and a default page size of 30. Authentication for a private integration is HTTP Basic Auth, using the account email as the username and an API key as the password; the Gorgias docs reviewed here did not publish an exact numeric rate limit, so don't quote one.
The regulation that matters here
Because this process touches a customer's order and purchase history, the California Consumer Privacy Act / CPRA applies to any California resident in the flagged-order pool. The one requirement that matters for this build: the app's stored evidence (order history, risk notes, ticket references) needs a deletion path tied to the same customer-deletion process the rest of the store already uses, since it is derived personal data, not a separate system exempt from a deletion request. This is background, not legal advice.
The workflow: deterministic checks, agent judgment, human approval
Think of the pipeline as three layers stacked on top of each other, each doing only what it's suited for.
A deterministic step detects the trigger. Shopify's own Shopify Flow guidance for high-risk orders recommends the "Order risk analyzed" event over "Order created," precisely because the risk result may not exist yet at order creation. Major implements the equivalent by running a scheduled check against the orders query filtered by risk level, since this build does not rely on an undocumented webhook topic for risk completion.
A dedupe step comes right after detection. Before writing anything, the workflow checks whether this order already has an evidence record, so a repeated schedule run or a reprocessed webhook never creates two tickets for the same order. This is the same idea as a receptionist checking the sign-in sheet before buzzing someone in twice.
The evidence snapshot step pulls the risk assessment, the recommendation, and the customer's recent order history (respecting the scopes actually granted) and writes it to the app's database as a fixed record, not a live query the agent re-runs each time.
The agent reads that snapshot and drafts a recommendation: hold, release, or escalate, with a plain-language rationale tied to specific facts (a brand-new account placing a large order, a shipping address that doesn't match billing, a customer with twelve clean prior orders). It does not call any Shopify or Gorgias write endpoint itself.
A routing step decides what happens to that recommendation. Low risk with a clean history logs an advisory note with no action taken. Medium or high risk, or any case where the agent's confidence is low, opens a Gorgias ticket.
The reviewer resolves the ticket and records a decision back in the app; only then does a person take the actual hold, release, or cancel action inside Shopify. The safest first release of this workflow is advisory end to end: it tells the reviewer what it thinks, it never auto-releases or auto-cancels on its own.
The app, agent, and workflow architecture
Split the build into three pieces that each do one job.
The app is the system of record: a "Flagged Order Review" app with managed tables holding every order's evidence, the agent's recommendation, the Gorgias ticket reference, and the final human decision. Nothing about fraud risk should live only in a prompt or a chat transcript; it lives in rows the team can query, filter, and audit.
The agent owns exactly one judgment call: given the risk assessment and the bounded order history, is this order's story consistent, and if not, what to flag. It never decides fulfillment or payment; a decision field in the app, written by a person, is what actually changes an order's status.
The workflow is the deterministic glue: the schedule that checks for newly analyzed orders, the dedupe logic, the routing rule between "log it" and "escalate it," and the retry policy on any Shopify or Gorgias call that fails.

Shopify fraud-risk review architecture on Major
Proposed data model, as fields the app owns (not native Shopify fields):
- order_id: the Shopify order's GID, the dedupe key.
- risk_level: HIGH, MEDIUM, LOW, NONE, or PENDING, copied from the order risk assessment's risk level.
- riskrecommendationref: the order risk summary's recommendation value at the time of the snapshot.
- customer_id: the Shopify customer GID, when present.
- orderhistorysummary: a bounded text summary of recent orders for that customer, scoped to whatever history access the app's scopes allow.
- evidence_source and evidencecapturedat: where the snapshot came from and when, for audit purposes.
- agent_rationale: the plain-language explanation the agent attaches to its recommendation.
- recommended_action: hold, release, or escalate, as the agent's suggestion only.
- gorgiasticketid: populated only for escalated cases.
- reviewer, decision, decision_at: who actually acted and when.
- status: pending, advisory_logged, escalated, resolved.
This structure keeps the app useful even before any agent runs: it is a queryable ledger a risk manager can filter by risk level or status on its own.
How to build Shopify fraud-risk review on Major
Here is the build in order, from connecting the systems to rolling the workflow out to the team.
- Connect the Shopify and Gorgias connectors once, under IT's control. An administrator sets up the Shopify connector with a scoped access token (read access to Orders, and only the Customer history access actually approved) and the Gorgias connector with its API key, both stored in Major's central credential store rather than in any prompt or workflow step. You should see both connectors listed as active resources that any app, agent, or workflow in the org can reuse without re-entering credentials.
- Build the Flagged Order Review app with the data model above. Describe the app to Major in plain language and it generates the managed tables, a queue view filtered by status, and a detail view per order showing the risk assessment, the order history summary, and the decision fields. You should see a working table of flagged orders with a status column, even with zero rows yet.
"Build an app called Flagged Order Review with a table of flagged orders (orderid, risklevel, riskrecommendationref, customerid, orderhistorysummary, evidencesource, evidencecapturedat, agentrationale, recommendedaction, gorgiasticketid, reviewer, decision, decision_at, status). Add a queue view grouped by status and a detail view per order."
- Add the agent that reads from this app. Give it access to the app's endpoints for reading an order's evidence snapshot and writing its own recommendation and rationale fields, and nothing else. You should see the agent successfully write a draft recommendation to a test order's record without touching Shopify or Gorgias directly.
"You are the fraud-risk review agent. Read the evidence snapshot for the given orderid from the Flagged Order Review app. Compare the risklevel and risk facts against the orderhistorysummary. Write a recommendedaction (hold, release, or escalate) and a 2-3 sentence agentrationale citing specific facts. Never call any Shopify or Gorgias write endpoint. If the evidence is incomplete or the risklevel is PENDING, set recommendedaction to escalate."
- Set the workflow's trigger as a schedule, not a webhook on an undocumented event. Configure it to query Shopify's orders endpoint filtered by
risk_levelfor medium and high on a recurring interval you choose for your order volume, write new evidence snapshots, call the agent, then route by its recommendation. You should see the workflow run end to end once in a test, producing either a logged advisory row or a new Gorgias ticket. - Test with a handful of real flagged orders from a sandbox or low-risk batch before trusting it on the full queue. Confirm the dedupe logic skips an order already processed, and confirm a forced PENDING risk_level correctly re-queues instead of silently dropping. You should see the retry queue catch at least one simulated API error during this test.
- Roll out with role-based permissions: the risk manager and designated reviewers get write access to the decision fields, the rest of the team gets read-only visibility into the queue, and only a named, authorized person's actual Shopify action (not the app) triggers a hold, release, or cancellation. You should see an audit log entry for every decision change, tied to the reviewer's identity.
Governance and IT control
Credentials for both Shopify and Gorgias sit in Major's central credential store, scoped to exactly the access this workflow needs, and governed by IT rather than embedded in a workflow step or an agent prompt. That is the same model Major uses across every app and agent in the org: one connector, reused, never re-entered.
Role-based access on the Flagged Order Review app limits who can change a decision field versus who can only view the queue. SSO handles identity for every person who touches the app, so access follows the org's existing login, not a separate password.
Every write to the app, every agent recommendation, and every Gorgias ticket creation is a logged, attributable event. That audit trail is what makes a risk manager's decision defensible later, whether to a customer, a chargeback dispute, or an internal review.
Actions that change an order's real state, meaning the actual hold, release, or cancellation in Shopify, require explicit human approval in this build. The agent's output is a recommendation field, not a trigger for a Shopify mutation. That boundary should be deliberate and visible in the workflow's configuration, not just a design intention.
Data retention ties back to the CPRA point above: the order history summary and agent rationale the app stores are derived personal data, so a customer deletion request against the Shopify store should also clear or anonymize the matching app record, not leave it orphaned.
Failure handling and edge cases

Approval path: from pending risk analysis to recorded decision
A risk assessment that comes back PENDING is not a failure, it is an expected intermediate state. The workflow should re-check it on the next scheduled run rather than treating it as "no risk" or discarding the order.
A Shopify API error or a rate-limit response should trigger a backoff and retry, not a dropped record. Since neither Shopify's nor Gorgias's docs publish one fixed universal rate number for every plan, build the retry logic to read response status and back off progressively rather than hardcoding a request count.
Duplicate processing is the most common operational bug in a scheduled workflow like this. The dedupe check against order_id before writing a new evidence snapshot is what prevents two Gorgias tickets for the same order after a schedule overlap or a retry.
Missing or limited customer history is a signal in itself, not an error to hide. If the app's granted scopes only expose the past 60 days of orders, the agent's rationale should say so explicitly rather than implying a complete history was reviewed.
The agent must never auto-release, auto-cancel, or capture payment based solely on its own recommendation. That line is the core governance decision of this build, and it should be enforced in the workflow's structure (no write path from agent to Shopify mutation), not only in the agent's prompt.
When the agent itself fails or times out, the order should fall back to a default of "escalate to a person" rather than silently dropping into an unreviewed state. An unresolved case is better surfaced than hidden.
Worked example: one flagged order from detection to decision
This is an illustrative example only, not a measured customer result.
A mid-size Shopify merchant processes around 3,000 orders a month. On a Tuesday morning, the scheduled check finds an order flagged MEDIUM by Shopify's risk assessment: new customer account, first order, shipping address in a different state from the billing address on the card, order value $340.
The workflow writes the evidence snapshot: risklevel MEDIUM, the specific facts from the assessment, and an orderhistory_summary showing this is the customer's first-ever order with no prior history to compare against.
The agent reads the snapshot and recommends escalate, with a rationale noting that a new account plus an address mismatch is exactly the pattern Shopify's MEDIUM assessments are designed to catch, and that no order history exists yet to resolve the ambiguity either way.
The routing step opens a Gorgias ticket with the evidence summary attached. The risk manager reviews it in four minutes instead of the fifteen to twenty minutes a from-scratch manual lookup across Shopify and a spreadsheet might otherwise take, and decides to call the customer's phone number on file to confirm the shipping address before releasing the order.
The reviewer records "release" in the app's decision field, with a note, then manually releases the hold in Shopify. The full evidence trail, including the original risk facts, the agent's rationale, and the reviewer's own note, stays attached to that order_id for any later dispute or chargeback review.
What to measure
Track a small set of metrics the risk manager already watches, now visible in one place instead of scattered across a spreadsheet and a Shopify tab.
- Time from risk-flagged to reviewed, to see whether the queue is keeping pace with order volume.
- Share of flagged orders auto-logged as low-risk advisories versus escalated to Gorgias, to see if the routing threshold needs adjusting.
- Reviewer override rate, meaning how often a person's decision disagrees with the agent's recommendation, as a signal of whether the agent's rationale is actually useful.
- Duplicate or retried evidence snapshots caught by the dedupe check, as a health signal on the workflow's schedule overlap.
- Orders released versus held versus cancelled, tracked against outcome (confirmed fraud, confirmed legitimate, no further signal) over time.
None of these numbers are meaningful on day one. Give the workflow a few weeks of real flagged orders before using any of them to tune the routing threshold or staffing on the review queue.
Frequently asked questions
How can Shopify automate fraud-risk order reviews?
Shopify's own risk analysis flags an order with a risk level, and a workflow built on top of that result can gather the order's evidence, compare it against the customer's history, and route ambiguous cases to a person for review. The automation handles detection and evidence gathering; a person still makes the hold or release call on anything uncertain.
When should a Shopify fraud workflow run?
Run it off Shopify's "Order risk analyzed" signal rather than "Order created," since the risk result can finish after the order exists. On Major, that means a scheduled check against orders filtered by risk level, checked on an interval you choose for your volume; don't treat any specific polling frequency as a platform guarantee.
Can AI automatically cancel or release a high-risk Shopify order?
The safer approach keeps those actions with a person. An agent can summarize the risk evidence and recommend hold, release, or escalate with its reasoning, but the actual cancellation, release, or payment capture in Shopify should require a named reviewer's confirmation, logged in the app.
How do Shopify and Gorgias fit into an order-review workflow?
Shopify supplies the order, the risk assessment, and customer order history. Gorgias is where a person actually works the ambiguous case as a ticket, with the evidence summary attached. Gorgias is the review surface, not the fraud decision engine.
What Shopify data does an order-review app need?
At minimum, read access to Orders (for the risk assessment and order details) and whatever Customer-history access has been explicitly approved for the use case. Older order history beyond Shopify's default 60-day window, or protected fields like name, email, and address, need separate access requests under Shopify's protected customer data process.
Before you go live
Run through this checklist with the risk manager and IT before pointing the workflow at live order volume.
- [ ] Shopify connector scoped to read-only Orders access, plus only the approved Customer history fields
- [ ] Gorgias connector configured with least-privilege ticket creation and lookup access
- [ ] Both credentials stored in Major's central credential store, not in any prompt or workflow step
- [ ] Test run completed against a sandbox or low-risk order batch, including a forced PENDING risk_level case
- [ ] Dedupe logic verified against a repeated or overlapping schedule run
- [ ] Role-based permissions set: who can change a decision field versus who has read-only queue access
- [ ] Retry and backoff behavior confirmed for a simulated Shopify or Gorgias API error
- [ ] Named owner assigned for reviewing escalated tickets and for any customer-deletion request touching stored evidence
Keep this list next to the workflow's configuration so a new reviewer can get oriented without re-reading the whole article.
More ecommerce AI automation use cases
- AI agents for ecommerce: every use case by tool
- AI agents for operations & revops: every use case by tool
- Automate Shopify NetSuite order-to-cash sync with AI exception handling
- Automate Klaviyo Recharge subscriber win-back flows with AI
- Automate Shopify ShipStation delayed-shipment alerts with AI
- Automate NetSuite sales order to production handoff with AI
Start building on Major
Build a Shopify fraud-review app that preserves risk evidence, routes ambiguous orders to Gorgias, and leaves order release under an authorized person's control.
Related articles
- Automate Shopify-NetSuite Order Sync with AI Exception Review
- Automate Salesforce Pipeline Briefs in Slack with AI
- Automate GitHub Security Triage into Jira
About the author
Ritvij Gautam is Head of Growth at Major.
Last reviewed November 10, 2026 by Ritvij Gautam.
Frequently asked questions
- How can Shopify automate fraud-risk order reviews?
- Shopify's own risk analysis flags an order with a risk level, and a workflow built on top of that result can gather the order's evidence, compare it against the customer's history, and route ambiguous cases to a person for review. The automation handles detection and evidence gathering; a person still makes the hold or release call on anything uncertain.
- When should a Shopify fraud workflow run?
- Run it off Shopify's "Order risk analyzed" signal rather than "Order created," since the risk result can finish after the order exists. On Major, that means a scheduled check against orders filtered by risk level, checked on an interval you choose for your volume; don't treat any specific polling frequency as a platform guarantee.
- Can AI automatically cancel or release a high-risk Shopify order?
- The safer approach keeps those actions with a person. An agent can summarize the risk evidence and recommend hold, release, or escalate with its reasoning, but the actual cancellation, release, or payment capture in Shopify should require a named reviewer's confirmation, logged in the app.
- How do Shopify and Gorgias fit into an order-review workflow?
- Shopify supplies the order, the risk assessment, and customer order history. Gorgias is where a person actually works the ambiguous case as a ticket, with the evidence summary attached. Gorgias is the review surface, not the fraud decision engine.
- What Shopify data does an order-review app need?
- At minimum, read access to Orders (for the risk assessment and order details) and whatever Customer-history access has been explicitly approved for the use case. Older order history beyond Shopify's default 60-day window, or protected fields like name, email, and address, need separate access requests under Shopify's protected customer data process.