ERP Selection Case Study: How an Ecommerce Brand Rejected the Wrong ERP

This post presents an ERP selection case study to guide you through the process.

1. The Demo Looked Perfect. The Evidence Told a Different Story.

This ERP selection case study shows how an ecommerce brand used a controlled conference room pilot to reject an unsuitable system before approving a wider rollout. The software looked capable in a sales demonstration. However, realistic transactions exposed gaps in inventory availability, warehouse fulfillment, returns, and accounting. As a result, the team stopped relying on promises and started using evidence.

The brand in this article is illustrative, not a verified customer account. Its operating model, test findings, scores, and final decision are hypothetical examples. Nevertheless, the test method reflects the practical questions that any growing product business can ask during an evaluation.

Imagine a company selling apparel and consumer goods through Shopify, Amazon, and wholesale accounts. In addition, it operates two warehouses and works with a third-party logistics provider. Its leaders want one dependable view of orders, inventory, purchasing, and finance. Instead, different applications report different answers when orders change or stock moves.

The vendor’s standard demonstration runs smoothly. Yet the buying team asks a harder question: Will the proposed configuration work when the business runs into the exceptions it sees every day? Therefore, the company sets measurable acceptance rules, runs six demanding tests, and reviews the failures before deciding whether to proceed.

2. Why the Brand Was Ready to Replace Its Software Stack

2.1 The ERP selection process began with conflicting inventory numbers

Initially, Shopify accepted online orders while a separate warehouse application handled picking and shipping. Meanwhile, accounting staff used QuickBooks, purchasing planners worked in spreadsheets, and wholesale orders arrived through a mix of email and EDI. Because each team updated different records, the company often needed manual checks before it trusted a quantity or an invoice.

For example, a buyer might assume 300 units would arrive next week because a purchase order remained open. However, the supplier may have canceled part of that shipment. Similarly, an ecommerce manager might publish warehouse stock before subtracting units already held for wholesale customers. Consequently, the same SKU could appear available in one tool and committed in another.

The problem was not simply slow reporting. Rather, each disputed number could affect customer promises, reorder decisions, cash commitments, or the monthly close. Therefore, leadership wanted to examine whether an integrated system could remove those breaks between departments.

2.2 Where ERP vendor evaluation demos create false confidence

During its first meeting, the shortlisted vendor showed dashboards, sales orders, purchasing screens, and warehouse operations. Additionally, the presenter moved through familiar workflows without delays. The team saw functions that matched its requirements list, so the proposal appeared strong.

Still, a demonstration usually reflects data and steps the presenter prepared. For instance, a clean sales order can pass even if a partial shipment later causes the wrong customer status. Likewise, a purchase receipt can look correct even if damaged units become sellable before inspection. Thus, feature coverage does not establish operating fit.

The team decided that an ERP selection case study should stand or fall on documented business outcomes, not product screenshots. Accordingly, it prepared transactions the vendor could not simplify without the buyer’s approval.

2.3 Turning recurring problems into acceptance requirements

First, the operations lead identified events with serious downstream effects: overselling, duplicate allocation, incorrect shipment completion, premature return release, supplier receipt differences, and unreconciled inventory value. Next, each business owner described the correct outcome in plain language.

For example, the ecommerce lead required the storefront to display only sellable stock. Finance required every tested inventory movement to follow approved posting rules. Meanwhile, the warehouse lead required partial fulfillment to leave the unshipped quantity open. Therefore, every rule had a measurable result and a named person responsible for approval.

3. ERP Selection Case Study: Define the Decision Before the Pilot

A conference room pilot tests whether configured business processes work together. However, teams often run it during implementation, while a pre-contract proof of concept focuses on the feasibility of a narrower requirement. In this example, the buyer negotiated a limited, configured evaluation before committing to a wider deployment. Accordingly, the team defined the stage, cost, and decision rights before testing began.

For a deeper explanation of testing stages, readers can consult the ERP conference room pilot guide. Here, the focus is different: the evidence that supports a rejection decision.

3.1 ERP vendor evaluation needs measurable pass-or-fail outcomes

Rather than asking whether the software “supports Shopify,” the buyer asked whether an order update could preserve the correct inventory state across the ERP, storefront, and warehouse. Similarly, instead of asking about returns management, the team required damaged units to remain unavailable until staff approved them.

Each test had a starting quantity, required actions, an expected result, and a severity rating. Moreover, the team documented what proof a vendor must show to pass. Consequently, disagreement about a result became a factual discussion of records and rules rather than a debate about what someone remembered seeing.

3.2 Mandatory controls cannot disappear into a score

The group marked four outcomes as non-negotiable: accurate sellable inventory, dependable partial shipments, enforced return quarantine, and reconcilable financial records. Although the vendor could earn points for usability and reporting, those scores could not cancel a critical control failure.

For example, an attractive dashboard would not offset publishing held inventory as available. Likewise, a fast purchase-order screen would not make an unexplained accounting variance acceptable. Therefore, leaders agreed to separate the weighted score from mandatory approval gates.

3.3 Document the scope of the configured pilot

Before the test, the parties agreed which integrations were live, which transactions were simulated, and which limits applied to the trial. In addition, they defined who would provide configuration, resolve defects, and pay for extra development. This kept failures from becoming surprise change requests.

Importantly, the buyer did not assume that a configured pilot replaced later user acceptance testing. Instead, the pilot answered whether the proposed solution merited further investment. If it passed, a separate implementation plan would still need training, migration, security, and formal acceptance work.

4. Prepare People, Data, and Evidence That Can Be Rechecked

4.1 Who owns the ERP pilot testing results?

The COO sponsored the decision, while functional leaders owned the results. For example, the ecommerce manager validated channel orders and published availability. The warehouse manager reviewed allocation, picking, shipments, and returns. Meanwhile, purchasing checked receipts, finance traced postings, and an integration specialist followed messages between systems.

Although the vendor consultant helped configure and operate the software, the relevant business owner gave the final sign-off. Consequently, no single presenter could approve every workflow based on a successful screen share. The company also kept one issue log, so each owner could see how problems affected other teams.

4.2 Build test data that reflects the awkward days

Instead of using only clean product records, the pilot included sizes, colors, bundles, warehouse locations, open customer orders, and supplier receipts. In addition, the team loaded synthetic wholesale prices, held stock, draft-order reservations, and delayed shipping confirmations. Because this ERP selection case study focuses on risk, the testers deliberately added cases that a normal demo would skip.

For instance, one order needed two locations, while another required a partial shipment. A return entered inspection before it could rejoin available stock. Likewise, a supplier delivered less than the purchase-order quantity and reported damaged goods. Consequently, these tests linked software behavior to operating risk.

The team also masked customer data and controlled access to the evaluation environment. Therefore, it could test meaningful business rules without copying unnecessary personal information into a vendor’s trial system.

4.3 Make results repeatable instead of anecdotal

Before each scenario, the owner recorded opening balances and a test ID. Next, testers saved timestamps, user actions, relevant messages, and the final inventory or accounting records. If a defect appeared, the vendor could reproduce the exact sequence rather than guess how it happened.

Furthermore, the group tracked configuration changes. As a result, each passing retest used identifiable rules. For connected operations, teams can inspect Xorosoft integrations as one potential architecture, while still requiring evidence for the actual connections and mappings their business needs.

Free ERP Readiness Assessment: Start by listing your three most expensive workflow failures. Then define what a vendor must demonstrate, who will approve the result, and which gaps you cannot accept. An initial assessment should clarify these requirements before a detailed demo.

5. ERP Selection Case Study: Six Tests the Demo Never Showed

The following results are hypothetical, and “Vendor A” does not identify a real software company. Nevertheless, these examples show how an ERP selection case study can turn uncertainty into a documented go/no-go decision. Each test compares expected behavior with an illustrative observed result.

5.1 Shopify availability: on-hand stock is not sellable stock

The ecommerce manager started with 100 units on hand: 25 committed to existing orders and 15 unavailable for quality or other holds. Therefore, the test expected 60 available units, not 75 or 100. Shopify defines on-hand quantity as the sum of available, committed, and unavailable inventory; incoming stock remains separate. Its inventory-state documentation explains these distinctions.

However, the illustrative ERP mapping sent 75 units to the storefront because it ignored the unavailable balance. Next, the team created another reservation and repeated the synchronization. The wrong result appeared again, so the group recorded the mismatch as a critical defect.

As a result, customers could buy unavailable stock and receive unreliable delivery promises. Therefore, the buyer required the vendor to correct the mapping and prove that quarantined stock stayed excluded after later updates.

5.2 ERP pilot testing across two warehouse locations

For the second test, an order requested 12 units while the preferred warehouse held only five available. Another location held 40. According to the buyer’s allocation policy, the ERP should direct the order to the second location without creating a duplicate reservation at the first.

Instead, the configured workflow required an employee to override the default location manually. However, that manual override was fragile during order surges. Consequently, the team marked the issue for remediation instead of accepting the demo as proof.

The ERP selection case study also checked what happened when a transfer was canceled or delayed. For example, inventory should not become available at the receiving warehouse merely because a transfer exists. A mature evaluation must distinguish physical stock, stock in transit, and stock that sales teams can actually promise.

For companies reviewing warehouse execution, XoroWMS offers a relevant product reference. However, this ERP selection case study does not assume that any vendor’s published feature list proves the tested allocation rules.

5.3 A partial shipment that looked complete everywhere else

Next, a wholesale customer ordered 12 units, but the 3PL shipped only seven. Therefore, the expected state was seven shipped and five still open. The order also needed correct tracking, billing treatment, and inventory deductions. Meanwhile, the unshipped balance had to remain visible to service and operations teams.

The illustrative integration instead sent a completed-order status to the sales channel. As a result, customer service believed the shipment was finished even though the warehouse still owed five units. Furthermore, the first test did not prove what would happen if the 3PL resent the same confirmation.

Next, testers replayed the message and shipped the final five units. These checks mattered because duplicate messages can create duplicate fulfillment events or stock changes when an integration lacks safe retry controls. Therefore, the acceptance rule required both correct partial statuses and idempotent processing where applicable.

A generic order-management demonstration would not necessarily reveal this weakness. By contrast, a test that starts with a real business exception exposes the gap between a feature and an end-to-end outcome.

5.4 ERP vendor evaluation of damaged returns and quarantine

For the fourth test, a customer returned four damaged units. Warehouse staff placed them in a non-sellable inspection state. Accordingly, the ERP and storefront needed to keep those four units unavailable until an authorized employee approved their release or disposal.

The warehouse screen showed the hold correctly. However, the illustrative outbound feed restored all four units to sellable inventory. Consequently, a shopper could buy goods that had not passed inspection. The tester then approved just one unit for resale and verified that the other three remained unavailable.

This ERP selection case study treats return quarantine as a control, not just a status label. In addition, the test checks who can release stock, whether the event leaves an audit trail, and whether repeated messages create extra available units. For apparel, sporting goods, food, and other product businesses, those checks support both customer confidence and inventory integrity.

5.5 Supplier receipts that did not match the purchase order

The purchasing lead ordered 100 units and received 70, including four damaged pieces. Therefore, the expected result was 66 sellable, four held, and 30 still outstanding. The team also needed the invoice-matching rules to follow the company’s approved process rather than silently treating all 100 units as received.

Initially, Vendor A released all 70 units into available stock. However, its consultant found a missing inspection setting, corrected the configuration, and reran the same transaction. This time, the system maintained the right quantities and open purchase balance. As a result, the buyer accepted the retest evidence.

This illustrates why an initial failure does not always prove the product is wrong. Sometimes the problem is configuration or test setup. Nevertheless, the team should approve only the corrected outcome, not an assurance that the setting will be fixed later.

5.6 ERP pilot testing for financial reconciliation

Finally, finance followed one shipment, a returned item, an inventory adjustment, and the related credit through the operational records and general ledger. First, the controller documented the company’s accounting policy and expected posting dates. Next, the team compared actual entries with those expectations.

The illustrative ERP recorded the customer credit but left a return-related inventory adjustment requiring manual repair. Moreover, its audit trail did not clearly connect the source event to the correction. Finance requested a supported fix and repeated the test. However, the retest still produced an unexplained variance.

Therefore, finance withheld approval. This failure carried more weight than a preference for screen layout because it threatened timely close and confidence in reported inventory value. The business did not demand one universal accounting design. Instead, it demanded that the proposed design work consistently under its documented rules.

Scenario	Expected result	Illustrative finding	Status
Shopify sellable stock	60 units available	75 units published	Critical fail
Two-warehouse allocation	Correct location and reservation	Manual override required	Open defect
Partial 3PL shipment	Seven shipped, five open	Order marked complete	Critical fail
Damaged return	All four held until approval	Four released as sellable	Critical fail
Partial supplier receipt	66 sellable, four held, 30 open	Passed after configuration change	Retest passed
Inventory accounting	Reconciled and traceable entries	Unresolved return adjustment	Critical fail

Because four mandatory controls remained unresolved, the team could not approve the solution merely by pointing to the passed receipt scenario. Instead, it moved to defect analysis and a formal decision review.

6. Separate Fixable Defects From a Poor Business Fit

A failed test needs investigation before it becomes a rejection reason. However, that investigation should have limits. In this ERP selection case study, the team gave the vendor a documented chance to correct each material issue, then required a repeatable result. Consequently, neither side could declare success by changing the acceptance standard after a failure.

6.1 ERP pilot testing should classify the root cause

First, testers distinguished incorrect master data from product behavior. Next, they separated missing configuration from integration defects and workflow design gaps. Finally, they identified requirements that could be met only through new custom development. Each category led to a different decision.

Defect type	Example	Sensible response
Data problem	Duplicate SKU or wrong starting balance	Correct the record; rerun
Configuration gap	Quarantine location rule disabled	Enable the rule; prove the result
Integration problem	Shipment mapping sends the wrong status	Repair the interface; retest end to end
Process design gap	Allocation logic conflicts with approved policy	Agree on a supported design
Structural limitation	Critical control requires unsupported code	Assess risk, cost, and potential rejection

For example, the supplier-receipt problem had a supported configuration fix. By contrast, the financial variance remained unresolved after retesting. Therefore, the team considered both the severity of the control and the evidence that a fix actually worked.

6.2 ERP pilot testing needs proof after every fix

A vendor may offer a patch, a consultant-led workaround, or a future release. Nevertheless, the buyer should ask who owns the change and when users can test it. Furthermore, a corrected scenario should not break a related workflow.

For instance, fixing a return hold should not stop an approved return from becoming sellable after inspection. Likewise, correcting partial fulfillment should not duplicate shipment confirmations when the 3PL retries a message. Consequently, every material fix needs a regression check, not just a successful screenshot.

Microsoft’s test execution guidance supports recording test results and managing issues through retesting. Accordingly, the team kept an auditable record of the changes and their outcomes.

6.3 The issue log made disagreements easier to resolve

The team recorded every open issue with a test ID, expected result, actual result, owner, severity, workaround, and retest date. As a result, executives could distinguish a minor user-interface issue from an unresolved risk to customer promises or accounting accuracy.

Moreover, the log exposed hidden costs. A workaround might require five minutes per transaction, a specialist’s approval, or a nightly manual reconciliation. Although each task looks small, the effort grows with order volume. Therefore, the buyer included ongoing labor and control risk in its software-fit decision.

7. ERP Selection Case Study: How the Team Rejected Vendor A

7.1 A weighted scorecard supported the discussion

A scorecard helped the committee compare business functions on the same scale. For example, inventory and warehouse performance carried more weight than interface preferences because errors in those areas directly affect orders and stock. However, the committee did not let a numerical average decide whether mandatory controls worked.

Evaluation category	Weight	Illustrative score (0–5)	Weighted points
Channel inventory controls	25%	3	15
Warehouse and fulfillment	25%	4	20
Accounting and reconciliation	20%	3	12
Purchasing and returns	15%	5	15
Integration reliability	10%	4	8
Ease of use and readiness	5%	4	4
Total	100%		74 / 100

These numbers are illustrative, not measured vendor performance. Nevertheless, they show why the ERP selection case study needed both weighted scores and mandatory gates.

7.2 Why mandatory gates overruled the overall score

Suppose the buyer used 70 out of 100 as a screening threshold. Vendor A scored 74 in the example. Yet four mandatory tests remained unresolved. Therefore, the system did not pass the decision gate even though the total score exceeded the provisional minimum.

The lesson is not that any minor defect should eliminate a vendor. Rather, buyers should identify the failures they genuinely cannot accept before testing begins. In addition, they should allow reasonable fixes and document residual risks. Then the final decision can reflect both functional performance and business impact.

7.3 ERP vendor evaluation evidence reached the executive team

First, the warehouse manager demonstrated the partial shipment and reservation gaps. Next, ecommerce staff showed the wrong published stock. Meanwhile, finance explained why the unresolved posting variance could delay reconciliation. Consequently, the COO had a record of observable issues instead of competing opinions from sales meetings.

In the ERP selection case study, the committee then compared the projected cost of fixes, custom-code dependency, and ongoing work. Although some gaps were repairable, the combined risks did not meet the company’s agreed acceptance standards. As a result, leadership issued a written no-go decision for Vendor A’s proposed solution and reopened the shortlist.

Importantly, Vendor A is fictional and does not refer to any named competitor. In addition, rejection under one business’s rules does not prove that a system is unsuitable for every company. The decision reflects this hypothetical buyer’s requirements and test evidence.

8. What to Compare After Rejecting a Vendor

An ERP selection case study should not imply that rejecting one system makes the replacement obvious. Instead, the business needs to evaluate the next candidate with the same tests. Moreover, it should check whether the current stack could be improved at a lower total cost before approving any migration.

8.1 Begin with Xorosoft as a connected-system candidate

For an inventory-driven ecommerce shortlist, Xorosoft is a practical first option to examine. Its XoroONE platform brings together inventory, purchasing, warehouse operations, manufacturing, and accounting. Likewise, XoroERP provides a reference point for evaluating connected operational and financial workflows.

However, an advertised capability is not proof that every customer rule will work without setup. Therefore, the buyer should repeat the identical Shopify availability, multi-warehouse, 3PL, quarantine, receiving, and accounting tests. The Xorosoft Shopify App Store listing is useful context for merchants, but the acceptance decision still belongs to the business team.

Watch Demo: Review how a connected platform organizes orders and stock, then ask for a demonstration built around your own exception cases. Above all, require evidence for the workflows that create the largest business risk.

8.2 Use the same ERP selection process for other options

After examining the first candidate, the company could evaluate NetSuite, Acumatica, Microsoft Dynamics 365 Business Central, or a focused combination of specialist systems. However, each candidate should receive the same test data, severity rules, and opportunity to correct configuration problems. This is fairer than comparing one vendor’s prepared demo with another vendor’s most difficult transaction.

For buyers explicitly comparing suites, the ERP comparison hub and the Xorosoft–NetSuite comparison can provide starting questions. Nevertheless, use those pages to plan tests, not to assume either platform will pass a company-specific scenario.

8.3 Sometimes the existing stack is worth improving

The right alternative might be better master data, a stronger inventory app, or a revised warehouse integration. For example, a company with one simple fulfillment location may not need a major system replacement. Conversely, an organization with recurring disputes between order, stock, purchasing, and finance records may benefit from shared operational data.

Therefore, compare three choices: keep and improve the stack, add a specialist tool, or deploy an integrated system. In addition, price training, integration upkeep, workarounds, migration, downtime, and internal support. A credible ERP selection case study should explain those alternatives rather than assume that buying another platform is always the answer.

9. Does Every Business Need a Conference Room Pilot?

9.1 More complexity means more ERP vendor evaluation risk

A full pilot makes sense when many departments share the same transaction. For instance, a fashion brand may juggle size variants, returns, and seasonal allocations. Similarly, a furniture distributor may deliver one order in several shipments. Meanwhile, a food business may require lot and expiry controls, while a wholesaler may need customer-specific pricing and EDI documents.

Therefore, buyer-defined tests should reflect actual operating complexity. For teams comparing these needs, the industries served by Xorosoft illustrate the different product-business contexts that can shape requirements. Still, an industry label alone never replaces a passing test.

9.2 When a smaller proof of concept may be enough

A small merchant may have one warehouse, few integrations, and simple accounting. Accordingly, a scripted demonstration plus a narrow proof of concept may be proportionate before signing. Nevertheless, the business still needs written requirements and later acceptance testing, especially for stock availability and financial accuracy.

Match testing effort to the potential harm of a failure. For example, a business with minimal returns may need fewer quarantine scenarios than a high-volume apparel seller. However, neither company should approve a critical workflow based solely on a sales promise.

9.3 Signs the company has outgrown disconnected tools

Recurring overselling, manual inventory adjustments, spreadsheet-based purchasing, delayed month-end closes, and inconsistent channel statuses can indicate a larger system problem. Furthermore, growth across warehouses and wholesale customers creates new allocation and pricing rules. Consequently, the operating model may become too complex for a collection of independent applications.

Still, an upgrade should follow a root-cause review, not a revenue milestone. The buyer should first identify which controls are broken and whether better processes could fix them. As this ERP selection case study illustrates, the strongest buying signal is a proven operational need paired with a solution that passes realistic tests.

10. ERP Selection Case Study Lessons for the Next Buyer

10.1 A practical ERP selection process checklist

Before meeting vendors, write a short list of failures that most affect customers and finance. Then turn those risks into repeatable scenarios with named owners. In addition, agree on acceptance standards, the test environment, and the records that count as proof. Finally, decide which defects trigger retesting and which block approval outright.

Buyer question	Evidence to request
Does sellable stock stay accurate?	Inventory states and channel update logs
Can fulfillment remain partial?	Order balance and 3PL confirmation history
Do returns remain under control?	Quarantine, release, and user audit records
Can receiving handle shortages?	Receipt, hold, and open PO quantities
Does finance reconcile?	Subledger, journal, and source transaction trail
Will fixes survive later updates?	Supported configuration and regression-test results

This checklist turns the ERP selection case study into a repeatable buying tool. Moreover, it reduces the risk that one confident demonstration will overpower the concerns of the teams responsible for daily operations.

10.2 Hidden costs can change the apparent winner

An ERP proposal may show licensing, setup, and training fees, but the business also needs to model operating workarounds. For example, five extra minutes on every exception can become a staffing problem as order volume grows. Similarly, custom integration code may create long-term testing and maintenance obligations.

Therefore, ask each vendor to separate standard features, configuration, supported extensions, third-party tools, and custom development. In addition, request clear ownership for ongoing support. Although a low headline price may seem attractive, unresolved dependencies can make the real cost much higher.

10.3 Questions to ask before final approval

First, can department owners repeat every critical test without relying on the vendor’s presenter? Next, does the system preserve correct inventory, fulfillment, and accounting records when messages are delayed or duplicated? Furthermore, can staff explain the audit trail behind adjustments and exceptions?

If any answer remains uncertain, continue testing or revise the project scope. By contrast, when a solution passes the agreed gates and the remaining risks have owners, the buyer can proceed with greater confidence. Ultimately, a careful ERP vendor evaluation should produce a defensible decision, even if the answer is no.

11. Conclusion: Choose Evidence Over a Perfect Demo

This ERP selection case study shows why buyers should treat difficult operational tests as a safeguard, not as a delay. A product can have an impressive interface and still fail the business rules that protect stock, shipments, and financial records. Therefore, the team should define the rules first, test realistic exceptions, document each defect, and verify the fixes before approving a rollout.

If the current stack is creating repeated gaps, an integrated platform deserves consideration. However, no ERP should win solely because its presentation looks polished. Instead, put Xorosoft and other candidates through the same acceptance process and let real results guide the decision.

Book a Demo: Book a personalized Xorosoft demo and bring your hardest Shopify, warehouse, purchasing, and accounting scenarios to the discussion.