Seven Hard-Earned Lessons from Enterprise AI Pioneers

The companies that got the most out of AI early did not treat it as a side project. They tested models against their own work, put AI inside their products, and gave the people closest to the work the tools to use it. OpenAI’s AI in the Enterprise guide describes seven practices from companies including Morgan Stanley, Klarna, BBVA and Mercado Libre. The figures below come from those case studies.

This article walks through the seven lessons with what they mean for your team, then looks at where generative AI creates an advantage a competitor cannot copy, and where it does not.

1. Start with evaluations

Morgan Stanley ran structured evaluations before rolling AI out to its advisors. It tested translation, summarization and advisor support against expert benchmarks, and the results decided which use cases went first. In a compliance-heavy firm, that evidence also built the trust needed to deploy at all.

What to do: Collect real examples of the work, with correct answers your experts agree on. Score every model, prompt or vendor against that set before deployment and after every change. An evaluation set is the cheapest way to settle arguments about whether AI is good enough.

2. Embed AI in the product

Indeed used generative AI to explain to job seekers why a job was recommended to them. The explanation, not just the match, increased applications and hires.

What to do: Look for places in your product where a user has to interpret something, such as a recommendation, a report or a set of options, and let AI explain it in their terms. Smaller fine-tuned models often handle a narrow product feature at lower cost and latency than a large general model.

3. Start early, because value compounds

Klarna put an AI assistant on customer service. Within months it handled two-thirds of service chats, and resolution times fell from 11 minutes to 2. By the time of the case study, 90 percent of Klarna’s employees used AI in their daily work.

What to do: The support result matters less than the second figure. Teams that use AI daily find the next use case themselves. Start with one production deployment now, and give staff access and training alongside it.

4. Customize and fine-tune your models

Lowe’s had inconsistent product data across a large catalog. Fine-tuned models improved product tagging accuracy by 20 percent and error detection by 60 percent.

What to do: Fine-tune when you need the model to follow your taxonomy, terminology or output format consistently, as with catalog tagging, document classification and internal search. When the model mostly needs your facts, retrieval is usually cheaper to keep current. Our comparison of RAG and fine-tuning covers the choice.

5. Put AI in the hands of domain experts

BBVA rolled out ChatGPT internally and let staff build their own custom GPTs. In five months, employees created about 2,900 of them. Uses included faster credit risk assessment, a legal assistant that has answered more than 40,000 queries, and sentiment analysis of customer feedback.

What to do: The people who run a process know where the time goes. Give them a governed internal tool to build with, then promote the best of what they build into supported systems. Central teams should set the guardrails, not decide every use case.

6. Unblock your developers

Mercado Libre built an internal platform that combines language models, APIs and security controls so its 17,000 developers could ship AI features without rebuilding the basics. On it, teams catalogued 100 times more products, reached close to 99 percent accuracy in fraud detection, and generated localized product descriptions and personalized notifications.

What to do: Once more than a couple of teams are building with AI, shared infrastructure pays for itself: model access, retrieval, evaluation, logging and guardrails in one place. Each new feature then starts from the business problem, not from setup work.

7. Set bold automation goals

OpenAI automated thousands of its own support tasks each month, such as emails, tickets and account updates, with an internal platform that works across email, knowledge bases and CRM.

What to do: Do not stop at speeding up one step. Ask which whole workflows a person should not have to touch, start with high-frequency, low-complexity work, and redesign the process around the agent. Our guide on autonomous AI agents for business automation shows how those systems are built and measured.

Summary table

Lesson	Key insight	Example
1. Start with evals	Structured testing decides what ships and builds trust	Morgan Stanley evaluated before rollout
2. Embed in products	AI that explains results improves the customer experience	Indeed explained job matches to applicants
3. Start early	Daily use across the company compounds value	Klarna handled two-thirds of service chats
4. Fine-tune	Customization improves accuracy and consistency	Lowe’s improved product tagging and error checks
5. Empower experts	Staff build tools for the problems they know	BBVA staff built about 2,900 custom GPTs
6. Unblock developers	A shared AI platform raises output per engineer	Mercado Libre built one for 17,000 developers
7. Automate boldly	Redesign whole workflows, not single steps	OpenAI automated thousands of support tasks

Where generative AI creates a competitive advantage

Every competitor can buy the same models. Generating text, images or code is not an advantage on its own. The lessons above point to where the advantage actually comes from: your data, your workflows and how fast your organization learns. A Deloitte analysis of generative AI use cases maps similar patterns across industries.

It creates advantage when it runs on data only you have.

• Retail and ecommerce. Recommendations, product copy and search built on your catalog and your customers’ purchase history. Lowe’s and Mercado Libre both started by fixing and enriching their own product data.
• Finance. Fraud detection, risk assessment and stress-test scenarios grounded in your transaction history and risk policies, with an audit trail.
• Healthcare and life sciences. Treatment planning support and compound screening on clinical and trial data, under human review.
• Manufacturing. Generative design of components within your materials and production constraints, and maintenance planning from your own equipment history.

It creates advantage when it sits inside a workflow. A model that drafts an answer saves minutes. An agent that drafts the answer, checks policy, updates the CRM and routes the exception changes how a team works. The leader’s guide to AI agents covers how to choose and govern that kind of system.

It creates advantage when the organization learns faster. Klarna’s and BBVA’s figures on daily use matter more than any single feature, because each team that uses AI finds the next improvement.

It does not create advantage when the output is something anyone could produce with the same prompt: generic marketing copy, stock imagery or a chatbot over public information. Those may still save money, but they will not set you apart, so treat them as cost savings and pick the cheapest reliable option.

AI success takes time

The organizations in these case studies treat AI as a long-term capability, not a one-off project. They measure before they ship, put AI where customers and staff see the difference, and build shared platforms so the next project starts further ahead.

For production deployment strategies, read our guide to AI model optimization techniques. For retail specifically, see our research on AI in retail and commerce.

Want to apply these lessons to your own AI roadmap?

Lightrains helps companies set up evaluations, fine-tune models on their own data, and build the internal platforms and agents that take AI from pilot to daily use.

Talk to Lightrains about enterprise AI development

This article originally appeared on lightrains.com