Do you need an internal AI assistant (RAG), or is a ChatGPT Business license enough?
Most SMEs do not need to build an assistant on their own documents to get started. They need a team license and clear rules. RAG earns its keep when precise, large, frequently changing internal knowledge must give everyone the same answer.
· 9 min read
The question has come up in almost every conversation with SME leaders over the past year: should we have an internal AI assistant built on our procedures, quotes, after-sales notes and product catalog, or is it enough to buy the team ChatGPT Business (formerly ChatGPT Team), Microsoft 365 Copilot, Claude Team or Gemini seats?
Short answer: for most SMEs, a well-configured team license plus three or four written usage rules covers the bulk of the need (drafting, summarizing, occasional document analysis). A RAG assistant becomes worth it when three conditions stack up: a large internal document base that keeps changing, repetitive questions whose answer must be exact and identical for everyone, and a real cost of error (a wrong price on a quote, a wrong after-sales instruction). In between, the connectors built into team licenses close part of the gap.
This piece is for founders and operations leads in small and mid-sized companies (roughly 10 to 250 people) who already see their staff using AI and wonder what to put in place. It does not cover AI built into a product you sell, nor large groups with an IT department and a data team. For regulation, see our AI Act article; for the gap between usage and gains, see the one on productivity.
Three stable definitions before comparing
Team license: an AI subscription the company buys for its staff, with an administered workspace. ChatGPT Business, Microsoft 365 Copilot, Claude Team and Gemini in Google Workspace are the usual examples. The difference from a personal account is rarely the model, which is often the same. It is the contract, the administration (who has access, which connectors are allowed) and how data is handled.
RAG assistant (retrieval-augmented generation): an assistant that, before answering, retrieves the relevant passages from a document base chosen by the company, then writes its answer from those passages and cites them. The model does not learn your documents; it looks them up on every question. Quality therefore depends mostly on the base it searches and on how that base is split, indexed and kept current.
Shadow AI: staff using AI tools outside any framework provided by the company, typically through free personal accounts or subscriptions paid out of pocket. It is not primarily an IT security issue in the classic sense. It is a signal: the need is real, and the official tool is missing or falls short.
One useful nuance: team licenses now do a form of RAG themselves. When ChatGPT Business searches your SharePoint, or Copilot draws on your Microsoft 365 files, that is retrieval before generation. So the real question is not 'RAG or no RAG'. It is 'the generic retrieval bundled with a license, or an assistant designed for one corpus and one job'.
The starting point: your staff already use AI, the question is which account
Before talking about a custom assistant, look at what is already happening. According to a YouGov survey for Microsoft France published on 12 February 2026 (657 managers and executives in French private companies, surveyed in January 2026), 61% of people who use AI at work use generative AI through personal accounts at least once a week, 38% of them daily. The same survey says more than seven managers in ten have had no AI training. The study was commissioned by a vendor that sells team licenses, so read the figure as an order of magnitude. It does match what we see in SMEs.
In practice, quotes, customer records and margin sheets are already going through personal accounts, with privacy settings the company does not control. Banning it works poorly: usage simply moves to phones. The fastest and cheapest response is to provide an official tool that does at least as well as the personal account. That is why the team license almost always comes before a RAG assistant: the license addresses shadow AI, a RAG assistant does not.
On the OpenAI side, the team plan is now called ChatGPT Business. OpenAI's help center, checked on 28 September 2026, lists a standard seat at $25 per user per month billed monthly, or $20 billed annually, with a two-seat minimum; prices vary by country and currency. OpenAI says it cut that price by $5 on 2 April 2026.
On data, OpenAI's enterprise privacy page states that ChatGPT Business data is not used to train its models by default, unless the organization explicitly opts in. That is the most concrete difference from a free or Plus personal account, where it is up to the employee to switch that setting off. Workspace admins manage members and choose which connected apps are allowed. For internal knowledge, ChatGPT Business offers projects, custom GPTs shared across the workspace, and a 'company knowledge' feature that searches connected apps (SharePoint, Google Drive, Slack, HubSpot and others) within each user's existing permissions and cites its sources, according to OpenAI's release notes.
For SMEs already on Microsoft 365, Microsoft 365 Copilot (which Microsoft's documentation now calls Microsoft Copilot) is grounded in Microsoft Graph: the emails, files, meetings and chats the user can already access. Microsoft's documentation states that prompts, responses and data accessed through Graph are not used to train foundation models. The Copilot Business add-on, for organizations of up to 300 users, sits around $20 per user per month in the US on an annual commitment, with a first-year discount advertised until 31 December 2026, on top of the existing Microsoft 365 subscription.
Claude Team (Anthropic) and Gemini in Google Workspace follow the same logic: an administered workspace, contractual commitments on data, connectors to company tools. We do not quote their prices here; they move fast and depend on the plan. The choice between these licenses depends mostly on the office suite you already run, not on a model leaderboard.
Where the license stops, and where a RAG assistant earns its place
License connectors search your files as they are. If your SharePoint folder holds four versions of the price list, two of them outdated, the tool may lean on the wrong one. It does not know which version is authoritative, or that the 2023 product sheet has been replaced. For individual use (finding a document, summarizing a file), that is fine: the person checks. For an answer that goes out to a customer, it is not.
A RAG assistant designed for a specific job differs on four points. The corpus is chosen and bounded (for example: validated after-sales procedures, the current catalog, terms and conditions). Documents are prepared for retrieval (chunking, versions, validity dates). Behavior is constrained (answer only from sources, say 'I don't know' otherwise, always cite). And answers are tested against a set of real questions before go-live.
Typical cases where it pays off: industrial after-sales, where technicians hunt for the right procedure among hundreds of manuals and service histories, with product references that look alike. A sales team that must price quickly from a catalog of thousands of items and discount rules, where a pricing error costs margin. A front desk or hotline answering the same procedural questions all day, where you want a newcomer to answer like a veteran.
Typical case where it does not: a fifteen-person team that mostly wants to draft emails, summarize meeting notes, rephrase offers and analyze a spreadsheet now and then. A custom assistant would cost more to build and maintain than it returns. A license, two or three well-prepared projects and a few rules are enough.
There is also an often overlooked middle floor: a shared project or custom GPT inside the license, fed with a dozen reference documents chosen and kept current by a named person. It is not an industrialized RAG assistant, but it often delivers most of the benefit for a fraction of the effort. It is a good test before investing.
The decision grid: five questions, count the yeses
Ask these five questions about one specific use case, not about 'AI in the company' in general. Count the yeses.
1) Is the corpus bigger than what one person can know by heart? For example several hundred procedure documents, a catalog of thousands of items, or years of after-sales history. 2) Do the same questions come up every day, from several people, and must they get the same answer?
3) Does a wrong answer have a direct cost: a wrong price on a quote, a safety instruction passed on incorrectly, a contractual commitment made by mistake?
4) Do the documents change often (prices, product versions, procedures), to the point where you need to know which version is authoritative? 5) Are the people who need the answer different from the people who know the documents (new hires, field technicians, sales reps, or even customers)?
Zero or one yes: a team license and usage rules are enough. Two yeses: a team license with connectors, or a shared project fed with maintained reference documents, then reassess in six months. Three or more: a dedicated RAG assistant for that use case deserves proper scoping, starting with document quality and ownership. Either way, the team license stays useful for everything else: the two are not mutually exclusive.
What a RAG assistant will not fix
Wrong or outdated documents. A RAG assistant answers from what it finds. If the reference after-sales procedure is three years old and technicians do something else, the assistant will serve the old procedure with confidence, sources attached. That is worse than no answer, because the citation builds trust.
No owner. Every corpus needs someone who decides which version is authoritative, removes obsolete documents and approves new ones. Without that person, the base degrades within months and people go back to the old way: asking the colleague who knows. If nobody wants that role, do not start the project.
Knowledge that is written down nowhere. In many SMEs, the real discount rule or the real fault diagnosis lives in two people's heads. A RAG assistant cannot retrieve what does not exist. The first job is then writing, not coding, and that is often where the project creates the most value, with or without AI.
Poorly kept structured data. A document assistant is not a reporting tool. If the question is 'what is the real stock for this item' or 'what margin do we make on this customer', the answer lives in the ERP or a clean database, not in PDFs. That is a data topic (see our articles on data audits and on choosing between a data warehouse and better-kept Excel).
And adoption. An assistant nobody opens returns nothing. The right indicator is not the number of questions asked in week one, but how often, three months later, it saves a call to a colleague or a trip back to the binder.
The practical next step
If your staff already use AI without a framework, start with a team license aligned with your suite (Microsoft 365 or Google Workspace, otherwise the tool people already use most) and a one-page set of rules: which tools are allowed, which data never goes in (health data, sensitive personal data, client secrets under NDA), who reviews anything that goes to a customer, and who to ask.
Then pick one use case where the same question comes up every day, run it through the grid above, and test it first with a shared project or the license's connectors. Log the questions that fail and why: document not found, outdated version, information never written down. That list will tell you better than any quote whether a RAG assistant is justified.
If three or more yeses come out and the test stalls on answer quality, that is the time to scope a dedicated assistant, starting with the corpus and its owner, not with the technology.
Torn between a team license and an assistant on your documents?
We start from one real use case, look at your documents and how they are kept, then tell you plainly whether a license and rules are enough, or whether an internal AI assistant is justified, and under what conditions.