The AI Act has direct implications for customer service departments that use AI tools for customer interactions, automation, or decision-making. Depending on how you use AI, you’ll be subject to either light transparency requirements or stricter requirements for high-risk systems. In this article, we answer the most frequently asked questions about what the AI Act means for customer service and how to properly prepare your organization.
Which AI applications in customer service are covered by the AI Act?
Most AI applications in customer service fall under the “limited risk” or “minimal risk” categories under the AI Act. Examples include chatbots, virtual assistants, and automated email processing. These systems are largely unregulated, but they must comply with transparency requirements whenever they communicate with people.
In practical terms, this means that a chatbot or voice-based AI assistant must always clearly inform users that they are communicating with an AI system and not with a human. This will take effect on February 2, 2025, as the transparency requirements are already in force.
However, there are situations in which customer service AI does fall into the high-risk category. This is the case when the system:
- Profiling individual customers to make decisions about access to essential services, such as credit or insurance
- Automatically determines whether someone is eligible for a service or product based on personal characteristics
- Used for emergency calls or urgent services where errors have direct consequences for safety
- Emotion recognition applied in the workplace, for example, to monitor employees
AI systems that perform only routine tasks, such as forwarding messages or retrieving information from a knowledge base, generally fall outside the high-risk definition. However, as soon as a system makes independent decisions that directly affect customers, the classification changes.
What are the requirements for high-risk AI in customer interactions?
If an AI system used in your customer service is classified as high-risk, significantly stricter requirements apply. The most stringent requirements fall on the system provider, but as the deployer—the organization that implements the system—you also have clear responsibilities.
Requirements for Providers of High-Risk AI
Companies that develop and market high-risk AI systems must, among other things:
- Establish a continuous risk management system throughout the system’s entire lifecycle
- Maintain technical documentation in accordance with the requirements of Annex IV of the AI Act
- Enable automatic event logging
- Systematically incorporate human oversight into the system’s design
- Apply for CE marking and a declaration of conformity
- Registering the system in the European AI database
Requirements for Deployers in Customer Service
If your customer service department deploys a high-risk AI system from a third-party vendor, you are the deployer. Your obligations are more limited, but they are not without responsibility:
- Use the system as specified by the provider
- Assign human supervision to qualified and trained employees
- Retain logs for at least six months
- Informing employees about the use of the AI system before it is put into use (Article 26(7))
- Conduct a data protection impact assessment (DPIA) where applicable
In addition, under Article 86, customers have the right to request an explanation of decisions made about them by a high-risk AI system. So make sure you are able to provide that explanation.
When must a customer service department comply with the AI Act?
The AI Act will be implemented in phases. For customer service departments, three dates are most relevant. The prohibited practices and the AI literacy requirement will take effect on February 2, 2025. The requirements for high-risk systems will become fully enforceable on August 2, 2026.
Here is the specific timeline:
- Effective February 2, 2025: Prohibited AI applications are no longer allowed. Examples include manipulative techniques, emotion recognition in the workplace, and social scoring. The AI literacy requirement is also in effect, which means that employees must have a basic understanding of AI.
- Effective August 2, 2025: Requirements for providers of General Purpose AI (GPAI) models are in effect. Fines may already be imposed.
- Effective August 2, 2026: Most requirements for high-risk Annex III systems will become enforceable. This is the date by which customer service organizations that use high-risk AI must be fully compliant.
So don’t wait until 2026 to get started. Setting up an AI registry, assessing your systems, and training employees takes time. Organizations that are already taking action now will be in a much stronger position down the road.
How does the AI Act differ from the GDPR for customer service teams?
The AI Act and the GDPR complement each other but address different risks. The GDPR protects personal data and regulates how it is processed. The AI Act regulates AI systems themselves, regardless of whether they process personal data. For customer service teams, this means you must apply both frameworks in parallel.
A few specific differences:
- Scope: The GDPR applies to all processing of personal data. The AI Act applies specifically to AI systems and their impact on people, even when no personal data is involved.
- Risk Model: The GDPR is based on the principles of data minimization and purpose limitation. The AI Act uses risk levels to determine how strict the requirements are.
- Transparency: Both laws require transparency, but the AI Act adds a specific requirement: users must know when they are interacting with an AI, even if no personal data is being processed.
- Overlap in DPIA: For high-risk AI that also processes personal data, both an AI Act compliance assessment and a GDPR DPIA are required. These processes may overlap, and you can combine them effectively.
For the day-to-day operations of a customer service department, this means: the GDPR compliance measures you’ve already put in place are a good foundation, but they’re not enough. The AI Act requires additional documentation, risk assessments, and governance specifically related to your AI systems.
What do customer service representatives need to know about AI transparency?
Customer service representatives must understand that the AI Act establishes two types of transparency requirements: transparency toward customers and transparency within the organization. Both are relevant to day-to-day work practices.
When it comes to customers, any AI application that communicates directly with people must identify itself as AI. A chatbot that conducts conversations, a voicebot that handles incoming calls, or a system that sends automated emails must clearly indicate this. Employees who receive complaints about AI interactions must know how to handle them properly and where customers can exercise their rights.
The AI literacy requirement (Article 4) is already in effect. This means that organizations must train their employees in the basics of AI: what it is, how it works, and what its limitations are. Employees do not need to become technical experts, but they must:
- Understanding which AI systems they use every day
- Knowing When an AI Decision Requires Human Oversight
- Being able to accurately inform customers about the use of AI in customer interactions
- Recognizing signs that an AI system is producing inaccurate or biased results
Automation bias—the blind reliance on AI recommendations without critical thinking—is a specific risk that the AI Act aims to address. Employees must be trained to view AI as a tool and not as an infallible decision-maker.
How do you prepare a customer service department for the AI Act?
Preparing for the AI Act starts with a clear overview of all the AI systems you use. Without that inventory, you won’t know which obligations apply to you. A structured four-step approach will help you get started.
- Create an AI registry: Document all the AI systems you use, including their purpose, vendor, and the role your organization plays (deployer, provider, or something in between).
- Classify your systems: Determine the risk level for each system. Most customer service tools fall under the “limited” or “minimal” risk categories, but be sure to verify this explicitly for systems that make decisions about customers.
- Review contracts with suppliers: Ensure that suppliers of AI systems fulfill their obligations as providers. Request technical documentation, user manuals, and information on how human oversight is built into the system.
- Train your employees: Invest in AI literacy for everyone who works with AI systems. This is not only a legal requirement, but also a way to improve the quality of your customer interactions.
Organizations that use multiple providers for phone services, chat, WhatsApp, and email are at greater risk of compliance gaps. If systems do not communicate with one another and there is no central overview, it is also virtually impossible to properly implement AI Act requirements such as logging and human oversight.
How Pegamento Helps Ensure AI Act Compliance in Customer Service
As a customer service department, you want to use AI to better assist customers, not to get bogged down in compliance issues. We help organizations achieve both goals at the same time: smart AI implementation that also complies with the requirements of the AI Act.
Our Agentic AI for customer service goes beyond traditional automation. Whereas classic RPA bots follow instructions, our Agentic AI assistants take the initiative on their own: they think along with you, act proactively, and scale up when necessary. We don’t do this with expensive custom solutions, but with a smart combination of proven standard building blocks that we configure to fit your situation.
Here’s what we specifically offer to help you prepare for the AI Act:
- Transparent AI systems that identify themselves to customers, in accordance with transparency requirements
- Built-in human oversight, so employees can always take control
- Logging and reporting on all customer contact channels from a single central platform
- Support for classifying AI systems and preparing documentation
- Everything under one roof: from implementation to management and support, without a complex supplier structure
We are ISO 27001-certified for information security, supplemented by ISO 9001 and ISO 26000, which means that compliance and quality are systematically ensured at our company. Would you like to know how your customer service department is currently performing and what it takes to operate in compliance with the AI Act? Contact us, and we’d be happy to help you figure it out.
Frequently Asked Questions
What happens if my organization does not comply with the AI Act?
In the event of non-compliance with the AI Act, regulators can impose substantial fines. For violations of the most serious obligations, such as the use of prohibited AI practices, fines can reach up to 35 million euros or 7% of global annual revenue. For less serious violations, such as failing to comply with transparency obligations, the fines are lower but still significant: up to 15 million euros or 3% of revenue. In addition to financial penalties, you also risk reputational damage among customers who are becoming increasingly aware of their AI rights.
How do I know if an AI tool from a third-party provider is AI Act-compliant?
Explicitly ask your provider for technical documentation, a statement of compliance, and information on how human oversight is built into the system. For high-risk systems, providers are required to provide this documentation in accordance with Annex IV of the AI Act. Also verify whether the provider has registered the system in the European AI database and whether clear instructions for use are available. If a supplier is vague about this, that is a significant red flag to be critical about the partnership.
My organization uses a chatbot that sometimes uses human names. Is that allowed?
No, this violates the transparency requirements of the AI Act, which have been in effect since February 2, 2025. An AI system that communicates with people must always clearly indicate that it is an AI, even if it has a human name or personality. This applies to chatbots, voicebots, and automated email systems. Adjust your chatbot’s settings so that, at the start of every conversation, it explicitly states that the customer is communicating with an AI assistant.
How do I handle customers who ask for an explanation of an AI decision?
Under Article 86 of the AI Act, customers have the right to request an explanation of decisions made about them by a high-risk AI system, such as a rejection based on an automated assessment. Ensure that your customer service representatives know which AI systems make high-risk decisions and that they have access to the information needed to explain those decisions. Establish internal procedures for how this process will be handled and who is responsible for responding to such requests.
Does the AI literacy requirement also apply to employees who use AI only indirectly?
The AI literacy requirement (Article 4) applies to all employees who work with AI systems, even if only indirectly. Think of a team leader who reviews reports generated by AI, or an employee who processes customer files that have been pre-sorted by AI. The level of training does not have to be the same for everyone: employees who actively use AI on a daily basis need more in-depth knowledge than colleagues who only have indirect contact with it. Tailor the training program to the extent of AI use for each role.
What is an AI registry, and how do I start creating one?
An AI registry is an internal document in which you record all AI systems within your organization, including their purpose, vendor, risk level, and the role your organization plays. Start simple: make a list of all the tools you use for customer contact, automation, or decision-making, and note for each tool who the provider is and what the system actually does. Then, for each system, add the risk level based on the AI Act categories. This register is not only useful for compliance but also for internal governance and supplier management.
As a deployer, do I need to take any action if my AI supplier ceases operations or makes significant changes to the system?
Yes, as a deployer, you are responsible for ensuring the continuity and proper functioning of the AI system you deploy. If a supplier makes significant changes to the system, you must reassess whether the system still meets the AI Act requirements and whether the risk classification is still accurate. Ensure that contracts stipulate that suppliers inform you in a timely manner of any significant changes, updates, or the discontinuation of the system. This will help you avoid unexpected compliance gaps and allow you to take alternative measures in a timely manner.
