AI in Companies: How to Manage New Cyber Risks and Not Lose Control of Technology
AI in Companies: How to Manage New Cyber Risks and Not Lose Control of Technology
Why AI is fundamentally accelerating cyberattacks and how corporate defenses must change
How to safely implement AI in business processes and why quality data, access permissions and data sovereignty are key
How to set up AI governance, management accountability and control over AI agents with regard to security and regulatory risks
While not so long ago, companies primarily tested artificial intelligence, today they are increasingly connecting it with internal applications, documents and databases. AI assistants help employees search for information, prepare analyses or process customer requests. In addition, with the development of AI agents, there are more and more opportunities to automate entire parts of work processes.
However, it is the transition from experimentation to real operation that reveals new problems. Companies are gradually finding out that it is not enough to acquire quality technology. They also need to know what data they use, who is responsible for its operation, how its outputs are checked and what happens if the system starts working incorrectly.
These questions were also raised at the BDO expert breakfast in October focused on the use of AI and data in financial management. Experience from implementation projects shows that one of the biggest obstacles is not the technology itself, but the readiness of company data, processes and rules for its safe use.
Cyberattacks are accelerating – defenses must keep up
Today, artificial intelligence helps not only companies, but also cyber attackers. Generative tools make it possible to prepare convincing fraudulent messages faster, imitate the communication of specific people, or automate some phases of attack preparation. A significant risk is, for example, fraud using deepfake technology. An attacker can use a synthetic voice or image to imitate a member of the company's management and try to get employees to transfer money or hand over confidential information. But with the use of AI, not only the persuasiveness of attacks changes. Their speed is also crucial.At the BDO conference, American colleagues drew attention to their experience with attacks supported by AI agents, for which, according to their analysis, the security operations center had about 20 to 30 seconds left to react correctly. This is a specific experience, not a universal time limit, but it well illustrates how much automation can reduce the space for defense. In such situations, it may no longer be possible to rely solely on manual evaluation of security events. Companies will increasingly need security teams equipped with automated tools and AI defense agents that can quickly recognize, evaluate, and, in predefined cases, initiate an appropriate response.
At the same time, human oversight remains crucial, especially for decisions that can affect critical systems or the operations of a company. Therefore, the future of cyber defense will not lie in replacing experts with artificial intelligence, but in their ability to use it to speed up detection and response.
The biggest security risk doesn't have to come from the outside
In addition to attacks using AI, companies must also address the risks associated with its internal use. Let's imagine a company that introduces an AI assistant connected to the company's document storage. Thanks to it, employees can quickly search for information from contracts, internal directives or project documentation. However, if the system does not sufficiently respect access permissions, it may make information available to the user that they should not normally have access to.It is similar situations that BDO experts encounter when assessing the corporate environment. Connecting AI tools with large document repositories can reveal historically incorrectly set permissions or sensitive documents that the organization has not yet had sufficient insight into. Another risk is the uncontrolled use of publicly available AI tools by employees. In an effort to speed up work, they can enter client data, internal financial information, or other confidential documents into these services without the organization having sufficient visibility into the conditions under which they are processed.
In addition, with the development of AI agents, the scope for unwanted automated operations is expanding. Unlike conventional chatbots, agents can perform specific tasks through connected applications, such as editing records, processing requests, or running workflows.
Therefore, the company must know not only what data the AI has access to, but also what operations it is allowed to perform. Special attention is required for permissions to edit and delete data, work with backups, or the ability to make irreversible changes to information systems.
Without quality data, reliable AI cannot be expected
One of the most significant learnings from practical AI implementations is the importance of data quality and management. Businesses can invest in advanced models and automation, but if they are working with incomplete, inconsistent, or incorrectly defined information, the results may not be as expected. The problem does not have to be only missing or erroneous data. Often there are different definitions of the same indicators across the organization. A typical example would be the term “client”, “revenue”, or “profitability”, which is used differently by departments. If AI works with several different definitions without clear context, it can provide different results even with seemingly identical queries.At the BDO breakfast in October, it was said that data quality and its correct interpretation are among the recurring obstacles to the introduction of AI. In addition to technical security, data governance, i.e. clearly set rules for the management, ownership, quality and use of company data, is therefore gaining importance again. For organizations, this means the need to determine who is responsible for each data source, how the definitions of key indicators are unified, and how the origin of the information on which AI is based can be verified. This is especially important in financial management. If AI is to assist in the preparation of reporting or managerial decisions, it must be possible to distinguish between the validated calculation, estimation, and interpretation created by the language model.
The reliability of AI does not start with the model itself, but with the data and processes on which it is built.
Data sovereignty: where is company information and who has control over it?
With the expansion of cloud AI services, the issue of data sovereignty is also becoming increasingly important. Today, companies are increasingly dealing with where their information is physically stored, who can access it, and what legal regimes are subject to its processing. It's not just about meeting privacy requirements. Dependency on specific technology vendors, the availability of services, and the organization's ability to get its data back or switch to another solution also play an important role.This debate is also topical at the European level, where the issues of data control and digital infrastructure are reflected in the broader topic of technological and data sovereignty. From a business perspective, it is therefore important to assess not only whether a particular cloud service is sufficiently secure, but also what consequences its limitation, outage or change in terms of provision would have. One option is a hybrid approach, in which an organization uses external cloud AI services for less sensitive activities, while leaving selected confidential data and critical processes in its own or otherwise controlled environment.
However, such a solution is not automatically more secure. It requires the right architecture, access control, adequate technical security and clearly defined rules for working with data. Therefore, the choice between cloud, on-premises, and hybrid AI operations should be based on the company's specific security, legal, operational, and economic requirements.
AI governance: from individual experiments to controlled operations
With the growing number of AI tools and agents, companies are getting into a situation where it is no longer enough to solve their use individually at the level of individual teams. Employees can create their own assistants, automate workflows, and connect AI with other applications. However, if an organization does not have an overview of their number, purpose, and responsible persons, it gradually loses control over what technologies actually work in its environment.This is where AI governance comes into play, i.e. an artificial intelligence management system that combines security, responsibilities, internal rules and ongoing control. Experience from implementation projects shows that the entire life cycle of an AI solution is an important issue. It is not enough to decide on its implementation. Organizations must determine who will manage it, who approves changes, how new versions of models are tested, and who will intervene if the system starts to provide unexpected results. The economic side is also an important part. The operation of AI agents can be associated with variable costs, which depend on the frequency of their use, the complexity of the workloads, or the consumption of computing resources.
Therefore, it is advisable to assess not only the technical feasibility and safety, but also the real benefit of a particular solution. AI governance thus requires the cooperation of management, IT, cybersecurity, data specialists, finance, and business process owners. The goal is not to hinder innovation, but to ensure that individual experiments become a long-term sustainable and controlled part of the company's operations.
AI Act and regulatory requirements: safety as part of the responsible use of AI
The importance of AI management is also confirmed by European regulation. The AI Act establishes rules for the development, provision and use of AI systems, with specific obligations based mainly on the role of the organization and the risk of the given use. For selected categories of systems, the regulation puts emphasis on, for example, risk management, documentation, human oversight, reliability and cybersecurity. In addition to the AI Act, companies must also take into account other relevant regulations, especially the GDPR and, depending on the nature of their activities, also sectoral requirements.The financial sector, for example, pays increased attention to these issues. Banks and other regulated institutions must address ICT risk management, digital operational resilience and the security of sensitive information when using new technologies. The European DORA regulation also plays an important role here. However, it is crucial for companies to distinguish between formal compliance with regulatory requirements and the actual safety of the technologies used. An organization may have the necessary documentation processed, but at the same time it may not have access permissions, control of external integrations or procedures in case of compromise of the AI system correctly. Regulatory compliance and practical safety management must therefore be interlinked.
What should companies verify before deploying AI?
Safe AI adoption doesn't have to start with a large-scale transformation project. The first step should be to map the current use of the technology and identify areas where any bugs or security incidents can cause the most damage.In particular, organizations should focus on the following issues:
- Do we know where we use AI? The company should have an overview of approved and uncontrolled tools, AI agents and their connection to internal systems.
- Do we have control over company data? Data must be of sufficient quality, properly defined and protected. Do we know where it is processed and who can access it?
- Can we evaluate the risks of a specific application? We assess not only the functionality of the technology, but also its safety, operational, economic and regulatory impacts.
- Do we know who is responsible for AI? Responsibilities must be set up for approval, operation, output control, model changes, and possible system suspension.
- Are we prepared for an incident? Can we recognize unwanted AI behavior in time, limit its impact, and react quickly enough to automated attacks?
It is important that these measures do not remain only part of the documentation. Their functionality needs to be regularly verified and adapted to changes in technology and the way they are used.
The future of AI will not only be about the speed of innovation
Artificial intelligence will increasingly significantly influence the functioning of companies, their decision-making and the form of cyber threats. Therefore, organizations are faced with a task that goes beyond IT itself: to connect technological development with security, quality data, risk management and clear responsibility.Companies that implement AI without sufficient controls may inadvertently expand their own vulnerabilities. On the other hand, organizations that incorporate security into the design and implementation of new solutions will gain better prerequisites for their long-term and reliable use.
The decisive question is therefore not how quickly a company can implement artificial intelligence, but whether it can safely manage it even when it becomes a regular part of its business.
