5 May 2022
In April 2021, the European Commission proposed a landmark law to govern artificial intelligence. The “Artificial Intelligence (AI) Act” is intended to protect fundamental rights when AI systems are used. This Regulation is a world-first, and is likely to have global implications. ICCL has counselled co-legislators at the European Parliament and European Commission to encourage improvement in how the AI Act protects people and their fundamental rights.
The two European Parliament committees (IMCO and LIBE) leading the Parliament’s work on the EU AI Act have now set out their negotiating position in a formal report. Many of ICCL’s key amendments have been embraced, though there remains room for further improvements.
The positives
The report makes an important change to the definition of AI systems by removing the phrase "human-defined" so that objectives defined by AI systems autonomously are included. This change partially adopts ICCLs recommendation. We discuss below how the definition can be further improved.
In addition, most of our recommendations on enforcement have been embraced in the report, especially in the newly created Title VIII Chapter 3a and 3b. These adopted ICCL recommendations include strengthening the enforcement at Union level, giving individuals the right to complain and the right to judicial remedy. We note that the right to lodge a complaint to a supervisory authority has also been included in Axel Voss’ JURI report, albeit limited to the infringement of data protection. IMCO-LIBE report rightly eliminated this limitation.
The new Article 68f adopts ICCL’s recommendation to empower the Commission to "acquire samples related to AI systems, including through remote inspections, and the power to reverse engineer the AI systems". These powers are important for the enforcement to be effective.
Furthermore, when drawing up the technical documentation of AI systems, providers now need to provide "examples of scenarios for which the system should not be used." This takes account of another ICCL recommendation: that the providers should state the reasonably foreseeable misuses of the AI systems they place on the market.
Further change
There is still room to improve the AI Act. The definition of AI systems should be amended to remove “for a given set of objectives” so that AI systems whose uses are indeterminate are not left out. This change will address the issue of general-purpose AI systems without creating a new carve out, which will be in-line with the intention of the co-rapporteurs.
Furthermore, the Act is yet to consider the potential risks from the interaction of multiple AI systems. By treating AI systems as if they operate without interacting with each other, the Act ignores potentially significant risks to fundamental rights. This concern can be easily addressed by making minor but essential changes as we have suggested.
Although the report improves the enforcement regime in the AI Act, it does not strengthen the powers of Market Surveillance Authorities in Member States. In particular, the powers provided to the Commission in the new Article 68b (1) (d) and Article 68f should be given to MSAs in Member States as well. These strengthened powers, as we have mentioned before, would be equivalent to the minimum powers that MSAs have under other product safety regulations.
Finally, the report makes an important change by requiring "users who are public authorities or Union institutions, bodies, offices or agencies or users acting on their behalf" to register in an EU database. The use of the EU databased should be further strengthened:
- All users should be required to register in the EU database, not only public authorities.
- The database should include the serious incidents and malfunctioning that involve AI systems, but also near-misses, too. Publishing near-misses has the potential for operators of AI systems to fix their systems before a serious incident can occur. This kind of reporting has a broad societal benefit.