Infraestruturas de Portugal AI agent procurement: contract object

Yesterday I started this series of blogposts by some procedural aspects of the project. Today, we're moving on to the object of the contract, that is BIA the Artificial Intelligence Bot (or agent). And here things are interesting as well to say the least.

BIA is to be used for the procurement of goods and services, of which Infraestruturas de Portugal tenders around 800 contracts a year (in 2024). Public works are not part of the technical specifications for this first iteration but there is an expectation it can be used there in the future. Since there are no explanations about this exclusion, we are left to wonder the reasons for public works not being in scope. I would guess it is probably the fear the complexity (and political/reputational risk) of such projects being a factor here.

Here's what Infraestruturas de Portugal is looking to acquire in a nutshell:

(The Homer or as it is affectionally known among Simpsons fans, the Homermobile. It was known for trying to do everything and then some. You can guess how it ended.)

Simpsons jokes aside, the fact of the matter is that the contracting authority wants its AI bot - sorry, agent - to do a lot of heavy lifting in the processing of goods and services tendering procedures:

• Automate the documentary verification of tenders
• Perform assisted analysis of tenders
• Support the jury in preparing evaluation reports, ensuring traceability and transparency

Those are only the general objectives for the contract, and they are elaborated upon further. The specific objectives state:

"Objective 1 - Automation of tender pre-analysis
Implement an agent capableof performing the automatic pre-analysis of all proposals received, reducingthe time required for theinitial screening and ensuring compliance with the requirements of the procurement process.

Objective 2 - Efficient tender analysis
Develop a system that efficiently and objectively analyses the submitted proposals, highlighting the strengths and weaknesses of each one in relation to the procurement requirements.

Objective 3 - Reduction of operational costs
Significantly reduce the manual effort required to carry out the entire pre-analysis and comparison process for all proposals received within a given procedure, freeing up human resources for more strategic and higher value-added tasks.

Objective 4 - Improving selection precision
Increase accuracy in proposal selection by eliminating subjectivity and supporting human analysis, ensuring a fair evaluation consistent with the requirements of the procurement process. Increased transparency in procurement processes.

Objective 5 - Intelligent feedback
Implement a system that provides detailed and intelligent feedback on each proposal, helping proponents to better understand their evaluations and encouraging continuous improvements in the quality of future proposals.

Objective 6 - Adaptability and continuous learning
Develop an agent with continuous learning capabilities and adaptability, constantly improving its effectiveness and accuracy through the analysis of data and historical results from this process."

These are then further refined in even more detail, but I shall cite only the bit about the work the AI agent is supposed to do to help the jury of the procedure:

• Confirmation of the existence and validity of qualified signatures on all documents submitted; 
• Verification of the presence of documents that must be submitted for the procedure;
• Extraction of information present in the Invitation to tender and/or Procedure Programme, such as the identification of documents to be submitted at each stage of the procedure, for the purposes of possible acceptance/exclusion of bids by the jury; 
• Analysis of clarifications requested/provided with a view to identifying additional exclusion factors; 
• Evaluation of potential conflicts based on the analysis of the composition of the corporate bodies of the various competitors;
• Price consistency check: total price versus itemised prices,numerical values versus values in words;
• Summary of the proposals, identifying the strengths and the award criteria, to facilitate the jury's work.

This. Is. A. Lot. Especially when we look at what the AI agent is supposed to be doing within each of those objectives, ie the automation of tender pre-analysis, analysis (not pre) of financial and technical capacity, summary and intelligent feedback.

While the first two bullet points are probably a good use case for AI in the public procurement pipeline, the remainder are fraught with issues. I could go at length on any of those objectives or technical requirements but for now will mention just two that are obviously problematic for me: the summarisation and continuous learning.

AI summarisation

AI summarisation is problematic because of LLMs probabilistic nature and their inability to grasp context and nuance in complex texts. They have been described as "lossy encyclopedias." Articles like this are still valid today, even when using cutting-edge LLM models. And yes, they still hallucinate and steps must be taken to reduce (not eliminate!) hallucinations.

Cutting edge LLM models are good enough for many uses, namely where, well, good enough suffices. But here we are talking about legal risk and the possibility of liability arising from summarising a suite of documents incorrectly. This critique can be extended to the whole approach of extracting and parsing data coming from PDFs, bit more about that later on. For now, in a country prone to public procurement litigation the use of a tool like this to summarise a tender has the potential to increase litigiousness instead of reducing it. 

AI summarisation of tenders is problematic for another reason. As described above, the contracting authority wants the tool to identify strengths and weaknesses thus facilitating the work of the jury. 

Despite the best efforts of Infraestruturas de Portugal stating (correctly) that the jury will remain in control of any decisions...but then Tesla states the same about the full autonomous driving capability of their cars. In the context of public procurement, what this means in reality is that the members of the panel will not become proficient with the tenders and their documentation. There is a counter-argument here which is that they are probably under resource and time pressure anyway so it is not as if they can spend as much time with the docuemnts as they might have wanted to. This is not a counter-argument I can easily rebuke, especially knowing all too well how procurement works in Portugal.

There is another more pernicious issue here though. By using a summary to base its decision on, the jury is beign invited to offload its own higher cognitive functions into the tool itself. Once that happens the jury will be hostage to the tool's view of the bids and each member influenced not by the actual documents in front of them but the AI summary instead. That is inevitable. You cannot unsee what you have just seen, even if wrong. Furthermore, the juries will now have to become proficient at deploying techniques to query the bot and to check for hallucination and mistakes. Those are not skills jury members are currently expected to develop!

From a litigation perspective this carries two additional risks. First, the lack of knowledge of the jury about the elements of the proposals will reduce its effectiveness in explaining its decision-making process. I did a lot of procurement litigation in Portugal and know how this works. Lawyers will have to try and paper over the cracks in the holistic knowledge a jury should have about the justifications for the decisions it takes. One can say that already happens, but the lawyers having to explain decisions partially based on LLMs will have a more difficult time doing so than in the past...especially if they also get access to those nice summaries.

Second, all this paper trail needs to be kept, meaning the logs, inputs and outputs of the tool will have to be provided to the court in case of a legal challenge since they are now part of the 'administrative process' behind the award decision. It is standard practice that if a challenge is brought before the courts, the contracting authority will send all the paperwork in connection with the procedure to the court. It remains to be seen what the courts will make of the chain-of-thought logs from the tool that are expected to be used to explain its output.

Continuous learning

As for the continuous learning, I am not convinced Infraestruturas de Portugal knows how these AI systems work. They are not trainees that learn on the job by themselves. For learning to occur, someone somewhere must be tasked with training the tool with that additional data. Call it model post-training, finetuning or something of the like. As I will discuss in more detail in the blogpost about the underlying 'open source' models Infraestruturas de Portugal wants to use, while free as in beer, they do contain some licensing restrictions on finetuning and the development of future models. 

Nonetheless, any work to evolve a solution will cost money, a lot of money and it is not something that appears to have been accounted for as there is no indication of running or maintenance costs, let alone a continuous development of the tool. This is a nice segway into tomorrow's blogpost on contract value and cost.