National Audit Office publishes report on Brexit consultancy expenditure

Well, it looks like today is transparency day in the blog as the NAO has just published a report on the spend by Central Government with consultants for Brexit related matters. Predictably it makes for grim reading and here’s a bit of the press release (the final paragraph):

The NAO found that departments have not met the standards of transparency expected by government when publishing details of contracts for EU Exit consultancy. In December 2017, the Crown Commercial Service issued guidance to encourage greater transparency in government procurement. It recommended that departments publish basic information about the award of contracts within 90 calendar days. However, the NAO found that it has taken on average 119 days for basic details of EU Exit consultancy contracts to be published, compared to 82 days for all consultancy contracts. The NAO also found that in its review of contracts for EU Exit consultancy that some had not been published as recommended, and all that had been published were significantly redacted.


That’s one well buried lede.


Spain publishes guidance for low value contracts

Spain’s current public procurement law imposes strong transparency requirements for most contracts below the financial thresholds, restricting the use of non-transparent procedures to contracts value at below €15,000 (services) or €40,000 (works) only. My take is that overall this is a positive step and one that I regret Portugal not following.

That does not mean the move is painless or without difficulties. To clarify the requirements and operational implications of the move, Spain’s Procurement Regulator Body (something Portugal should have copied too…) published its first ever binding guidance specifically on Art 118 of the Spanish Public Contracts Law.

The guidance makes a very restrictive interpretation of the grounds enabling the use of the non-transparent procedure, from the requirements (deemed as cumulative) to the potential loopholes of contract splitting or recurrent yearly contracts with the exact same object. Julio Gonzales has a few extra comments (in Spanish) on the Global Politics and Law blog.

The Commission's 6 strategic priorities for public procurement policy

Continuing the short game of procurement tennis with Albert Sanchez-Graells on the Commission's Communication Making Public Procurement work in and for Europe COM(2017) 572 today I will be talking about the Commission's 6 strategic priorities for public procurement policy. These are:

  • Ensuring wider uptake of strategic public procurement
  • Professionalising public buyers
  • Improving access to procurement markets
  • Increasing transparency, integrity and better data
  • Boosting the digital transformation of procurement; and
  • Cooperating to procure together.

Albert's comments on these priorities can be found here. He went for the catchy trumpesque title, whereas I went for the more listicle like one. Our views, are fairly similar in most of the priorities and I am happy he picked up on my earlier bugbear of the choice of language and lack of precision in the terminology used. He made some very critical remarks on the Commission's introduction and picked up an underlying trend (visible in other areas) of using procurement to achieve industrial policy goals. That objective is orthogonal to the idea of a single market in procurement: we achieve either one or the other, but not both at the same time. Plus, I would refer readers once more to the mounting body of evidence that buying national simply does not work

Albert's assessment of the Commission's example of "best practice" in innovative cross-border procurement (section 2) is brutal, but fair. Too much "practice" is passing off as "best practice" simply because someone somewhere says so and in absence of any evidence to back up the claim. It reminds me of a comment I felt compelled to make once to someone who is now a DG Director and was unfortunate enough to mention about a decade ago the Vasco Da Gama bridge in Lisbon as a leading example of PPP/PFI, soon after the Portuguese Audit Court published a review criticising the 15 contract amendments and extensions which always left the economic operator better off.

Ensuring wider uptake of strategic public procurement

The Commission starts by clamouring for more strategic public procurement. Albert claims, correctly (once more) that there is zero empirical evidence to justify this objective. As far as I can tell this "ambition" remains firmly grounded in the world of "policy goals" that do not have a solid reason for it. Why is it desirable? Will it leaves all better off?

My view (as well as Albert's) is that some strategic public procurement can be compatible with the single market, thus leaving us better off. This is mostly the green procurement part, since I have long accepted that internalising the true cost of pollution leads to a more complete single market than one that ignores such cost. Innovative procurement can also be compatible with the internal market but on a different basis. It is possible for procurement to function as a lead buyer for innovative products and services, but having said that those will always be the exception and never the rule for day to day expenditure (just the OJEU for innovation partnerships...). All in all, they may lead to more complex (expensive) procurement processes and also more expensive goods and services, but as long as they do internalise externalities or lead to products/services that otherwise would not exist I see them as fundamentally compatible with the single market (in this I depart slightly from Albert's reservations about higher cost while agreeing on the possible downsides for SMEs). Oh, and the amendment to the Clean Vehicles Directive 2009/33/EC could not come quick enough, but its remit is much wider than public procurement.

The problem lies with social considerations as strategic objectives. These range from the wooly to those that appear to be common sense (ie, demanding wages above the minimum wage or imposing one's minimum wage on other countries workers) or even those that are down right protectionist (demanding local labour/apprenticeships). These tend to be an anathema towards the objectives of the single market which remains (at this moment in time) at the core of the Union. I won't even talk (for now) about the exporting of European labour standards to third countries and what that does to both EU consumers and those producers, but Dani Rodrik does on his new book (p.229 I'm told!). But my worries about the growing compliance culture in procurement can be found here

Professionalising public buyers

Most of our views about this priority can be found in the first entry to this procurement tennis match (here and here). There is however an important angle we omitted the first time around. By asking for the professionalisation of public buyers the Commission indirectly recognises that procurement is today difficult and complex - thus requiring the said professionalisation and specialisation in some instances as well. 

Should we not be aiming for the opposite then? Making procurement so simple that it actually does not require that much specialist knowledge?* After all our phones today are mightly complex but said complexity is abstracted away for the benefit of the user. This goes hand in hand with my usual analogy that procurement rules are just like operating systems. And the one we use has really old roots.

*no, I'm not thinking about principles and negative obligations like in the 60s to be the solution.

Improving access to procurement markets

I share Albert's disappointment with this part of the communication. It reads (again) as a general enunciation of ideas that lack coherence or fit.

This is one of the most disappointing aspects of the October 2017 Communication. The Commission indicates that improving access to procurement is mainly geared "to increase the SME share of public procurement in line with their overall weight in the economy", in particular "in view of promoting more cross-border procurement". However, the only specific actions mentioned by the Commission concern (i) the Remedies Directive (and, specifically, its criticisable decision not to review it, see here and here), (ii) the initiative on third country access to EU procurement markets (see here), and (iii) a sectorial initiative to increase SME participation in defence and security contracts. This is puzzling. 

While those initiatives can have some effect on increasing SME access to procurement markets, they are unlikely to facilitate a step change. Much more is needed in terms of guidance and best practice on facilitating SME access to procurement domestically and in an EU cross-border context (which the Commission should undertake), and there are obvious limitations derived from the cost of having the administrative (and language!) capacity needed to export. In that regard, the proposals in the Communication do not even brush the surface of what could be done at EU-level--starting with practical guidelines on how to maximise the advantages derived from the fact that, in the Commission's own terms, "[t]he 2014 directives include measures that should facilitate the access of companies including SMEs to public procurement, also cross-border". It would certainly be helpful for the Commission to flesh that view out in more detail.

But there is more to criticise on that quote. The Commission claims we need to increase the share of SME spend in line with their overall weight in the economy, but earlier in the document it was claimed SMEs win 45% of the aggregate value of contracts above the EU thresholds. According to the OECD the added value of SMEs in advanced economies lies between 50 and 60%, so we are not exactly a magnitude away *and that is only for contracts above the EU thresholds* which are not really small or SME friendly anyway. As the Commission is worried about SME cross-border success rates (rightly so) is it proposing to lower the thresholds or remove the non-tariff trade barriers on lower contracts? No such thing is to be found in the document though.

The Commission could (but has not) made the point to justify the increase by including only those situations where SMEs have won by themselves or in a consortium a given bidder. It has, however, decided to include in the 45% figure also sub-contracting. Sub-contracting does not really have much to do with procurement rules or (practice) since main contractors are free to decide who they want to contract with and crucially how they want to spend their contract money. So in reality the figure of SME participation rate in public procurement may effectively be much lower - but we are left in the dark by the use of a questionable statistical method by the Commission.

There are some other questionable points about SMEs and public procurement. There is no explanation why it is desirable or better to have more SMEs tendering (and therefore, increasing their transaction and opportunity costs). It appears an article of faith that more SMEs tendering is simply better but without taking into account the tradeoffs it entails.

The Commission also claims that remedies are important for SMEs, but that could be argued about any sort of economic operator taking part in public procurement. Furthermore, if they are so important maybe the Commission should have taken the opportunity to review the Remedies Directive as well.

The Commission revisits as well in this section its ongoing battle with international cross-border procurement and third country access. I do not see how the words included here ("restoring a level-playing field is more than ever necessary") make any significant difference. More so, bearing in mind the poor level of direct cross-border procurement within the single market, maybe discussions about third country access will make the Commission lose the forest for the trees. 

But all that is low level hanging fruit. There is no spark, or transformational initiative that could really make a significant difference in cross-border procurement. Here's a few:

  • What about creating a set of procurement INCOTERMS that would make it easier for economic operators to understand the obligations arising from the contract? If we have CPV codes why can't we have INCOTERMS for procurement obligations as well?
  • What about solving the issue of different contract performance regimes? Only larger economic operators can actually invest in having subsidiaries (or access to legal teams in another country) to justify the legal risks arising from contract performance. Why are we not seeing a clear pathway to harmonising public contracts rules?
  • What about language issues? Is it a coincidence that most cross-border public procurement is literally cross-border or between Member States sharing a language? Ironically the Commission flags up language as potential issue on cross-border cooperation in procurement, but not in regular procurement activity...

Increasing transparency, integrity and better data

The Commission is right that making more procurement data available is a step in the right direction and will propose the adoption of new e-forms, ie standard public procurement forms. Nothing is earth shattering new about this and it can be useful as long as the data is kept on a machine readable format and available with permissive licenses. Like, for instance the Open Contracting Data Standard which is absent from any reference...

The suggestion of mandatory public registers is in my view the correct one, but the Commission stops short of solving the biggest issue behind this idea. After all, sending contract data to the OJEU is already mandatory, it is at the compliance level that reality its the proverbial skids. There is a simple way (for contracting authorities that is) to ensure compliance: ensuring that whatever system is used to make the purchased is connected to a central system capturing the information automatically. As each country is essentially developing its own standards and approaches to electronic procurement this is sadly a pipe dream for now and - yet again - a missed opportunity to make real progress.

I do not agree that Directive 2014/24/EU has strengthened the provisions on conflicts of interest, but precisely the opposite of making it clearly legal economic operators can take part in the process even if they help out drafting the technical specifications. It remains to be seen if I am wrong about this...

The Commission is also correct in its conclusion that more data allows for collusion to be more easily spotted. That is true, since it is akin to keeping urine samples from the Olympics to re-test in the future as more advanced tests are developed. The opposite is true as well for markets where collusion is advanced. By making the data public cartel members will be tempted to develop collusive techniques that either do not show up in the data or that show in ways the enforcers have not spotted yet. Solve for the (new) equilibrium as they say...

In addition, for all the references to use more and better data, the Communication is fairly light in providing empirical data to sustain any of its proposed solutions...

Boosting the digital transformation of procurement

This is a particularly disappointing section. The Commission is correct when it states that the benefits of electronic procurement will only be reaped when the whole process undergoes a digital transformation...which means? Not much if one trusts this section. The actual solutions provided by the Commission are simply re-hashes of previous initiatives such as the Single Digital Gateway or the European services e-card. And not a word about the roll-out of the ESPD and its teething problems...

Which is a shame since integrating different systems and processes digitally can really make a difference. I think the Commission should be looking at integrated cross-border payment systems for inspiration on how the process could be re-organised in a way that functions across multiple jurisdictions. Once more, I think we should be looking at how the private sectors solves its own cross-border transaction issues.

There is also no indication the Commission understands how digital services are different from regular services and how those by definition should always be part of the single market, irrespective of their value.

Cooperating to procure together

The final section cover cross-border cooperation in procurement and exemplifies yet again the Commission interest in looking to the accessory at the expense of the principal. Cross-border cooperation in procurement will remain a niche area and one that needs significant prior work before it can justify this kind of attention. I will defer to Albert's comments about the lack of legal certainty arising from this idea.

The wooly language used doesn't help the Commission's case: 'Their [CPBs] role in the standardisation of public procurement processes and market insight also represents a key element for the professionalisation of public administrations and it enables SME-friendly procedures.' Right.

In other words, we should be solving other more pressing problems (cross-border procurement in general) before spending time with vanity projects. In a way, this is no different from all the bruah about procurement of innovation (hey, innovation partnerships, competitive procedure and competitive dialogue I'm looking at you) at the expense of the day-to-day operations in procurement. There is a lot more that needs to be done for 99% of the procurement projects *before* we should spend time on the 1%.

Appendix

Going through the Appendix I was surprised to see some suggestions that were not mentioned at all in the main body of the Communication, leaving us in the dark of what they may actually refer to.

For example, on section 3 (improving access to procurement markets) the Commission will apparently "launch pilots to boost SME participation via business intermediaries and innovation brokers" whatever that means.

On section 6 (cooperating to procure together) another pilot on SME-firendly policies in Central Purchasing Bodies, whatever that may be.

All in all, a missed opportunity.

Links I Liked [Public Procurement]

1. Big tech's grip loosens on UK.gov IT spend.

2. World Bank puts out it "Benchmarking Public Procurement 2017" report. Albert has the lowdown of it.

3. The best way to build big is to start small. Agreed, way too many initiatives in public sector are based in delusions of grandeur when there is plenty of low hanging fruit yet to be picked.

4. A more transparent public procurement (Catalan only). Mostly about how perspectives on public procurement have changed in Catalonia, albeit those reductions are fairly minor. 

5. Compliance and public procurement (Spanish only). I suspect this will be a bigger topic in the years to come as procurement gets dragged more and more into a compliance frameset.

Links I Liked [Public Procurement - UK edition]

1. Wales will have its own regulatory powers in public procurement. I do not know how this will pan out since the Public Contracts Regulations 2015 cover Welsh contracting authorities for the most part. Maybe their powers will be restricted to regulate the areas that are not already regulated by the PCR2015, ie contracts below-thresholds whose provisions do not apply to devolved contracting authorities. Either way I would expect the Welsh Government to push some of their policies, particularly SQuiD and community benefits.

2. Central Government wants to mandate apprenticeships in large construction contracts. Now that was quick. There was a rumour flying around for the last few years that Francis Maude was really not keen on social clauses in procurement. My opinion on this is clear: no one knows the costs involved by introducing social clauses (which raise complexity and imply transaction and opportunity costs). the impact on SMEs and that these are the gateway drug to offsets as they exist in defense. There are no free lunches, but at least the Central Government will apparently limit this obligation to contracts above £10M. For now.

3. What would a 'Brexit' mean for public procurement in the UK? Not much in the short run, particularly if the UK decided to join the GPA. A useful scapegoat would be lost however (oh those damn, pesky Directives, source of all evil..).

4. More mystery shopper results. This time for July/August 2015.

5. Data in Whitehall: which UK departments are the least and most open? It is so much easier to get on the high horse of claiming transparency than actually delivering it.

6. Edinburgh drops BT for CGI, claims to save £45M over 15 years. If it is true, another sign it pays to go to the market regularly for large projects.

Slovakia, public procurement and transparency

Transparency International put out a detailed report about the experience in Slovakia of mandatory advertising of contract award information. I am a huge fan of contract award transparency and am always on the lookout for things like this report which is full of interesting findings.

Apparently Slovakian contracting authorities have to publish the contract award information since 2011 and covers contracts both above and below the EU thresholds. The result is that a plethora of contract information is now available online and Slovaks are making good use of it: apparently the database website gets 54,000 visits...every month. That is a lot of people for such a small country. I wonder how many are from citizens and how many from suppliers trying to figure out how the market is working and what prices are being paid (I do not see this as negative, but Albert probably objects).

What about the effects of this radical transparency? On the one hand...

On the plus side the report claims the accountability brought by the interest citizens show on the information, plus the media is reporting more and more cases. This can be a double edged sword as Gustavo Piga told me: more information in the hands of good journalists is great, in the hands of bad journalists it's a nightmare. I can totally relate to that. As a lawyer one of the cases I was involved in was plastered all over the press. Obviously, the "journalist(s)" only presented one side of the story and had no interest in the actual truth. The authors also mention this as an issue.

Another positive finding claimed by the authors is that the radical transparency led to higher participation in tenders. There may be some correlation between transparency and more participation and perhaps suppliers are more confident that the increased transparency leads to reduced foulplay, but I saw no evidence of causation. In my view, the increased participation rate is probably due to a combination of factors wider than just transparency. Moving procurement online (cradle to grave e-procurement) seems to me as a bigger reason and one that can be married to the more difficult economic times which lead to suppliers having to compete more fiercely for business.

Increased transparency reduces the incentive to use non-transparent procedures as the benefit of the non-transparent procedure disappears if at the end you need to make the end result public. The authors mention the reduction in non-transparent procedures but make no claim about the connection to the transparency reform.

Another positive finding is a lower perception of corruption. I agree that putting all the information online can contribute to dispel foulplay myths surrounding many contracts, thus contributing to a reduction on corruption perception levels. Furthermore it apparently also led to a reduction in queries from the public (makes sense as well), thus reducing the burden on contracting authorities to provide that information time and time again with each request.

On the other hand...

The authors looked into costs and concluded that apart from startup costs (ie, systems and moving everything online) the marginal cost of uploading the information of each contract to the database is quite limited. Although it seems the data came from empirical research with only 4 authorities and it would have been preferable to test it wider, this seems a reasonable finding. I would add that we can reduce the marginal cost to zero with proper e-procurement software (ie, getting software to do it automatically). That is a hint for platform providers as other countries (such as parts of the UK) want to move in a similar direction.

I am less convinced however by the dismissal of collusion and cartels. The authors state that "As for fear of collusion in tenders or loss of interest of companies in dealings with the state, we find little evidence of their existence." Little evidence is one thing, lack of evidence another. I suspect they meant the latter. It is a well known fact that identifying cartels is difficult without whistle-blowing so the absence of evidence is not the evidence of absence. I am not sure the authors dug deep enough to reach the latter (Am I channeling Albert here?) and have made similar comments about Portugal in the psat. The contrarian argument made by the authors is that even with the added transparency, average tender participation numbers have crept up. In my view, it indicates that i) suppliers are discounting the fact the contract will be published; ii) in general, competition appears to have improved even with the added transparency. Additionally, as I mentioned here before more information tends to make markets better (by reducing arbitrage) not worse, except for the cases where there are underlying conditions for cartel formation. More information makes life difficult for the insiders who benefited from access to the decision-makers.

Issues

The authors highlight the lack of compliance monitoring as one of the current issues in the system. Compliance is still not 100%, contracts miss vital information and there are reports of contracts "disappearing" from the register. This will always be a problem with centralised systems, or any system that uses human intervention for that matter. I will take the bait and plug again my idea of a blockchain type of interface to deal with contract information and feedback.

 

Public Contracts Regulations 2015 - Regulation 55

Regulation 55 - Informing candidates and tenderers

Regulation 55 defines the timings and kind of information tenderers are entitled to receive at the end of their participation in a procurement procedure, particularly in case they were unsuccessful. The obligations for the contracting authority cover both the decision to conclude or not a procedure and the grounds on which the decision was taken. This Regulation is a perfect example of the underlying tensions between the principles of transparency and competition in public procurement.

This Regulation is quite similar to Article 55 of Directive 2014/24/EU as well as part of Regulation 32 of the Public Contracts Regulations 2006, although it is much cleaner and prescriptive than the latter. Regulation 55 includes three exceptions to the disclosure rule. First, in case it would impede law enforcement or would be against public interest. Second, that it would prejudice legitimate commercial interests of a particular economic operator (ie, the contract winner). Finally, in case it might prejudice the fair competition between economic operators. Personally, I think these will be over-used as in the past to avoid disclosure of information about contracts.

Explaining why a procedure was not taken to the end does not appear problematic. However, the main rule under Regulation 55  that tenderers are to be informed of the outcome of the procedure and a certain degree of information about the winning tender, raises some questions. Last week Albert criticised the amount of information provided on competition law grounds and provided excellent reasons why that is so. I disagree with him as we come from completely different perspectives. He believes that too much transparency about contract awards distorts the market and facilitates collusion (it does, at least in markets/sectors where collusion is already present or likely). Myself, on the other hand I believe that disclosing a lot more information will actually make the market less imperfect by providing the same level of information to *all* interested players in the market. It would significantly change the market dynamics. Furthermore, less information creates opportunity for arbitrage which, in my view, economic operators have been benefiting from over the years. Unless, perhaps economic operators have been always fair and charging different contracting authorities the same price for the same products/services, right? I believe that more information leaves markets *better overall*, even accepting that in some cases it will indeed help out colluders.

The test subject on this should be Portugal where since 2009 all (read: most) contracts awarded below thresholds are published online and have to follow e-procurement from cradle to grave. Technically contracts above thresholds should have been published as well but a clear obligation is more recent than 2009 and was part of the bailout programme Portugal benefited from. So far, no evidence coming from the country supports the claim collusion is rampant. Now, absence of evidence is different from evidence of absence, but there are two data points that are important to take into account: SME participation rates are going up and prices have come down in some sectors, indicating at least a modest increase in competition. I will try to get some more information about the country tomorrow as I will be at a conference in Porto.

I would heartily support a proper piece of research on this in Portugal. Any takers?

 

 

 

Public Contracts Regulations 2015 - Regulation 33

Regulation 33 - Framework agreements

In Regulation 33 we can find the rules applicable to framework agreements. This Regulation transposes Article 33 of Directive 2014/24/EU. Regulation 33 also starts the section on "Techniques and Instruments for Electronic and Aggregated Procurement". How exciting!

Framework agreements have become remarkably popular in the UK over the last few years and I have already covered some of their problems before. Albert had a lot to say today about competition (scathing, scathing!) and this is an area of intense interest for me in terms of research opportunities in the future.

For creation of a framework agreement, contracting authorities can use any procurement procedure included in part 1, thus meaning that they can even use the negotiated procedure assuming there was a benefit in doing so. One of most interesting uses of framework agreements I have seen over the last few years was done by the FiredUP project, where a competitive dialogue was used to create a framework agreement. Abby Semple gets full credit for that work.

Framework agreements can either be single or multi-supplier, with different rules for each type. Rules for the first are contained in paragraph 7 and state that any subsequent contracts must respect the limits set by the framework and that the operator may be consulted to supplement its tender if needed. One of the problems with these rules is that due to the lack of transparency after the institution of a framework it is nay impossible to know if these rules are being complied with.

Multi-supplier rules can be found in paragraph 8 and are divided into three sets: i) without reopening competition; ii) by partly reopening competition; iii) by completely reopening competition.

Contracts can be awarded without reopening competition if the terms and conditions of the framework agreement were detailed enough and in an objective manner in the procurement documents leading to framework agreement. The litmus test is thus: do the suppliers need any extra information to be able to perform the contract and does the contract authority has enough information from them, or more is needed?

Even if the framework agreement sets out all the necessary terms for awarding subsequent contracts, contracting authorities can still reopen competition if such possibility was mentioned in the procurement documents.

Finally, in case the terms governing the framework are not detailed enough, then the contracting authority is entitled to reopen competition within the framework to award subsequent contracts.

Issues with framework agreements

Framework agreements have a number of issues associated with them in my view: duration, market foreclosure, lack of transparency and collusion. I will focus on the first three and redirect you to Albert's excellent tirade on the last.

The first issue I see with framework agreements is their duration. According to Regulation 33, framework agreements can last for 4 years, which seems like a reasonable duration until it is obvious that the limitation is for the framework agreement in itself and not the subsequent contracts. In other words, it is possible for contracts to extend long after the supposed 4 year deadline. And as the UK does not have a 3 year limitation for the starting term of a contract as other Member States do, I have a feeling they can last for long. If they should last for long is another question, my view is that they should not (they should be called every year to reduce market foreclosure, collusion and keep suppliers on their toes).

The second issue is market foreclosure. This is particular important as the use of framework agreements is on the rise in the UK. Framework agreements may have a market foreclosure by restricting access to billions of pounds worth of spend every year to the few suppliers which are admitted to them. Now multiply that for at least four years and the potential effect is quite big. In my view, framework agreements represent the opposite of what procurement rules were originally designed for: transparency and opening the internal market.

The third problem is lack of transparency. Framework agreement proponents argue that they are subject to transparency, procurement rules and principles in their creation stage. They are right, but to a point. The problem is that by then not a single penny has been spent yet. So effectively transparency only applies to a "pre-award stage". Imagine that we apply the same standard to the open procedure: we would get transparency for a PIN notice or a selection stage, but no transparency whatsoever for the award stage. Would we say that the the principle of transparency had been served well then?

Lack of transparency has other pernicious effects. It is pretty much impossible to know what happens on a framework agreement once it is set in place. How are the calloffs done? How often are they done? What has changed from the original documents of the framework agreement? How many contracts have been awarded? How much money was spent via said contracts? I am not arguing that all practice is bad, but the lack of transparency ensures that is impossible to answer those questions and others.

Framework agreements are information black holes: we know they exist, they attract a lot of attention, but nothing ever leaks out.