e-Discussion [CLOSED] - Methodology of Indicator 4 of the GPEDC Monitoring Framework

PLEASE NOTE: the below consultation was open from 20 October until 10 December 2015 and the relevant background document can be found here.

Together with the outcomes of consultations held through the WP-STAT and the IATI Steering Committee with their respective memberships, outcomes of this consultation will form the  basis for a finalised proposed methodology for Indicator 4. The proposed final methodology with consolidated outcomes of the consultation will be submitted to the GPEDC’s Monitoring Advisory Group (MAG).  On the basis of MAG’s technical guidance, the finalised methodological approach will be presented for consideration and endorsement at the GPEDC’s Steering Committee meeting, expected to be held in February 2016 (or possibly earlier via online consideration). 

 

The Busan Partnership agreement has at its heart transparency, mutual accountability among partners and accountability to beneficiaries of development co-operation and all stakeholders. As part of this effort, the common standard was adopted for electronic publication of timely, comprehensive and forward-looking information on resources provided through development co-operation, taking into account the statistical reporting of OECD and complementary efforts by the International Aid Transparency Initiative (IATI). To this end, the Global Partnership for Effective Development Co-operation Monitoring Framework’s Indicator 4 (Information on development co-operation is publically available or the “Transparency Indicator”) aims to assess the state of implementation of the common standard by co-operation providers as an important tool to generate evidence-based policy dialogue on the transparency of development co-operation and to facilitate mutual learning and accountability at the country and global levels.


To this end, please find here a paper composed by the UNDP-OECD Joint Support Team (JST) of the GPEDC, which presents the proposed revised methodology for Indicator 4. The methodological approach is based on the OECD/DAC Secretariat’s refined approach for assessing data from the Creditor Reporting System (CRS) and the Forward Spending Survey (FSS); and the IATI Secretariat’s methodology and proposed scoring based on the publishing of statistics on the IATI Dashboard on which a final proposal will be presented at their upcoming Steering Committee meeting in December 2015.

 

To finalise the methodology for Indicator 4 and leverage the most meaningful aspects of each system to assess transparency in a coherent/consolidated manner, the JST is seeking views from relevant stakeholders on the following key questions. While any input on the methodology for assessing transparency is welcome during the consultation process, input is especially sought regarding those unresolved methodological points within the challenges set out below. 


Challenge 1:

The functional focus of the information produced by the three sources of data is different, posing a question regarding the weighting of each source

Data from the CRS is retrospective in nature (past behaviour) and data from the FSS reflects expected behaviour (forward-looking). IATI provides data that is current and forward-looking. The systems have been described as pursuing different purposes, with OECD-sourced data pursing an accountability/statistical reporting objective, while IATI data is consodered to be most useful for ongoing aid management.

The challenge is to what extent the assessments on transparency based on information from these different systems should be aggregated together, presented separetely, or combined to some degree. Should information on each donor be presented separately or as one composite (aggregating all systems together)? Should the information be presented separately or as one OECD classification (combining CRS and FSS) and one IATI classification? For a more technical explanation of the methodology, please click here.          

 

Challenge 2:

OECD-sourced and IATI-sourced transparency measurements include different dimensions of transparency

The discussions in Busan (2012) called for a measurement of transparency on the basis of three dimensions (timeliness, completeness/comprehensiveness, and forward-looking nature) of the reported data, under the assumption that all the data reported was publicly disclosed. Further developments since that time have highlighted the value of including measurements of data accuracy and data use as other key elements of data transparency that could be considered in the future.

Methodological and reporting challenges arise from the fact that the two OECD-sourced reporting systems and the one from IATI differ in terms of the transparency dimensions that they cover. In particular, the OECD-sourced reporting system incorporates an accuracy dimension as part of their transparency dimensions. The issue is then whether or not one should try and have the same dimensions within each system when classifications are formed to allow for more consistent comparison between two groups.  Until all systems are in a position to include a measurement of accuracy, the question is whether it should be included just for CRS/FSS when forming the categorical classification during the current round of monitoring, present available accuracy information separately, or exclude this dimension until future rounds of monitoring when the methodology can be applied more consistently across the systems? 

 

 

Comments

Anna Whitson a UNDP-OECD GPEDC Joint Support Team from United States Fri, December 11,2015

Dear Colleagues,

Many thanks to all who provided inputs into this e-Discussion, which has now closed. Please check back in the coming days for a summary of the conversation and response from the GPEDC UNDP-OECD Joint Support Team.

Best,

GPEDC UNDP-OECD Joint Support Team

Mari Miura from Japan
Fri, December 11,2015

Japan appreciates the GPEDC Joint Support Team and the Monitoring Advisory Group for their efforts to refine the Indicator 4.  Our comments on the issues raised by the JST are below.

 

Challenge 1: The functional focus of the information produced by the three sources of data is different, posing a question regarding the equal weight of each source

 

Japan supports the Alternative C (as indicated in P.10 of the methodology document), which presents the three systems (CRS, FSS, IATI) separately.  Considering that the three systems are technically independent, and the system each provider report to is different, there is no ground for aggregating all or some of results from the systems.  If the GPEDC envisage to future engagement of new providers in the monitoring, particularly the transparency indicator, the methodology should leave sufficient space for future possible incorporation of other approaches to increase transparency of development cooperation providers.  In light of this, the GPEDC should also present providers’ efforts of transparency other than CRS, FSS, and IATI in a narrative form, so that it could take into consideration some qualitative factors of transparency which are not necessarily quantitatively reflected in the three systems.  This could encourage new providers to participate in the monitoring, as well as contribute to knowledge sharing among providers.

 

Challenge 2: OECD-sourced and IATI-sourced transparency measurements include different dimensions of transparency

 

Japan supports to include the accuracy dimension of CRS/FSS from this monitoring round.  There is no doubt about the importance of data accuracy in addition to timeliness, comprehensiveness, and predictability.  DAC CRS and FSS have accumulated practices on methods of data accuracy to this day, therefore, incorporating the accuracy dimension of CRS/FSS in this monitoring round could provide useful inputs to IATI and the GPEDC for the future adaption of the accuracy dimension as transparency indicator.

 

Thanks!

Ellen Kelly a Policy Officer, "Policy Officer, European Commission" from Belgium
Fri, December 11,2015

The European Commission has already posted comments to this consultation some time ago. We remain concerned that the proposed option C is too complex to be easily comprehensible, particularly for those who are not familiar with the technical details of the transparency agenda.  This means it is not likely to be an effective driver of behaviour change in terms of incentivising ambition amongst donors and other publishers.  As the consensus seems to be that option C is the best one, we wonder whether an additional element could be added of broad categories of publishers e.g. Excellent, Good, Average, Poor or something similar.  This could then be used to push for behaviour change (which should be one of the main aims of the GPEDC monitoring process), while retaining the technical detail.

Andrew Ditmanson
Thu, December 10,2015

Comments from the United States on the proposed methodology:

  • Challenge 1: The U.S. support alternative C, presenting all three systems separately with a separate score for each. We believe this is the most transparent option. We agree that this approach will also require an aggregate score for each provider. How each system is weighted in the aggregate score should be articulated as clearly as possible ​during the comment period noted in the fourth bullet below, as well as for the record​.
  • Challenge 2: The U.S. supports alternative B or C. We strongly support including the accuracy dimension, whether through a complementary approach or by accepting the different approaches in each system.
  • On forward-funding, it will be important that non-programmable humanitarian assistance is excluded from the assessment.
  • On next steps, the U.S. expects the proposed methodology and the synthesis of comments the Joint Support Team provides to the Monitoring Advisory Group as well as the report the MAG provides to the GPEDC Steering Committee to be made publicly available ​in a timely way before meetings of the two bodies​, and that the ​methodology  will be open for public comment prior to the February 2016 meeting of the Steering Committee.​
Anna Whitson a UNDP-OECD GPEDC Joint Support Team from United States
Thu, December 10,2015

Dear Colleagues,

Many thanks to those of you who have already provided input into this lively discussion!

Please note that the e-discussion will end tonight (10 December) at 11:59pm EST (GMT -5).

Best,

UNDP-OECD Joint Support Team of the GPEDC

Claire Devineau from France
Thu, December 10,2015

Please find below the comments from the French Treasury (French ministry of economy and finance), in line with the view already expressed in the latest formal WP stat meeting.

We welcome the work in order to improve the monitoring of the transparency indicator.

Regarding the first challenge, we would be in favor of the option C in order to preserve the nature of each system. The transparency assessment should be presented separately.  We think it is not relevant to consider the principle of composite index for the transparency assessment, as long as the IATI system will not be really complementary to the  CRS and FSS system.

Regarding the challenge 2, if the individual assessments of the CRS, FSS and IATI were monitored separately, we think each system could be presented with their own dimensions (option C).

 

Miguel a Accountability and Integrity from United States
Thu, December 10,2015

Comments from the Global Partnership for Social Accountability (World Bank)

Thank you for the invitation to participate in this discussion.

In the case of ISSUE 1, the GPSA believes Option C (data from the 3 system separately) is the best option.  Even though we recognize there are important advantages related to options A and B, we believe that by aggregating the systems we might undermine the quality of the information produced. Option C allows organizations and practitioners to have access to different sources and to consume and customize the information depending on the nature of the question they are trying to answer and taking into consideration the characteristics of each system.   

In the case of ISSUE 2, the GPSA believes “accuracy” is a relevant dimension. Option B offers the possibility of including this dimension but marginal to the discussion, which may overestimate the state of affairs in countries where development aid does not represent an important share of the national budget. Therefore Option C is the one the GPSA supports.

Ann-Christin Solas a Policy Advisor Aid/Development Effectiveness from Germany
Thu, December 10,2015

Please find below the comments from the Federal Ministry for Economic Cooperation and Development (BMZ), Germany:

Germany welcomes the ongoing consultation process on the revised methodology for the transparency indicator and the opportunity given to comment on the proposed methodology. Our comments below are in line with our position expressed during the latest WP-STAT meeting: 
 
  • The revised methodology seems to be an improvement to the former approach as it allows for equal consideration of the complementary systems (CRS, FSS and IATI) and accounts for differences in the nature and functionality of the different data sources. In this regard Germany would favor Option C regarding Challenge 1 i.e. to present the data from the three systems separately. 
  • Referring to Challenge 2 Germany likes to stress the importance of data quality and accuracy for the actual use of data. Germany therefore strongly supports the inclusion of an accuracy/quality dimension in the overall assessment, even when assessments on quality can differ among systems.
Sofie Habram a Sofie Habram, Swedish International Development Cooperation Agency, Sweden from Sweden
Thu, December 10,2015

Sweden's comments for the GPEDC consultation on Indicator 4

 

We think that the best option is the one that is the best for our partner countries. We believe that transparency and access to open data is key in aid effectiveness. In respect to that, we believe that the most important issue must be the relevance and use of data at the country level for aid management. Regardless of how the score is designed for Issue 1, all systems have shortcomings. CRS data is not used for aid management in partner countries and FSS coverage is complete only for partners' priority countries. IATI data, being real-time and multi-stakeholder, has the potential to be an important tool for aid management but tools and training are needed in order for it to be used by partner country governments. As there is no consistent method for aggregating the three systems presented, we prefer Option C, but ask if there are not any other options that would be consistent with the Busan commitment on a ‘common standard’ for reporting on resources provided through development co-operation.

 

With regard to Issue 2 we believe that accuracy can never be a bad thing as long as you are measuring data quality. We should not underestimate the value of reliable and accurate data and this could be reported in the same index or separately (alternatives B or C).

 

We believe that our partner countries would benefit most from keeping their present AIMS where they seek contributions from FSS, local embassies and representations but the AIMS should be constructed or modified so that they can benefit from IATI data.

Farida Bena from France
Wed, December 09,2015

Comments posted on behalf of the CSO Partnership for Development Effectiveness (CPDE)

 

CPDE values GPEDC’s efforts to finalise the transparency indicator by consulting its members. At the same time, we are deeply concerned about the delays and challenges in completing this work, a full four years after the Busan Partnership Agreement was agreed. With the monitoring process for the Sustainable Development Goals (SDGs) being established in 2016, it is of utmost urgency that this matter be resolved in order to ensure the complementarity and utility of GPEDC monitoring. We also wish to recall the Busan commitment to implement a common, open standard for the electronic publication of timely, comprehensive and forward-looking information on development co-operation resources by December 2015”.[1](BPA, 23c). Providers should do whatever they can to honour this commitment.

 

Concerning the first challenge, CPDE recognises there are fundamental differences between the OECD and IATI classification, which would be challenging to capture in one composite score during this monitoring round. A couple of CPDE members have already laid out options to calculate this score should this approach be followed. Alternatively, CPDE recommends presenting information on each donor separately through two different scores, with an explanation of what each score represents. Similarly, OECD and IATI classification could be presented as two distinct sets of information, with a similar explanation of why it is not feasible to merge the two classifications at this point. It is essential that the final scores and corresponding calculation of the monitoring process be open, transparent and publicly available.

 

In addition, CPDE strongly recommends that the second Progress Report clearly summarise the process undertaken by OECD and IATI to agree a common, open standard and highlights the key future steps that will be necessary to make progress on this agenda.

 

Concerning the second challenge, CPDE appreciates the importance of accuracy in assessing the quality of data provided on development co-operation resources. We strongly recommend that both IATI and CRS/FSS develop robust and transparent methodological approaches to assessing accuracy or data quality in consultation with development stakeholders. This assessment should be incorporated into future rounds of monitoring. The 2016 Progress Report should give priority to the key components of timeliness, comprehensiveness and forward-looking nature of the information provided in presenting the data, in accordance with article 23 c) of the Busan Partnership Agreement.



[1] BPA article 23c) 

Fahmeeda Wahab a Policy Analyst from Canada
Wed, December 09,2015

 

Canada’s comments for the GPEDC consultation on Indicator 4

[ CFO-STATS (SWS) input ]

 

General Comment: We find the current proposal represents a sound effort to qualify donors’ efforts against the Busan transparency objectives.  We especially, welcome the recognition that the systems which make up the common standard fulfill a different, but complementary purpose.

 

Challenge 1: To what extent the ‘transparency assessments’ generated by these 3 systems should be aggregated together, presented separately, or combined to some degree? Same weight? 

While we recognize that a single score is attractive in terms of communicating and ranking donor performance, we believe that mixing or aggregating the assessment scores will over-simplify the evaluation against each component of the standard (FSS/IATI/CRS), and there is a risk aggregation would provide a less accurate score, potentially creating more confusion.

For the reasons above, we have a strong preference for Alternative C – which would present all 3 systems separately, without grouping donors according to their system choices.  This presentation is clear and transparent, because it clearly shows the systems to which individual donors report, without losing the details of each assessment. 

We would caution against arbitrarily weighing a system over another.  In our view, the weighing is subjective, and depends on the end-user’s point of view.  For example, a fiscal planner in a partner country’s Finance Ministry could assign more weight to FSS as more important, because FSS provides a macro, long-term overview of public inflows.  On the other, a development project manager may be more interested in finding specific project’s disbursement schedule (IATI).  An audit officer may be equally interested in checking plans against actuals (completeness - FSS/IATI/CRS). The weighing ultimately depends on the eye of beholder, and it would be important to remain as neutral as possible in the monitoring round. As long as all the relevant facts are presented, our view is that readers will be able to draw their own conclusions on individual donors’ level of effort vis-à-vis the Busan objectives. 

 

Challenge 2: Shall we include the new ‘accuracy dimension’? How shall we report on ‘accuracy’?

As others have already pointed out, the agreement in Busan did not contain an “accuracy” dimension, and so it would be difficult to justify its inclusion in this monitoring round.  However, given that transparency without accuracy would not serve the intended purpose of Busan, and given that data providers have had the opportunity to ramp up and improve reporting under all systems, we encourage an accuracy dimension in subsequent monitoring rounds.    We therefore, have a preference for Option B - reporting of accuracy dimension separately, if available, and preferably in an annex.

Regarding additional dimensions

We would oppose the inclusion of Public Disclosure and Use as stand-alone dimensions. In our view, Public Disclosure is only relevant to FSS and can be integrated into the assessment.

As for Use, it’s an entirely different issue that has nothing to do with donor behaviour. What may be more important to highlight, perhaps in future assessment, is how the common standards and associated systems are able to take into consideration partner countries’ requirements.  This would align well with Busan 23c: “this standard must meet the information needs of developing countries and non-state actors, consistent with national requirements”.    The current proposal on the Budget ID at the WP-Stat is good example how donors and data standards can work together and take into consideration partners’ needs. 

 

Edite Singens
Wed, December 09,2015

Portugal's comments on Methodology of Indicator 4 of the GPEDC Monitoring Framework : 

Portugal welcomes the work in order to fine-tuned the monitoring of the transparency indicator. On the questions raised we have the following position already expressed in the former WPSTAT meeting:

 

Portugal supports Alternative C , a presentation in parallel of the three systems, a position in line with the WPSTAT Secretariat ´s assessment.

Portugal strongly supports the Accuracy dimension to be integrated in the assessment. As already previously expressed an “assessment of quality” must be core in any assessment on transparency. 

09 December 2015

Attachment(s) Portugal comments Indicator 4_2nd monitoring round09_12_2015 .pdf
Emily BOSCH a Head of DAG Secretariat and Policy Specialist, UNDP from Ethiopia
Tue, December 01,2015

Thanks for the opportunity to comment on this discussion. As this indicator is measured at the global level, it may be strange to weigh in at the country-level with even more questions! However, I do believe that the ultimate goal of all these systems apart from accountability to different constituencies in the north should also be ease of use of data at the country-level for aid management. In that respect, regardless of how the score is fashioned for Challenge 1, all systems have shortcomings: CRS data is at "outdated"; FSS coverage is complete only for partners' priority countries; and IATI data, although "real-time" really has yet to be used by partner country governments to manage aid at the country-level. In Ethiopia, Government's annual bulletin on development finance includes only data from the aid management platform, which is based solely on disbursement information entered by development partners into a database managed by the Ministry of Finance. Similarly, data from the aid management platform complements CRS data when we compile partner profiles or look at division of labor, joint programming, etc.

Should a provider of development assistance that reports to CRS, FSS and IATI score higher? Will a composite number on transparency rather than 3 separate scores make a difference? Why not just indicate who reports to each of the three (CRS, FSS, IATI)?  It cannot hurt to publish data in many different forms, but whether it really make a difference in terms of the actual use of the data is not always obvious – this is the part of the equation that is often left out of the discussions on transparency since Busan. It may well be that this indicator really is just another mechanism for ranking amongst peers with the assumption that the more that is in the public domain, the more potential there is for use of the data somewhere by someone. If that is the case, however, we should just state this rather than pretend that all stakeholders (north and south) are equally implicated and affected by a transparency indicator that may have little to do with the true disclosure of aid commitments and disbursements to Governments at the country-level.

With regard to Challenge 2, as long as you’re measuring data quality, I would not underestimate the value of having reliable and accurate data, whether this is reported in the same index or separately (alternatives B or C). There is no shortage of unverified data available everywhere, so why not at least try to provide some certification that the data is as accurate as possible?

As long as transparency at the country-level is not explicitly part of the equation, partner countries understandably have less say on this indicator. Ultimately, as previous contributors have mentioned, transparency is political and perhaps all the more so when negotiations are taking place between a Government and provider of development assistance on future programming.

Bill Anderson
Tue, December 01,2015

Emily, UNDP has reported $35m 2014 spend for Ethiopia through IATI. For 2015 to date it has reported $30m budget and $19m spend. Has UNDP provided this data to the aid management platform? In total almost $2 billion  of 2015 spend to Ethiopia is currently available via IATI. Isn't it time that misconceptions about the lack of useful IATI data are put to bed?

DELALANDE Guillaume a Policy Analyst from France
Tue, November 24,2015

Dear all,

please find below the link to the paper presenting the "WP-STAT's Secretariat's view on the GPEDC methodology for monitoring the Busan commitment on transparency":

http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=DCD/DAC/STAT/RD(2015)5/RD2&docLanguage=En

Best regards,

Anna Whitson a Masters Student from United States
Mon, November 23,2015

Dear Colleagues,

Due to popular demand, please be advised that we have extended this consultation until 10 December 2015 at 11:59pm EST (UTC -5).

Many thanks to all those who have provided input thus far and we look forward to hearing from many others!

Best,

The GPEDC UNDP-OECD Joint Support Team

Liz Steele a EU Representative Publish What You Fund from United Kingdom
Sun, November 22,2015

Introduction

Publish What You Fund and Transparency International welcome the opportunity to provide comments and recommendations to strengthen the methodology of the transparency indicator for the GPEDC’s second round of monitoring. We strongly agree that the development community should hold itself to account for the commitments made at Busan and the transparency indicator is a crucial part of this peer review process and the Global Partnership’s monitoring framework. In its annual Aid Transparency Index[1] Publish What You Fund has successfully developed an independent and alternative methodology for monitoring donor commitments. We believe this has been a significant factor in driving progress in this sector.

Therefore we are deeply concerned that basic methodological issues around the development of the indicator still remain unresolved, a full four years after the Busan Partnership Agreement was agreed. With the monitoring process for the Sustainable Development Goals (SDGs) being established this coming year, it is of utmost urgency that this matter is resolved in order to ensure their complementarity and utility. Currently the value of the GPEDC monitoring exercise to measure progress and incentivize change is under question, in particular given that donors committed to “implement a common, open standard for the electronic publication of timely, comprehensive and forward-looking information on resources provided through development cooperation…with the aim of implementing it fully by December 2015”.[2] The results of the 2015 Aid Transparency Reviews[3] show that many providers are currently off-track to meet this commitment. They also show that progress is possible in a short period of time, where the political will exists.

We urge donors to take every step to make their development cooperation transparent by the end of this year.We also urge the GPEDC to ensure that existing commitments on transparency are broadened and strengthened at the 2016 High Level Meeting to meet the ambition of the SDGs, with a particular focus on data quality and its use at country level. As part of this process, new indicators will need to be developed.

The current methodological proposal for the transparency indicator is an attempt to provide a technical solution to an inherently political problem. If traditional international information provision systems such as the OECD-DAC’s traditional Creditor Reporting System (CRS) and Forward Spending Survey (FSS) had already been providing sufficient transparency, there would have been no need for new international commitments in 2011. While they have various benefits, the CRS and FSS were never designed to meet the needs of multiple stakeholders for timely, comprehensive and forward-looking information. This is why the international community made ambitious commitments at Busan to improve the transparency of their development cooperation and why the International Aid Transparency Initiative (IATI) Standard was developed.

Looking to the future, we need to ensure that a robust indicator is selected which is relevant to the implementation of the Sustainable Development Goals (SDGs) and financing for development framework; one which measures financing flows beyond aid and which recognizes the role that is being played by non-traditional providers of development finance within the broader context of the 2030 sustainable development agenda.

 

Remaining true to Busan’s intentions

The indicator’s methodology therefore needs to reflect the intentions behind the Busan commitment’s fundamental principles of timeliness, comprehensiveness and the forward-looking nature of information.

At country level, partner countries have consistently made it clear that they require “comprehensive, timely, reliable and useful information on aid commitments, disbursements and actual use flows in useful formats and in an accessible manner[4] in order to support the budget planning process. The IATI open data standard is currently the only viable mechanism for meeting these commitments because it is:

  •  Timely: Providers of development cooperation are encouraged to publish and refresh their data monthly and at least quarterly.
  • Comprehensive: The IATI Standard enables the publication of detailed information at the project and transaction level. There is scope for full traceability of financing flows, matching published data with what is received at country level, and with impacts, outcomes, outputs, and results. There is also the option to link information to recipient country budget classifications.
  • Forward-looking: Information published to the IATI Standard should be current activity-level data and for the next 2 years in advance.
  • Open and accessible: IATI is an open data standard and effectively responds to the UN’s call for data revolution. IATI data is comparable, shareable, re-usable, accessible and timely through the IATI Registry. All IATI data is published to an open data licence. This means that people can re-use the data if they credit whoever is publishing it (“attribution”).

Table 1 (below) shows a comparison of the three systems and how they match partner countries’ stated requirements.

 

Principle

Partner country requirements[5]

IATI

CRS/FSS

Timeliness

Data should be no more than three months old and updated quarterly.

Yes. Data can be updated and refreshed as often as publishers wish, ideally monthly and at least quarterly.

No. CRS data is between 11 and 24 months out of date and only released once per year.

Comprehensiveness

Detailed project-level information on budgets, actual disbursements, results and sub-national geographic information

Yes. Provides detailed information at the project and transaction level, with many more information fields than CRS. There is scope for full traceability of aid, matching published data with what is received at country level, and with impacts, outcomes, outputs, and results. There is also the option to link information to recipient budget classifications.

No. Provides high-level information on projects, often aggregated to programme level rather than disaggregating by each subcomponent, or each sector or recipient country. Does not provide a way of matching donor spending with what is received in country or with results.

Forward-looking nature

Annual project budgets three years ahead

Yes, detailed indicative forward-looking budgets - as much as publishers are able to provide. Information should be current and at least 2 years in advance.

No. Should provide information up to 3 years in advance, however underlying data from this survey is not always accessible and is not sufficiently detailed.

 

Remaining relevant to the 2030 agenda

The methodological approach should ensure that the indicator remains relevant within the 2030 sustainable development agenda. The Addis Ababa Action Agenda[6] recognises the need for high-quality disaggregated data for smart and transparent decision-making[7]. It also highlights the need for transparency of budgets, aid, tax, and extractives and the potential of open and joined-up data standards in achieving, monitoring and reviewing the SDGs at sub-national, national and international levels.

Being able to readily compare machine-readable information across different financing flows will be a pre-requisite for better predictability and greater accountability in the context of the 2030 agenda. The IATI Standard currently captures different financing flows from development banks, climate funds, the private sector and civil-society organisations[8] and could easily be adapted and extended to cover more. It offers an ideal basis for developing a fully comprehensive standard for publication of information on all development finance flows and is well-placed to support countries in monitoring and implementing coherent and integrated national financing frameworks.

Open data and open data standards such as IATI will be central to implementing the global goals and to monitoring commitments on financing development. If the GPEDC wishes to play a meaningful role as part of the 2030 framework this is where it should be focusing its future energy and resources.

 Recommendations for this consultation

For the transparency indicator methodology to be useful, the results of the monitoring process need to be usable by its inclusive multi-stakeholder community. The final scores should enable donors to see how well they are meeting their existing commitments and what more they need to do, and for partner countries, civil society organisations and other accountability stakeholders to hold them to account. This means that the results from the monitoring process should be:

  • Comparable across aid providers and not split into different categories;
  • Simple to understand and communicate;
  • Open and transparent to the public.

 For these criteria to be fulfilled we recommend that:

  1.  Each system (FSS, CRS, IATI) is responsible for determining its own methodological approach and determining how good quality information is measured. The results from the systems should be presented separately as three separately numerical scores and then aggregated to offer a single composite result. This means that scores from different publishers can easily compared.
  2. As detailed in the table above, more updated, disaggregated and forward looking information is being made available in the IATI system. This should be reflected in the weighting system to calculate the composite result in order to ensure consistency with the Busan principles and to reward these publishers. FSS data (when publicly available) could be added as a smaller proportion of the final result in recognition of its effort to fulfill the forward-looking dimension of the Busan commitment. Finally, the last proportion of the aggregate score could take into account OECD DAC CRS data. In practice, this may mean receiving higher scores for IATI publication than CRS reporting. But there is no reason why IATI and CRS should be scored the same if one system meets the Busan principles and another system does not.
  3. A blanket “accuracy” dimension should not be included in the indicator methodology in the GPEDC’s second monitoring round as this is not one of the Busan principles. However we strongly recommend that for future monitoring all systems develop a robust and transparent methodological approach to measuring the quality of the data being published/reported in an open and inclusive manner and as part of their own processes. This will enable the IATI community to define and agree a measurement of good quality data. It also means that the OECD-DAC and DAC donors are able to include an “accuracy” dimension for the FSS and CRS without imposing a one-size-fits-all approach.
  4. Final scores and the corresponding calculations of the monitoring process must be open and transparent and publicly available for others to examine and re-use.

 

 



[1] See http://ati.publishwhatyoufund.org

[2] BPA article 23c)

[3] See http://roadto2015.org/progress

[4] See CABRI position on aid transparency:  http://www.cabri-sbo.org/resources/publications/policy-briefs/109-policy-brief-cabri-position-on-aid-transparency

[5] See 2014 IATI Country Survey: http://www.aidtransparency.net/wp-content/uploads/2013/01/Paper-4a-Country-Survey-of-AIMS.pdf

[6] See http://www.un.org/esa/ffd/wp-content/uploads/2015/08/AAAA_Outcome.pdf

[7] AAAA Para. 125

[8] See the IATI dashboard for progression over time http://dashboard.iatistandard.org/publishers.html

Ellen Kelly from Belgium
Tue, November 10,2015

Comments from the European Commission on the proposed revised transparency indicator methodology:

  • In general, the methodology proposed seems to be an improvement on the previous version.  It attempts to balance the task of assessing different systems whilst combining them.  However, there is a risk that the results will still be too complex for users to understand.  The end product should aim to be as easily understood as possible, by a range of end users.

  • It is welcome and important that the most recent data published to each system will be assessed, rather than data from the same years.  IATI data publication has changed significantly over the past few years, so current data will be very different to that from two years ago.

  • Separation of three systems: it would make sense for the overall assessment to give a higher “score” / better result to those publishers who publish data to all three systems, rather than just one or two.  This is implied, but not explicitly stated how it will be achieved.

  • Coverage.  At the moment, it seems as if publishers to just one system e.g. IATI only will benefit from this situation, as their score will not be “adjusted” down to reflect the coverage ratio.  This needs to be considered.  A methodology to calculate the coverage rate of non-DAC donors, multilaterals, NGOs etc is mentioned.  This is vital, if other donors reporting to all systems are not to be inadvertently penalised.

  • Coverage 2: If data from different years is used from CRS & IATI, how will the coverage ratio be calculated?  Challenge 1: We would favour Alternative B: Presenting an aggregate CRS / FSS classification along with an IATI classification.  This will keep the two data sources separate, whilst remaining relatively uncomplicated.  Alternative C, of presenting all 3 systems separately, would also be a possibility but runs the risk of being too complicated for stakeholders to understand.

  • Challenge 2: We would favour Alternative C: Including the accuracy dimension, even if this is assessed slightly differently in the two different systems. 

  • Divergence in scoring method.  Does the proposed categorical scale cover all three reporting systems, and as such it is an overall rating?  This would be useful, to add some simplicity into the proposed approach.

  • Year of reference for reported data.  Baseline year – what will the baseline year be used for this (it is currently unclear)?

  • Coverage of actors.  Will all IATI non-government publishers be assessed as well as governmental ones?

  • IATI data.  Forward-looking nature.  Planned disbursements should also be considered as forward-looking data at activity level in IATI datasets.

Bill Anderson a Technical Lead, IATI Secretariat from United Kingdom
Fri, October 30,2015

For the information of IATI Steering Committee members:

The IATI Secretariat will share its proposals for a revised methodology (as reflected in the JST document) next week (w/c 2 Nov) for consultation and refinement during November, and for discussion and approval at the IATI Steering Committee meeting in Copenhagen on 2-3 December 2015.

This will be a public consultation and I will post details here next week.

Bill Anderson
Wed, November 11,2015

The IATI Dashboard now contains the proposed Indicator Scoring and underlying statistics. 

We hope that in the next three weeks of consultation and fine tuning a consensus can be reached that will allow the Steering Committee to formally approve a methodology during the December meeting that is acceptable to all.

Comments and discussions regarding the methodology are taking place publicly on the IATI Discuss site or privately via email to consult@iatistandard.org. We will also be arranging a number of calls for consultation, and these will be announced on the same site.

Bill Anderson
Thu, November 12,2015

The IATI consultation is maintaining a running list of outstanding  issues and recommendations here

Type forum
Date Created Tue, October 20,2015
Created By Anna Whitson
Original Space Global Partnership for Effective Development Cooperation Community
Cross posted in Global Partnership for Effective Development Cooperation Community
# of Teamworks Views 1483
# of Teamworks Recommendations 0
Visibility Public
Domain of origin Teamworks