CEDTAP Assessment Summary

1. INTRODUCTION

The Community Economic Development Technical Assistance Program (CEDTAP) is a four-year pilot project funded by The J.W. McConnell Family Foundation and managed by the Centre for the Study of Training, Investment and Economic Restructuring (CSTIER) in the Faculty of Public Affairs and Management at Carleton University in Ottawa. CEDTAP was established to:

• Strengthen communities
• Strengthen the capacity of organizations providing technical support for CED
• Enhance the legitimacy and effectiveness of the CED sector
• Address equity issues related to women, youth, First Nations, disabled and ethnic communities within the context of CED
• Test a program model that will apply and disseminate innovative technical assistance.

2.PURPOSE

An assessment of the CEDTAP program was carried out after two years in order to:

(a) assist the program to improve its performance, (b) make recommendations on programming directions and priorities for the remainder of the pilot project, (c) identify issues for subsequent phases, and (d) identify initiatives that have potential for application in other settings.

The evaluators were asked to concentrate on results achieved by the program, for the providers, the organizations receiving grants, the field of CED and, where possible, the communities. Efficiency issues related to program management were assessed only in relation to the degree of satisfaction expressed by the grant recipients and providers about working with the CEDTAP program/Secretariat.

3.RESULTS FRAMEWORK

Since this was the first time that the McConnell Family Foundation had worked with an intermediary organization to implement a program, the foundation established a number of learning goals related to the CEDTAP model. These learning goals were taken into account in developing a framework of results expected for the program, against which the actual results could be assessed. The results framework is included in Appendix 1 of the complete report.

4.METHODOLOGY

Information for the assessment was collected through: document review; consultation with the CEDTAP Secretariat and the McConnell Family Foundation; input from approximately 1/3 of CEDTAP’s approved technical assistance providers; and on-site visits/ consultation with representatives of 20 initiatives funded by the CEDTAP program.

5.OVERALL FINDINGS ON RESULTS

5.1 Satisfaction with the program

CBOs generally appreciate the opportunity to get funding for Technical Assistance (TA). Very few other funding programs offer them the flexibility that CEDTAP does in terms of types of activities funded and timing. The majority of CBOs are satisfied with the providers and the assistance given, and with the fact that the provider manages the CEDTAP grant.

The majority of providers are also satisfied with the program. They feel that CEDTAP has enabled them to improve their skills by working in the field and to make valuable connections with providers and other people interested in CED across the country.

5.2 Leveraging

The CBO’s contribution (both cash and in-kind) represents an important investment. In addition, CEDTAP support makes recipient groups more visible and increases their chances to access funding from other sources. Most CBOs indicated some form of leveraging for the CEDTAP grant. Quebec and BC are noteworthy in the number of sources of funding available for CED-related initiatives.

5.3 Characteristics of funded initiatives

There is a fairly broad range among the types of CBOs that received CEDTAP funding. Grants have gone to a greater number of early-stage organizations (groups that are not yet active in CED) than was initially expected. For most of the funded organizations, TA is necessary in order to strengthen their organizations before undertaking CED in the community. However, early-stage organizations require a longer time frame and a greater range of support activities than is available under the program’s current design. CEDTAP’s funding limitations and short-term approach make it difficult to provide adequate support for early-stage organizations. These restrictions also make the program unsuitable for organizations that are in the start-up phase (i.e. groups that are very small or just working out their ideas).

In a number of cases, grants have gone to umbrella organizations that provide consulting services to local groups. When funding goes to intermediary groups rather than the groups that are directly involved in CED, it is difficult to identify and assess results at the local level that derive from CEDTAP support.

Most organizations chose on-site technical assistance rather than other types of interventions. Participants also rated exchanges very highly as a way to learn from others’ experience, clarify their own thinking, and identify what might work in their community.

5.4 Characteristics and role of technical assistance providers

Distribution of projects among TA providers was fairly equal. Of the 30 technical assistance providers approved in the first intake, 27 had worked with at least one CEDTAP-funded project during 1998-99. Five providers worked with 3 projects, two worked with four and one worked with five projects. The concentration of projects for providers in BC and Quebec was much higher than in other provinces. The main factor affecting provider selection in Quebec was the limited number of francophone providers; in BC, knowledge of the local context and funding sources were important criteria.

The skills offered by providers cover a wide range, reflecting the diversity among providers and within the CED sector in general. The providers play an important role in linking CBOs with CEDTAP through activities such as: (a) introducing groups to the program, (b) helping to develop the project and define the nature of the TA required, (c) providing information to help CEDTAP assess the capacity of CBOs applying for funds or establish priorities for a region, and (d) suggesting components to be included in a CEDTAP-funded project.

6.SPECIFIC RESULTS ACHIEVED

Overall, the CEDTAP program has achieved some noteworthy results for participating CBOs and technical assistance providers, as well as for the CED sector in Canada. Since most CBOs were at an early stage and CEDTAP initiatives were relatively small, results at the local level did not always appear to be very significant. However, CBO representatives generally consider that CEDTAP support has made a big difference to specific aspects of their development. Technical assistance providers were also positive in their comments on the benefits of the program, for them personally and for development of the CED sector.

It should be noted, however, that the results achieved to date are far from the long-term objective of economic development and will be affected by a variety of external factors (e.g. general economic climate, access to additional funds). Not all the initiatives will achieve the necessary conditions to produce expected results related to job creation and enterprise development.

Most projects did not include an analysis of gender issues or specific components to address gender equality. Many CBOs indicated that it was not an issue as women already play key roles in the organization.

6.1 Major results achieved for the providers

Capacity Development

• Increased knowledge of CED approach and methodologies.
• Improved capability in CED field – better understanding of development process and projects - improved capacity to facilitate, assess and evaluate.
• Strengthened providers’ businesses by providing income

Partnerships Developed

• Expanded knowledge of other service providers – opportunities created for networking through Providers’ Forum and projects that involved several providers.
• Opened up opportunities that would otherwise have been difficult to get started (e.g. links with a multi-cultural community association in BC).
• Opened new territories for providers who were selected by CBOs outside their natural networks (geographic or cultural).

6.2 Major results achieved for the CBOs

Capacity Development

• Strengthened organizations – interventions helped CBO representatives (Board and staff) to improve their capacity to define goals and directions for the group. In some cases, the project acted as a catalyst for greater community participation in the CBO.
• Increased knowledge of CED and clarification of mission. This result is particularly important for early-stage organizations that had an ‘idea’ but limited knowledge of how to implement it (e.g. where to start, what resources needed etc.).
• Development of strategic management tools that should increase the CBO’s efficiency and capacity to leverage funds (e.g. business plan).

Constituency Strengthened

• Recognition from the members of the community (e.g. participation, in-kind support) and from local or even national groups.
• Basis created for greater community participation in the project or the CBO (e.g. community consultation on tourism strategies).

Leverage Capacity Strengthened

• Identification of new funding sources and strategies (e.g. local, provincial, federal).
• Greater access to funds through development of business plans or overall community strategy for CED (e.g. tourism strategy).

Partnerships Developed

• Exchanges between groups helped them learn from one another’s experience.
• Sharing of materials and approaches. Providers working with more than one organization were more likely to use materials they had developed, and adapt them to the group’s needs.
• New linkages with organizations not funded by CEDTAP that had related experience.
• Requests from other groups for the CBO to facilitate a process similar to the CEDTAP project.

6.3 Major results achieved for the CED sector

Results for the CED sector were anticipated as Outcomes – visible by the end of the CEDTAP program. However, after only two years some results were already apparent because of CEDTAP’s focus on providers as one target group.

Networking

• Increase in the number of providers working together on projects.
• Increased exchange of information and knowledge on CED issues among individuals from different parts of the country as well as among diverse groups (e.g. co-ops, universities, consultants, CBOs/CEDOs).
• Facilitation of active discussion on gender issues through the Gender and CED learning group.

Dissemination of Lessons Learned

• Successful CEDTAP-funded experiences have been promoted on CEDTAP’s website and through Making Waves, a publication disseminated across Canada to CED practitioners.
• The Providers’ Forum enables providers from across the country to exchange experience, discuss issues related to CED, and learn about successful initiatives funded by CEDTAP or other donors.
• The Gender and CED Learning Group investigated electronic means for information-sharing with mixed results.

Visibility

• CEDTAP funding increased the visibility of local initiatives and helped to improve access to other funders especially in BC and Quebec.
• The fact that CEDTAP is a national program increases the visibility of the sector as a whole. Some providers support CEDTAP precisely because it is a pilot project that has the potential to attract other donor’s attention to CED initiatives.
• By providing opportunities for practitioners to get together, the CEDTAP program has hastened the development of the Canadian CED Network. CEDNet organizers have been able to piggyback their activities with CEDTAP events, opening up the possibility for practitioners from across the country to become involved in discussions about the future of the CED sector.
• The existence of both CEDTAP and CEDNet increases the availability of information on community economic development in Canada and makes it easier for new organizations and practitioners to become involved.

7.CONCLUSIONS

7.1 CEDTAP has been successful as a pilot project because it has demonstrated that:

• There is a constituency of local groups interested in working in CED.
• There is a need for technical assistance to support the development of these groups
• Resources are available within the CED sector to provide effective TA.
• A national program is feasible but requires adequate resources.
• CEDTAP’s flexible approach (placing responsibility in the hands of the CBOs) generally works well and is appreciated by the groups.
• Short-term targeted interventions produce results for participating CBOs but it is difficult to assess their long-term effects in relation to CED, especially for early-stage organizations.
• Support to providers has helped to build the CED sector.
• CEDTAP interventions increase an organization’s capacity to leverage other funding. They do not create any financial dependency on the program.
• CEDTAP interventions, in and of themselves, are not sufficient to build the CED sector to a point where local initiatives produce significant economic benefits. The interventions are too small, too short-term and too focused. Organizations that want to seriously pursue CED must be able to access sources of long-term capital funding, in addition to receiving the kind of technical support that CEDTAP has provided (e.g. planning, management, marketing etc).

Factors of success to date include:

• funding for activities that other donors will not fund
• flexibility in project definition to meet local needs
• access to a national roster of CED specialists with varied skills and approach
• good use of regional resources to identify priorities (BC and Quebec)
• good communications at a personal level with CEDTAP Secretariat
• commitment to funding projects in under-served areas and with disadvantaged groups

Limitations/ constraints to date include:

• insufficient resources to effectively implement a national program
• a very broad mandate with diverse target groups
• steep learning curve for Secretariat in a field that includes a broad range of approaches
• communications problems related to providing services in two languages
• limited knowledge of how to use new technologies effectively
• limited resources to make effective use of computer networking

7.2 CEDTAP’s mandate and scope were very ambitious for a pilot project.

The expectation that CEDTAP could demonstrate results in a short time for the three target groups (providers, community organizations and the CED sector) was overly ambitious. Providers have indicated some benefits at a personal level and for the sector (increased knowledge, networking). Benefits for funded CBOs are not necessarily related to CED. Given the large number of early-stage organizations involved in the program, and the fact that many are new to CED, results for the sector will take much longer to materialize (if the groups continue to actively pursue CED). In addition, results for the sector will be difficult to link with a CEDTAP intervention.

The number of communities targeted for interventions was also ambitious. A more limited number might have allowed CEDTAP to test a variety of approaches to program delivery, including providing some organizations with longer-term (i.e. multiple year) support in a series of strategic interventions designed to build their overall capacities.

7.3 As the CEDTAP program is operating now, there appears to be some tension between the goals for providers and those for the CBOs. For example :

• Providers are playing a major role in project definition and financial management. Removing these responsibilities from the CBO is, in the opinion of the evaluators, contradictory to the long-term development of management skills.
• Short-term interventions are better for providers than for development of the CBO. A variety of planned interventions over a longer period of time would be more effective in strengthening the capacity of CBOs, especially early-stage groups.
• The pre-selection of providers is designed to ensure that a CBO receives good quality technical assistance. However, limiting a CBO’s choice of provider means that (1) they may not be able to work with the best person for their needs, (2) they may have limited control over the approach used, (3) activities may be sub-contracted to a different provider, and (4) they may not get the results they were expecting.

7.4 Providers and community groups have been generally satisfied with CEDTAP’s management of the program.

7.5 Completion of the current Phase I without a decision on funding Phase II may be counter-productive to future results.

CEDTAP currently has limited funding for the last 1½ years of the project (10 new projects are projected). To complete Phase I, the Secretariat will have to change the selection process, making it more competitive and restricted. As the number of active projects decreases over the next 1½ years, the number of staff at the Secretariat should also be reduced. This will result in a general decline in services, information etc. CEDTAP runs the risk of losing the momentum created in Phase I. If a second Phase is approved, CEDTAP will have to incur ‘start-up’ costs to re-generate interest in the program. A decision on Phase II at this time would allow CEDTAP to address issues related to the program model and management now, and maintain the changes for Phase II.

8.RECOMMENDATIONS

• The McConnell Family Foundation should consider the time completed as adequate to demonstrate the effectiveness of CEDTAP as a pilot project. Rather than allowing the program to decline over the next 1 ½ years, a decision should be made as soon as possible on funding for Phase II. If approved, funding should be made available to maintain CEDTAP at its current level (at least). Any restructuring/ reorientation of the program should be carried out as soon as possible with a view to maintaining the revised approach during Phase II. Approval of Phase II at this time will allow CEDTAP to maintain its momentum and credibility in the CED community (with CBOs, providers and other donors).
• The evaluators support the decision to place a significant focus on more mature organizations that have CED initiatives underway. However, we also recommend that: (1) CEDTAP should re-invest in promising CBOs that received funding during the pilot phase in order to build on the results achieved by the initial grant, and (2) early-stage organizations should still be considered a legitimate target group if a different approach is taken to funding.
• It is clear that results related to job creation and local economic development are far in the future for many recipient organizations. A revised approach that provided greater funding per project over a longer period of time might prove more effective in promoting CED.
• The low level of organizational development/ capacity demonstrated by most of the recipient CBOs suggests that CEDTAP’s goals at the community level, and its target for project assistance, should be reviewed.
• CEDTAP should take a stronger developmental approach towards building the capacities of CBOs. Groups should be given greater responsibility (and support if necessary) to manage all aspects of their project including selection of the provider, financial management, and documentation of results. Building a broad range of management skills will contribute to an organization’s long-term sustainability.
• CEDTAP should establish a process to review the skills of a proposed provider who is not pre-selected, against publicly available selection criteria. This would enable CEDTAP to maintain control over who is providing TA at the same time as the CBO is given greater responsibility for this key component of the project.
• CEDTAP should establish a process to coordinate the use of multiple providers and should ensure that the expertise provided is complementary.
• CEDTAP should be pro-active in working with other donors to ensure that complementary funding is available for long-term support of CED initiatives.

9.ISSUES FOR CEDTAP’S FUTURE

9.1 Issues related to the model

• National Scope:
  • Adequate resources to cover: (1) additional costs for isolated regions, (2) developing and maintaining electronic networking capacity.
  • Cost-effectiveness of centralized administration versus some form of de-centralized network for promotion, project identification, monitoring and reporting.
  • Clarification of CEDTAP’s role as a national network/resource for providers.
  • Clarification of CEDTAP’s role in relation to CBOs.
• Target Groups and Level of Support:
  • Clarification of priority of target groups – providers and CBOs.
  • Consideration of alternative models of support: greater funding per project/ longer time frame/ fewer initiatives.
• Providers:
  • Extending providers’ list versus improving quality/ expertise of those already selected through mentoring or other approaches.
  • Best way to support provider development – e.g. more frequent regional workshops versus national Providers’ Forum

9.2 Issues related to management

• Program Direction and Management:
  • Participation of providers in project selection, program evaluation and strategic direction.
  • Improving CEDTAP’s capacity for project monitoring and evaluation.
  • CEDTAP’s role in policy development, dissemination, and research in partnership with the providers and CBOs.
  • Promotion of gender and CED.
• Role of Providers in Projects:
  • Provider’s responsibility for monitoring and reporting to CEDTAP.
  • Who manages/ coordinates inputs of multiple providers.
  • Quality control if provider sub-contracts to another person.
• Role of CBOs:
  • Greater flexibility for CBOs to manage key components of project (providers, finances) as part of a learning process.
• Financial Management:
  • Administrative costs for provider.
  • Standardization of fees.
• Communications:
  • Improving communications between Secretariat, CBOs and providers, especially related to project design and monitoring.
  • Management of website and other forms of electronic communication.