Anecdotes are Cute, but They Don’t Prove Success
I will never forget the first face-to-face meeting I had with a program officer at a major national grantmaking foundation. It was 2002 and I had flown to New York City with my boss for a three-day foundation meeting spree. I was brand new to the nonprofit sector and the greenest of green in terms of fundraising. My then boss, while very charming, was also not the best fundraiser (or mentor). We both knew why he had invited me to join him, though neither of us said it out loud – I was there to be the brown employee face to help bring legitimacy to the organization’s racial justice work. At the time, I was all too happy to be the token if it meant a free trip to New York and the opportunity to get some real experience in major grant work. We can talk about racial bias and tokenism in the nonprofit sector another time…but today, friends, I’m here to talk with you about handling your program evaluation business so you can prove your success to your board and funders… So, there I was, young and green, sitting in front of the program officer and shaking like a leaf while she peppered me with questions about the work I was doing, helping people with prior felony convictions regain their voting rights. We talked about the conversations I was having with my state’s legislators, testimony I had given in legislative hearings, research I had conducted about what was happening on this issue in other parts of the country. But when it came to answering questions about the impact of my work, all I had was a few stories to tell. I shared stories of individual successes. I talked about the positive feedback I had received facilitating legislative testimony from people impacted by the criminal justice system. I spoke about letters I received from people who had registered or voted for the first time thanks to the work my organization was doing. And on and on like such with the stories. But what I failed to do was show any actual, scalable strategy or any actual, scalable impact. That’s not to say that what I was doing wasn’t making any impact. But if it was, I had not come prepared to prove it. All I had were stories. And at the end of the day, friends, stories are nothing more than anecdotes. Anecdotes are cute. Anecdotes pull heartstrings. Anecdotes can hit the emotions like nobody’s business. But they can never replace the value of cold hard facts when it comes to proving success and impact. If only I had known then what I know now, I would have arrived in New York prepared with all the data I needed to show the value and impact of my work. I would have had statewide voter data from before the passage of the voter restoration law we had advocated for; I would have had output numbers on how many community forums and voter registration drives we had hosted since the law’s passage; I would have had numeric and demographic information on those we had registered to vote; I would have had even more statewide voter data at 6 months post-passage, 12 months, 18 months, and so on, to demonstrate any positive trends resulting from our network of nonprofit partners who were doing similar work around the state. You get my point, friends. But instead, all I had was a bunch of cute stories. Too many nonprofit leaders rely too heavily on anecdotes when talking about their successes. But when it comes to actual data to back up the anecdotes and show real impact, things tend to get sparce. I cannot stress enough the importance of defining and measuring the impact of your programs and services. Real success must be backed up with data. Period. I have written before about the importance of program evaluation. Process evaluations are helpful for understanding a program’s efficiency, service delivery mechanisms, and overall cost-effectiveness. Process evaluations can help you make internal improvements to enhance the delivery of your services, reduce costs and enhance efficiency. But outcome evaluations are the key to proving your impact and success. Outcome evaluations are all about the numbers…and analyzing the numbers over time to track whether and how your work has created the change you intend it to. So, what sorts of data should the nonprofit leader track for a meaningful outcome evaluation? It really depends on the program, the target population, and the goals. Your programmatic goals will dictate what you should be tracking in order to prove your success. Take a close look at your goals and consider all of the different pieces of information you can collect and analyze over time to show the impact of your work. Please don’t misunderstand me, friends. Stories and anecdotes have their value and can play a significant positive role in your community outreach and fundraising. They can even come in handy when you’re talking to a program officer at a major grantmaking organization. Just remember to keep the anecdotes in their proper place – as examples of the great work you do, but never as proof of your success. Nothing can take the place of proper data collection and tracking when it comes to showing your organization’s actual impact. Period.
You Think You’re Making a Difference…But Are You?
Nonprofit leaders and volunteers are passionate, busy people. There is so much to be done and in many understaffed and under-resourced organizations, not enough time or money with which to do it. Clients need to be seen, grants need to be written, board members need to be informed, staff needs to be managed…and the list goes on. But while everyone is busy with the day-to-day work of running the organization and its programs, who is paying attention to how these programs are being managed & implemented? Who is looking at the impact of these programs on the intended client base or the broader community? How does the nonprofit leader know if the services being provided are reaching the intended clientele? How does the board of directors know if the organization’s outputs are translating into the desired outcomes in the strategic plan? You probably know where this conversation is heading. So, before you give me the side-eye and tell me there’s no time left in your day to think about nonprofit program evaluation, just hear me out. I swear, it’s one of the best things you can do for your organization. Everyone in leadership and management should know and understand the importance of conducting regular nonprofit program evaluations. These are periodic reviews that nonprofits use to determine the efficacy of specific programs, the efficiency of fiscal strategies, and/or to answer critical questions about the organization and its work. Nonprofit evaluations are key to understanding how well programs are being run and, equally importantly, what impact an organization’s programs have in relation to their goals and objectives. A lot can be learned from regular program evaluations, and this is why I recommend them for looking at everything from individual fundraising efforts to year-end outcomes. And while there is no single model for conducting a nonprofit program evaluation, there are two main types to keep in mind: Process-Based Evaluations – This type of evaluation is used to measure and analyze program implementation. Process evaluations allow nonprofit leaders to understand and assess program outputs, costs, and efficiency of service delivery. These evaluations look at what and how much is being done, what the program/service costs to deliver, how many clients are being served, and what problems have been encountered during service delivery. Process evaluations allow nonprofit leaders to understand the how and how much of their work so they can make improvements in process and efficiency as needed. Outcomes-Based Evaluations – This type of evaluation is used to measure and analyze program results and effectiveness. Outcome evaluations allow nonprofit leaders to understand the impact of their services or programs on their intended client base or the broader community. If done consistently over time, outcomes-based evaluations can be extremely valuable in the evolution and growth of a nonprofit organization, particularly as it relates to attracting and retaining larger sources of funding. While process evaluations are valuable tools for internal use – particularly for helping organizations become more efficient and effective in their program implementation – outcome evaluations enable organizations to understand their impact and demonstrate their value to the broader community. Regardless of your organization’s size or age, every nonprofit leader should conduct process and outcomes evaluations at least annually as an exercise in good nonprofit management. To do this effectively, evaluation should be built into your strategic and annual plans from the very beginning. Creating your organizational plans with evaluation in mind enables you to identify what data you will need to track to successfully evaluate when the time comes. For example, if your strategic plan calls for expansion of your services into a new geographic market, an evaluation-proactive approach might dictate the collection of zip code data in your client intake process so that the expansion’s success can be analyzed. Nonprofit program evaluation may seem like a daunting task, especially for under-resourced and understaffed nonprofits already operating on shoe-string budgets. But at the end of the day, program evaluation is key to ensuring your organization is functioning efficiently, effectively, and in alignment with its mission, goals, and objectives. Treating program evaluation as part of your overall program creation and implementation cycle is one of the best ways to set your organization up for long-term success, attract more of your target population, and make your nonprofit more attractive to donors and grant funders.