"Without adequate evaluation to inform us, we are walking in a fog and don't know what we are missing."
That simple premise underlies much of the professional work and personal obsession of Carolyn Cumpsty Fowler, Ph.D., MPH, assistant professor and evaluation coordinator at Johns Hopkins University School of Nursing. Fowler led a two-part interactive session encouraging attendees to think about evaluation as a type of critical thinking that applies to all stages of any project and to realize how evaluation strategies help create effective connections with other people.
Challenges and dangers
Fowler starts with three big challenges often faced with any project: limited resources, non-strategic choices and lack of collaboration. Fowler says that the most important resources are not financial, and that one of the biggest dangers is a well-funded, well-resourced program; too many resources inhibit creativity and introduce complacency that tends to overlook the need for evaluation as a critical part of any program.
Another danger Fowler notes is non-strategic intervention choices. Something may seem like an amazing opportunity or a good idea at the time, but if it doesn't meet the objectives or help reach the end goal of the program, it isn't a benefit. An evaluation of the opportunity before accepting it will tell you it's not a wise use of resources.
Lack of collaboration Fowler calls "silo-sightedness." People and skills are resources that need to be invested to earn interest, she says. if you keep them in your silo they dont earn a return. Connections should be created early and often. For example, common challenges like turf wars can be avoided with proper pre-evaluation. "People don't get engaged in a turf war unless they care about something," says Fowler. Evaluation in advance can lead to collaboration with another passionate individual or organization instead of later conflict.
These three big challenges are completely interrelated and have a multiplicative effect that can be either positive or negative. If you overcome two of the three challenges, it makes it much easier to overcome the third; if you are struggling at two of the challenges, it makes it hard to maintain success on the third.
Evaluate early and always
"The biggest fraud that has been perpetrated on our community is the idea that outcomes evaluation is what matters," says Fowler. "If you are not evaluating until outcomes, it is often too late to change anything."
Evaluation is needed throughout all phases of a program: planning, implementation and outcome. "Evaluation incorporates formative evaluation--this is my total obsession in life," says Fowler.
Problems will happen with every project -- that will be part of every outcome evaluation -- but evaluation used during planning and implementation increases your ability to influence the outcome.
Evaluation before the program gives you a sense of the terrain and the context you are entering into.
Fowler outlines five "need to know" formative phases: pre-project, start-up, implementation and project modification, maintenance and sustainability, and replication and policy. Each of these stages should be thought about in advance and each can benefit from the thought-work leading up to it.
Key concepts
Concept 1: "The connections you create in the program design phase are critical to achieving outcomes." Strategic connections are "planned, tactical, calculated, deliberate, premeditated, considered intentional associations we form with others."
Concept 2: Before we can create strategic connections, we need to ask a lot of questions. It isn't always obvious what we need to know or why we need to know it. Asking questions is a large part of evaluation and planning -- it helps us understand the communities we are in and the communities we can collaborate with, and this can extend beyond just the obvious circles of relationships and influence.
Fowler reminds attendees that "community is not a geographic place; it is a shared place of being." And a part of resource development is recruiting the right people to share that place. All connections must prioritize relationships, and all connections must be evaluated.
Fowler shares a couple of favorite quotes to help illustrate these key ideas:
- "If the first button of one's coat is wrongly buttoned, all the rest will be crooked," Giordano Bruno (1548-1600).
- "The first step towards getting somewhere is to decide that you are not going to stay where you are," John Pierpont Morgan (1837-1913).
What types of connections are necessary for you to have a successful program? How could evaluation help you to maintain and strengthen your connections?