Learning to See Clearly

Jamie Clearfield's Musings on Evaluation and Development


3 Comments

Thoughts from Ghana: And it is Good

Excited for Data Visualization Training!

Excited for Data Visualization Training!

Growing up I loved history and social studies. Coming from a family of history buffs (my grandfather has stacks and stacks of military history books in his house and my father once penned an unpublished murder mystery set during the construction of the Brooklyn Bridge), I loved learning about and imaging the past. Today I still love nothing more than roaming old ruins or cities, imaging the day to day life – the clothes, shouts, colors of a by-gone era. Thanks to an amazing AP US History teacher in high school, I fell in love with the Guilded Age and Progressive Era following it – especially those fierce muckrakers who took pen to paper to uncover and publicize society’s injustices. Perhaps having always been bookish and quieter, I can more easily imagine and identify with those muckrakers of old – many of whom never claimed fame or fortune, but used what talent they had to work for the common good.

These muckrakers come to mind as I work on different projects here in Ghana. My evaluation experience has given the skills and training to look critically at programs, my past experiences internationally and my mentorship with great leaders and thinkers like Abdul Kassim at KGSA and Karen Boatman at Boston University have helped me to learn to analyze and break things down. My belief that through working and learning together, hard work, a lot of experimentation, and hope has also helped me to believe that we can build things up, we can create positive change, even if it is a long time coming. ..

A challenge for evaluators, both internal and external, is how much of a muckraker to be – how honest an answer is your client or organization looking for? And the self-doubt is there, as is the knowledge that there is no sure way forward, everything is an experiment of sorts. Is it better for an organization to pour money into cataract outreach if no one is tracking complication rates? Would it be better to expend more to train surgeons or screening personnel, or invest more heavily in monitoring and evaluation? Is anything better than nothing, even if the “anything” may bring its own set of challenges?

Waiting for cataract surgery

Waiting for cataract surgery

There is no easy answer, perhaps there is no answer at all. There are small breakthroughs – demonstrating how visualizing data differently can help with decision making, doing small capacity-building trainings in Excel, touting Anne Emery’s blog (and hoping the wifi here is connected soon so people can access it!). So I continue to struggle if I be a muckracker or a simple blogger, the sun shines bright here in Kumasi, the children shout “Obruni, Obruni how are you?” as we walk to and fro from the hospital, and there is the feeling of the rhythm and flow and feeling that there is work here and it is good…


Leave a comment

Thoughts from Ghana: Living on the Edge

It’s been awhile since I’ve written – we’ve shifted two countries since my last post. We spent about 3.5 weeks in Madurai, India with the Aravind Eye Center – an amazing place of efficiency and compassion and the largest eye center in the world. I was observing with their training and outreach divisions in preparation for our jump over to Africa. As of last Monday, we are Kumasi, Ghana at the Komfo Anokye Teaching Hospital (KATH) Eye Center…

We are always on the periphery. Living so transiently, it is hard to do anything but skim the surface. This is especially the case working and often living in a hospital setting as we did in Nepal and India. Kathmandu

Observing at a Children's Vision Screening Camp with Aravind

Observing at a Children’s Vision Screening Camp with Aravind

allowed us to live closer to reality. We lived about a mile or so from the hospital and given our love of walking, the safety of the city, and its amazing sites, we learned the narrow alleyways and side streets. During our 4 weeks there, I loved the feeling of the city map connecting in my head, the chaos forming into logic. However in both far western Nepal and in Madurai, India where we spent time at the Aravind Eye Center, we lived within the hospital campus. A literal and figurative wall separated us from daily life with only all too brief chance to voyeur out and glimpse life.

We are learning to live on the periphery and negotiate this challenging precipice. It is far too easy to convince yourself that you belong in one realm over the other, to forget that you are straddling worlds, cultures, beliefs, values, to forget that only those that you claim as your own are real in that moment. It is far too easy to forget yourself and your role on the edge, to let yourself be convinced of your insignificance and to be lulled into a sense of false purpose holed up in your little world. I found this in Far Western Nepal. There I distanced myself even further from the world convincing myself of the “need” to complete one project, only to find myself too distracted, discontented and full of self-doubt to finish anything. Instead of throwing myself into my present surroundings, drinking in all I could, I hid myself, holed up with my computer and convinced myself this was productive. While the beauty of the place will stay with me, those memories fade overtime as do pictures. Regret will linger on. Engagement, friendship, and knowing that your weariness at the end of the day comes from an earnest attempt to connect –those feelings drive you forward and bring reality closer.

My spouse now struggles here in Ghana, with similar feelings of underutilization and a sense of uselessness. It is compounded by the extreme need we see around us and the lack of efficient systems in place to allow him to practice and work effectively as a clinician. We are learning how to support each other, to balance the ups and downs of our two unique souls within the space of our shared guest house room, and our shared experiences. Ghana feels more familiar to me, having worked so long in East Africa – other former British colonies. Despite the separation of thousands of miles, there are similarities of history and experience despite the vastly different cultures. We are too much on the periphery here, too separated from the local culture and people by circumstances and expectations (both ours and the teaching hospital where we are working). We stay at a guesthouse run by the hospital for its visiting doctors and nurses, and are currently living with 3 other Americans.

And some things don't make sense...A brand new desk in the eye center...

And some things don’t make sense…A brand new desk in the eye center…

The hospital wants us to a car and driver they provide to walk the mile to and from the guesthouse. This we have politely declined, and use the time to walk and observe, though it is too short a walk to truly engage. This we are unsure of how best to do, and so far have been too exhausted by our attempts to process our daily work to do much of anything except head back and forth to the eye clinic.

Being an evaluator internationally is challenging since by definition I am to be the outsider yet offer recommendations and advice… there is so much weariness though in never understanding. Not knowing cultural difference places me as the ultimate outsider, often too ignorant to appreciate the complexities at work, and thus my recommendations I fear are ill-advised or laughable. There are also many things that simply just do not make sense. Here in Ghana I am focused much more on operational systems and helping the hospital use data in its decision making processes. A lot of time is spent crunching numbers, a lot of time observing patient flow, and also time spent waiting for someone. I sit writing this post now lying in wait for the eye clinic director in his clinic room so we can finalize a proposal. He has been in and seen several patients in my presence and left again, “We’ll meet in 5 minutes…” he says rushing out the door some 10 minutes ago…

Every experience builds on another, and to have courage to learn from past mistakes is a major focus now. Someone told us recently, “Manage expectations- that is the key to living here.” It is true for so much of life and work, both domestically and internationally. We learned so much from our time in Nepal and India, perhaps most importantly to be filled with compassion, for those around us and also with ourselves. It is so easy to forget, to look away, to be focused on the project instead of the life around it. Ghana is shaking us up and reminding us how much on the edge we are, and as we struggle forward to find our footing, we can only try to our best to follow a mentor’s advice to me, “Go out there and do some good…”


1 Comment

Nepal Thoughts from the Field: Trying It on for Size

Life is a mix of things here in Nepal – overwhelming beauty, grace, poverty, hope, things known and unknown. After three weeks in Kathmandu, Z and I are currently in far western Nepal based at a rural eye hospital. Kathmandu was everything a developing capital city coming off a decade long civil war could be expected to be – a mix of chaos and blaring horns with temples and shrines on nearly every street corner, millennium-old Buddhist stuppas and glossy new malls beckoning the emerging middle class. Tradition and culture continually struggling against rapid urbanization and globalization; we wondered a lot what Kathmandu will look like in 10 years time.

A stuppa we stumbled upon when we got lost coming home from work...

A stuppa we stumbled upon when we got lost coming home from work…

In Kathmandu, We were attached to Tilganga Eye Hospital – a renown eye care center that provides efficient and effective eye care services in Nepal. Their model is similar to the Aravind Eye Care System has perfected over the past thirty years – a high volume cost recovery model where everyone receives the same quality of eye care, though those who cannot pay are not turned away. Instead those who are able to pay or who wish to have certain amenities (such as a private hospital room, an appointment time rather than walk-in service, after-hours service, etc.) can do so. I could on and on regarding the Aravind model, but I suggest for those interested, to read the case studies and other reports others, much more eloquent than I, have already written.

While my spouse was attached to various specialty clinics and the operating room to learn new a different cataract removal technique widely used in the developing world, I was attached to the hospital’s academic and training department to review their training programs for international ophthalmologists and allied personnel. A major focus for the hospital is developing and broadening the skills and knowledge of medical personnel, both in Nepal and internationally. Doctors, nurses, technicians, biomedical equipment engineers, and more from Ethiopia, Ghana, Bhutan, Indonesia, North Korea, East Timor, Vietnam, Cambodia as well as Australia and the United States have come to Tilganga for a wide variety of training and surgical exposure. Managing all the different logistics and visa difficulties of getting these personnel into Nepal is an incredible feat in of itself. Training so many different people, personalities – each with their own expectations and experiences – is another accomplishment all together. Nearly 150 ophthalmologists from over 20 countries have received specialized cataract training since 2007, in addition to over 70 nurses and scores of other paramedical and nonclinical staff.

Despite the efficiencies already in place to keep the hospital running and solvent, little evaluation work has been conducted with the training programs. While the hospital has a monitoring and evaluation office (currently a department of one!), it is mainly focused on the clinical aspect of the hospital, and ensuring that the numbers are there for the monthly reports to donors. This is not something new or even surprising – as we all know, educational evaluations are involved and are not straight forward. Determining what metrics to use, how to collect them, and how to determine the impact of education is an ongoing challenge as we see from the continuing education testing saga in the United States. To date, the training programs at Tilganga have participants complete a feedback form. The form is reviewed briefly as the person leaves. It is then filed in a 2-ring binder labeled with their country of origin; the binder than sits on a shelf.

In speaking with the training officer, there has yet to be a longitudinal review of the feedback forms, or any type of major follow-up of participants once they return to their home countries beyond a few emails with their sponsoring agency. I had found my job… even if it did mean digitizing 6 years worth of old feedback forms…

Back to the world of paper evaluations...

Back to the world of paper evaluations…

Currently, in the jungles of far western Nepal, I am reviewing and making sense of the data. The  hospital also asked me to help put together some of the training departments strategic planning  notes, for which the feedback forms will be helpful as well. The data proves interesting, the days  here are slow – cups of tea and a backup generator allow for the work to go on…

Coming in to Tilganga has been an educational and if truth be told, awkward experience. For years  working abroad, I chose positions where I would be present for a significant period of time, be able  to stay and live in the community to build relationships and trust. This type of work and  evaluation smacks a bit of everything I learned “not to do”. Not fitting into any label that the  hospital is used to giving to a visiting foreigner (Doctor, medical student, journalist…) – my  presence at Tilganga was stilted though over the weeks we developed a good rapport of sorts.  Reading the strategic planning notes and trying to make heads or tails of meeting minutes from a  workshop this past spring is like reading a book with the pages all mixed up; a not uncommon  challenge of evaluation and I suspect external evaluators globally. It was exciting to sit with Ravi,  the M&E officer and pass ideas back and forth. He loved getting links to EvalCentral, AEA, and other  evaluation blogs and resources (though I’m not sure how well Chris Lysy’s incredible evaluation  humor translates!). And yet… seeing so many people in need of immediate tangible care, while I sit  at my computer day after day, it is easy to be swayed that there is perhaps a better use for my  time here. My previous roles internationally were much more varied and engaged on the ground–   teaching, developing and giving workshops, implementing, as well as conducting monitoring and evaluation.  Having chosen one                     label is a new and interesting challenge, one I am currently unsure of but am trying on for size…

Thoughts and comments as always are welcome! Feel free to shoot questions about Nepal and our work overseas my way, I’ll answer the best I can.

Himalayas peaking through the monsoon clouds...

Himalayas peaking through the monsoon clouds…


1 Comment

Evaluation Transitions: Nepal Bound…

transition
I apologize for the silence on the blogosphere…it has been a hectic few months. And while my thoughts are always churning on evaluation, they sometimes take awhile to formulate into something that is worth writing or comprehensible. I have been thinking a lot about transitions, since that is where I am in my personal life. My husband I are about to embark on a 6 month international stint working with the Himalaya Cataract Project. The Himalaya Cataract Project (HCP) works to cure and combat preventable blindness globally, focusing specifically on sustainability. They invest in infrastructure and more importantly education and training of doctors, nurses, techs, and other medical personnel needed to make their work as cost effective and efficient as possible. Through innovations by Dr. Sanduk Ruit and many others, cataract surgery can be performed for as low as $20 (compared to over a $1000 in the United States). This is due in part to advances in surgical technique and local production of necessary technology, but also organizationally focusing on efficiency and training people up to perform the highest level task that they can with their given education. This has allowed the hospital supported by HCP, Tilganga Eye Institute of Ophthalmology, and others throughout India and Nepal (most notably Aravind) to operate on a cost recovery model. Within this model, everyone receives the same high quality opthalmologic care; extra amenitites cost extra. Using different figures based on local GDP, generally a percentage of people seeking care pay above cost, some at-cost, and the rest are far below or covered for free.

We’re incredibly honored and tremendously excited to embark on this trip which is taking us to Nepal, Ethiopia, Ghana, and potentially India. My husband Z will be working on the medical side of things, I will be working on a number of different evaluations with the organization.

Transitioning from the United States overseas is always an adventure. I am also playing some mind games thinking about the differences in working with a healthcare organization versus an educational program. While I have previously worked in public health, it was with an HIV education program. Direct services were not provided, rather critical support services and education. I am particularly struct by how easy it is to discuss what we will be doing; there is no hard “sell” for curing blindness. Watching videos from the field and from cataract camps of people who may have been blind for years suddenly able to see their children, parents, friends, homeland once more and be able to work and participate again in communal life once more – that moment is awe-inspiring, a rebirth.

What is harder for HCP and so many other organizations is to find the funding and support for everything else that must take place to get a patient to the operating table.The backstage of any non-profit or NGO is frought with highs and lows, crisis and triumphs. I have seen this cycle with Kibera Girls’ Soccer Academy and other education organizations, where the end result cannot be truly defined and any measure you do use takes a long time to see any result. This transition to HCP is revolutionary for me, yet I know there is so much to learn and so much more to try to understand.

So we’re off … I will do my best to post updates on evaluations from the field, as well as pictures. I look forward to embarking on this new adventure through a broader evaluative lense, and to learning, living, experiencing all that these new transitions will bring!


1 Comment

Thinking about the What and the Why: Explaining Evaluation to Others

As I go further into thinking of myself and calling myself an evaluator, I am having the talk with more and more people. “So what is it that you do exactly?” Thinking about these conversations and building off of Chris Lysy’s brilliant cartoons, I have found explaining what evaluation is and why it is important to be two separate but equally interesting (and at times challenging) conversations. For those in the field, you know this can be quite a long and involved answer to describe the data analysis, writing, critical and systems thinking, technical innovation, instrument development, counseling, mentoring, facilitating, capacity-building, learning, researching, and many other things that comprise our day to day life. Sometimes the work is slow, and sometimes chaotic. You push way overtime on one project to find yourself with rewrites and revisions that you simply don’t have time for. Developing strong relationships and building trust with clients, particularly smaller organizations is a dance of uncertainty – one step forward, two steps back, shuffle left and hopefully bound forward. For external evaluators, we are also always grappling with the ever present conflict of interest of providing clear, rational information that our clients may not necessary want or be ready to hear, and trying to do so in a way so they’ll listen and perhaps even hear what we’re saying. What we do as evaluators is ever changing, dynamic, important, real.

Thus explaining the what of what we do as evaluators is complicated. However I have found that getting across the what is perhaps more challenging than the why of evaluation. Those of us who fall into the field, through whichever path, continue to work and improve skills because we believe that evaluation is one among many tools to helping others, making our world a little bit easier to understand, and getting close to, even if not completely, right. Explaining the why perhaps is easier because it is so deeply connected to what I believe and it is the passion behind our craft that comes through even in the midst of lengthy explanations and discussions. For so many in the field, evaluation makes as much sense to us as looking both ways before crossing the street, it is engrained in our thinking, our actions, our day to day lives. Evaluation is a pathway to improvement and growth, not just for programming but for organizations and individuals. The why of evaluation is the passion behind our craft, and it is the knowledge and curiosity that something amazing may already exist, and if not, that the potential is there. The why of evaluation is in the belief in hope.

I am relatively new to the evaluation field (and even more recently begun thinking of myself as an evaluator, but that’s another future post), and make no claim to being an expert. However I do know from my work overseas and domestically, people are drawn to and can find common ground with those who are passionate and genuine with their craft, regardless of the language barrier. And so, as I learn and grown and develop within evaluation, and as I have more and more discussions about what it is I do exactly, I am learning to start with explaining the why.  For it is in the why that I am finding that the moment of connection is made, the essence understood.

Thoughts, comments? How do you describe what you do as an evaluator? Contribute below!


2 Comments

Thoughts from the Field: An Uncomfortable Truth

An uncomfortable truth – education is not free. I say uncomfortable because many of us in the development world and in broader society want it, wish it, trust it to be so. I say this working with a “free” girls’ high school in the Kibera slums (Kibera Girls Soccer Academy). While the school is free for the girls attending, that does not mean that there is not a constant battle, headache, and heartache to raise operating costs, pay teachers something close to a livable and respectable wage, and provide for the equipment and supplies our students need to thrive. KGSA has always pushed for providing not just sufficient but quality education. Slowly, surely through a lot of work, sacrifice, and dedication they are doing it. Graduates are entering university and thriving there. They are coming back and working in their communities to provide other girls and children with similar opportunities….

I digress, and it is so easy to digress when discussing our passions and stories of programs that are succeeding. Yet for every KGSA, Shining Hope for Communities, or Educate!, there are many other community-based and independent schools and organizations struggling and simply disappearing. Within the United States, while education is a protected civil right, it is likewise not free. The continuing saga (rather than debate since there has yet to be any real dramatic shift except perhaps the emergence of charter schools in how the United States finances public schools in decades) over school finance means that while every child in America has the ability and must attend school, the quality of their education is dependent on zip code and their own parents’ ability to be their advocate.

While I don’t advocate for equating money with quality, success, or respect, it is hard to achieve those three values without investment of resources, both human and capital.  Yet it is at times a confusing struggle. As a young student I fervently believed that education must be free since it is so crucial to our existence, to ending poverty, and creating a more just and equal world. I still believe fervently in the power of education, but realize now that we live in a very complicated world. A world where there are very few straightforward corollaries. Politics, power dynamics, the human psyche all play a fantastic role. Working in Kibera and Arusha and small villages, time and time again I heard from people there, “If something is free, it has no worth.”

Poetry Workshop

In Uganda, the kids would chant, “Bazungu give me my money,” (Note: this was the only place I had traveled where kids actively chanted for money). They didn’t necessarily understand fully the English words, but it wasn’t hard to understand why they said them. The village had a lot of aid dropped in from abroad, mostly in the form of school buildings with little follow-up. Thus the secondary school was majestic yet students and families were still struggling to pay school fees, equipment was hard to come by, and teachers continued to struggle to make a living. Another major consequence: parents – largely subsistence farmers – were intimidated by the school compound and only came to the school if summoned to pay school fees. Instead of place of unity, it became a place of separation and differentiation, and while many parents were proud to have their child attend, they did not think they could be involved in their child’s education, or for the child to remain involved in daily life on the farm.

It is an uncomfortable truth – education is not free. I am struggling to fully understand this and what this means in the development and evaluation worlds.  As I reading the mounting frustration of teachers in America as regulations and evaluation to some extent undermine their legitimacy and field and as I continue to read the moving memorials to Anne Smedinghoff, and think of so many who are continuing the good fight in the classroom and elsewhere, both in the United States and abroad to encourage, enlightened, learn, and create possibilities, I am troubled, I am inspired – where do we go from here?

Thoughts? Comments? Have you come across any uncomfortable truths in your work? Let’s broaden the discussion, and find a path forward together….


4 Comments

Lessons Learned from the Field Part 2: Build Strong Organizations

Greetings, I’m Jamie Clearfield. Lately I’ve been reflecting on the different organizations I have worked with and how they have impacted me as an evaluator.

I first worked with a community-based organization in Kenya (you can read that post here. I next worked with a rural HIV education and testing organization, Support for International Change in Tanzania. The organization does tremendous work in leadership development with young Tanzanians and also working with over 100 villages in providing quality HIV education and testing services, as well as access to care to those living with HIV, where others fear to travel.

SIC is not afraid of the long hours, non-existent roads, alternating choking dust and sinking mud if it means that one more person can learn to protect themselves or access care. Moreover, as an organization SIC is focused not only on the work but on developing strong internal leaders and providing staff opportunities to be heard and grow. This means that a Tanzanian university student volunteering on an education campaign can later become a full time staff member. Regular and open communication, working with local culture, and the development of not only a strong strategic plan and goals, but actively working towards those goals were all key features that I now recognize as elements of a robust and innovative organization.

At the end of each year, SIC held a 3 day long planning meeting – setting aside time to review the years goals and focus on how they were met. What worked well? What didn’t work well? All staff was included, from the managing director to the drivers who drove the trucks filled with field staff and equipment to and from the villages. The strategic plan was reviewed, goals were set for the coming year, goals that made sense because they were developed by the very people who know what it takes to accomplish them.  Data was reviewed from the past year – data that was systematically collected and regularly reviewed at monthly staff meetings. None of the data was new, but being analyzed within a broader context of how it helped the organization meet its broader mission.

The time this organization took to review, revise, praise, and come together set a broader tone of mission and a recognition that their work is important and on-track.

My lesson: Building a strong internal organization is as important as the work the organization does, if not more so. A strong internal team, working together on shared mission and goals, means work on the ground has a focus, is determined, and of quality. As an evaluator, this is crucial for our evaluation work as it is for the organizations we work with. As an evaluator, maintaining focus on my own internal mission and my organization’s mission is crucial.

And perhaps our role as an evaluator should also be to help organizations become stronger internally. Not only will it make our work as evaluators easier, but it might just mean better results.

What lessons have you learned through your work as an evaluator or working with organizations? What are some of your personal mantras when you’re working in evaluation?


Leave a comment

Lessons Learned from the Field, Part 1: Define Your Mission

Greetings, I’m Jamie Clearfield. I’m an evaluator with CEAC (Collaborative for Evaluation and Assessment Capacity) and am new to the evaluation blogging world*. I’ve been having a lot of discussions lately with many different people about evaluation and organizations. Having attended a recent networking event with the New York Consortium of Evaluators (NYCE) featuring David Hunter, it got me thinking a lot about the different organizations I have worked with and how they have impacted me as an evaluator.

I got my start working with a community-based organization in the Kibera slums of Kenya, interning during a study abroad program. The experience was and continues to be transformative – nearly 7 years later I’m proud to be still involved with them and seeing how they have transformed their community through providing free secondary school education for girls. Working with the Kibera Girls’ Soccer Academy was an education in the power of what communities can achieve working together.

No MBA, no doctorate degree can take the place of dedicated people who are willing to experiment, fail, reevaluate and transform day after day no matter the challenges. And there were challenges – within 2 months after I arrived, the school was evicted from the bare two rooms it had acquired unless it began charging the girls tuition. The school’s director and founder, Abdul Kassim (check out his Nairobi TEDx audition below)- a charismatic man, a leader with a simple vision and a heart full of love for his community and for seeing it become a place where everyone has a chance – and I had many long walks and talks about the nature of challenges, remaining true to convictions and facing the unknown.

In the end, he went to the people who mattered most – the students – and asked what he should do. The school chose eviction and everyone together – students, teachers, and a small intern – worked together to procure space and finances to build new classrooms.

Today the school has graduated over 100 girls, educates 120 more every year, has graduates attending university, and more important is a symbol of hope, change, and locally driven development in Kibera. The habits of self-reflection, mission-driven evaluation remain a core focus of the school. While organizationally the school is learning, growing – its continual focus on remaining dedicated to its original purpose of providing free secondary education for girls remains a priority.

My lesson: Remaining true to your mission is crucial and worth the challenges. This is as true as an evaluator as it is for organizations.

What lessons have you learned through your work as an evaluator or working with organizations? What are some of your personal mantras when you’re working in evaluation?

Abdul Kassim: Real Change Requires Listening

*Many thanks to Ann Emery, Chris Lysy, and Karen Anderson for encouragement in getting this blog going!

Follow

Get every new post delivered to your Inbox.

Join 554 other followers