Learning to See Clearly

Jamie Clearfield's Musings on Evaluation and Development


Thoughts from Ghana: And it is Good

Excited for Data Visualization Training!

Excited for Data Visualization Training!

Growing up I loved history and social studies. Coming from a family of history buffs (my grandfather has stacks and stacks of military history books in his house and my father once penned an unpublished murder mystery set during the construction of the Brooklyn Bridge), I loved learning about and imaging the past. Today I still love nothing more than roaming old ruins or cities, imaging the day to day life – the clothes, shouts, colors of a by-gone era. Thanks to an amazing AP US History teacher in high school, I fell in love with the Guilded Age and Progressive Era following it – especially those fierce muckrakers who took pen to paper to uncover and publicize society’s injustices. Perhaps having always been bookish and quieter, I can more easily imagine and identify with those muckrakers of old – many of whom never claimed fame or fortune, but used what talent they had to work for the common good.

These muckrakers come to mind as I work on different projects here in Ghana. My evaluation experience has given the skills and training to look critically at programs, my past experiences internationally and my mentorship with great leaders and thinkers like Abdul Kassim at KGSA and Karen Boatman at Boston University have helped me to learn to analyze and break things down. My belief that through working and learning together, hard work, a lot of experimentation, and hope has also helped me to believe that we can build things up, we can create positive change, even if it is a long time coming. ..

A challenge for evaluators, both internal and external, is how much of a muckraker to be – how honest an answer is your client or organization looking for? And the self-doubt is there, as is the knowledge that there is no sure way forward, everything is an experiment of sorts. Is it better for an organization to pour money into cataract outreach if no one is tracking complication rates? Would it be better to expend more to train surgeons or screening personnel, or invest more heavily in monitoring and evaluation? Is anything better than nothing, even if the “anything” may bring its own set of challenges?

Waiting for cataract surgery

Waiting for cataract surgery

There is no easy answer, perhaps there is no answer at all. There are small breakthroughs – demonstrating how visualizing data differently can help with decision making, doing small capacity-building trainings in Excel, touting Anne Emery’s blog (and hoping the wifi here is connected soon so people can access it!). So I continue to struggle if I be a muckracker or a simple blogger, the sun shines bright here in Kumasi, the children shout “Obruni, Obruni how are you?” as we walk to and fro from the hospital, and there is the feeling of the rhythm and flow and feeling that there is work here and it is good…

1 Comment

Nepal Thoughts from the Field: Trying It on for Size

Life is a mix of things here in Nepal – overwhelming beauty, grace, poverty, hope, things known and unknown. After three weeks in Kathmandu, Z and I are currently in far western Nepal based at a rural eye hospital. Kathmandu was everything a developing capital city coming off a decade long civil war could be expected to be – a mix of chaos and blaring horns with temples and shrines on nearly every street corner, millennium-old Buddhist stuppas and glossy new malls beckoning the emerging middle class. Tradition and culture continually struggling against rapid urbanization and globalization; we wondered a lot what Kathmandu will look like in 10 years time.

A stuppa we stumbled upon when we got lost coming home from work...

A stuppa we stumbled upon when we got lost coming home from work…

In Kathmandu, We were attached to Tilganga Eye Hospital – a renown eye care center that provides efficient and effective eye care services in Nepal. Their model is similar to the Aravind Eye Care System has perfected over the past thirty years – a high volume cost recovery model where everyone receives the same quality of eye care, though those who cannot pay are not turned away. Instead those who are able to pay or who wish to have certain amenities (such as a private hospital room, an appointment time rather than walk-in service, after-hours service, etc.) can do so. I could on and on regarding the Aravind model, but I suggest for those interested, to read the case studies and other reports others, much more eloquent than I, have already written.

While my spouse was attached to various specialty clinics and the operating room to learn new a different cataract removal technique widely used in the developing world, I was attached to the hospital’s academic and training department to review their training programs for international ophthalmologists and allied personnel. A major focus for the hospital is developing and broadening the skills and knowledge of medical personnel, both in Nepal and internationally. Doctors, nurses, technicians, biomedical equipment engineers, and more from Ethiopia, Ghana, Bhutan, Indonesia, North Korea, East Timor, Vietnam, Cambodia as well as Australia and the United States have come to Tilganga for a wide variety of training and surgical exposure. Managing all the different logistics and visa difficulties of getting these personnel into Nepal is an incredible feat in of itself. Training so many different people, personalities – each with their own expectations and experiences – is another accomplishment all together. Nearly 150 ophthalmologists from over 20 countries have received specialized cataract training since 2007, in addition to over 70 nurses and scores of other paramedical and nonclinical staff.

Despite the efficiencies already in place to keep the hospital running and solvent, little evaluation work has been conducted with the training programs. While the hospital has a monitoring and evaluation office (currently a department of one!), it is mainly focused on the clinical aspect of the hospital, and ensuring that the numbers are there for the monthly reports to donors. This is not something new or even surprising – as we all know, educational evaluations are involved and are not straight forward. Determining what metrics to use, how to collect them, and how to determine the impact of education is an ongoing challenge as we see from the continuing education testing saga in the United States. To date, the training programs at Tilganga have participants complete a feedback form. The form is reviewed briefly as the person leaves. It is then filed in a 2-ring binder labeled with their country of origin; the binder than sits on a shelf.

In speaking with the training officer, there has yet to be a longitudinal review of the feedback forms, or any type of major follow-up of participants once they return to their home countries beyond a few emails with their sponsoring agency. I had found my job… even if it did mean digitizing 6 years worth of old feedback forms…

Back to the world of paper evaluations...

Back to the world of paper evaluations…

Currently, in the jungles of far western Nepal, I am reviewing and making sense of the data. The  hospital also asked me to help put together some of the training departments strategic planning  notes, for which the feedback forms will be helpful as well. The data proves interesting, the days  here are slow – cups of tea and a backup generator allow for the work to go on…

Coming in to Tilganga has been an educational and if truth be told, awkward experience. For years  working abroad, I chose positions where I would be present for a significant period of time, be able  to stay and live in the community to build relationships and trust. This type of work and  evaluation smacks a bit of everything I learned “not to do”. Not fitting into any label that the  hospital is used to giving to a visiting foreigner (Doctor, medical student, journalist…) – my  presence at Tilganga was stilted though over the weeks we developed a good rapport of sorts.  Reading the strategic planning notes and trying to make heads or tails of meeting minutes from a  workshop this past spring is like reading a book with the pages all mixed up; a not uncommon  challenge of evaluation and I suspect external evaluators globally. It was exciting to sit with Ravi,  the M&E officer and pass ideas back and forth. He loved getting links to EvalCentral, AEA, and other  evaluation blogs and resources (though I’m not sure how well Chris Lysy’s incredible evaluation  humor translates!). And yet… seeing so many people in need of immediate tangible care, while I sit  at my computer day after day, it is easy to be swayed that there is perhaps a better use for my  time here. My previous roles internationally were much more varied and engaged on the ground–   teaching, developing and giving workshops, implementing, as well as conducting monitoring and evaluation.  Having chosen one                     label is a new and interesting challenge, one I am currently unsure of but am trying on for size…

Thoughts and comments as always are welcome! Feel free to shoot questions about Nepal and our work overseas my way, I’ll answer the best I can.

Himalayas peaking through the monsoon clouds...

Himalayas peaking through the monsoon clouds…

1 Comment

Evaluation Transitions: Nepal Bound…

I apologize for the silence on the blogosphere…it has been a hectic few months. And while my thoughts are always churning on evaluation, they sometimes take awhile to formulate into something that is worth writing or comprehensible. I have been thinking a lot about transitions, since that is where I am in my personal life. My husband I are about to embark on a 6 month international stint working with the Himalaya Cataract Project. The Himalaya Cataract Project (HCP) works to cure and combat preventable blindness globally, focusing specifically on sustainability. They invest in infrastructure and more importantly education and training of doctors, nurses, techs, and other medical personnel needed to make their work as cost effective and efficient as possible. Through innovations by Dr. Sanduk Ruit and many others, cataract surgery can be performed for as low as $20 (compared to over a $1000 in the United States). This is due in part to advances in surgical technique and local production of necessary technology, but also organizationally focusing on efficiency and training people up to perform the highest level task that they can with their given education. This has allowed the hospital supported by HCP, Tilganga Eye Institute of Ophthalmology, and others throughout India and Nepal (most notably Aravind) to operate on a cost recovery model. Within this model, everyone receives the same high quality opthalmologic care; extra amenitites cost extra. Using different figures based on local GDP, generally a percentage of people seeking care pay above cost, some at-cost, and the rest are far below or covered for free.

We’re incredibly honored and tremendously excited to embark on this trip which is taking us to Nepal, Ethiopia, Ghana, and potentially India. My husband Z will be working on the medical side of things, I will be working on a number of different evaluations with the organization.

Transitioning from the United States overseas is always an adventure. I am also playing some mind games thinking about the differences in working with a healthcare organization versus an educational program. While I have previously worked in public health, it was with an HIV education program. Direct services were not provided, rather critical support services and education. I am particularly struct by how easy it is to discuss what we will be doing; there is no hard “sell” for curing blindness. Watching videos from the field and from cataract camps of people who may have been blind for years suddenly able to see their children, parents, friends, homeland once more and be able to work and participate again in communal life once more – that moment is awe-inspiring, a rebirth.

What is harder for HCP and so many other organizations is to find the funding and support for everything else that must take place to get a patient to the operating table.The backstage of any non-profit or NGO is frought with highs and lows, crisis and triumphs. I have seen this cycle with Kibera Girls’ Soccer Academy and other education organizations, where the end result cannot be truly defined and any measure you do use takes a long time to see any result. This transition to HCP is revolutionary for me, yet I know there is so much to learn and so much more to try to understand.

So we’re off … I will do my best to post updates on evaluations from the field, as well as pictures. I look forward to embarking on this new adventure through a broader evaluative lense, and to learning, living, experiencing all that these new transitions will bring!

1 Comment

Thinking about the What and the Why: Explaining Evaluation to Others

As I go further into thinking of myself and calling myself an evaluator, I am having the talk with more and more people. “So what is it that you do exactly?” Thinking about these conversations and building off of Chris Lysy’s brilliant cartoons, I have found explaining what evaluation is and why it is important to be two separate but equally interesting (and at times challenging) conversations. For those in the field, you know this can be quite a long and involved answer to describe the data analysis, writing, critical and systems thinking, technical innovation, instrument development, counseling, mentoring, facilitating, capacity-building, learning, researching, and many other things that comprise our day to day life. Sometimes the work is slow, and sometimes chaotic. You push way overtime on one project to find yourself with rewrites and revisions that you simply don’t have time for. Developing strong relationships and building trust with clients, particularly smaller organizations is a dance of uncertainty – one step forward, two steps back, shuffle left and hopefully bound forward. For external evaluators, we are also always grappling with the ever present conflict of interest of providing clear, rational information that our clients may not necessary want or be ready to hear, and trying to do so in a way so they’ll listen and perhaps even hear what we’re saying. What we do as evaluators is ever changing, dynamic, important, real.

Thus explaining the what of what we do as evaluators is complicated. However I have found that getting across the what is perhaps more challenging than the why of evaluation. Those of us who fall into the field, through whichever path, continue to work and improve skills because we believe that evaluation is one among many tools to helping others, making our world a little bit easier to understand, and getting close to, even if not completely, right. Explaining the why perhaps is easier because it is so deeply connected to what I believe and it is the passion behind our craft that comes through even in the midst of lengthy explanations and discussions. For so many in the field, evaluation makes as much sense to us as looking both ways before crossing the street, it is engrained in our thinking, our actions, our day to day lives. Evaluation is a pathway to improvement and growth, not just for programming but for organizations and individuals. The why of evaluation is the passion behind our craft, and it is the knowledge and curiosity that something amazing may already exist, and if not, that the potential is there. The why of evaluation is in the belief in hope.

I am relatively new to the evaluation field (and even more recently begun thinking of myself as an evaluator, but that’s another future post), and make no claim to being an expert. However I do know from my work overseas and domestically, people are drawn to and can find common ground with those who are passionate and genuine with their craft, regardless of the language barrier. And so, as I learn and grown and develop within evaluation, and as I have more and more discussions about what it is I do exactly, I am learning to start with explaining the why.  For it is in the why that I am finding that the moment of connection is made, the essence understood.

Thoughts, comments? How do you describe what you do as an evaluator? Contribute below!

Leave a comment

Lessons Learned from the Field, Part 1: Define Your Mission

Greetings, I’m Jamie Clearfield. I’m an evaluator with CEAC (Collaborative for Evaluation and Assessment Capacity) and am new to the evaluation blogging world*. I’ve been having a lot of discussions lately with many different people about evaluation and organizations. Having attended a recent networking event with the New York Consortium of Evaluators (NYCE) featuring David Hunter, it got me thinking a lot about the different organizations I have worked with and how they have impacted me as an evaluator.

I got my start working with a community-based organization in the Kibera slums of Kenya, interning during a study abroad program. The experience was and continues to be transformative – nearly 7 years later I’m proud to be still involved with them and seeing how they have transformed their community through providing free secondary school education for girls. Working with the Kibera Girls’ Soccer Academy was an education in the power of what communities can achieve working together.

No MBA, no doctorate degree can take the place of dedicated people who are willing to experiment, fail, reevaluate and transform day after day no matter the challenges. And there were challenges – within 2 months after I arrived, the school was evicted from the bare two rooms it had acquired unless it began charging the girls tuition. The school’s director and founder, Abdul Kassim (check out his Nairobi TEDx audition below)- a charismatic man, a leader with a simple vision and a heart full of love for his community and for seeing it become a place where everyone has a chance – and I had many long walks and talks about the nature of challenges, remaining true to convictions and facing the unknown.

In the end, he went to the people who mattered most – the students – and asked what he should do. The school chose eviction and everyone together – students, teachers, and a small intern – worked together to procure space and finances to build new classrooms.

Today the school has graduated over 100 girls, educates 120 more every year, has graduates attending university, and more important is a symbol of hope, change, and locally driven development in Kibera. The habits of self-reflection, mission-driven evaluation remain a core focus of the school. While organizationally the school is learning, growing – its continual focus on remaining dedicated to its original purpose of providing free secondary education for girls remains a priority.

My lesson: Remaining true to your mission is crucial and worth the challenges. This is as true as an evaluator as it is for organizations.

What lessons have you learned through your work as an evaluator or working with organizations? What are some of your personal mantras when you’re working in evaluation?

Abdul Kassim: Real Change Requires Listening

*Many thanks to Ann Emery, Chris Lysy, and Karen Anderson for encouragement in getting this blog going!