We know that money will be coming to Minnesota for digital equity based on the BEAD Digital Equity Plan that the Office of Broadband Develop has created That money will be doled out to local digital equity organizations. The Digital Equity Research Center has just release a report (Developing a Digital Equity Theory of Change with Tech Goes Home) that looks at how to measure the impact of digital equity work in a meaningful way…

The purpose of the project was to understand how participatory action research could be used to develop a theory of change and an evaluation framework to benefit Tech Goes Home, its community, and the larger digital equity field. To inform the project, the research sought insights from digital inclusion and digital equity organizations across the United States to understand how they articulated what success looks like to them and their communities. The research team examined how these theories of change help organizations use data to evaluate their programs and services. The ultimate goal of the project was to gather data to assist Tech Goes Home in developing a theory of change for the organization that could inform how the organization measures the outcomes and impacts of their digital equity work.

I thought that their recommendations might be helpful for the local digital equity community as well as researchers and policymakers…

For Digital Equity Organizations

• Allocate time, money, and intentional effort to capture insights and expertise from community members. Recent discussions in the digital equity field, both at practitioner conferences and in other spaces where digital equity researchers are gathering, have argued that organizations must pay community members to gain their expertise and participation in program evaluations.
• Engage evaluation participants in their native languages. Digital equity organizations that serve learners who speak multiple languages should work to ensure that their needs are considered in program evaluation efforts. These populations are likely to face compounded barriers to both digital inclusion and survey participation, and their experiences are therefore crucial to accurately understand the impact of programs.
• Work with funders to balance reporting requirements with sensitivity to participants’ privacy and attention to self-defined measures of success. Other digital equity organizations identified that one of the significant challenges they face is the need to gather data to show funders the outcomes and impacts of their financial support, while also showing community members why these activities are mutually beneficial. There can also be challenges in marrying what a funder requests and what the organization knows is perhaps a more representative indicator of success. Organizations should work with funders to establish respectful guidelines that ensure program evaluation is compelling, representative, and responsive to community needs.

Advice from Peer Organizations

• Digital equity organizations should stay focused on what they do well. Because the priority of funders can change over time, thus making program evaluation a moving target, we heard from our interviewees that it’s important for organizations in the digital equity field to stay true to their mission and the communities they serve.
• Provide support to community partners when they are asked to gather sensitive information. Digital equity organizations–like Tech Goes Home and others who we interviewed for this project who work with community partners to provide their programs and services–should co-create meaningful and respectful ways to address privacy concerns mentioned above. One way to do this is to take the lead on gathering this information and avoid placing the burden on community partners.
• Listen to your community, ask them for advice. Most of the people from peer organizations who we interviewed for this study relied on the knowledge, expertise, and wisdom of their community members and partners. Particularly in helping to define what success of their digital equity programs look like. Therefore, it’s essential that digital equity organizations develop deep ties and connections with people and leaders in their communities to ensure that digital equity programs address their needs and inform further opportunities to work towards digital equity and social justice.

For State and Federal Policymakers

• The success of broadband infrastructure programs relies on digital equity funding. Community members cannot access the digital world without affordable and reliable internet service, internet-equipped hardware, and hands-on support in making use of it. When evaluating the success of broadband deployment, administering entities must also consider whether communities have access to the digital equity programs necessary to make use of broadband access.

• Set aside funding that organizations can use to conduct program evaluation. Digital equity organizations have been arguing for years that if funders expect them to show the outcomes and impacts of their investments, the organizations should be compensated to do this work. Because program evaluation is time-intensive particularly when engaging covered populations, state and federal entities should allocate funding that can be used by digital equity organizations to measure the success of their state or federally funded programs as part of contracts.

• Provide technical assistance on program evaluation for digital equity organizations. In addition to funding, state and federal entities should provide technical assistance to support digital equity organizations in conducting outcomes[1]based evaluation. Counting outputs, such as numbers of digital skills classes offered or numbers of computers distributed, is a much easier task for organizations. However, if digital equity organizations are required to show the mid and longer[1]term outcomes, as detailed in the Tech Goes Home logic model in this report, then technical assistance must be provided by state and federal entities to help local organizations, particularly under-funded nonprofits, with this work

• Allow and encourage organizations to use government funding to compensate community members for their expertise. State and federal agencies should require grantees that receive funding to implement and evaluate digital equity programs to provide evidence that community members were engaged in determining what the success of these programs look like. State and federal agencies should also ensure that compensating community members for their expertise be allowable as a budgetable expense. If this unprecedented federal funding is truly to make an impact, then those most impacted by digital inequalities must be included as partners in the creation, implementation, and evaluation of these programs.