School districts as partners in research efforts

Research intended to improve teaching and learning — not just the agenda of researchers — can yield powerful results.

School districts are starting to realize that strong partnerships with local universities can play a role in developing a culture of data-driven and research-based decision making. That wasn’t the case some time ago. While there are solid reasons for working with external researchers, districts also have doubts that external researchers can provide useful and practical information that can guide continuous improvement processes.

At Jefferson County (Ky.) Public Schools (JCPS), we’ve learned that we can benefit from partnerships with universities and research laboratories if we have the right focus with researchers. Such collaborations can promote a data culture that will result in improvements in teaching practices and, ultimately, student outcomes.

We have learned from some wonderful research partnerships. Higher education institutions like the University of Louisville, Bellarmine University, Spalding University, University of Kentucky, Western Kentucky University, Vanderbilt University, and Harvard University have shown us that success comes from enabling (vs. preventing) conditions such as:

Coherence and alignment with our strategic plan vs. researchers following their own research agendas;
Active stakeholder involvement in the delineation of the research on the front end;
Ongoing feedback loops that allow for midcourse corrections vs. reporting only end-of-year results;
Reporting in a way that practitioners can understand vs. jargon-heavy and lengthy reporting designed for statistics-savvy audiences;
Timely reporting;
Follow-up after reporting so users are aided in taking appropriate actions; and
Protecting the rights of students and teachers involved in research vs. thinking that partnership research allows for a waiver of human subject protection regulations.

For example, JCPS took an innovative approach to supporting researchers through a partnership with the University of Louisville’s College of Education and Human Development. In 2007 and again in 2013, the organizations created a program for teachers and administrators to pursue doctoral degrees that culminated in research around the JCPS strategic plan. In order to ensure a focus on practice (i.e., relevance) and not just theory (i.e., rigor), the doctoral program’s leadership included the authors, who instructed courses and worked closely with the university’s Institutional Review Board (IRB).

The agreement between the two organizations and the high level of communication and coordination required of the partnership reduced many of the challenges common to more informal relationships between school districts and researchers. For example, many of the steps that school districts take to inform researchers — such as communicating the research agenda — are embedded in the college’s process for creating a research proposal. The approval process also overlaps across organizations: Students first propose their research project to the university’s IRB and then to the school district’s IRB. This, in turn, helps expedite the approval process within the school district. Some of the other redundancies that researchers generally experience are also reduced. For example, JCPS helped construct the cohort’s coursework, which includes training in data use, statistical analysis, human subject protection, and research methods. Having already approved the college’s training, the school district does not supplement with its own training and materials. The organizations also combine their efforts to oversee the research.

If the burden of data collection out-weighs the value of the research, then school systems have the right to decline requests.

JCPS has found that the work produced by participants in this program is higher quality than most of its research partnerships and that researchers are more motivated throughout the process because of the focus on keeping the work relevant and grounded in practice within its schools.

Other successful collaborations we’ve had include The Strategic Data Project with Harvard University (http://cepr.harvard.edu/sdp/), the 8-year, design-based Middle School Mathematics and the Institutional Setting of Teaching project with Vanderbilt University (http://peabody.vanderbilt.edu/departments/tl/teaching_and_learning_research/mist/index.php), and the Literacy Academy Project with Bellarmine University.

Data-access framework

To maximize their relationships with researchers, school districts should start by developing a data-access framework, which allows them to share existing data. The framework includes a common understanding of how the district will manage data access, make decisions about access, and use research results for organizational improvement.

The following steps contribute to a comprehensive workflow system for handling data-access requests:

Help researchers understand agency data and the data request process.
Create effective request forms for researchers.
Review data requests strategically.
Manage the data request process efficiently.
Release data appropriately.
Monitor data use.
Use research findings in the education institutions (National Forum on Education Statistics, 2014).

The data-access framework becomes the foundation for responding to researcher requests. School districts with experience working with researchers typically have structured procedures that facilitate the entire data-access process — from assisting researchers before their formal data access request through the completion of their research. In return for their support of researchers, school districts have clear expectations of what they want from that research: customized reports that will likely enable them to use the research results. If done well, researchers may help advance the school district’s own research agenda. In this regard, let’s be clear: If the burden of data collection outweighs the value of the research, then school systems have the right to decline requests.

Before researchers approach school districts typically they must be vetted by their own IRB. These entities classify and approve research at three levels that tend to correlate to their levels of intrusion and payoff. The classifications are:

Exempt: Research not involving more than minimal risk to students or staff, meaning the probability and magnitude of harm or discomfort anticipated in the research are not greater than those ordinarily encountered in daily life or during the performance of routine tests;
Expedited: Research involving greater-than-minimal risk but presenting the prospect of direct benefit to individual students or staff; and
Full board reviews: Research involving greater-
than-minimal risk and no prospect of direct benefit to individual students or staff but is likely to yield generalizable knowledge about the student or staff condition. (For more details, see the Education Department General Administrative Regulations [EDGAR], www2.ed.gov/policy/fund/reg/edgarReg/edgar.html.)

Unfortunately, the authors have fielded more than one data request that only focused on the researcher’s own agenda without considering the district’s research agenda. The fit between the researcher’s agenda and the district’s strategic agenda is clearly important and must be addressed upfront and not as an afterthought.

Processing research proposals

When requesting data from districts, researchers must understand key elements of how school officials will process the request. They review multiple features of the research proposals, including the purpose of the research and, more important, how the research supports the district’s strategic plan.

Key elements of the data request review include:

The proposal’s alignment with the district’s research agenda;
The approval letter by the institutional review board of the researcher’s institution;
The level of intrusiveness when collecting or retrieving the data; and
The compliance with important regulations such as the Family Educational Rights and Privacy Act (FERPA), the Protection of Pupil Rights Amendment (PPRA), and the National School Lunch Act (NSLA).

The common denominator of the federal, state, and local district regulations and policies stems from concerns over protecting data privacy for vulnerable populations. For example, school officials typically focus on which data must be masked, de-identified, and/or altered to protect individual privacy, particularly for vulnerable populations, including students with disabilities, English language learners, pregnant females, and students involved with the juvenile justice system.

A variety of technical tools have been devised to help protect the privacy and confidentiality of vulnerable populations and staff data even after release. These include:

Removing all sensitive information from a data report so the report may be distributed to a broader audience without disclosing sensitive components;
Masking specific data elements in a dataset to ensure that sensitive data are replaced with reasonable, but not real, data so sensitive information is inaccessible to unauthorized users; and
Removing personally identifiable information (PII) so the remaining information doesn’t identify an individual, and there is no reasonable basis to believe the information could be used to identify an individual — although a master database may be securely maintained by the school district to link back to the data.

In addition, the type of media on which data are released is also important to data security. For example, email is secure only when data are appropriately encrypted and otherwise protected before attachment and delivery. A web-based data request management system that requires a user name and password enhances the level of data security.

School district screener strategy

Researchers must understand the district’s research agenda, the circumstances in which requests may be approved, and how to appropriately access data. When they do, researchers are more likely to access, interpret, use, and manage data appropriately. Ideally, in return for granting researchers access to new and existing data, the school district will receive useful and valid information about education policies, processes, and practices.

School districts want research that’s useful, feasible, ethical, and technically sound. In screening, we ask, for example:

Does the research proposal address an important topic for improving teaching and increasing learning?
Does the research proposal align with the school district’s specific research priorities as delineated in the strategic plan?
Will the research result in practical rather than theoretical implications?
Will the benefit of the research justify the expenditure of resources?
Is there a burden on students or staff who might be asked to participate in data collection activities?
Is the research proposal compliant with FERPA, NSLA, and PPRA?

This process applies to all persons requesting data and proposing research, including school employees working on theses, capstones, or dissertations. In the past, in Jefferson County Public Schools, requests for data access often were serviced by the school officer who was approached with a data request. For example, a request to access special education data was received, reviewed, and, if approved, fulfilled by the special education department without involving any other district staff. But this was ineffective. Now, we expect all requests to be channeled to a central point of contact, typically the data management and research department, which has specialized knowledge regarding data protection and evaluates all requests in light of standard procedures, i.e., research agenda, data governance, and data-sharing policies.

Having a primary point of contact between researchers and the district ensures consistent practices are being used to determine the appropriateness of the proposed research methodology and approach. This office also can determine when legal counsel might be needed to review questions regarding the legality of a request, determining if, for example, a request complies with applicable laws and whether granting access to the data is required or prohibited by law. The review also may assess whether giving access to the data is consistent with the district’s existing contractual obligations or memoranda of understanding concerning access to data.

Vendors or service providers add complexity to this process. Like any other data access request, vendors must go through the formal request process and their research should be expected to align with the school district research agenda. In addition, the district should watch for any third-party organizations collecting data on behalf of a researcher; similar to vendors, these organizations should be expected to follow the proscribed data-access process.

Human subject protection

In general, key criteria for approving research in school districts involve carefully reviewing the following:

The risk to students and staff are minimized;
Risks to students and staff are reasonable in relation to anticipated benefits to subjects and the importance of the knowledge that may reasonably be expected to result;
Selection of students and staff is equitable and vulnerable populations are protected;
Informed consent is sought from each participant’s parent or legal guardian, or directly from staff while making sure to keep documentation for three to five years; and
Adequate provisions are followed to protect student and staff privacy and to maintain the confidentiality of data.

The protection of student privacy in examination, testing, and/or treatment is a delicate matter. Sensitive topics include asking students to reveal:

Political affiliations;
Mental and psychological problems potentially embarrassing to the student or his/her family;
Sex behavior and attitudes;
Illegal, antisocial, self-incriminating, and demeaning behavior;
Critical appraisals of other individuals with whom the student has close family relationships; and
Income, other than that required by law to determine eligibility for receiving financial assistance under a program.

Less sensitive are instances of what we call “directory information” — meaning information in a student’s education record that would not generally be considered harmful or an invasion of privacy if disclosed. Such examples include:

Student’s name;
Address;
Telephone number;
Date of birth;
Major field of study;
Dates of attendance, and
Degrees and awards received.

See FERPA for more details because directory information should not be confused with personally identifiable information. Directory information may be disclosed, unless parents/student opt out; however, personally identifiable information is any sensitive data other than directory information that may be released only with IRB approval. In general, active parent consent is required before the school district can provide a student’s education records. Even school employees should not have access to personally identifiable information unless the district has determined that staff has a legitimate educational interest or right to access the information.

Conclusion

The primary mission of a school district is teaching and learning in a safe environment while preparing a well-rounded student who can be a contributor to society. School districts do not exist to be research sites. In that regard, a key feature in good district practice is the protection of instructional time. As a result, school officials often question researchers about the level of obtrusiveness or disruption that the proposed research will have to the district’s data management and research department (if secondary data) and/or to teaching and learning (if primary data).

The common denominator of the federal, state, and local district regulations and policies stems from concerns over protecting data privacy for vulnerable populations.

School districts that provide data access to researchers say that too often they never hear back from researchers after the project is complete. If a school district has taken the time to review and approve the proposal, researchers should share the progress of their research and the results of their studies. While FERPA doesn’t explicitly require the school district to obtain research findings, it does require the researcher to be working on behalf of the educational entity. If researchers don’t share their findings, then why should districts allow researchers to have access?

Best practices suggest that researchers and school district officials decide if and how findings will be communicated to stakeholders, including administrators, practitioners, and parents. The district may require that the researcher present the findings in other formats, such as an executive summary and/or a short managerial report using nontechnical language to make it easier to share with stakeholders.

Researchers tend to be comfortable with presenting the results, but districts want more than that: We want guidance for our daily practice. In fact, if given a choice, school districts would prefer less information about the technical accuracy and more about the usefulness (i.e. practical implications) of the research. The more clearly and succinctly findings from quality research can be communicated, the more likely they are going to be actionable and benefit students.

This work is really about understanding the rights and responsibilities of a true partnership. On one hand, school districts want research that serves the goals and priorities of their strategic plans; to this end, districts can find value in mutually beneficial collaborations with researchers. On the other hand, researchers should pursue an active, engaged, and collaborative relationship with the school district, which will result in better research that is more likely to benefit students.

Those of us in the school district research community occasionally hear complaints from researchers about access to school data. From our perspective, we always think that it is possible, even recommended, that promoting student learning be at the core of their data request. Otherwise, why use our resources and time entertaining a data request that takes time away from instruction?

Our mission is to prepare all students to graduate and be successful in college and life, to reach their full potential, and contribute to our society throughout their lives. We need help with research that has potential to support data-driven decision making and improve student outcomes, not research that distracts and takes instructional time from kids. Research is a means toward an end, not an end in itself — at least that is how we see it from a school district perspective. Student development and learning is and should continue to be our primary goal. We hope researchers will realize that this is about codesigning, coimplementing, and coreporting research to solve problems of practice that affect student growth. This is about research informing practice, not about practice serving the interest of researchers. We need to balance rigor, relevance, and timeliness to rally around student needs. School districts can be great research partners as long as researchers focus on improving teaching and learning that provides actionable information.

Reference

National Forum on Education Statistics. (2014). Forum guide to supporting data access for researchers: A local education agency perspective. (NFES 2014-801). Washington, DC: U.S. Department of Education, National Center for Education Statistics.

CITATION: Muñoz, M.A. & Rodosky, R.J. (2015). School districts as partners in research efforts. Phi Delta Kappan, 96 (6), 42-46.