Data Mining

Originally published in Washington City Paper on July 26, 2018.
——

37789980_4338740114195_5682263499502256128_n

In the wake of a series of DC Public Schools scandals, Ward 3 Councilmember Mary Cheh came forward with an idea: an independent research collaborative that would conduct studies on the city’s public schools, including charters. This collaborative, outlined in draft legislation, would have an advisory board comprised of 16 education stakeholders who would drive the research agenda.

Cheh’s concept has precedent. Other cities, like Chicago, San Diego, and Houston, have similar research collaboratives, commonly referred to as “research practice partnerships” or RPPs. Local education advocates and Cheh’s colleagues on the Council have come out in strong support of her proposal.

But Cheh’s plan also has detractors, and many of them are the appointees of Mayor Muriel Bowser. At a six-hour public hearing held on July 13, several officials tapped by Bowser spoke out against this so-called “Education Research Collaborative.”

And at the same hearing, the public learned that the executive branch was exploring the launch of its own separate education research consortium with the Urban Institute, a national think tank located in D.C. The news sparked concerns that Bowser was seeking to undercut the Council’s push for independent oversight.

At the core of all this politicking: Who gets access to data about D.C.’s public schools, and how do they get to use it?

****

Cheh’s bill, introduced in April, has eight other co-sponsors, a Council supermajority which could override a potential veto from the mayor. The Council set aside $500,000 in its most recent budget for the auditor to “incubate” this pilot research consortium. (That funding becomes available in October, when fiscal year 2019 begins.) It would be launched initially in the Office of the DC Auditor, an agency outside of the executive branch. Supporters say that after a few years they would look for a new home—be it a local think tank, university, or its own independent agency.

The chair of the education committee, At-Large Councilmember David Grosso, has not yet taken a position on the bill, but in May he tried to steer the dedicated $500,000 to after-school programs instead. His effort failed 12-1.

The research collaborative was conceived of in response to the host of education scandals which emerged over the last year, including news that high school graduation rates were massively inflated and that the public schools chancellor knowingly violated a school choice policy he himself wrote. While local and national leaders have long looked to D.C.’s education reforms as a model for the nation, today many parents, community members, and even elected officials have voiced a lack of confidence in the gains reported by the school system, fearing information has become too politicized under mayoral control.

“I call the information that we get from our education agencies ‘PR,’” says At-Large Councilmember Robert White. “It can be very difficult to get hold of unbiased data.”

“Our hope is to get accurate, reliable, credible data, and then to use this data in a research partnership to understand whether the policies we are pursuing are really working,” says Cheh.

The idea has generated enthusiasm from local education activists who see it as a way to strengthen city schools. “From a parent and teacher perspective, everybody thinks this could be a huge benefit if it’s transparent, community-based, and has a real commitment to practitioners,” says Danica Petroshius, the co-vice president of the Capitol Hill Public Schools Parent Organization. “This is a chance to do something different, to really embrace a new way of doing business.”

Jeffrey Noel, who served as the data management director for D.C.’s state education agency—the Office of the State Superintendent of Education—between 2011 and 2015, also supports the proposed research collaborative.

“One of the things OSSE is known for in the state education community is having really focused on building its data validation and data quality tools … which means those things are really accurate, but it also means that the amount of data OSSE focuses on is really small,” says Noel. “There is a huge amount of raw data that exists that is just not being leveraged, that would help us to actually do better. But there’s a limit to how quickly OSSE can improve what it does internally, so some level of external effort would help.”

Despite this momentum, some significant hurdles remain for the Council’s idea.

At the July 13 hearing, current representatives from D.C.’s three education agencies—District of Columbia Public Schools (DCPS), the DC Public Charter School Board (PCSB), and OSSE—emphasized their support for conducting independent research, but testified that linking the collaborative to the auditor would conflate the lines between accountability and research, thereby crippling its chance for success.

Hanseul Kang, OSSE’s state superintendent, testified at the hearing that the auditor’s work should not be “commingled” with policy evaluation and research management “because these are two distinct functions” and doing so could “distract from the very real need for research that drives instructional practice.”

Ahnna Smith, the interim Deputy Mayor for Education, argued that housing the collaborative in the auditor’s office “would inherently politicize the research agenda” as the auditor is an arm of the Council. She also testified that the Council’s proposal “conflates the functions of audit and oversight with that of research” and would ultimately “prevent us from achieving the goals of gathering quality, accurate data” that could drive school improvement research.

Rick Cruz, the board chair of the DC Public Charter School Board echoed Kang and Smith’s concerns, testifying that while he supports both robust research and auditing, intertwining those two things could have unintended consequences. He also cautioned that schools “may be hesitant to work as cooperatively than they otherwise may be” if the research collaborative were housed in the auditor’s office, given “the inherent watchdog nature of the office.”

Supporters stressed that aside from an initial audit of all relevant data, the collaborative’s advisory board, not the auditor, would determine the research agenda. Plus, supporters say, it wouldn’t be permanently housed in the auditor’s office, but “incubated” there to get it off the ground. The primary goal, Cheh emphasized, is to ensure the effort launches in a trusted space that’s independent from the executive branch. She added that after two years, “we’ll be looking to see if there’s a better place to house it.”

D.C.’s auditor, Kathy Patterson, dismissed the criticism that housing it in her office would politicize it. The Council appointed Patterson to a six-year term in December 2014. “I don’t think it would be politicized at all,” she testified. “[My office] is seen as part of the legislative branch, but we also have a huge amount of independence.”

Michael Feuer, the dean of GW’s Graduate School of Education and Human Development testified that while he supports transitioning the collaborative to somewhere else after a while—including potentially his own university—starting it within the auditor’s office makes sense. “My hunch is placing the new consortium anywhere would provoke political and partisan concerns,” he said, adding that “if we wait until we’ve got the definitive evidence for a system that would be fool-proof and satisfying to everyone, we won’t get anywhere.”

Mary Levy, a longtime budget analyst for D.C. schools, is more blunt. “This idea is an infant in the cradle,” she tell City Paper. “And if you don’t put it in the auditor’s office it’s going to die in its cradle.”

***

Witnesses and councilmembers rarely uttered the word “subpoena” at the hearing, but the draft legislation endows the research consortium with subpoena power—undoubtedly one of the more contentious features of the proposal.

Josh Boots, the executive director of EmpowerK12, an education data support nonprofit, was one of the few witnesses to directly address the issue. “We are extremely concerned about the prospects of quality research … when that research is being conducted by an entity with the threat of subpoena power,” Boots said. “Will the [local education agency] leaders, principals, and teachers have the trust that is required to share candid quantitative and qualitative data under this scenario? We’re not sure.”

Supporters say that in an ideal world such legal authority would not be included, but that it’s needed in a city where schools and advocates have long fought against disclosing certain types of information.

“You have the subpoena as a backstop,” Councilmember Cheh tells City Paper. “When I was looking into the Department of General Services, for example, the mayor didn’t want me to launch a formal investigation with subpoena power, but then I could say, ‘Well OK, but then promise me when I seek information we’ll get it.’”

Others say a subpoena function is necessary because the Council has failed to sufficiently exercise its own oversight authority.

“The less muscle and oversight the education committee employs, the more empowered the education agencies are to not be transparent,” says Petroshius. “I think Council oversight has gotten much, much worse, and I’m done with excuses. There are many things at their disposal and it’s not being done, and that’s a deliberate choice.”

Grosso, the education committee chair, defends the council’s oversight. “I do hearings, meetings, legislation, performance and budget hearings, I write letters, I go out and tour buildings and engage in community conversations,” he tells City Paper.

Levy says that while the council still requests and publishes plenty of data, it lost a lot of incentives to police education since the switch to mayoral control. Before 2007, the council would sketch out the school district’s finances, but councilmembers could not control how those funds were actually spent. This often put councilmembers in the position of being blamed for the public school system’s struggles, yet they had few ways to address the problems. As a result, Levy says, the council would enforce tougher oversight, so it could better justify not ponying up funds all the time.

After 2007, though, DCPS’s failures were no longer seen as the fault of the Council, and councilmembers also benefited from the narrative that the city’s school reforms were a great success. “Over the years there are a lot of questions they just didn’t ask,” says Levy.

Questions community members say the education agencies have not responded to, and the Council has not aggressively pursued, include understanding what teacher turnover looks like at the individual school-level, scaled test score results, and more detailed analyses of “at-risk” subgroups within and between schools.

Following a 2015 independent evaluation of D.C. schools that found the city lacked comprehensive, accessible education data, Grosso pushed for an $11 million investment in OSSE’s data-collection capabilities. But today residents and researchers complain that the city’s education agencies still routinely withhold information from the public.

Leo Casey, the executive director of the Albert Shanker Institute, a think tank affiliated with the American Federation of Teachers, tells City Paper that when his center was conducting a national study on the racial composition of the teaching force, D.C. was the only major city to not provide them with the granular data they requested.

“Their story was they didn’t have the data on race and ethnicity that we needed,” says Casey. “And let me just say, the only reason one would not collect race and ethnicity data is if one didn’t want to know it.”

In June, Ruth Wattenberg, an education policy analyst and the Ward 3 representative on D.C.’s state board of education, argued in The Washington Post that the mayor’s “unchecked control” over education data hurts students. While other cities have mayoral-controlled school systems, no jurisdiction other than D.C. has its state superintendent also appointed by the mayor.

Wattenberg proposed making OSSE separate from the executive branch to give the agency more political independence. “We can’t control every aspect of our non-state status, but we can blame only ourselves if we eschew the checks and balances that exist in every state and consign ourselves to constant data spin and data darkness,” she wrote.

Grosso tells City Paper that over the last six months his staff has been closely exploring this idea of making OSSE more independent. “The mayor and her team could also just take their thumbs off the scale,” he says. “There’s nothing stopping the mayor from pulling politics out of schools.”

***

Cheh was the one to bring up the Urban Institute proposal at the July 13 hearing, grilling witnesses from D.C.’s education agencies and the think tank for more details and explanation. Discussions around this alternative RPP had, up to this point, been held in private—and Urban Institute’s pursuit was fresh news to many in the room. While education officials emphasize that discussions are still preliminary, some councilmembers and community advocates characterize it as an attempt to undermine Cheh’s initiative.

Matthew Chingos, the director of the Urban Institute’s education policy program, says their vision would be to establish a research entity modeled on the University of Chicago Consortium on School Research, including a data warehouse component for other independent researchers to use. “The goal is to create opportunities for research that researchers want to be doing, and that’s also to the benefit of local policymakers and practitioners,” he says.

While the Urban Institute would ensure that education researchers could publish their findings, no matter how positive or negative they might be, the D.C. education agencies would get to greenlight what research questions could be pursued in the first place. This is also how it’s done in Chicago, according to Bronwyn McDaniel, a spokesperson for Chicago’s research consortium.

Jeffrey Noel says that the reason Chicago’s RPP doesn’t require more independence from its public school district is because the state of Illinois “has a sufficient amount of independence to do critical oversight and research already.” D.C’s state education agency lacks a comparable level of independence.

“At the end of the day, the District does have some say because it’s their data,” Chingos tells City Paper, speaking of the potential new research consortium within Urban Institute. “So I think the question is how do you balance the need for independence with the need for collaboration and partnership, and how do you create an environment where folks feel comfortable going after hard questions.”

At the Council’s July hearing, Smith, the deputy mayor for education, said community input would be “critical” if the city’s education agencies moved forward with an Urban Institute partnership.

“The Urban Institute situation is exactly what parents and teachers fear,” says Iris Bond Gill, a parent advocate and a former OSSE official. “The city has been discussing a partnership for months in private, behind closed doors, and then they say, ‘Oh of course we want stakeholder engagement.’ But it’s always an afterthought.” (A Bowser spokesperson did not return City Paper’s request for comment on the Urban Institute RPP.)

For now Council leaders disagree over whether the Urban Institute initiative would hamper their proposed research effort.

White says it absolutely would undermine the Council. “It would seem very redundant if the Urban Institute or any other hand-selected research agency was working with our education agencies to collect data,” he says.

Grosso, the education committee chair, was more non-committal, saying it’s not clear that “what Urban wants to do is necessarily going to be the same thing that the Council wants to do.”

Council Chairman Phil Mendelson suggests that Urban’s initiative would not be independent—and therefore not directly competing with their proposal—if it were a partnership with the executive branch. “The issue here is an independent research collaborative, and what was discussed with the Urban Institute was hardly that,” he says.

Chingos emphasized that his think tank does not want to undermine the Council and “certainly want[s] the different efforts to be complementary.” At this point, he added, “everyone’s just figuring out how this all fits together, and we want to see how the legislative process evolves.”

In submitted testimony, Noel, OSSE’s former data director, warned of issues that could arise from establishing two separate research consortiums, with both entities competing for data, funding, educator engagement, and collaboration from other researchers. “Given this risk, I think it is important to outline policy options that could either result in sequenced consortium activities or legitimately limit the need for creation of two,” he wrote.

Noel also warned of housing too much education data outside of local government, where it’s “outside the purview of protections like FOIA and whistleblower protection.” (FOIA, or the Freedom of Information Act, allows residents, and often journalists, to request existing data sets and documents.)

Markup for the Council’s research collaborative will take place after the Council’s summer recess period, and a vote is likely to occur before the end of the calendar year.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s