Search nctq.org

Or search by choosing a tag
Training and Recruitment

Teacher Prep, Licensure, Student Teaching

Staffing

Hiring and Assignment, Supply and Demand, Teacher Diversity, Work Day

Evaluation and Support

Evaluation, Professional Development, Retention and Tenure

Compensation

Pay, Benefits

State Data Systems: California

Identifying Effective Teachers Policy

Look at National Findings
Goal

The state should have a data system that contributes some of the evidence needed to assess teacher effectiveness.

Does not meet goal
Suggested Citation:
National Council on Teacher Quality. (2015). State Data Systems: California results. State Teacher Policy Database. [Data set].
Retrieved from: https://www.nctq.org/yearbook/state/CA-State-Data-Systems-71

Analysis of California's policies

It does not appear that California's longitudinal data system for providing evidence of teacher effectiveness is mandated or that data system use is required in state policy.

California does not have a teacher of record definition, nor does it have a process in place for teacher roster verification. 

California publishes an annual report entitled Teacher Supply in California, which includes data on the number of teachers who received credentials, certificates, permits and waivers and addresses issues regarding the supply of teachers newly available to teach in California classrooms. Specifically, the report breaks down the number of credentials by those earning multiple subject, single subject and education specialist certifications. It also includes a table that compares the number of teaching credentials to permits issued for each authorization; these numbers include documents for individuals recommended by California institutions as well as for those who completed an out-of-state program. 

However, no connection is made between these data and district-level hiring statistics, and consequently this report provides an incomplete analysis of teacher production in California. In fact, the report includes the caveat that "these data are presented for comparison purposes only. No inference may be made regarding the shortage or surplus of teachers for specific credential areas as information was not available regarding the numbers of teaching positions in each credential area, numbers of credential holders currently serving in schools, or the availability of newly credentialed teachers for vacant positions in schools."

Citation

Recommendations for California

Ensure that the longitudinal data system is connected to teacher effectiveness.
Although California has a data system with the capacity to provide evidence of teacher effectiveness, the state should strengthen its policy and mandate the use of this system.

Develop a definition of “teacher of record" that can be used to provide evidence of teacher effectiveness.
To ensure that data provided through the state data system are actionable and reliable, California should articulate a definition of teacher of record that reflects instruction rather than grading and require its consistent use throughout the state.

Strengthen data link between teachers and students.
California should put in place a process for teacher roster verification. This is of particular importance for using the data system to provide evidence of teacher effectiveness. California should also ensure that its teacher-student data link is able to connect more than one educator to a particular student in a given course.

Publish data on teacher production. 
From the number of teachers who graduate from preparation programs each year, only some of those certified are actually hired in the state. While it is certainly desirable to produce a big enough pool to give districts a choice in hiring, the substantial oversupply in some teaching areas is not good for the profession. California should look to Maryland's Teacher Staffing Report as a model whose primary purpose is to determine teacher shortage areas, while also identifying areas of surplus. By collecting similar hiring data from its districts, California will form a rich set of data that can inform policy decisions. 

State response to our analysis

California asserted that NCTQ’s assertion that “only some of those certified are actually hired in the state” is misleading and inaccurate. California does not have the authority to dictate that a newly credentialed teacher must immediately seek employment, or dictate where a teacher finds employment, or dictate that an employer must hire a particular graduate. Newly credentialed candidates who want to find employment in California public schools are able to do so.

California added that its annual Teacher Supply Report is being augmented in 2015-2016 with data on teacher demand, including expected openings across statewide public school districts and shortage areas.

The state's response only pertains to NCTQ's first recommendation. 

Last word

Nothing in NCTQ’s recommendation suggests that the state is responsible for or should dictate when and where new teachers are hired. The point is that making information about new teacher hiring publicly available would provide important data about areas where supply and demand are aligned and areas where they are not.

    • Best practice
    • Meets goal
    • Nearly Meets goal
    • Meets goal in part
    • Meets a small part of goal
    • Does not meet goal

  • California

    California map
Evaluation of Effectiveness
Select another topic
Delivering Well Prepared Teachers
Expanding the Pool of Teachers
Identifying Effective Teachers
Retaining Effective Teachers
Exiting Ineffective Teachers
Pensions
Research rationale

It is an inefficient use of resources for individual districts to build their own data systems for value-added analyses.
States need to take the lead and provide districts with state-level data that can be used for the purpose of measuring teacher effectiveness.  Furthermore, multiple years of data are necessary to enable meaningful determinations of teacher effectiveness. Value-added analysis requires both student and teacher identifiers and the ability to match test records over time.  Such data is useful not just for teacher evaluation but also to measure overall school performance and the performance of teacher preparation programs. 

Additional elements are needed to use data to assess teacher effectiveness.
States need to have some advanced elements in place in order to apply data from the state data system fairly and accurately to teacher evaluations. State must have a clear definition of teacher of record that connects teachers to the students they actually instruct and not just students who may be in a teacher's homeroom or for whom the teacher performs administrative but not instructional duties. There should also be in place a process for roster verification, ideally occurring multiple times a year, to ensure that students and teachers are accurately matched. Systems should also have the ability to connect multiple educators to a single student. While states may establish different business rules for such situations, what it is important is that the mechanism exists, in recognition of the many possible permutations of student and teacher assignments. 

State Data Systems: Supporting Research
The Data Quality Campaign tracks the development of states' longitudinal data systems by reporting annually on states' inclusion of 10 elements in their data systems. Among these 10 elements are the three key elements (Elements 1, 3 and 5) that NCTQ has identified as being fundamental to the development of value-added assessment. For more information, see http://www.dataqualitycampaign.org.

For information about the use of student-growth models to report on student-achievement gains at the school level, see P. Schochet and H. Chiang, "Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains", July 2010, U.S. Department of Education, NCEE 2010-4004; as well as The Commission on No Child Left Behind, Commission Staff Research Report: Growth Models, An Examination Within the Context of NCLB, Beyond NCLB: Fulfilling the Promise to Our Nation's Children, 2007.

For information about the differences between accountability models, including the differences between growth models and value-added growth models, see P. Goldschmidt, P. Roschewski, K Choi, W. Auty, S. Hebbler, R. Blank, and A. Williams, "Policymakers' Guide to Growth Models for School Accountability: How Do Accountability Models Differ?" Council of Chief State School Officers' Report, 2005 at: http://www.ccsso.org/Documents/2005/Policymakers_Guide_To_Growth_2005.pdf

For information regarding the methodologies and utility of value-added analysis see, C. Koedel and J. Betts, "Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique", Education Finance and Policy, Volume 6, No. 1, Winter 2011, pp. 18-42; D. Goldhaber and M. Hansen, "Assessing the Potential of Using Value-Added Estimates of Teacher Job Performance for Making Tenure Decisions." The Urban Institute/Calder, February 2010, Working Paper 31, and S. Glazerman, S. Loeb, D. Goldhaber, D. Staiger, S. Raudenbush, and G. Whitehurst, "Evaluating Teachers; The Important Role of Value-Added." Brookings Brown Center Task Group on Teacher Quality, November 2010; S. Glazerman, D. Goldhaber, S. Loeb, S. Raudenbush, D. Staiger, G. Whitehurst, and M. Croft, Passing Muster: Evaluating Teacher Evaluation Systems, The Brookings Brown Center Task Group on Teacher Quality, April 2011; D. N. Harris, "Teacher value-added: Don't end the search before it starts," Journal of Policy Analysis and Management, Volume 28, No. 4, Autumn 2009, pp. 693-699. H.C. Hill, "Evaluating value-added models: A validity argument approach," Journal of Policy Analysis and Management, Volume 28, No. 4, Autumn 2009, pp. 700-709; T.J. Kane and D.O. Staiger, "Estimating teacher impacts on student achievement: An experimental evaluation". National Bureau of Economic Research, Working Paper No. 14607, December 2008.

There is no shortage of studies using value-added methodologies by researchers including T.J. Kane, E. Hanushek, S. Rivkin, J.E. Rockoff, and J. Rothstein. See also T.J. Kane and D.O. Staiger, "Estimating teacher impacts on student achievement: An experimental evaluation". National Bureau of Economic Research, Working Paper No. 14607, December 2008; E.A. Hanushek and S.G. Rivkin, "Generalizations about using value-added measures of teacher quality." American Economic Review , Volume 100, No. 2, May 2010, pp. 267-271; J. Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement."The Quarterly Journal of Economics, Volume 125, No. 1,February 2010, pp. 175-214; T.J. Kane and D.O. Staiger, "Estimating teacher impacts on student achievement: An experimental evaluation". National Bureau of Economic Research, Working Paper No.14607, December 2008. S.G. Rivkin, E.A. Hanushek, and J.F. Kain. "Teachers, Schools, and  Academic Achievement." Econometrica, Volume 73, No. 2, March 2005, pp. 417-458; E.A. Hanushek, 2010, "The Difference is Great Teachers," In Waiting for "Superman": How We Can Save America's Failing Public Schools, Karl Weber, ed., pp. 81-100, New York: Public Affairs.

See also NCTQ's "If Wishes Were Horses" by Kate Walsh at: http://www.nctq.org/p/publications/docs/wishes_horses_20080316034426.pdf and the National Center on Performance Incentives at: www.performanceincentives.org.

For information about the limitations of value-added analysis, see Jesse Rothstein, "Do Value-Added Models Add Value? Tracking, Fixed Effects, and Causal Inference." Princeton University and NBER. Working Paper No. 159, November 2007 as well as Dale Ballou, "Value-added Assessment: Lessons from Tennessee," Value Added Models in Education: Theory and Applications, ed. Robert W. Lissitz (Maple Grove, MN: JAM Press, 2005). See also Dale Ballou, "Sizing Up Test Scores," Education Next, Volume 2, No. 2, Summer 2002, pp. 10-15.

We use cookies

We use cookies to recognize your repeat visits and preferences, as well as to measure the effectiveness of campaigns and analyze traffic. To learn more about cookies, including how to disable them, view our Privacy and Cookie Policy. By clicking "Accept" or by continuing in the site you consent to the use of cookies unless you have disabled them.

Dismiss Accept