DATA PLAYBOOK California Health and Human Services Agency
Planning Part B: What Data You Will Need
Table of Contents:
1. For Analysts
2. For Managers
With your goals and strategy successfully outlined, you can now think about what data or measurements you need to collect to answer your guiding questions, as well as the data you need to determine if you are ready to proceed with data collection. If you’re a manager, you’ll also need to define your outcome measures and performance/self-assessment metrics to maintain the integrity of your project and ensure you’re supporting your team and stakeholders as best you can.
For Analysts
Before proceeding, you should go through a Readiness Checklist to ensure you’ve considered your own strengths, weaknesses, and that of your manager and team. Get the support or learning you need now to prevent misunderstandings or frustrations later in the process.
Step 1: Readiness Checklist
Ask yourself: Do you have the Support, Knowledge, and Resources to Complete your Project?
| - Do my managers/directors have the bandwidth to support me? |
| - Do I/my team have enough expertise to complete this project? |
| - Who is my department’s Data Coordinator? (The individual responsible for knowing the data assets held by your department) - Contact CHHS@osi.ca.gov to find your Departments Data Coordinator |
| - Do I have access to the data I need to complete the project? |
| - Do I know the statistical methods required to analyze my data? |
Step 2: Review Your Program Data
Your program data is the core data of this project — it’s the specific measurements that you need to collect in order to answer the project’s guiding questions. As a review, your guiding questions are the purpose of this project as a whole, and spending some time thinking about your project’s purpose statements will help you determine what data you need and how you should collect it.
Example Purpose Statements:
- I need to decide how to allocate resources to different programs based on which is the most successful
- I want to improve or refine an existing program or model to be more effective
- I want to create product or service that positively impacts a community
- I want to look at existing data to find trends and patterns that people care about
It can be useful to review all your data assets with these questions in mind. Contact your department’s data coordinator for more information about the types of program data you collect in your department by emailing CHHS@osi.ca.gov.
For Managers:
Managing a team at CHHS is challenging — in addition to setting and working toward your own personal goals, you must also assess the performance of your team and support their continuing learning; set the broader goals that guide larger initiatives, programs, or departments; and work toward capacity building in analytics, data literacy/governance, and much more.
The following section is written for a wide range of manager roles, including the larger cohort of managers who supervise analysts and technical employees (SSM1s) to the smaller cohort of branch-level directors or managers working on capacity, vision, and strategy of their department.
Part 1: Assessing Capability
As a manager, you may be in charge of managing the overall performance and strategy of the project or program; you also may need to assess the performance of the team itself, and the department’s resources. This requires defining and measuring outcome data, monitoring your team’s or program’s performance, and assessing your department’s current data assets and analytic capabilities.
| The following section contains a number of frameworks and resources to assess your Team’s Capabilities… |
|---|
| …related to projects and programs |
| - Assessing Readiness: considering the scope, risks, limitations of your data project - Measuring Performance: Setting Key Performance Indicators (KPIs) for the project and your team - Determining outcome measures: benchmark, baseline, and comparative data |
| …at the department level |
| - Strategic Use of Data: how effectively does the department utilize data to inform decisions and strategy? - Capacity Building: Improving internal capacity, assessing management strategy & organization - Data Governance & Management: Management & Security of Data, Improving Data Literacy, data de-identification guidelines |
Managers of Projects and Programs: Assessing Readiness
Before planning your data collection, go through the following readiness checklist to ensure you are capable of successfully carrying out this data project. You should catalog your assets and resources regularly throughout your project to identify areas of weakness or gaps in resources.
The Readiness Checklist:
- How do programs or stakeholders use data currently? What do they do with it? How do they use it to make decisions or produce products for external stakeholders?
- What are limits to either the data or the implementation solution?
- What are the risks and issues with the current data? What value is not being realized?
- Identify the current workflow for collecting, processing, and publishing data. Are there dependencies to collecting, processing, and publishing the data?
Remember, if you do not have the resources you need, you and your team will likely encounter problems in your data project. Address weaknesses early and be on the lookout for areas you can improve throughout your project.
Measuring Performance and Outcome
This is the data you need to collect after deploying your product or service to determine whether or not it met your goals and was successful. A useful framework to reference is the Key Performance Indicators (KPIs) framework described here. KPIs measure your performance relative to your goals.
- Check out this resource to learn all about KPIs: what they are, why they work, and how to set them effectively.
Managers of Departments: Strategic Use of Data
It is imperative for managers to regularly assess and improve how effectively they use their data assets to inform their strategic planning and organizational structure, as well as improve their offered programs and services. We will root our assessment in Harvard’s Strategic Use of Data Self-Assessment Guide, a useful framework for understanding how strategically your department uses data and how to improve. A few examples from the guide:
- Effective Budgeting and Financial Planning practices driven by data
- Assessing organizational strategy and goal-setting
- Measuring accountability at all levels of your team
Building Analytic Capacity
For managers interested in these types of assessments or improving the current heat the of their data structures, check out additional resources on building Capability and Capacity in your department (such as the Analytics Capability Assessment for Human Service Agencies.
Note: For more concrete recommendations to build analytic capacity, check out this Roadmap to Capacity Building in Analytics. It will cover:
- All staff/Human Resource Needs for a successful team
- Executing a successful Program or Initiative
- How to integrate best practices in Data Governance
- Training Resource Topics to Provide to Analysts (Data Processing Methods)
You may also be tasked with assessing the quality of your department’s data management and data governance, or working on capacity-building frameworks to improve data literacy and analysis skills.
