In ediscovery, your first duty is defensibly protecting your company from risk. The necessity of this role has meant that, historically, Legal could operate outside the discretion of corporate budgets. But soaring data volumes and complexity are driving up the costs of ediscovery, and there is increasing pressure to adopt a more business-centric approach.
To control costs without increasing risk requires a clear understanding of how your ediscovery teams operate, what are the opportunities for greater efficiency, and how you are performing over time.
In other words, you need metrics.
Getting Started with Ediscovery Metrics
Developing strong, meaningful metrics for your ediscovery processes can sometimes fall under the “easier said than done” category. After all, there are hundreds of activities and outputs you might measure; how do you decide which levers to pull to generate better outcomes?
Before your ediscovery team commits to specific key performance indicators (KPI), it’s important to take stock of your operating model. Here are some things to keep in mind as you work through your initial assessment.
Metrics should be personal
The ultimate purpose of metrics is to solve problems for your ediscovery team. These challenges will be unique for every organization, so ask yourself: What causes our biggest headaches? What part of our approach is the most costly? Takes the most time? This will guide you to a starting point that is best for your team.
Target common issues
Not all problems are created equally; some have a disproportionate impact on the success of your team. This might be because they come up frequently or because so many other things are contingent upon them. Legal holds are one example. Bad legal hold processes can cause downstream impacts on collection, review, production, and presentation. Try to identify these kinds of recurrent roadblocks as a starting point.
Partner with IT
Once you’ve isolated a few high-impact areas of focus, you may discover that you don’t have the right data to measure them. IT should be your best friend on the data/metrics journey. Not only will they have a valuable perspective on technology-related challenges in ediscovery, but they’ll also be able to help you gather the information you need to measure performance and track improvement.
Monitor data on a regular cadence—but keep an eye on the big picture
When designing your metrics and reporting programs, it’s critical to monitor them consistently and often. Of course you want to identify opportunities for efficiency and improvement, but you also want to ensure you’re measuring the right things.
Monitoring for anomalies can point you toward solutions or alert you that you’re on the wrong track. But once you’ve established the right metrics to track, don’t get too bogged down in minor fluctuations. Instead, watch for long-term patterns.
Analyze patterns for underlying causes
This is where metrics create real improvements. The data can show you what’s happening, but you and your fellow practitioners will provide the necessary context to understand why it’s happening.
Say, for example, you notice that your per-GB processing costs have skyrocketed for a specific case. What are the variables that might have caused this? Does this matter have a unique data source or file type that’s more challenging to process? Is it something about your process that caused the change—and is there something you can do about it in the future? This kind of analysis will help you isolate operative variables, and your data can help you test the hypothesis.
Specific Metrics for Ediscovery Dashboards
Now that you’ve taken the time to develop your strategic approach to metrics and reporting, it’s time to build some practical dashboards. Again, the dashboards you choose will depend on your organization’s unique challenges and needs, but it’s helpful to have a sense for the kinds of things you might track.
Jennifer Courchaine, CEDS, Vice President of Customer Experience and Thought Leader at Zapproved, recommends the following categories and dashboards to help you track and improve a wide range of ediscovery processes.
Operations dashboards
These essential metrics are a great starting point to measure your team’s operational strengths and potential weaknesses:
- Overall Case Metrics
- Number of custodians (on preservation, with collected data, under active review)
- Number of active legal holds
- Average length of cases
- Amount of data in review
- Data Governance
- Catalog of legacy/retiring systems
- Catalog of incoming systems
- Systems on hold
- Costs
- Number of outsourced cases
- Size of outsourced cases
- Average length of outsourced cases
- Monthly service provider/outside counsel fees (by case type and by firm)
- Internal systems costs including maintenance
Risk management dashboards
Once you understand the metrics that matter for your legal operations, it’s important that you also establish measurements for risk, especially those most relevant in ediscovery: spoliation and excessive data.
Spoliation/Data Loss
Courchaine notes that, “If your challenges are more on the left of the EDRM, then you’re almost certainly going to want to focus on process management” when it comes to what you measure here. This means tracking how effective your systems are, adoption rates, and trainings:
- Training Programs
- Legal hold training as part of annual compliance training
- Legal hold training as part of new-hire onboarding
- Legal Hold Technology
- Standardized legal hold language
- Custodian compliance on legal holds
- Systemized Preservation In Place
- Automated preservation of core assets (Office365, Google Workspace, etc.)
Excessive Data
Excess data may come from a number of factors, including legacy system data and outdated deletion policies. Here’s some things to track to control these and other associated risks:
- New Systems
- Incoming systems
- Accessibility of data
- Retention policies
- Type of data/users (including how unique that data is)
- Legacy Systems
- Types of data/users (including how unique that data is)
- Accessibility of data
- Retention policies
- Planned obsolescence
- Data Outside of Standard Retention
- Amount
- Time frames/keywords to search
- Number of custodians (total and unique)
- Data Provided Externally
- Which firms have your data
- Volumes/unique data
- Firm destruction policies
Change management dashboards
As you grow more comfortable analyzing and using your metrics, you’ll begin to notice growth opportunities—new processes, new technology, or new roles that lead to efficiencies and cost savings. But that will also require excellent change management if you want to realize returns on your efforts.
Effective training and support is critical in this stage. Courchaine recommends tracking these variables related to change management:
- Administrative training
- Expected data
- Actual date
- On track: Yes/No
- Remediation plan (if necessary)
- User Training
- Expected date
- Actual Date
- On track: Yes/No
- Remediation plan (if necessary)
- First Case In Application
- Expected date
- Actual date
- On track: Yes/No
- Remediation plan (if necessary)
- Cost Effectiveness
- Cost per case prior to roll-out
- Expected cost per case post roll-out
- Actual cost per case post roll-out
Stakeholder visibility
Another important aspect of great metrics and reporting is that it provides a clear picture of ediscovery activity for stakeholders across the business. The following dashboards are useful for General Counsel, partners in IT or HR, and any other business units that may benefit from stronger ediscovery:
- Legal Holds
- Number of legal holds
- Number of custodians
- Number of unique custodians
- In-Place Preservation
- Volume of data overall and by matter
- Released custodians/standard retention
- Collected Data
- Volume of data overall and by matter
- Data in Review
- Number of reviewers
- Volume of data overall and by matter
- Process/System Changes
- Standard operating procedures, new technology implementation, additional insourcing
- Cost Reduction
- Overall spend, spend per matter, anomalies
- Risk Considerations
- Areas of greatest risk, user access audits, retention policies and process for released employee returning to standard retention, termed employee
Make Ediscovery Software Your Single Source of Truth for Metrics
Technology has made tracking metrics and improvement over time easier than ever. Zapproved’s ZDiscovery Platform can help you track important information across the ediscovery process, create reports and references for key stakeholders, and develop greater efficiency with automation and task-tracking.
