Kraft v. Essentia Health
Kraft v. Essentia Health
2022 WL 16699450 (D.N.D. 2022)
February 25, 2022
Senechal, Alice R., United States Magistrate Judge
Summary
The court granted the plaintiffs' motion to compel Essentia to review documents identified through fifteen low-hit search strings and to produce the identified documents which are responsive to plaintiffs' requests for production. The court also directed plaintiffs to consider possible additional modifications to their proposed search strings and prioritize their requests as to the proposed high-hit search strings. The documents in question are ESI, and the court found the search strings to be sufficiently narrowed, relevant, and proportional to the needs of the case.
Additional Decisions
Jessica Kraft, et al., Plaintiffs,
v.
Essentia Health, et al., Defendants
v.
Essentia Health, et al., Defendants
Case No. 3:20-cv-121
United States District Court, D. North Dakota
Signed February 25, 2022
Counsel
McLain J. Schneider, Schneider, Schneider & Schneider, Fargo, ND, Brian Marty, Pro Hac Vice, J. Barton Goplerud, Pro Hac Vice, Hudson, Mallaney, Schindler, & Anderson P.C., West Des Moines, IA, Elizabeth Fegan, Fegan Scott LLC, Chicago, IL, Brooke Achua, Melissa Ryan Clark, Pro Hac Vice, Fegan Scott LLC, New York, NY, for Plaintiffs.Briana L. Rummel, Vogel Law Firm, Bismarck, ND, Robert B. Stock, MacKenzie Lee Hertz, Angela Elsperger Lord, Vogel Law Firm, Fargo, ND, for Defendants.
Senechal, Alice R., United States Magistrate Judge
ORDER
*1 In this putative class action, plaintiffs allege defendants Essentia Health and Innovis Health, LLC (Essentia) sold and administered injectable pharmaceuticals—time and temperature sensitive pharmaceutical products (TTSPPs)—that had been stored outside the proper temperature range. Plaintiffs contend the improper storage of TTSPPs resulted in “reduced vaccine potency and the increased risk of vaccine-preventable diseases.” (Doc. 28, p. 2). Essentia's answer admits that in April 2020, it advised patients who had received certain TTSPPs that the TTSPPs may have been subjected to a temperature excursion, potentially rendering them less effective. (Doc. 30, p. 2). The TTSPP's in question were stored at defendant Dakota Clinic Pharmacy (DCP), which is owned in part by Essentia.
Plaintiffs move to compel Essentia to apply certain search strings in its identification of electronic documents responsive to plaintiffs’ requests for production of documents and to produce documents responsive to those search strings. Essentia contends review of documents identified by the disputed search strings would be unduly burdensome and disproportionate to the needs of the case. Further, Essentia argues the searches it has conducted to date satisfy its obligations under a March 1, 2021 Order Governing Discovery of Electronically Stored Information (ESI Order). (Doc. 46). Both plaintiffs and Essentia describe their disputes concerning search strings at some length, but rather than tracing the source of the current dispute, the court focuses on the dispute itself.[1]
Broadly described, plaintiffs requested that Essentia produce documents showing (1) Essentia's organizational structure and its relationship to DCP and to other persons and entities involved in manufacturing of or administration of the TTSPPs, (2) Essentia's policies and practices regarding quality and safety of the TTSPPs, (3) all documents related to the alleged temperature excursion, (4) Essentia's internal and external communications concerning the alleged temperature excursion, and (5) records of patients who received the TTSPPs. (Doc. 158-3).
To respond to plaintiffs’ requests for production of documents, Essentia used an electronic document search process. Pursuant to the ESI Order, Essentia collected documents from 24 record custodians and identified locations of documents of those custodians. That resulted in identification of a pool of 2,917,752 total files. (Doc. 163, p. 2). The parties discussed various proposed search strings but did not reach agreement on all proposals. Essentia developed and applied 8 search strings which identified 5,951 “unique resources.” Id. at 6. A unique resource is one that is responsive to one search string but not responsive to any other searches in the system. Id. at 2. Unique resource counts do not include related resources, for example, attachments to emails.
*2 In total, plaintiffs proposed 138 search strings. Essentia ran each of the proposed search strings and reviewed all unique resources identified in response to 108 search strings proposed by plaintiffs and the 8 search strings Essentia developed on its own. Id. Essentia provided a hit list showing the number of unique responsive resources for its 8 search strings and for plaintiffs’ 138 proposed search strings. (Doc. 162, p. 4). Forty-seven of plaintiffs’ 138 proposed search strings identified no unique resources. Id. at 8 n.7.
Thirty-one of plaintiffs’ proposed search strings remain in dispute, 15 of which plaintiffs characterize as “low-hit searches.” (Doc. 158-2). As to those 15 search strings, plaintiffs contend “the number of unique documents is so small that including them creates little burden and removing them would have no material impact on Essentia's review.” (Doc. 158, p. 9). Each of those 15 low-hit search strings yielded fewer than 950 unique resources. (Doc. 162-1).
With their motion, plaintiffs filed a chart detailing asserted relevance of each of the 15 low-hit search strings. (Doc. 158-2). Essentia responded with a chart detailing asserted lack of relevance, (Doc. 162-1), and another chart describing its objections to plaintiffs’ proposed search strings and describing any modified versions of the proposed search strings Essentia has run. (Doc. 163-2). With their reply, plaintiffs filed yet another chart addressing differences between their requested search strings and search strings Essentia considers to be alternatives to plaintiffs’ requests. (Doc. 167-1).
A declaration of Essentia's counsel states the 31 disputed search strings encompass 152,601 unique resources and estimates Essentia would incur $1,150,663 in attorney fees for review of those unique resources, without considering the time necessary to review related resources. (Doc. 163, p. 6). Plaintiffs respond that Essentia's estimate is based on an unreasonable length of time for review of each document. (Doc. 167, p. 5 n.2).
Law and Discussion
There is no question that a process of keyword searching can satisfy a party's obligation to search for documents responsive to a request for production under Federal Rule of Civil Procedure 34. Were that not true, Essentia could be required to review each of the 2,917,752 documents in the identified pool.
Though keyword searching is commonly used, it is not a perfect tool for distinguishing between relevant and irrelevant documents. In considering effectiveness of keyword searching, both recall and precision are important. Recall refers to, of the total number of relevant documents in a document collection, how many are correctly retrieved through the keyword search process. Precision refers to the percentage of retrieved documents that are actually relevant. As described in one commentary:
Importantly for the practitioner, there is typically a trade-off between precision and recall. One can often adjust a system to retrieve more documents—increasing recall—but the system achieves this result at the expense of retrieving more irrelevant documents—decreasing precision. Effectively, one can cast either a narrow net and retrieve fewer relevant documents, along with fewer irrelevant documents, or cast a broader net and retrieve more relevant documents, but at the expense of retrieving more irrelevant documents.
The Sedona Conference, The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 15 Sedona Conf. J. 217, 237-38 (2014). “Whether search terms of ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.” United States v. O'Keefe, 537 F.Supp. 2d 14, 24 (D.D.C. 2008). Developing and implementing effective search terms, or search strings, requires careful planning by persons with an understanding of the process and of the product or vendor used to implement the search process. To effectively identify relevant ESI, search strings must be carefully composed and multiple levels of searches may be necessary. E.g., UnitedHealthcare of Fla., Inc. v. Am. Renal Assocs. LLC, No. 16-cv-81180, 2017 WL 4785457 (S.D. Fla. Oct. 20, 2017). Ideally, the parties would cooperatively develop the procedures and search strings to be utilized. As described in the ESI Order,
*3 The parties are to meet and confer concerning search methodologies, including without limitation, the use of keyword search terms and/or the use of technology assisted review....
Each responding party will collect documents from custodians and from the locations agreed to between the parties or pursuant to Court order.
The parties are to discuss and attempt to reach an agreement on search methodologies. Agreement on a search methodology does not relieve a party of its obligations under the Federal Rules of Civil Procedure to conduct a reasonable search, interview relevant witnesses and custodians, and produce all relevant and responsive documents of which it is aware, regardless of whether such documents are located with the agreed-upon search methodology, and without waiting for an agreement on or implementation of such search methodology.
The parties are to meet and confer about methods to search ESI in order to help identify ESI that is subject to production in discovery and filter out ESI that is not subject to discovery. The parties are to establish search methodologies with the goal of limiting the scope of review for production, minimizing the need for motion practice, and facilitating production in accordance with the deadlines set by the Court or agreed upon by the parties....
The parties will meet and confer to discuss search validation provisions, regardless of the search methodology implemented.
(Doc. 46, pp. 4-7). The court then considers whether plaintiffs’ proposed search strings are within the scope of discovery allowed by Federal Rule of Civil Procedure 26(b)(1).
1. Federal Rule of Civil Procedure 26(b)(1) Analysis
At its core, this dispute is governed by Rule 26(b)(1)’s definition of the scope of permissible discovery:
Unless otherwise limited by court order, the scope of discovery is as follows: Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party's claim or defense and proportional to the needs of the case, considering the importance of the issues at stake in the action, the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit. Information within this scope of discovery need not be admissible in evidence to be discoverable.
As movants, plaintiffs have the burden of showing relevance and proportionality of the discovery they seek. Essentia, the party resisting discovery, has the burden of showing the burden of the requested discovery outweighs its likely benefit. See Vallejo v. Amgen, Inc., 903 F.3d 733, 742 (8th Cir. 2018); see also NoSpill, LLC v. Scepter Canada, Inc., No. 18-cv-2681-HLT-KGG, 2021 WL 4860556 (D. Kan. Oct. 19, 2021); Hall v. Rent-A-Center, Inc., No. 4:16cv978, 2018 WL 4293141, at *3 (E.D. Tex. Aug. 31, 2018).
Rule 26 directs consideration of the importance of issues at stake in the action. Though Essentia characterizes the case as “in simplest terms ... a putative class action between private parties by which the Plaintiffs seek money damages,” (Doc. 162, pp. 11-12), the putative class numbers over 50,000 health care consumers. The court views that factor as weighing in plaintiffs’ favor.
*4 The parties have very different views of the amount in controversy. Plaintiffs assert potential damages of the putative class totaling tens of millions of dollars while Essentia views total potential damages as a small fraction of that amount. The court views the amount in controversy as weighing in plaintiffs’ favor.
Rule 26 also considers relative access to information in assessing proportionality. Essentia points to the more than 3,500 pages of documents it has produced to date and the more than 8,000 responsive documents over which it asserts privilege as showing lack of information asymmetry. The court cannot accept that assertion. Rule 26 considers relative access to information, not the volume of documents one party may have produced. See No Spill, 2021 WL 4860556, at *8. Other than through discovery, plaintiffs have no access to the information now at issue. That factor weighs strongly in plaintiffs’ favor.
As to the parties’ resources, plaintiffs argue Essentia is a well-resourced health system and suggest that factor weighs in plaintiffs’ favor. Essentia, citing Lawson v. Spirit AeroSystems, Inc., argues a party's resources cannot justify addressing unlimited discovery requests to that party. No. 18-1100-EFM-ADM, 2020 WL 3288058 (D. Kan. June 18, 2020), aff'd, 2020 WL6939752 (D. Kan. Nov. 24, 2020). But the court has no evidence of resources of either plaintiffs or of Essentia and so does not consider that factor to weigh in either side's favor.
As to the importance of the discovery in resolving issues in the case, the information plaintiffs seek in this motion concerns a central issue—Essentia's liability. That factor weighs strongly in plaintiffs’ favor.
Finally, Rule 26(b)(1) directs the court to consider “whether the burden or expense of the proposed discovery outweighs its likely benefit.” In analyzing that factor, the court separately considers the proposed low-hit search strings and the proposed high-hit search strings.
2. Plaintiffs’ Proposed Low-Hit Search Strings
Plaintiffs assert the 15 disputed low-hit search strings would each yield a number of unique resources “so small that including them creates little burden and removing them would have no material impact on Essentia's review.” (Doc. 158, p. 9). Essentia responds that the 5,750 unique resources—not including related documents—identified by the low-hit search strings total nearly as many unique resources as were reviewed through Essentia's original searches. Further, Essentia argues the low-hit searches are not sufficiently tailored and seek information targeted by searches Essentia has already performed. (Doc. 162, pp. 13-14). The critical question is whether the additional search strings plaintiffs propose are likely to produce relevant documents not already produced.
Essentia did not separately estimate fees it would incur for review of documents identified through the low-hit search strings. It states 5,750 unique resources were identified through the low-hit search strings and 146,311 unique resources were identified through the high-hit search strings. Assuming the same number of documents reviewed per hour, review of 5,750 unique resources identified through the low-hit search strings would require less than 5% of the time required to review 146,311 unique resources identified through the high-hit search strings. In their reply, plaintiffs argue that, even at “Essentia's below-average estimate” of documents that could be reviewed per hour, review of documents identified through the low-hit search strings could be accomplished by two reviewers in fewer than two-and-one-half weeks. From plaintiffs’ perspective, that time is reasonable and proportionate to the needs of the case in light of the terms included in the disputed low-hit search strings. (Doc. 167, p. 2).
*5 Essentia describes building of search strings as a complex technical process that is time intensive and expensive. (Doc. 162, p. 4 n.1).[2] Essentia asserts aspects of its search platform make some of plaintiffs’ proposed search strings impractical, specifically that its platform allows application of only one numerical connector[3] per search string. In running plaintiffs’ proposed search strings, Essentia used “or” or “and” as connectors rather than the numerical connectors plaintiffs had proposed. Plaintiffs respond that none of the disputed search strings include more than one numerical connector. Essentia's response did not explain how the limitation on numerical connectors restricts its searches, given that none of the disputed search strings includes more than one numerical connector.
According to Essentia, the total unique resource count is underinclusive of the total number of resources it reviewed because its review included related resources, e.g., attachments to emails. Essentia cites an approximate 30% increase in resources returned by the 8 searches it initially ran when related resources are included in the number of documents it reviewed. Further, Essentia estimates the 8 search strings it developed “account for over 90 percent of the responsive, but privileged, resources in this case.” (Doc. 163, p. 6).
Essentia describes qualitative sampling it did of documents identified by the disputed search strings and contends the disputed search strings recovered “excessive numbers of non-responsive documents.” (Doc. 162, p. 5). Essentia states it provided plaintiffs examples of the non-responsive documents it had discovered and offered to provide those documents for the court's review. Id. In Essentia's view, the information that could be gathered through use of the disputed low-hit search strings “has already been targeted by more appropriately tailored searches that Essentia has already performed.” Id. at 14.
As to some of the disputed low-hit search strings, Essentia asserts tenuous relevance because DCP rather than Essentia was responsible for TTSPP storage. But, as plaintiffs reply, there is a close relationship between Essentia and DCP: Essentia owns part of DCP, Essentia identified the pharmacist-owner of DCP as the pharmacy manager for Innovis Health, LLC—which owns Essentia—and any temperature excursion impacted Essentia's patients.
Considering all the Rule 26 factors, the court concludes the disputed low-hit search strings are sufficiently narrowed, relevant, and proportional to the needs of the case. Essentia will be ordered to review the documents identified through each of the fifteen low-hit search strings and to produce the identified documents which are responsive to plaintiffs’ requests for production.
3. Plaintiffs’ Proposed High-Hit Search Strings
When Essentia ran the 16 high-hit search strings, identified unique responses ranged from 1,184 to 110,123. (Doc. 163-2). Review of the 146,311 unique resources identified through the original disputed high-hit search strings would entail significant expense to Essentia. Plaintiffs’ proposed modification of some of the high-hit search strings, but the court understands from the parties’ briefing that Essentia has not run all of the modified searches so the number of unique responses to each is not yet known. Review of plaintiffs’ modifications to the high-hit search strings confirms that none include more than one numerical connector, and impact of the “one numerical connector” restriction of Essentia's search platform is therefore not apparent to the court.
*6 Because of those uncertainties, the court will not order that Essentia produce documents responsive to the sixteen proposed high-hit search strings at this time. The court will, however, order that Essentia determine the number of unique resources that would be identified through each of those search strings, as modified by plaintiffs, and provide that information to plaintiffs. As it concerns plaintiffs’ high-hit search strings, the motion to compel will be denied without prejudice to filing a later motion after Essentia has provided data on unique resources identified through plaintiffs’ modified proposed high-hit search strings. In the interim, plaintiffs are directed to consider possible additional modifications to their proposed search strings that would address Essentia's stated objections. Plaintiffs are further directed to prioritize the search strings. It is unlikely, for example, that the court would order that Essentia review documents in a search string such as #52, which identified 110,123 unique resources.
Conclusion
For the reasons discussed above, plaintiffs’ motion concerning search terms, (Doc. 157), is GRANTED IN PART. Within 30 days of the date of this order, Essentia is directed to review the documents identified through each of the 15 low-hit search strings and to produce the identified documents which are responsive to plaintiffs’ requests for production.
Concerning plaintiffs’ proposed high-hit search strings, within 30 days Essentia is directed to determine the number of unique resources that would be identified through each of those search strings, as modified by plaintiffs, and provide that information to plaintiffs. In the interim, plaintiffs are directed to consider possible additional modifications to their proposed search strings that would address Essentia's stated objections and to prioritize their requests as to the proposed high-hit search strings.
To the extent plaintiffs’ motion is DENIED, that ruling is without prejudice to plaintiffs filing a later motion after the steps described above have been completed.
IT IS SO ORDERED.
Dated this 25th day of February, 2022.
Footnotes
The parties participated in a dispute resolution conference as required by Civil Local Rule 37.1, and following that conference, the court authorized plaintiffs to file the current motion. (Doc. 116, p. 2).
Essentia states it ran all 138 search strings that plaintiffs originally proposed. (Doc. 162, p. 8). Essentia has not explained any additional “complex technical process” that might be required if the court were to order review of documents identified through plaintiffs’ originally proposed search strings, though the court recognizes that additional search strings would need to be developed to address any modifications to the search strings that have already been run.
A numerical connector is a search function that requires search terms to appear within a specified number of terms of each other, e.g., “injectable w/3 medication.” (Doc. 162, p. 4 n.3).