Kraft v. Essentia Health
Kraft v. Essentia Health
2022 WL 18621531 (D.N.D. 2022)
September 29, 2022
Senechal, Alice R., United States Magistrate Judge
Summary
The court ordered Essentia to review documents identified by nine searches, with the court ordering that Essentia review a minimum of three percent (3%) of the documents identified through each search and submit for in camera review a randomly selected minimum of three percent (3%) of the reviewed documents from each search which it asserts are nonresponsive. The court also denied plaintiffs' request for attorney fees, giving considerable weight to Essentia's assertion of estimated fees in connection with sampling.
Additional Decisions
Jessica Kraft, et al., Plaintiffs,
v.
Essentia Health, et al., Defendants
v.
Essentia Health, et al., Defendants
Case No. 3:20-cv-121
United States District Court, D. North Dakota
Filed September 29, 2022
Counsel
Scott A. Haider, McLain J. Schneider, Schneider, Schneider & Schneider, Fargo, ND, Brooke Achua, Pro Hac Vice, Melissa Ryan Clark, Pro Hac Vice, Fegan Scott LLC, New York, NY, Elizabeth Fegan, Fegan Scott LLC, Chicago, IL, J. Barton Goplerud, Pro Hac Vice, Hudson, Mallaney, Schindler, & Anderson P.C., Brian Marty, Pro Hac Vice, West Des Moines, IA, for Plaintiffs.Angela Elsperger Lord, MacKenzie Lee Hertz, Robert B. Stock, Vogel Law Firm, Fargo, ND, Briana L. Rummel, Vogel Law Firm, Bismarck, ND, for Defendants.
Peter W. Zuger, Ian Ronald McLean, Morgan Lyn Croaker, Serkland Law Firm, Fargo, ND, for Defendant Dakota Clinic Pharmacy, LLC.
Senechal, Alice R., United States Magistrate Judge
ORDER
*1 This is the second order addressing disputes about defendants’ obligations to search for electronically stored information responsive to plaintiffs’ discovery requests. As with the first motion, the parties’ briefs on this motion detail the history of the dispute at some length. Each side blames the other for the current dispute. The current dispute is compounded by defendants’ assertion of one or more privileges over the vast majority of ESI resources it has reviewed, resulting in production of a very small percentage of the ESI resources the search process has identified as responsive to plaintiffs’ discovery requests.[1] Rather than focusing on the parties’ competing rancor, the court focuses on the current dispute itself.
Background
This is a putative class action potentially involving more than 50,000 class members. Plaintiffs allege defendants Essentia Health and Innovis Health, LLC, (collectively, Essentia) sold and administered injectable pharmaceuticals—time and temperature sensitive pharmaceutical products (TTSPPs)—that had been stored outside the proper temperature range. Plaintiffs contend the improper storage of TTSPPs resulted in “reduced vaccine potency and the increased risk of vaccine-preventable diseases.” (Doc. 28, p. 2). Essentia's answer admits that in April 2020, it advised patients who had received certain TTSPPs that the TTSPPs may have been subjected to a temperature excursion, potentially rendering them less effective. (Doc. 30, p. 2). The TTSPPs in question were stored at Dakota Clinic Pharmacy (DCP), which is owned in part by Essentia.[2]
Broadly described, plaintiffs requested that Essentia produce documents showing (1) Essentia's organizational structure and its relationship to DCP and to other persons and entities involved in manufacturing of or administration of the TTSPPs, (2) Essentia's policies and practices regarding quality and safety of the TTSPPs, (3) all documents related to the alleged temperature excursion, (4) Essentia's internal and external communications concerning the alleged temperature excursion, and (5) records of patients who received the TTSPPs. (Doc. 158-3).
In their earlier motion, plaintiffs classified disputed searches as either “low-hit” or “high-hit,” with those classified as low-hit identifying 950 or fewer resources.[3] A February 25, 2022 order required Essentia to (1) review resources identified through fifteen low-hit searches and to produce responsive resources identified through those searches to plaintiffs; and (2) determine the number of unique resources that would be identified by sixteen high-hit searches, if those searches were modified in the manner plaintiffs had requested. (Doc. 173, p. 11). Additionally, the February 25 order directed plaintiffs to (1) consider possible additional modifications to proposed searches in order to address Essentia's stated objections and (2) prioritize their requests regarding the high-hit searches. Id. Essentia states it fully and timely complied with the February 25 order, (Doc. 208, p. 5), though plaintiffs disagree, (Doc. 205, p. 4).
*2 At some point subsequent to February 25, Essentia recognized various “technical issues” in the search process its vendor had been utilizing and repeated the searches it had performed earlier.[4] When repeated, some of the searches’ hit counts were significantly higher and some were significantly lower than on the first search. Essentia's brief describes a number of factors contributing to the hit count disparities.[5] Of the searches previously categorized as low-hit, Essentia's most recent data shows two searches (24 and 63) with hit counts significantly higher than 950, which was the highest number of hit counts for any of the searches identified as low-hit for purposes of the February 25 order. Essentia has not yet reviewed all resources now identified by searches 24 and 63.
In the current motion, plaintiffs seek an order addressing eleven proposed searches. In its brief, Essentia explained that after plaintiffs’ motion was filed, Essentia learned from its vendor that “quotation marks were not applied to certain split and hyphenated terms” in searches 64, 67, and 68 and that after correction of that error, searches 67 and 68 both returned fewer than 1,000 unique resources. (Doc. 208, p. 13 n.18). Essentia states it therefore no longer objects to reviewing resources identified by searches 67 and 68, so nine searches—rather than eleven—remain at issue. Essentia states search 64 identified 21,085 unique resources after the correction was made and maintains it objection as to review of resources identified by search 64. Id.
Essentia estimates it has reviewed over 100,000 resources identified through the search process to date but has not stated how many of those resources it considers responsive to plaintiffs’ discovery requests.[6] Essentia has asserted one or more privileges as to more than 9,000 documents. Id. at 2.
Plaintiffs request that Essentia be ordered to review resources returned by five searches: 12 (as to which plaintiffs propose a modification of the search prior to any review), 24, 52, 84, and 138. Plaintiffs propose search 12 be modified to substitute “w/250” for “AND,” asserting the modification would ensure references to “temperature” would appear closer to any references to the affected medications. (Doc. 205, p. 10).
As to the other remaining disputed searches (63-66), plaintiffs propose that (1) Essentia be ordered to review randomly selected subsets of resources returned by two of those searches; (2) Essentia provide plaintiffs examples of nonresponsive resources from each search; and (3) if Essentia believes the searches return an unreasonable number of nonresponsive resources, the parties then engage in discussion about potential modification of the searches still in dispute. Additionally, plaintiffs request an award of attorney fees incurred in connection with both this motion and their previous motion concerning ESI searches. Id. at 13-17.
*3 Essentia responds that it has agreed to review resources in every search that has currently resulted in identification of 1,000 or fewer unique resources, as well as the 1,078 resources identified by search 37. Id. at 7. Essentia calculates that task will entail review of an additional 9,522 unique resources. Further, Essentia contends the additional review of documents returned through searches 12 (without considering plaintiffs’ proposed modification), 24, 52, 84, and 138 would require review of 29,692 additional unique resources and estimates that review would result in attorney fees of more than $102,000. Id. at 8 n.10.
As to plaintiffs’ request for sampling of the resources identified by searches 63-66, Essentia states it previously agreed to do that, though the court understands Essentia did not agree to the details of sampling plaintiffs now request. Essentia contends the searches now in dispute duplicate those it has already run and argues the review it has performed to date confirms that the “vast majority” of responsive resources have already been identified and additional review would reveal excessive numbers of nonresponsive resources because plaintiffs propose “unreasonably broad” searches. Id. at 3, 7.
Plaintiffs’ brief includes the following description of the nine searches now at issue, including the hit count data Essentia provided for each of the disputed searches in August 2021 and the revised hit count data Essentia provided in June 2022:
(Doc. 205, pp. 6, 9-10) (tables combined and search numbers in numerical order).
Law and Discussion
The February 25 order described principles applying to keyword searching:
Though keyword searching is commonly used, it is not a perfect tool for distinguishing between relevant and irrelevant documents. In considering effectiveness of keyword searching, both recall and precision are important. Recall refers to whether, of the total number of relevant documents in a document collection, how many are correctly retrieved through the keyword search process. Precision refers to the percentage of retrieved documents that are actually relevant. As described in one commentary:
Importantly for the practitioner, there is typically a trade-off between precision and recall. One can often adjust a system to retrieve more documents—increasing recall—but the system achieves this result at the expense of retrieving more irrelevant documents—decreasing precision. Effectively, one can cast either a narrow net and retrieve fewer relevant documents, along with fewer irrelevant documents, or cast a broader net and retrieve more relevant documents, but at the expense of retrieving more irrelevant documents.
The Sedona Conference, Best Practices Commentary on the Use of Search & Information Retrieval methods in E-Discovery, 15 Sedona Conf. J. 217, 237-38 (2014) (hereafter “Best Practices”). “Whether search terms of ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.” United States v. O'Keefe, 537 F. Supp. 2d 14, 24 (D.D.C. 2008). Developing and implementing effective search terms, or search strings, requires careful planning by persons with an understanding of the process and of the product or vendor used to implement the search process. To effectively identify relevant ESI, search strings must be carefully composed and multiple levels of searches may be necessary. E.g., UnitedHealthcare of Fla., Inc. v. Am. Renal Assocs. LLC, No. 16-cv-81180, 2017 WL 4785457 (S.D. Fla. Oct. 20, 2017). Ideally, the parties would cooperatively develop the procedures and search strings to be utilized. As described in the ESI Order,
*4 The parties are to meet and confer concerning search methodologies, including without limitation, the use of keyword search terms and/or the use of technology assisted review....
Each responding party will collect documents from custodians and from the locations agreed to between the parties or pursuant to Court order.
The parties are to discuss and attempt to reach an agreement on search methodologies. Agreement on a search methodology does not relieve a party of its obligations under the Federal Rules of Civil Procedure to conduct a reasonable search, interview relevant witnesses and custodians, and produce all relevant and responsive documents of which it is aware, regardless of whether such documents are located with the agreed-upon search methodology, and without waiting for an agreement on or implementation of such search methodology.
The parties are to meet and confer about methods to search ESI in order to help identify ESI that is subject to production in discovery and filter out ESI that is not subject to discovery. The parties are to establish search methodologies with the goal of limiting the scope of review for production, minimizing the need for motion practice, and facilitating production in accordance with the deadlines set by the Court or agreed upon by the parties....
The parties will meet and confer to discuss search validation provisions, regardless of the search methodology implemented.
(Doc. 173, pp. 4-5). Like the earlier search term dispute, this matter is at its core a question of application of Federal Rule of Civil Procedure 26(b)(1).
1. Federal Rule of Civil Procedure 26(b)(1) Analysis
Rule 26(b)(1) defines the scope of permissible discovery:
Unless otherwise limited by court order, the scope of discovery is as follows: Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party's claim or defense and proportional to the needs of the case, considering the importance of the issues at stake in the action, the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit. Information within this scope of discovery need not be admissible in evidence to be discoverable.
As movants, plaintiffs have the burden of showing relevance and proportionality of the discovery they seek. Essentia, the party resisting discovery, has the burden of showing the requested discovery outweighs its likely benefit. See Vallejo v. Amgen, Inc., 903 F.3d 733, 742 (8th Cir. 2018); see also NoSpill, LLC v. Scepter Can., Inc., No. 18-cv-2681-HLT-KGG, 2021 WL 4860665 (D. Kan. Oct. 19, 2021); Hall v. Rent-A-Center, Inc., No. 4:16cv978, 2018 WL 4293141, at *3 (E.D. Tex. Aug. 31, 2018).
The court's analysis of five of the six Rule 26 factors has not changed since the February 25 order was issued. As in the earlier motion, given the putative class numbering over 50,000 health care consumers, the court views the importance of the issues as weighing in plaintiffs’ favor. Likewise, given plaintiffs’ assertion that potential damages of the putative class total tens of millions of dollars, the court views the amount in controversy as weighing in plaintiffs’ favor. Other than through discovery, plaintiffs have no access to the information now at issue, so that factor continues to weigh strongly in plaintiffs’ favor. Since the court has no evidence of resources of either plaintiffs or of Essentia, the court does not consider the parties’ resources. And importance of the discovery in resolving issues in the case continues to weigh in plaintiffs’ favor.
*5 Thus, the crux of the Rule 26(b)(1) analysis is again whether the burden or expense of the proposed discovery outweighs its likely benefit, i.e., what is the likelihood that review of resources identified through the nine disputed searches will result in production of additional responsive resources and, if so, at what cost to Essentia.
Plaintiffs state they “have no confidence that Essentia has applied search terms accurately and will have to rely on a to-be-determined search validation method that Essentia refused to negotiate at the outset of this case.” (Doc. 205, p. 15). The repeated problems with Essentia's search process and resulting changes in reported hit counts for some of the searches are very troubling to the court. Nonetheless, the court must accept Essentia's most recent hit count data as the best data currently available.
A. Search 12
Plaintiffs assert search 12 is “highly relevant” since it seeks resources regarding temperature guidelines and records for “Affected Medications” at the heart of the case.[7] Id. at 9. Essentia argues inclusion of words such as “med!” and “inject!” in search 12 result in identification of an excessive number of resources. Essentia also asserts it has already reviewed searches that target temperature guidelines and records for the affected medications. (Doc. 208, p. 11). It appears, however, Essentia has not run plaintiffs’ proposed modified search 12. Essentia will be ordered to do so within ten days and to report the results to plaintiffs within fourteen days.
B. Searches 24, 52, 84, and 138
According to Essentia's latest data, searches 24, 52, 84, and 138 returned hit counts ranging from 1,743 to 3,061 and the total number of unique resources identified by those four searches is 10,687. Essentia estimated the cost of reviewing 29,692 documents would be more than $102,000, and the court infers review of 10,687 of those 29,692 documents would result in fees of approximately 36% of that estimate or almost $37,000.
Plaintiffs describe search 24 as “plainly relevant,” seeking “documents about temperature logging, requirements, contingencies, and parameters, for example, all of which target documents about Essentia's years of failing to properly maintain, monitor, or document medication temperatures.” (Doc. 205, pp. 10-11). In their earlier briefing, plaintiffs described search 24 as “calculated to capture topics and events at the very heart of this case.” (Doc. 158, p. 10). Given the importance of the search, the court finds plaintiffs have met the burden of showing relevance and proportionality. Essentia did not specifically address the burden of reviewing only resources identified by search 24. The current data shows a hit count of 2,840 for search 24, and the court infers Essentia would incur less than $10,000 in attorney fees to review those 2,840 resources. But Essentia states its sampling of resources identified by search 24 confirms the identified resources are “excessively nonresponsive.” (Doc. 208, p. 12). As to search 24, Essentia will be required to provide examples of non-responsive resources for in camera review.
*6 As to searches 52, 84, and 138, plaintiffs state each is “very targeted” to the facts of the case and each reaches “multiple core topics” that could have been broken into multiple searches to result in lower hit counts. (Doc. 205, pp. 8-9). The current hit count data shows the resources identified by the three searches total 7,847. The court infers Essentia would incur approximately $27,000 in attorney fees in reviewing all resources identified by the three searches. Essentia suggests it provide examples of non-responsive resources for in camera review. (Doc. 208, p. 12 n.15). The court will order that it do so.
C. Searches 63-66
Current data shows hit counts ranging from 21,085 to 77,715 for searches 63-66. See id. at p. 13 n.18 (describing search 64). Essentia had initially provided hit counts ranging from 6 to 547 for those same searches, but when issues regarding use of wildcards were corrected, the hit count numbers increased astronomically.
Plaintiffs describe this group of searches as relating to location of the affected medications, using either specific drug names or categorical names for the affected medications. Plaintiffs propose statistically random sampling of the resources identified by these searches. Essentia states it has agreed to sample resources returned by searches 63-66, though it objects to the extent of sampling plaintiffs propose. Essentia estimates that the sampling plaintiffs propose would result in attorney fees of $7,000. Id. at 8 n.10. Essentia will be required to perform more limited sampling and to submit examples of non-responsive resources for in camera review as described below.
2. Sampling
Plaintiffs request that Essentia be required to review 1,000 randomly selected documents from each of two searches—searches 63 and 65—and that Essentia be required to produce responsive documents identified by the two searches. Further, plaintiffs request that Essentia provide data regarding the number of responsive documents identified and at least ten examples of non-responsive documents identified by each search, “with a goal toward selecting examples that are representative of categories of irrelevant documents so that the parties can assess ways to exclude such categories of documents.” (Doc. 205, p. 17). If after completing that sample review, Essentia believes an unreasonable number of nonresponsive documents are identified by the two searches, plaintiffs ask that Essentia be required to propose modifications to the searches and confer with plaintiffs to discuss possible modifications of the two search terms.
Essentia responds that it has agreed to sample resources recalled by its “Rerun Searches and to provide examples of the nonresponsive results as required by the ESI order.” (Doc. 208, p. 13). The ESI order does not specifically address sampling or providing examples of documents Essentia deems nonresponsive. Nor does Essentia specifically identify the extent of sampling it has agreed to undertake.
This order requires Essentia to sample resources identified through eight of the nine disputed searches—24, 52, 63, 64, 65, 66, 84, and 138. Essentia's most recent hit-count data for those eight searches totals 182,645 resources. Accepting and inferring from Essentia's attorney fee estimates, ordering review of even ten percent of the documents would result in Essentia incurring more than $50,000 in attorney fees. Given Essentia's assertion of “excessively nonresponsive” documents having been returned, Essentia has demonstrated burdensomeness of plaintiffs’ request for sampling.
Case law provides little guidance for determining an appropriate scope of required sampling. The court gives considerable weight to Essentia's assertion of estimated fees in connection with sampling. At this time, the court will order that Essentia review a minimum of three percent (3%) of the documents identified through each of the eight searches identified in the previous paragraph and that it submit for in camera review a minimum of three percent (3%) of the reviewed documents from each search which it asserts are nonresponsive. Selection of the documents reviewed, and of the documents submitted for in camera review, must be randomized within each search.
3. Plaintiffs’ Request for Attorney Fees
*7 Federal Rule of Civil Procedure 37(a)(5)(7) provides that if a motion to compel “is granted in part and denied in part, the court may ... after giving an opportunity to be heard, apportion the reasonable expenses for the motion.” “In determining a reasonable apportionment of fees, the court will look to the relative degree of success of the party seeking fees.” Est. of Wright v. Forgey, No. 2:13-CV-333, 2016 WL 2956699, at *2 (N.D. Ind. May 23, 2016) (citation and internal quotation marks omitted). “The court also will look to the degree to which the objecting party was justified in refusing greater cooperation.” Id. “District courts possess wide latitude in ... evaluating the reasonableness of attorney's fees requested.” Id. Given disposition of the motion, the court will not order payment of attorney fees at this time.
Conclusion
This motion requires the court to consider the burden plaintiffs’ requested resolution would impose on Essentia. That task is made more difficult by the repeated issues involving Essentia's search process to date and by Essentia's broad assertions of privilege. At this time, the court orders Essentia to take the following actions:
As to search 12, run plaintiffs’ proposed modification within ten days of the date of this order and report the results to plaintiffs within fourteen days;
As to searches 24, 52, 63, 64, 65, 66, 84, and 138, within thirty days of the date of this order, randomly select and review a minimum of three percent (3%) of the documents identified through each search and submit for in camera review a randomly selected minimum of three percent (3%) of the reviewed documents from each search which it asserts are nonresponsive.
IT IS SO ORDERED.
Dated this 29th day of September, 2022.
Footnotes
A separate motion challenging defendants’ privilege assertions is also pending. (Doc. 160).
Dakota Clinic Pharmacy has been dismissed from the action. (Doc. 56).
The earlier order referred to disputed “search strings.” Because the parties’ briefs refer to “searches” or “search terms,” this order uses the parties’ terminology.
Essentia states it “took it upon itself to rerun the original version of every search” after learning of the issues with its search process. (Doc. 208, p. 6).
Changes in hit count data appear to have resulted from several factors, including restriction or confusion over use of numerical connectors, resources Essentia had already reviewed not being excluded when searches were repeated, and an inappropriate application of wildcard terms. (Doc. 208, p. 6). Essentia also states that its current hit counts, unlike the original hit counts, include related resources such as email attachments. Id. at 2 n.1.
Essentia asserts minutes of a July 19, 2022 status conference noted it had reviewed “over 10,000 documents” but that the “estimated number is indeed closer to 100,000.” (Doc. 208, p. 2 n.1). On review of the recording of that conference, the court notes Essentia's counsel stated it had reviewed “thousands of documents, frankly well over 10,000 documents.”
Plaintiffs define “Affected Medications” as “the TTSPPs subject to the Disclosed Temperature Excursion,” i.e., “the Temperature Excursion ... publicly disclosed by Essentia in or about April 2020.” (Doc. 148-2, pp. 3-4).