*1 This multi-district litigation concerns an alleged conspiracy to reduce supply and increase prices for methylene diphenyl diisocyanate (“MDI”) and toluene diisocyanate (“TDI”), precursor ingredients for the manufacture of polyurethane foam and thermoplastic polyurethanes. The parties have undertaken extensive discovery thus far and currently are operating under discovery and other deadlines set forth in the Court's Case Management Order entered on April 27, 2022 (ECF No. 653) and modified on March 23, 2023 (ECF No. 890). Presently before the Court are Plaintiffs’ Responses and Limited Objections (ECF No. 804) to the Special Master's most recent report and recommendation pertaining to Plaintiffs’ motion to compel Defendants to use certain search terms and technology assisted review methodologies for their discovery productions.

I. BACKGROUND[1]

Plaintiffs’ pending objections are the latest in an ongoing series of discovery challenges involving Plaintiffs’ dissatisfaction with Defendants’ search terms and technology-assisted review, or “TAR,” methodologies. Plaintiffs filed their first Motion to Compel Defendants to Use Certain Search Terms and TAR Methodologies on March 25, 2021. (ECF No. 455). Defendants cross-moved for a protective order to proceed with the search terms and TAR protocols they had selected. (ECF Nos. 470, 471). After extensive briefing, expert reports, expert testimony, and oral argument, Special Master Hon. James C. Francis IV (Ret.) recommended denying both motions, determining, inter alia, that Defendants’ proposed methodologies contain serious flaws that would preclude them from certifying their discovery responses to be reasonable under Rule 26(g), and that Plaintiffs “failed to demonstrate that their proposed search terms are the only reasonable option.” (ECF No. 529 (In re Diisocyanates Antitrust Litigation, Misc. No. 18-1001, 2021 WL 4295729, at *11-13 (W. D. Pa. Aug. 23, 2021)) (“First R&R”)). Plaintiffs objected. Defendants, for their part, did not object and instead unilaterally modified their TAR procedures to address concerns expressed by the Special Master. (ECF Nos. 532; 532-6; 899, ¶ 14). The Court overruled Plaintiffs’ objections[2] and adopted the First R&R on September 21, 2021. (ECF No. 549). In adopting the First R&R, the Court directed Defendants to proceed as they proposed, outlined a process for Defendants to validate the completeness of their TAR searches, and afforded Plaintiffs an opportunity to seek judicial review if they disagreed with Defendants’ assessment as to when their searches were complete. (ECF No. 549).

Defendants subsequently performed testing to identify search terms that would capture sufficient responsive documents and to determine whether using disputed search terms would improve the search efficacy. (ECF Nos. 604, ¶¶ 32-34; 606, ¶¶ 11-25; 607, ¶¶ 14-22; 608, ¶¶ 24-30; 609, ¶¶ 9-18). In doing so, the parties engaged in extensive conferrals, leading Defendants to add several search terms, including certain of Plaintiffs’ proposed terms without modification and adopting some of Plaintiffs’ proposed terms in modified formats. (ECF No. 584, ¶ 16). Unsatisfied, however, Plaintiffs continued pressing for a substantially broader set of search terms and search strings and sought a new hearing with the Special Master, prompting Defendants to move for a protective order permitting them to use their selected terms (ECF Nos. 582, 583), while Plaintiffs moved to compel Defendants to use Plaintiffs’ search terms (ECF Nos. 589, 592). These motions were referred to the Special Master. (ECF No. 596).

*2 Upon review, the Special Master recommended that Plaintiffs’ second motion to compel be denied and that Defendants’ motion for protective order be denied as moot. (ECF No. 611 (In re Diisocyanates Antitrust Litigation, Misc. No. 18-1001, 2022 WL 173678 (W.D. Pa. Jan. 7, 2022)) (“Second R&R”)). The Special Master determined that the Court's prior Order adopting the First R&R foreclosed further adjudication of the search term dispute at that juncture but did not “immunize the [Defendants’] production from challenge at the end of the day,” and that Plaintiffs would have an opportunity to challenge both the search terms selected by Defendants and the point at which Defendants chose to halt the TAR process based on the validation information that Defendants had committed to providing. (Second R&R at *4). Neither Plaintiffs nor Defendants lodged objections,[3] and the Court adopted the Special Master's Second R&R in its entirety on January 19, 2022. (ECF No. 619 (In re Diisocyanates Antitrust Litigation, Misc. No. 18-1001, 2022 WL 170036 (W.D. Pa. Jan. 19, 2022) (adopting the Second R&R and stating that “at the appropriate time, Plaintiffs will be permitted to challenge search term inadequacies they believe rendered the search unreasonable”))).

Thereafter, Defendants substantially completed their productions and provided Plaintiffs with information regarding the validation of their TAR methodologies and search terms.[4] (ECF No. 707, ¶¶ 22-23 & Exhibits 3-8 (ECF Nos. 707-4 to 707-9)). Consistent with the Second R&R and the Court's Order adopting it, Defendants stopped their TAR review after achieving recall rates[5] of at least 70%. By way of example, WCA stopped its TAR review upon estimating that it had located 73.81% of the responsive documents and the responsiveness rate of the final batches was under 15%. (ECF No. 899, ¶ 22).

Unsatisfied, Plaintiffs conferred with Defendants over the course of several weeks. (ECF No. 707, ¶¶ 35, 38). Then, upon reaching impasse, Plaintiffs filed their third motion to compel, disputing yet again the adequacy of Defendants’ search terms and challenging whether Defendants’ TAR searches were reasonably complete. (ECF No. 705 (Plaintiffs’ Motion to Compel Defendants to Use Certain Search Terms and Further TAR Review)). Once again, the Court referred this motion to the Special Master (ECF No. 713), who issued a Report and Recommendation recommending that it be granted only to the extent that Defendants WCA, Covestro, and Huntsman be required to resume their TAR review, but that it be denied in all other respects. (ECF No. 798 (In re Diisocyanates Antitrust Litigation, Misc. No. 18-1001, 2022 WL 17668470 (W.D. Pa. Oct. 19, 2022)) (“Third R&R”)).[6]

*3 In formulating his recommendations in the Third R&R, the Special Master referenced his First and Second R&Rs, and then stated that the “time has come” to address Plaintiffs’ challenge to Defendants’ search terms, concluding that the search terms utilized by the Defendants “have not been demonstrated to be unreasonable,” and that these search terms “adequately captured the critical issues that [P]laintiffs have identified.” (Id. at *7, 11). As for Defendants’ cessation of their TAR review, the Special Master noted that Defendants followed their proposed procedures as directed by the Court, while also noting that neither their proposed process nor the Court's directive adopting it included “specific metrics for deciding when the review had been adequate.” (Id. at *11). The Special Master undertook detailed qualitative and quantitative analyses and concluded:

(Id. at *12). The Special Master then addressed Defendants’ argument that additional TAR review would impose a disproportionate burden on them, concluding that, “[o]n balance, then, the proportionality factors favor requiring WCA, Huntsman,[7] and Covestro to continue their TAR review.” (Id. at *13). Importantly, the Special Master supplied an additional recommendation for determining when enough is enough, stating, “To reduce the probability of further dispute and to provide the parties with a benchmark, I recommend that each review may presumptively be terminated when the last two batches reviewed contain no more than 10% responsive documents.” (Id. at *13). Plaintiffs’ Objections and Notice of Supplemental Authority followed. (ECF Nos. 804, 819). Thereafter, Defendants filed a Response to Objections. (ECF No. 812).

II. DISCUSSION

Plaintiffs object to the Special Master's recommended determination that Defendants’ search terms were reasonable because those search terms were not individually evaluated. Plaintiffs also object to the Special Master's recommendation that Defendant Huntsman should not be required to resume its TAR review. Upon careful de novo review of the entire record, the Court overrules Plaintiffs’ objections and fully adopts the Special Master's recommendations set forth in the Third R&R. See Fed. R. Civ. P. 53(f).

The applicable legal standards are not in dispute. First, Federal Rule of Civil Procedure 26(b)(1) provides, in relevant part, “Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party's claim or defense and proportional[8] to the needs of the case.” Moreover, Federal Rule of Civil Procedure 26(g) requires a party responding to discovery requests to conduct “a reasonable inquiry” but does not require perfection, especially in the realm of electronic discovery. See First R&R at *6 (citing cases). As correctly stated by the Special Master:

*4 (Third R&R at *6) (internal quotation marks and citations omitted). The Special Master continues:

Id. at *7) (internal quotation marks and citations omitted). Applied to the complexities of TAR, as is the case here, the principle of reasonableness incorporates an obligation for the producing party to validate its searches. (First R&R at *6). Moreover, the producing party has the right in the first instance to decide how it will produce its documents, so long as its search and collection methods are reasonable. Id. Discovery is also informed by the related concepts of cooperation and transparency. Id. at *7 (citing Youngevity International Corp. v. Smith, No. 16-cv-00704, 2019 WL 1542300, at *12 (S.D. Cal. Apr. 9, 2019)).

In their objections to the Third R&R, Plaintiffs principally complain that the Special Master did not evaluate each disputed search term individually and assess the reasonableness of any consequent omissions from Defendants’ TAR review. However, Plaintiffs point to no controlling authority that would require such a painstaking scrutiny of hundreds of disputed search terms and their efficacy if applied, or not, to millions of pages of documents through TAR methodologies. Notably, when adopting the Special Master's First R&R, the Court affirmed the holding that Defendants are not compelled to adopt the Plaintiffs’ search terms or TAR methodologies. (ECF No. 549). The Court also ordered Defendants to proceed with the validation methodologies suggested by the Special Master and subsequently adopted by Defendants. Id. Notwithstanding these prior rulings, Plaintiffs seem to be unrelenting in their pursuit of an elusive and unreasonable standard of perfection. Even so, the Special Master did quantitively and qualitatively scrutinize Defendants’ search terms and validation methodologies and concluded that Plaintiffs failed to establish that the search terms applied by Defendants were unreasonable.

*5 Contrary to Plaintiffs’ criticism that the Special Master did not consider disputed search terms individually, the Court notes that the Special Master did review the list of disputed search terms, the chart of “phrases” that the Plaintiffs derived from the disputed terms to illustrate the types of communications they contend were excluded, and the list of documents from Defendants’ productions that the disputed terms and the terms actually used both hit, concluding that Defendants’ search terms were “robust enough to capture many of the same documents that the disputed terms would have, without bringing in volumes of additional non-responsive documents.” (Third R&R at *10-11). Moreover, the Special Master qualitatively scrutinized each specific document identified by Plaintiffs from the null set as having been omitted from the TAR review based upon application of Defendants’ search terms and found that none of these omitted documents demonstrates that the application of Defendants’ search terms was unreasonable. The Special Master discerned that some documents were duplicates or near duplicates of documents otherwise produced by Defendants and none were individually unique or consequential. (Id. at *10). To build upon the Special Master's “mosaic” parlance, these documents presumably would amplify portions of the picture already assembled, but would not complete or otherwise add to portions of the picture not yet assembled. Think of a mosaic in more simplistic terms such as a paint-by-numbers art project, and assume that the entire picture is broken into segments numbered 1 through 5 to differentiate five different colors that make up the entire picture. Assume further that the documents generated from Defendants’ search terms complete the sections of the picture numbered 1, 2, and 3, leaving sections 4 and 5 uncolored. Extending this analogy, consider the documents from the null set derived from one or more disputed terms as also being placed in sections 1, 2, or 3 because, as the Special Master determined, they are duplicates, near duplicates, or otherwise not unique or individually consequential. These additional documents, and others like them, may certainly be relevant to the whole picture as they reinforce the sections that already are filled-in, thus potentially making those portions of the picture more vivid. But these new documents do nothing to fill-in any empty spaces[9] and are not useful towards completing the mosaic as Plaintiffs suggests. Indeed, Plaintiffs fail to establish that the exclusion of disputed search terms systematically omits certain types of circumstantial evidence that would, if produced, complete the whole picture either directly or by inference.

The Special Master's conclusion that the search terms applied by Defendants are reasonable is further reinforced by the Special Master's scrutiny of pertinent quantitative measures for assessing completeness. The Special Master notes that Defendants’ search terms achieved between approximately 74% and 89% recall, thus surpassing recall rates between 70%-80% that are often considered reasonable by courts. (Third R&R at *8). Plaintiffs’ objections assert that the Special Master ignored the declarations of their experts who critiqued the statistical methodology used by Defendants to calculate the recall rates, opining that Defendants’ calculations are “overly rosy” because “highly qualified independent reviewers will disagree on relevance about 30% of the time.” (ECF Nos. 774, ¶ 29; 804 at 12). However, the Special Master did consider Plaintiffs’ experts’ opinions but nonetheless concluded that accounting for human error does not undermine the quantitative reasonableness of Defendants’ search terms and TAR methodologies. (See Third R&R at *8). It is noteworthy, too, that the Special Master previously addressed Plaintiffs’ human error argument in his First R&R, noting that while “failure to account for reviewer error will artificially inflate the recall statistic,” the Special Master also determined that “courts have generally not mandated the incorporation of such reviewer error in calculating the recall statistic.” (First R&R at *12-13). That decision was adopted by the Court. (ECF No. 549). Plaintiffs’ attempt to resuscitate[10] this argument again now is unavailing.

Finally, Plaintiffs object to the Special Master's recommendation that Huntsman be permitted to cease its TAR review because its last two batches of reviewed documents contained 9.1% of responsive documents, which is less than the 10% benchmark the Special Master set for establishing a presumptively reasonable search. (Supplemental Third R&R). Plaintiffs assert two bases for their objections: first, that the responsiveness rate of Huntsman's final batch was 13.8%; and second, that the Special Master reached his conclusion without considering the quality of the responsive documents in Huntsman's final two batches. The Court finds both arguments unpersuasive.

Plaintiffs originally advocated for a quantitatively presumptive responsiveness standard of 10% (ECF No. 460-3), and Huntsman was instructed to evaluate the responsiveness rate for its last two baches in accordance therewith and is presumed to have undertaken a reasonable search for documents if the responsiveness rate for those batches is below 10%. Huntsman satisfied that standard. Plaintiffs now seek to move the goal posts by decoupling these final two batches. Plaintiffs also seemingly argue that Huntsman's results should be disregarded or otherwise discounted because the 4.4% responsiveness rate of its penultimate batch is, in Plaintiffs’ view, “anomalous,” and that the decision to terminate the review should not be based on a “single sample of 500 documents.” (ECF No. 804 at 13). Plaintiffs mischaracterize both the process Huntsman used and the standard by which it is to be measured. As set forth above, Huntsman's 9.1% responsiveness rate satisfied the prescribed standard of 10% for its final two batches. Moreover, that result is based upon a review of 1,000 documents, not 500 documents, just as Plaintiffs advocated for and just as directed. (ECF No. 743, ¶ 9). Moreover, this standard for measuring the completeness of the overall TAR review was developed in the context of evaluating those rates end-to-end, so Plaintiffs’ attempt to mischaracterize what has been reviewed in the final two batches, and to divorce those batches from each other and from the overall review does nothing to undermine the presumptive reasonableness of Huntsman's search. The presumption of reasonableness is firmly supported here. Notably, in addition to the average rate of responsiveness of 9.1% for the last two batches, the average rate in the null set and unreviewed TAR population is 3.417% and the end-to-end recall rate is 84.5%. (ECF No. 743, ¶ 14). Plaintiffs simply have not proffered any supportable quantitative basis to conclude that Huntsman's TAR review or its decision to cease that review when it did was unreasonable.

*6 Finally, contrary to Plaintiffs’ contention, the Special Master did undertake a qualitative review of the responsive documents from these last two review batches[11] and concluded that they “do not appear to be materially different in kind from the responsive documents located in the null set” and that they are “not dramatic enough to warrant further review.” (Third R&R at *12). Plaintiffs’ objections identify no specific documents returned from the searches or from the null set to support a contrary determination. Again, Plaintiffs simply have not proffered any qualitative basis to conclude that Huntsman's TAR review or its decision to cease that review when it did was unreasonable.

III. CONCLUSION

Based upon the forgoing and for the reasons expressed by the Special Master, the Court adopts the Special Master's Third R&R (consisting of the R&R and the Supplemental R&R) in its entirety. Accordingly, Plaintiffs’ objections are overruled, and Plaintiffs’ motion to compel is granted to the extent that WCA and Covestro are required to resume their TAR review,[12] and the motion is denied in all other respects.

THEREFORE, this 7th day of November, 2023, IT IS HEREBY ORDERED as follows:

1. The Special Master's Report and Recommendation, as modified by his Supplemental Report and Recommendation, (ECF Nos. 798 & 800), is hereby ADOPTED in its entirety as the opinion of this Court.

2. Plaintiffs’ Responses and Limited Objections to the October 19, 2022 Report and Recommendation of Special Master Hon. James C. Francis IV (Ret.) (ECF No. 804) are OVERRULED.

3. Plaintiffs’ Motion to Compel Defendants to Use Certain Search Terms and Further TAR Review (ECF No. 705) is GRANTED IN PART AND DENIED IN PART. The motion is GRANTED to the extent that Defendants WCA and Covestro are required to resume their TAR review in accordance with this Court's Opinion, and the motion is DENIED in all other respects.