United States District Court,
M.D. Georgia,
Valdosta Division.

NATIVE AMERICAN ARTS, INC., Plaintiff,
v.
BUD K WORLD WIDE, INC., Defendant.
Civil Action No. 7:10?CV?124 (HL).
May 18, 2012.

ORDER
HUGH LAWSON, Senior District Judge.

*1 This case is before the Court on Defendant's Motion to Exclude the Survey, Report, Supplemental Report, and Related Testimony of James T. Berger (Doc. 96). For the reasons discussed below, the motion is granted.

I. BACKGROUND AND FACTS
At issue in this case is the Indian Arts and Crafts Enforcement Act of 2000 (“IACEA”), 25 U.S.C. ? 305, et seq. The IACEA creates a civil cause of action against “a person who directly or indirectly, offers or displays for sale or sells a good ... in a manner that falsely suggests it is Indian produced, an Indian product, or the product of a particular Indian or Indian tribe or Indian arts and crafts organization....” 25 U.S.C. ? 305e(b) . Plaintiff alleges that Defendant has “directly or indirectly offered or displayed for sale or sold multiple quantities of various goods,” and has “advertised, marketed, offered and displayed for sale and sold goods at its stores, via catalogues and on the Internet through its interactive website, budk.com,” in a manner that falsely suggests the goods are Indian produced, an Indian product, or the product of a particular Indian or particular Indian tribe or Indian arts and crafts organization resident within the United States. The goods include tomahawks, artworks, crafts, and knives in a traditional Indian style or medium. (Second Am. Compl., ?? 8?12; Doc. 94, pp. 3?9).

Plaintiff has retained James T. Berger to provide expert testimony in this case. Berger was asked to conduct a consumer survey where “consumers and potential consumers of Native American craft products were asked to determine if a variety of products shown suggested that the products were authentic (Indian-made) Native American Indian product [sic].” (Report of James T. Berger, p. 2; Doc. 97?1, p. 2).

The survey was conducted over the internet between November 14 and 18, 2011. The survey was directed to men and women, ages 18 and over, who have either purchased or shopped for Native American products during the last 12 months, or intend to shop for or purchase Native American products in the next 12 months.

After asking for the state in which the respondent resided, two “knockout” yes or no questions were asked: “Have you shopped for Native American Indian products at retail, mail order or over the Internet in the last 12 months?” and “Do you intend to shop for Native American Indian products at retail, mail order or over the Internet in the next 12 months?” If the respondent answered “No” to both questions, he was “knocked out” of the survey and could not continue. If the respondent answered “Yes” to one of these questions, he was allowed to proceed.FN1

FN1. Another knockout question was whether the respondent or anyone in his household worked for a marketing research firm, an advertising agency, or any firm or organization that sells Native American products.

The qualified respondents were then asked: “Do you believe that Native American Indians make products for sale such as tomahawks, knives, dolls and jewelry?” The respondent had the option of answering “Yes,” “No,” or “Don't Know.” This question was not a knockout question. According to Berger, it was just an informational question that did not have any bearing on the survey results. (Deposition of James T. Berger, p. 100).

*2 The respondent was then shown 23 products, including knives, jewelry, tomahawks, and dolls. Each survey page contained the picture of a product and text taken from a website or catalogue. For each product, the respondent was asked the question: “Do you believe this illustration suggests that the product shown is an authentic (Indian-made) Native American Indian product?” The respondent could answer “Yes,” “No,” or “Don't Know.”

The pictures and text used in the survey were provided to Berger by Plaintiff's counsel. Berger then used a photo studio to “transform” the pictures of the products so they all looked similar in size and no particular product stood out. (Berger dep., p. 101).

The products tested in the survey came from Plaintiff, Defendant, Mangalick Enterprises, Peter Stone Company, and Atlanta Cutlery Corporation.FN2 Five of Plaintiff's Indian-made items were used as the control group: Looking Face Doll (Item 9); Cherokee Knife (Item 10); Zuni earrings (Item 13); Navajo Tomahawk (Item 17); and Ho?Chunk Knife (Item 18). The control items were selected by Berger and Plaintiff's counsel. Seven of Defendant's products were tested: Bear Hunter Fixed Blade Knife (Item 5); Cherokee Tomahawk (Item 6); Pipe of Peace, Axe of War (Item 7); Cherokee Bone Handle Tomahawk (Item 8); Functional Plains Indian Dagger with Sheath (Item 11); Native American Bowie Knife (Item 22); and Indian Tribal Tomahawk (Item 23). These seven items were selected by Berger for inclusion in the survey. He believed them to be representative of the other products at issue in the lawsuit, and it is his opinion that any other products selected would yield similar survey results. (Berger report, p. 7; Doc. 97?1, p. 7).

FN2. Plaintiff also has lawsuits pending against Mangalick Enterprises, Peter Stone Company, and Atlanta Cutlery Corporation. Berger was retained as an expert for those lawsuits, and his survey will presumably be used in those cases as well.

The first 50 surveys constituted a pilot study. A pilot study is when the survey is given to a small group as a precautionary control. If the results of the pilot study reflect a flaw or are not what the researcher was looking for, the survey can be abandoned or altered. The researcher can determine from the pilot study if the results of the survey are reliable. With this particular survey, Berger deemed the results reliable as he saw a very consistent pattern in the responses given in the pilot study. (Berger dep., p. 110).

As the pilot study results were deemed reliable, they were included in the final survey results. In total, 262 men and women participated in the survey. The results for Defendant's products were as follows FN3 :

FN3. Again, the question asked was: “Do you believe this illustration suggests that the product shown is an authentic (Indian-made) Native American Indian product?”

Product Name	Yes	No	Don't Know
Bear Hunter Fixed Blade Knife	53%	37%	10%
Cherokee Tomahawk	39%	43%	17%
Pipe of Peace, Axe of War	44%	45%	11%
Cherokee Bone Handle Tomahawk	58%	32%	10%
Functional Plains Indian Dagger with Sheath	53%	31%	16%
Native American Bowie Knife	48%	38%	14%
Indian Tribal Tomahawk	56%	36%	8%

According to Berger, 20 percent is the generally accepted threshold for likelihood of confusion. (Berger dep., pp. 13?14). Based on the survey responses, Berger concluded in his expert report that “every product or advertisement shown in In [sic] this survey suggests it was made by Native American Indians.” (Berger report, p. 10; Doc. 97?1, p. 10).FN4

FN4. This was Berger's conclusion for all of the products, not just those from Defendant.

*3 At the request of Plaintiff's counsel, Berger also prepared a supplemental expert report regarding Senate Report 106?452 , which relates to the IACEA. Berger was asked to review the Senate Report and provide his opinion as it relates to damages incurred by Plaintiff.

The Senate Report, which was drafted in October of 2000, states in part as follows:

[The] growing influx of inauthentic Indian arts and crafts has dramatically affected the Indian arts and crafts market by driving down prices, and tainting consumer confidence in and the cultural integrity of the market.

........
To compete [with imitation Indian arts and crafts], traditional Indian artisans are required to reduce their prices and reduce their profit margin.
........
Another result of the rapid rise in imitation Indian arts and crafts is that consumer confidence in this market is declining. Lack of consumer confidence will reduce the demand for Indian arts and crafts as consumers shift their preference to goods with greater consumer protections.
The flood of inauthentic Indian arts and crafts also damages traditional Indian heritage and culture.

S. REP. NO. 106?452, at *2 (2000) .
In his supplemental expert report, Berger states that after examining the products at issue, based on his marketing experience, he

believe[s] that the inflow of non-genuine products has a detrimental effect to genuine Native American artisans and Native American Arts in a number of ways:

a. It drives down prices and forces the genuine artisans such as Native American Arts to reduce its prices and therefore reduce its profits.

b. It undermines consumer confidence because consumers can not be sure if they are purchasing genuine or non-genuine products.

c. It enables makers of non-genuine products to trade on the goodwill of Native American artisans.

(Supplemental Report of James T. Berger, p. 3; Doc. 97?3, p. 17).
Berger goes on to state that his survey research supports Senate Report 106?452 in that the survey “verified the ‘flood of inauthentic Indian arts and crafts' were, in effect, perceived to be made by Native American Indians.” (Berger supplemental report, p. 3; Doc. 97?3, p. 17). In effect, the supplemental expert report states that Plaintiff has been injured and damaged by Defendant because Defendant is selling products that are thought by consumers to be authentic Indian-made products, when they in fact are not. (Berger dep., pp. 138?39).

Defendant has moved to exclude Berger's survey, both expert reports, and his testimony under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786 (1993) , Federal Rule of Evidence 702 , and Federal Rule of Evidence 403 .

II. ANALYSIS
A. Daubert /Federal Rule of Evidence 702

In federal court, expert opinions must meet the admissibility guidelines announced by the Supreme Court in Daubert, 509 U.S. at 579, and Federal Rule of Evidence 702.

*4 Rule 702 states:

If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.

Fed.R.Evid. 702 .
The district court generally engages in a three-part inquiry to determine the admissibility of expert testimony. Specifically, the court considers: (1) whether the expert is qualified to testify about the matters he intends to address; (2) whether the methodology used by the expert to reach his conclusions is sufficiently reliable; and (3) whether the experts testimony will assist the trier of fact, through the application of scientific, technical, or specialized expertise, to understand the evidence or determine a fact in issue. McCorvey v. Baxter Healthcare Corp., 298 F.3d 1253, 1257 (11th Cir.2003) (citations omitted). District courts have a duty under Rule 702 to Censure that any and all scientific testimony or evidence admitted is not only relevant, but reliable. Daubert, 509 U.S. at 589 . Plaintiff bears the burden of laying the proper foundation for the admission of its expert's testimony, and admissibility must be shown by a preponderance of the evidence. Allison v. McGhan Med. Corp., 184 F.3d 1300, 1306 (11th Cir.1999) (citation omitted). The same standard applies to all expert testimony, including testimony regarding scientific, technical, and other specialized matters. Kuhmo Tire Co. v. Carmichael, 526 U.S. 137, 147, 119 S.Ct. 1167 (1999) .

However, it is not the role of the district court to make ultimate conclusions as to the persuasiveness of the proffered evidence. Quiet Tech. DC?8, Inc. v. HubelDubois UK Ltd., 326 F.3d 1333, 1341 (11th Cir.2003) . The district court's gatekeeper role is not intended to supplant the adversary system or the role of the jury. Id. Instead, vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence. Id.

B. Federal Rule of Evidence 403

Federal Rule of Evidence 403 provides another means for the exclusion of expert testimony. Rule 403 provides that the court “may exclude relevant evidence if its probative value is substantially outweighed by a danger of one or more of the following: unfair prejudice, confusing the issues, misleading the jury, undue delay, wasting time, or needlessly presenting cumulative evidence.” Fed.R.Evid. 403. Survey evidence “must be excluded under Federal Rule of Evidence 403 where it is so flawed in methodology that its probative value is substantially outweighed by its prejudicial effect.” Trouble v. Wet Seal, Inc., 179 F.Supp.2d 291, 307 (S.D.N.Y.2001).

1. Internet survey and expert report
*5 The IACEA restricts the manner in which products may be displayed, offered for sale, or sold to the public. The ultimate issue for the jury to decide in this case is whether Defendant has offered, displayed for sale, or sold a good in a manner that falsely suggests it is Indian produced, an Indian product, or the product of a particular Indian or Indian tribe or Indian arts and crafts organization.

Expert testimony must assist the trier of fact, through the application of scientific expertise, to understand the evidence or determine a fact in issue. In the Court's opinion, Berger's testimony as related to the internet survey and initial expert report will not assist the trier of fact or be helpful to the jury. The fact that 139 survey respondents thought the “illustration” of Defendant's Bear Hunter Fixed Blade Knife suggests that the product shown is an authentic Native American Indian product is really irrelevant to the ultimate issue in this case. The question is whether the 12 jurors picked to hear the case find that Defendant offered, displayed for sale, or sold a good in a manner that falsely suggested it was Indian-made. While an expert may testify to an ultimate issue for the trier of fact, Fed.R.Evid. 704 , his testimony still has to be helpful to the jury. Berger's testimony that his “research data shows that every product or advertisement shown in [ ] this survey suggests it was made by Native American Indians” tells the jury what result to reach. Even in light of Rule 704 , that remains improper. See Owen v. Kerr?McGee Corp., 698 F.2d 236, 240 (5th Cir.1983) ; Fed.R.Evid. 704 advisory committee's notes (1972).

Further, Berger does not offer any opinion on matters beyond the competence of an average juror. FExpert testimony is admissible if it concerns matters that are beyond the understanding of the average lay person. United States v. Frazier, 387 F.2d 1244, 1262 (11th Cir.2004). “[E]xpert testimony generally will not help the trier of fact when it offers nothing more than what lawyers for the parties can argue in closing arguments.” Id. The average lay juror will not need the benefit of Berger's opinion to determine whether Defendant falsely suggested products were Indian-made.

Aside from whether the expert testimony will assist the jury, the Court finds that the survey must be excluded for other reasons. To assess the validity and reliability of a survey, a court should consider a number of criteria, including whether:

(1) the proper universe was examined and the representative sample was drawn from that universe;

(2) the survey's methodology and execution were in accordance with generally accepted standards of objective procedure and statistics in the field of such surveys;

(3) the questions were leading or suggestive;

(4) the data gathered were accurately reported; and

(5) persons conducting the survey were recognized experts.

Louis Vuitton Malletier v. Dooney & Bourke, Inc., 340 F.Supp.2d 415, 433 (S.D.N.Y.2004) (citation and alterations omitted), vacated on other grounds by Louis Vuitton Malletier v. Dooney & Bourke, Inc., 454 F.3d 108, 117 (2d Cir.2006); see also Shari Seidman Diamond, Reference Guide on Survey Research, Reference Manual on Scientific Evidence 359, 364, 373?418 (Fed.Jud.Ctr.3d ed.2011) (hereinafter “Diamond”).
*6 Plaintiff is correct that the Eleventh Circuit has held that alleged technical deficiencies in a survey go to the weight of the survey, not its admissibility. See Jellibeans, Inc. v. Skating Clubs of Ga., Inc., 716 F.2d 833, 845 (11th Cir.1983) . However, that is not an absolute rule, as a fundamentally flawed survey may be excluded under Rule 702 or Rule 403 . See Citizens Fin. Group, Inc. v. Citizens Nat'l Bank, 383 F.3d 110, 118?21 (3d Cir.2004) ; THOIP v. Walt Disney Co., 788 F.Supp.2d 168 (S.D.N.Y.2011) ; Hodgdon Powder Co. v. Alliant Techsys., Inc., 512 F.Supp.2d 1178, 1181 (D.Kan.2007) (“[W]hen the deficiencies are so substantial as to render the survey's conclusions untrustworthy, a court should exclude the survey from evidence.”)

In order to be considered reliable, a survey must resemble the way consumers would view the products in the marketplace. “Although no survey can construct a perfect replica of ‘real world’ buying patterns, a survey must use a stimulus that, at a minimum, tests for confusion by roughly simulating marketplace conditions.” Trouble, 179 F.Supp.2d at 308 . A survey must be “designed to examine the impression presented to the consumer by the accused product. Therefore, [it] must use the proper stimulus, one that tests for confusion by replicating marketplace conditions.” Conopco, Inc. v. Cosmair, Inc., 49 F.Supp.2d 242, 253 (S.D.N.Y.1999) .

Defendant argues that the survey does not replicate marketplace conditions, as Defendant does not sell its products from a single image, but from an interactive website and detailed catalogue. Defendant states that the website and catalogue “contain numerous contextual clues which would easily clear up any confusion as to whether the relevant products are authentic Native?American made.” (Doc. 97, p. 9). Defendant complains that its items are taken out of context in the survey, and that the survey pages should have included additional information that is shown on the website and in the catalogue, including prices, brand logos, and other suggested products.

In response, Plaintiff states that all seven of Defendant's products were exhibited in the survey the exact same way they were exhibited on the website or in the catalogue. The pictures and narratives were not altered in any way. Plaintiff contends the survey “exactly replicates the presentation of the product picture and narrative by Bud K to the potential consumers as reflected in the marketplace.” (Doc. 105, p. 9). Plaintiff further argues that there was no reason to add information about prices, logos, or references to other products because those were not directly part of the advertisement and are not relevant to the purpose of the survey.

The Court has been provided with a copy of the survey, so it knows what the survey respondents were shown. Also in the record are printouts from Plaintiff's website and Defendant's website and catalogue for the following items contained in the survey: Cherokee Tomahawk (Item 6?Defendant), Cherokee Bone Handle Tomahawk (Item 8?Defendant), Cherokee Knife (Item 10?Plaintiff), Functional Plains Indian Dagger with Sheath (Item 11?Defendant), Navajo Tomahawk (Item 17?Plaintiff), Ho?Chunk Knife (Item 18?Plaintiff), Native American Bowie Knife (Item 22?Defendant), and Indian Tribal Tomahawk (Item 23?Defendant). A comparison of the survey and the printouts shows that Berger did include in the survey the product descriptions from the websites and catalogue FN5, but he did not include prices, brand logos, additional pictures of the products, or other recommended products as shown on the websites and in the catalogue.

FN5. For instance, Item 6 is Defendant's Cherokee Tomahawk. Both the survey and website printout describe the product as follows: “Features a hardwood handle with a decorative carbon steel black-coated blade. The traditional style and classic elegance creates the perfect setting for your collection. 15? overall. 4 3/4? edge.” (Doc. 97?1, p. 19; Doc. 97?3, p. 7).

*7 As discussed supra, the IACEA is concerned with the “manner” in which one offers for sale, displays for sale, or sells a good. Here, the manner in which Defendant offers for sale, displays for sale, or sells goods encompasses all of the information provided on the website or in the catalogue. A consumer in the marketplace would not view one picture of a product and a few lines of text. Instead, he would see a plethora of additional information-information that Berger did not include in the survey for the respondents to view. An image from a website “cannot meaningfully test for confusion if it is not presented in the way that an Internet user would actually encounter it.” Fancaster, Inc. v. Comcast Corp., ??? F.Supp.2d ????, 2011 WL 6426292, at *17 (D.N.J.2011) . Further, although the survey was conducted on a computer, the respondents were not allowed “to interact with [the websites] as they ordinarily would in the marketplace.” Id. “The results of a survey that does not adequately simulate how a consumer would encounter a trademark are neither reliable nor probative.” Id. The Court finds that Berger's failure to present the complete manner in which Defendant displays, offers to sell, or sells its goods renders the survey unreliable under Rule 702 and misleading and confusing under Rule 403 .

The questions asked in the survey pose another problem. Defendant argues that the survey presented ambiguous, leading, and suggestive questions, thereby corrupting the results. Defendant contends that asking “Do you believe this illustration suggests that the product shown is an authentic (Indian-made) Native American Indian product?” plants the idea that the product is an “authentic (Indian-made) Native American Indian product” in the respondents' minds, which is improper. Defendant further argues that the introductory question “Do you believe that Native American Indians make products-for-sale such as tomahawks, knives, dolls and jewelry?” improperly suggested to the respondents that the “tomahawks, knives, dolls and jewelry” that follow are made by Native Americans.

Plaintiff responds that the question regarding the products was not suggestive, which is reflected in the high number of “No” and “Don't Know” answers to each of Defendant's products. Plaintiff also argues that the introductory question is irrelevant because it had no bearing on Berger's ultimate data calculation.

“[T]he wording of a question, open-ended or closed-ended, can be leading or non-leading, and the degree of suggestiveness of each question must be considered in evaluating the objectivity of a survey.” Diamond at 393. Open-ended questions require the respondent to express an answer in his own words. Closed-ended question provide the respondent with set responses. Diamond at 393?94. The question regarding the products and the introductory question are both closed-ended questions, as the respondents were limited to answering “Yes,” “No,” or “Don't Know.”

*8 Closed-ended questions are not necessarily leading or improper. The inclusion of the “Don't Know” option works as a “quasi-filter question to reduce guessing,” which lends credibility to the survey. Diamond at 390. “By signaling to the respondent that it is appropriate not to have an opinion, the question reduces the demand for an answer, and as a result, the inclination to hazard a guess just to comply.” Diamond at 390.

The Court agrees with Defendant that the question regarding the products is leading and slanted. It implies the answer to the ultimate question the survey was designed to test, which is improper. The respondents should have been asked “Who makes this product?” or “Who do you believe makes this product based on the picture and text provided?” As for the introductory question, it improperly suggests to the respondents that they should believe there is some connection between Native Americans and the products shown. Whether Berger ultimately relied on the responses to the introductory question is irrelevant.

In addition to the leading nature of the questions, the Court believes the question regarding the products is ambiguous, requiring the exclusion of the survey. The respondents were asked if they believed the “illustration” suggested the product is an authentic Native American product. At no point in the survey is “illustration” defined. What constitutes the “illustration” referred to in the question? Is it the picture of the product? Is it the text? Is it the picture and the text? In the Court's opinion, the answer is not clear.

“When unclear questions are included in a survey, they may threaten the validity of the survey by systematically distorting responses if respondents are misled in a particular direction, or by inflating random error if respondents guess because they do not understand the question.” Diamond at 388. “If the crucial question is sufficiently ambiguous or unclear, it may be the basis for rejecting the survey.” Id. The Court can assume Berger meant for the respondents to review both the picture and the text, but that is nothing more than an assumption. The term “illustration” generally brings to mind a picture or diagram, not text. It is not out of the realm of possibility that the respondents only looked at the picture of the product when answering the question, which would distort the survey responses. Because a layperson could interpret the term “illustration” in several ways, the Court concludes the question was ambiguous.

For the reasons discussed above, the Court finds that Berger's survey and his testimony relating to the survey are unreliable and unhelpful. Exclusion of Berger's survey and expert report, as well as his testimony relating to the survey is warranted under both Daubert and Rule 403 .

2. Supplemental expert report

The Court finds that Berger's supplemental expert report must also be excluded. Berger's opinion contained in the supplemental expert report is based at least in part on his survey, which has been excluded. Further, the conclusions contained in the supplemental report are little more than a parroting of the findings in the Senate Report from 2000.

*9 Plaintiff argues that the supplemental report is admissible because it is based on Berger's extensive survey and marketing expertise. The Court does not disagree that experience may provide a sufficient foundation for expert testimony. See Fed.R.Evid. 702 advisory committee's notes (2000 amends.). This does not mean, however, that “experience, standing alone, is a sufficient foundation rendering reliable any conceivable opinion the expert may express.” Frazier, 387 F.3d at 1261 (emphasis in original). “If the witness is relying solely or primarily on experience, then the witness must explain how that experience leads to the conclusion reached, why that experience is a sufficient basis for the opinion, and how that experience is reliably applied to the facts.” Id. (citing Fed.R.Evid. 702 advisory committees notes (2000 amends.)). Berger did none of those things. “The trial courts gatekeeping function requires more than simply ‘taking the experts word for it.’ “ Id. (citing Fed.R.Evid. 702 advisory committees notes (2000 amends.)). The lack of a connection between Berger's experience and the facts of this case leaves the Court with nothing but his word. Rule 702 and Daubert require more. Berger's supplemental expert report is inadmissible.

III. CONCLUSION
The proponent of expert testimony always bears “the burden to show that his expert is ‘qualified to testify competently regarding the matters he intend[ed] to address; [ ] the methodology by which the expert reach[ed] his conclusions is sufficiently reliable; and [ ] the testimony assists the trier of fact.’ “ Frazier, 387 F.3d at 1260 (citations omitted). Plaintiff has not met this burden with regard to Berger. His survey, opinion testimony, and both expert reports are inadmissible under Rules 403 and 702 and Daubert.

Defendants' Motion to Exclude (Doc. 96) is granted.

SO ORDERED.

M.D.Ga.,2012.
Native American Arts, Inc. v. Bud K World Wide, Inc.
Slip Copy, 2012 WL 1833877 (M.D.Ga.)