Table 6

ISS results: parent matching

Team

Run

Total

TP

Unique

TP (unique)

Percentage correct

Percentage correct (unique)

MRR


4

1

372

51

361

51

0.1371

0.1413

-

4

2

372

71

361

70

0.1909

0.1939

-

6

1

2,497

147

2,072

117

0.0589

0.0565

0.5525

11

1

18,385

360

5,156

131

0.0196

0.0254

0.6594

11

2

18,371

376

5,270

145

0.0205

0.0275

0.6253

11

3

18,371

387

5,252

156

0.0211

0.0297

0.6416

14

1

634

13

579

12

0.0205

0.0207

0.8718

14

2

458

10

422

10

0.0218

0.0237

0.8167

14

3

560

13

514

11

0.0232

0.0214

0.8718

27

1

1,420

37

1,386

36

0.0261

0.0260

0.4653

28

1

3,028

150

3,001

148

0.0495

0.0493

0.3740

28

2

2,249

127

2,231

126

0.0565

0.0565

0.3696

28

3

5,448

352

3,210

191

0.0646

0.0595

0.3392

36

1

4,515

232

3,407

169

0.0514

0.0496

0.5731

36

2

11,827

571

7,526

343

0.0483

0.0456

0.5813

36

3

4,083

247

3,018

161

0.0605

0.0533

0.5476

43

1

3,691

111

3,117

97

0.0301

0.0311

0.4083

43

2

1,507

69

1,383

63

0.0458

0.0456

0.3524

43

3

3,674

148

3,257

131

0.0403

0.0402

0.3449

47

1

7,934

278

4,975

159

0.0350

0.0320

0.5232

47

2

7,633

274

4,835

156

0.0359

0.0323

0.5205

47

3

8,355

290

5,172

163

0.0347

0.0315

0.5329

49

1

21,431

590

10,422

285

0.0275

0.0273

0.3785

60

1

2,243

104

2,019

91

0.0464

0.0451

0.3460

60

2

4,714

157

3,932

130

0.0333

0.0331

0.3959

60

3

7,780

229

6,293

192

0.0294

0.0305

0.3998


This table reflects the baseline evaluation of the submissions received for interaction sentences subtask (ISS). Here, the submitted passages were compared with the previously manually selected passages reflecting the best interaction evidence. The 'Total' column indicates the total number of evaluated passages (note that submissions of articles for which the curators could not find a suitable evidence passage where excluded from evaluation). True positive ('TP') is number of correct passages (mapping the manually annotated ones). The 'Unique' column shows the number of unique passages per run (after removing duplicate passages). The 'TP (unique)' column indicates the number of correct passages (mapping the manually annotated ones) in the collection of unique passages. 'Percentage correct' is the fraction of predicted passages corresponding to the 'best' previously extracted passages. The 'Percentage correct (unique)' is the fraction of unique predicted passages corresponding to the 'best' previously extracted passages. 'MRR' is the mean reciprocal rank of the correct passages. Note that in the case of team 4, the MRR should not be taken into account, because all of the submitted passages here were labeled with rank 1 by this team.

Krallinger et al. Genome Biology 2008 9(Suppl 2):S4   doi:10.1186/gb-2008-9-s2-s4

Open Data