Task 3 Results

Team	F2	Precision	Recall	MAP	R@5	R@10	R@30
JNLP_RUN1	0.8365	0.8037	0.8744	-	-	-	-
CAPTAIN.H2	0.8301	0.8333	0.8516	0.6721	0.7174	0.7935	0.9130
CAPTAIN.H3	0.8204	0.8002	0.8584	0.6721	0.7174	0.7935	0.9130
CAPTAIN.H1	0.8103	0.8196	0.8311	0.6721	0.7174	0.7935	0.9130
JNLP_RUN2	0.7863	0.7272	0.8402	-	-	-	-
JNLP_RUN3	0.7861	0.7420	0.8265	-	-	-	-
INFA	0.6917	0.7671	0.6826	0.6463	0.7065	0.7717	0.8913
mpnetAIIRLab	0.6674	0.3562	0.8858	0.8012	0.8696	0.9130	0.9674
OVGU3	0.6041	0.6347	0.6142	0.7134	0.8152	0.8587	0.9674
mistralRerank	0.5962	0.3196	0.7900	0.6916	0.8261	0.9022	0.9022
OVGU2	0.5959	0.6096	0.6027	0.7466	0.7826	0.8696	0.9565
NVAIIRLab	0.5836	0.3014	0.7854	0.7468	0.7609	0.8261	0.9348
UIwa	0.5816	0.5856	0.5890	0.6656	0.6739	0.7500	0.7609
UImeta	0.5793	0.5788	0.5890	0.6720	0.7283	0.7609	0.7609
UIthr	0.5723	0.6027	0.5685	0.6656	0.6739	0.7500	0.7609
OVGU1	0.4672	0.4635	0.4795	0.7134	0.8152	0.8587	0.9674
UA-mpnet	0.2540	0.0986	0.4361	0.3435	0.3913	0.4674	0.6196
UA-gte	0.2517	0.0986	0.4292	0.3244	0.3913	0.4783	0.6304
UA-bm25_allMini	0.2113	0.0795	0.3699	0.3168	0.3152	0.4457	0.6196
*NOWJ.H1	0.0137	0.0137	0.0137	0.0268	0.0217	0.0652	0.1304
*NOWJ.H2	0.0137	0.0137	0.0137	0.0268	0.0217	0.0652	0.1304
*NOWJ.H3	0.0137	0.0137	0.0137	0.0268	0.0217	0.0652	0.1304
	Total Teams:	7
	Total submissions:	22

Results marked with * from NOWJ are unofficial submissions due to late submission. Since long list submissions of JNLP are not the ranked list for the final submission results, evaluation results for the long list are not included to avoid confusion.

Note: One question was removed from the final evaluation due to a mistranslation issue from Japanese to English.

Last updated on April 19, 2025

Task 2 Task 4