r/MachineLearning • u/No-Lab4175 • 22h ago
News ICLR 2027 Initial Review Scores ranked by University [N]
Crawled from OpenReview, titles matched to arXiv, then affiliated with OpenAlex
5
u/No-Lab4175 22h ago
Data & Original Picture: https://GitHub.com/clareLab/posteLab/posters
More Posters: https://clareLab.moe/gallery
Also on my X: https://x.com/c1areLab/status/2121674375367901292
7
u/yannbouteiller Researcher 21h ago
How comes the number of submissions lives outside the realm of natural numbers?
EDIT: Oh nvm, it is weighted by author
8
u/No-Lab4175 21h ago
Good question
Addendum: Why isn't the submission count an integer? Because, for example, if a paper is affiliated with 100 universities, I can't simply add 1 to the count for all 100 universities. Therefore, a normalization is performed based on the number of institutions, and the credit is weighted and distributed among each affiliated institution (weighted based on the number of authors under each institution) to ensure that the submission count is always 1.
8
u/No-Lab4175 21h ago
For example
A paper:
Author 1: Tsinghua, MIT
Author 2: MIT
Author 3: MIT
Then:
MIT: (1/3)/2 + 2/3
Tsinghua: (1/3)/2
2
3
u/GoodRazzmatazz4539 Researcher 22h ago
How do you know from which university the papers are? Aren’t we still in double blind review mode?
11
u/No-Lab4175 22h ago
Crawled from OpenReview, titles matched to arXiv, then affiliated with OpenAlex
5
u/GoodRazzmatazz4539 Researcher 22h ago
Can you say how many did not had a matching Arxiv paper yet? And is this correlated with score?
5
u/No-Lab4175 22h ago
Only about 1/4 to 1/3 is affiliated, but I don't think it will affect too much, since the mean score is 2.19 here, against the 2.02 overall, not much difference. But yeah, not precise of course
5
u/GoodRazzmatazz4539 Researcher 22h ago
There probably is a bias since lower score papers are likely less polished or they are just less likely to post their paper after receiving low scores, so it make sense that their avg. is lower.
Would be interesting to see if date posted on Arxiv and higher score correlate3
u/No-Lab4175 22h ago
There are already posts trying to correlate the score and the date submitted on ICLR 2027 if I remember correctly (the result is that early submission tends to receive higher score). And yes, it would be interesting to correlate with arXiv
2
2
49
u/Thrumpwart 21h ago
This is nerd football. Colleges can start trading PhD students. Should they be allowed to get paid by sponsors though?
4
16
8
u/mmarkDC 21h ago
The bottom ones weren't what I was expecting. Of institutions with >=10 matched papers, Iowa State had only 14% with above-median reviews. Virginia Tech 2nd worst at 20%. Even with noisy reviews, those are so much below most other large universities that it raises an eyebrow. I would've expected the bottom places to be different kinds of institutions than those two. Either Iowa-State and VTech researchers are submitting some real garbage, or they got very unlucky with the reviewer draw.
3
2
1
10
u/evarol 22h ago
How do you assign each paper to an organization?
by last author, first author, majority of authors?
a lot if not most papers have authors from multiple orgs
Cool visualization!