Mark Sanderson
@IR_oldie
Followers
4K
Following
2K
Statuses
3K
I'm a search engine researcher. I tweet about information retrieval (IR) and uni work. I'm at RMIT University, these are my personal opinions.
Melbourne
Joined March 2010
Just to be clear, I'm not saying that this doesn't work, I just want to think more about it. Getting a system to measure itself sounds weird, but seems to work here.
0
0
2
NeuroPhysiological Approaches for Interactive Information Retrieval (NeuroPhysIIR'25) CFP is out #chiir2025
0
0
2
"Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges". Inject unrelated toxic content into a relevant passage, an LLM judge still says the passage is relevant from @ManveerTamber & @lintool
1
0
8
That's fantastic news, congratulations! @RMITComputing
"Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment" w/@juschnab @danulaera @umbrellacorpn accepted at #webconf2025! 😱🌟 We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification. #SIGIR2025
0
0
7
Building on four successful workshops in the last few year, it was been decided to create the Inaugural SEASON conference of the Search Engines and Society Network, starting in 2025. Submissions, April 30th. Held in Hamburg.
0
1
5
Thanks @claclarke for pointing me to this
@IR_oldie @softwaredoug We have a comparison of pairwise relevance assessment with other methods in our LLMEval workshop paper from SIGIR 2024.
0
0
0
Thanks for pointing me to this paper @macavaney
@IR_oldie @softwaredoug The pairwise setting is a really nice way to augment minimal human labels! If you know a result is relevant, you can use a pairwise LLM to predict if another result is *more* relevant. More on this in our SIGIR’23 paper :)
0
0
2
Using LLMs for pairwise relevance decisions, I haven't seen that tried before. Great article from @softwaredoug
Local LLMs might unlock a new age of democratized search relevance. Here I talk about using Qwen 2.5 locally as a local pairwise search evaluator
3
0
7
RT @claclarke: If you can't access the book directly, a draft of our chapter is available on arXiv.
0
2
0
RT @claclarke: @Marwah_k @NegarEmpr @IR_oldie and I contributed a chapter on Generative IR Evaluation to this recently released book on inf…
0
4
0
There have been a number of investigations on whether LLMs can replace humans for relevance assessment. I think the evidence is showing LLMs have a strong role, but won't replace. This recent paper from @claclarke and @lauradietz99 supports this view
0
3
19
Best plushie ever!
And our Baby Yoda paper received the Best Paper Honorable Mention award 🏅. I was also pleased to get a copy of @tetsuyasakai's great book that I’ve wanted to read for a long time, plus some adorable Waseda University goodies. Truly honored, huge thanks to my supervisors @IR_oldie, Falk Scholer and @pt_ir for their support and guidance.
0
0
9
What a team of keynote speakers. I must confess seeing that Steve Robertson will be there is a thrill. One of the legends of information retrieval reflecting on the field.
🎅🏻🎅🏻 #sigir2025 incredible keynotes - Ophir Frieder on "Digitized Health"; - @IGurevych on "Please meet AI, our dear new colleague. In other words: can scientists and machines truly cooperate?"; - Steve Robertson on "BM25 and All That - A Look Back"
0
2
14
"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", short paper accepted #chiir2025. Led by our Masters student @rankun203 with @Marwah_k @IR_oldie @damiano10. Awesome result Kun! @AdmsCentre @RMITComputing @ACM_CHIIR
1
7
30