https://www.youtube.com/watch?v=ddCYORu41Xs Talking about ways to use an off-the-shelf language model to solve a multiple-choice task. Covering: - Intro to the Kaggle competition - Benchmarking with GPT3.5 - Using the OpenAI function calling API to enforce structure on answers - Using Llama2 as a classifier by examining the logits (next token predictions) - Using perplexity to evaluate question-answer pairs Notebook using the OpenAI API to test GPT3.5: https://www.kaggle.com/johnowhitaker/. Llama2 demo notebook: https://colab.research.google.com/dri. (quickly made for this video, don't trust the calculations, rather start with the below notebook) Notebook testing different open models with the perplexity approach: https://www.kaggle.com/code/takamichi. (a good template to start experimenting since it shows how to run as a submission.
www.bilibili.com