What is it about?
We label ~1000 sentences from FOMC statements and see how accurately different language models can detect the sentiment contained within each sentence. We also experimented with prompt engineering techniques that make structured JSON parsing more reliable.
Featured Image
Why is it important?
This is important as FOMC minutes and other forms of central bank communication are used by financial markets to judge the current state of the economy and to anticipate future interest rate changes.
Perspectives
The paper shows that GPT-4 tends to be too conservative in its sentiment predictions. It over-reports neutral sentiments and is reluctant to decide on positive or negative valences. Llama-3-70B did a better job of correctly distinguishing between sentiments. Smaller models underperform Llama-3, but they may be useful if processing time and cost are paramount.
Jan Spörer
Read the Original
This page is a summary of: Is Small Really Beautiful for Central Bank Communication? Evaluating Language Models for Finance: Llama-3-70B, GPT-4, FinBERT-FOMC, FinBERT, and VADER, November 2024, ACM (Association for Computing Machinery),
DOI: 10.1145/3677052.3698675.
You can read the full text:
Contributors
The following have contributed to this page
