IEEE

Benchmark Text Preprocessing Techniques in Natural Language Processing

Abstract: Text preprocessing is a key step in Natural Language Processing (NLP) that deals with the cleaning, tokenization and structure of text before building models. A comparison of the recent ...
GitHub

A methodology to pre-process text data for preventing lookahead bias in Large Language Models (LLMs), particularly in financial text analysis.

Entity Neutering is a text preprocessing technique designed to anonymize financial documents and news articles to prevent LLMs from using prior knowledge about specific companies, industries, or time ...
IEEE

Contextual Spelling Corrector for Indonesian Text Preprocessing: A Comparative Analysis of Large Language Models

Abstract: Spelling error are a major problem in Natural Language Processing (NLP), particularly when it comes to Indonesian text, since they might reduce the ...