Automatic metrics for humorous texts. Currently supporting English, Spanish, French, Portuguese and Mandarin Chinese.
| Metric | Method | Reference |
|---|---|---|
| Quantum Uncertainty / Incongruity | quantum_uncertainty, quantum_incongruity |
Liu and Hou (2023) |
| Surprise Ratio | surprise_ratio |
He et al. (2019) |
| Ambiguity / Distinctiveness | kao_ambiguity, kao_distinctiveness |
Kao et al. (2016) |
Humetrix is a Python library that provides a collection of automatic metrics for evaluating humor in text. Quantum Entropy metrics apply to any text with a setup and a punchline, while Surprise Ratio and Ambiguity/Distinctiveness are designed for puns and need the pun and alternative words.
All resources are downloaded automatically on first use, but the repository also contains the scripts used to train our models (see docs/skipgram.md)
Humetrix requires Python 3.12 or above:
pip install humetrixPyTorch is installed from PyPI by default. For a CPU-only install, install PyTorch first following the official instructions.
Here is a simple example of how to use Humetrix:
from humetrix import HumorAnalyzer
# Needed resources are downloaded automatically on first use
analyzer = HumorAnalyzer(language='en')
sent = 'Not all math puns are bad. Just sum of them.'
sign = 'sum'
alt_sign = 'some'
qi_score = analyzer.quantum_incongruity(sent)
qi_xlm_score = analyzer.quantum_incongruity(sent, backend='transformer')
qu_score = analyzer.quantum_uncertainty(sent)
qu_xlm_score = analyzer.quantum_uncertainty(sent, backend='transformer')
sr_score = analyzer.surprise_ratio(sent, sign, alt_sign)
kao_amb_score = analyzer.kao_ambiguity(sent, sign, alt_sign)
kao_dist_score = analyzer.kao_distinctiveness(sent, sign, alt_sign)Metrics return None when they cannot be computed.
Use HumorAnalyzer(<language>, device="cpu") too choose where neural models run, and transformer_model_name to choose the model for the transformer backend.
Full API Documentation is available at https://nlp-cisuc.github.io/Humetrix/.
Embeddings, skip-gram models, n-gram tables, spaCy and transformer models are downloaded on first use and cached locally, so the first call of each metric may take a while. They are hosted at NLP-CISUC/humetrix-resources.
To reproduce the experiments of the paper, clone the repository and use uv:
uv syncThis project is licensed under the MIT License - see the LICENSE file for details.
The resources have their own licenses, listed in the resources repository.
To be announced