Entry

essentia オーディオ/音楽解析ライブラリ

Simple Title

Dmitry Bogdanov, et al. 2013. ESSENTIA: an open-source library for sound and music analysis. In Proceedings of the 21st ACM international conference on Multimedia (MM '13). Association for Computing Machinery, New York, NY, USA, 855–858. DOI:https://doi.org/10.1145/2502081.2502229

Description

https://github.com/MTG/essentia

Type

Tool

Year

https://github.com/MTG/essentia

Posted at

May 26, 2021

Overview - 何がすごい?

音楽情報処理を念頭においたオーディオの解析ライブラリ Essentia
Python, JavaScriptなどで使える
TensorFlowベースの学習済みモデルも合わせて公開されている

Machine learning models - Essentia 2.1-beta6-dev documentation

Essentia includes algorithms for running inference with two types of data-driven machine learning models that can be used for high-level annotation of music audio: We provide various pre-trained models of both types for various music analysis and classification tasks. Essentia provides wrapper algorithms for TensorFlow deep learning models, designed to offer the flexibility of use, easy extensibility, and real-time inference.

essentia.upf.edu

Technology

2021年5月現在公開されている学習済み機械学習モデルは以下

自動タグ付け
音のEmbedding (OpenL3, VGGish-AudioSet).
音源分離 (Spleeter)
テンポ推定 (TempoCNN)
転移学習モデル

音楽ジャンル (trained on 4 different datasets)
ムード: happy, sad, aggressive, relaxed, acoustic, electronic, party
調性あり/ 調整なし
danceability どのくらい踊れるか
歌あり / インスト
性別 (male, female singer)

自動タグ付け

転移学習を用いた識別モデル

Further Thoughts

インストールが意外とハマる... Ubuntuは `pip` でいけるが Macは自分でソースからコンパイルするのが良さそう.

Installing Essentia - Essentia 2.1-beta6-dev documentation

Essentia depends on (at least) the following libraries: All dependencies are optional, and some functionality will be excluded when a dependency is not found.