Audio Analysis V7 Classifier
Here is a description of new fields we have added with the new V7 classifier version
Advanced Genre
Similar to the V6 Genre classifier, but with more genres available. Each label has a score ranging from 0-1 where 0 (0%) indicates that the track is unlikely to represent a given genre and 1 (100%) indicates a high probability that track represents a given genre. The new genres are the following:
'afro', 'ambient', 'arab', 'asian', 'blues', 'childrenJingle', 'classical', 'electronicDance', 'folkCountry', 'funkSoul', 'indian', 'jazz', 'latin', 'metal', 'pop', 'rapHipHop', 'reggae', 'rnb', 'rock', 'singerSongwriters', 'sound', 'soundtrack', 'spokenWord'
Since music could break genre borders the genre classifier can predict multiple genres for a given song instead of only predicting one genre. A track could be classified with rapHipHop (Score: 0.9) but also reggae (Score: 0.8).
The genre can be retrieved both averaged over the whole track and segment-wise over time with 15s temporal resolution. In addition to the score, the API also exposes a list that includes the most likely genres.
AudioAnalysisV7Result.advancedGenreAudioAnalysisV7Result.advancedGenreTagsAudioAnalysisV7Segments.advancedGenre
Advanced Sub-genre
Again, very similar to the V6 Sub-genre fields. The set of sub-genres though is the same in this case:
bluesRock, folkRock, hardRock, indieAlternative, psychedelicProgressiveRock, punk, rockAndRoll, popSoftRock, abstractIDMLeftfield, breakbeatDnB, deepHouse, electro, house, minimal, synthPop, techHouse, techno, trance, contemporaryRnB, gangsta, jazzyHipHop, popRap, trap, blackMetal, deathMetal, doomMetal, heavyMetal, metalcore, nuMetal, disco, funk, gospel, neoSoul, soul, bigBandSwing, bebop, contemporaryJazz, easyListening, fusion, latinJazz, smoothJazz, country, folk
Each label has a score ranging from 0-1 where 0 (0%) indicates that the track is unlikely to represent a given sub-genre and 1 (100%) indicates a high probability that track represents a given sub-genre.
The sub-genre can be retrieved both averaged over the whole track and segment-wise over time with 15s temporal resolution. In addition to the score, the API also exposes a list that includes the most likely sub-genres.
Some tracks don't have any sub-genre. In this case the sub-genre tags is an empty array and averaged segments values are unavailable.
AudioAnalysisV7Result.advancedSubgenreAudioAnalysisV7Result.advancedSubgenreTagsAudioAnalysisV7Segments.advancedSubgenre
Advanced Instruments
The advanced instrument classifier predicts the presence of the following instruments:
percussion, synth, piano, acousticGuitar, electricGuitar, strings, bass, bassGuitar, brass and woodwinds.
It is possible to retrieve the presence of each instrument for each track segment, a list of the dominant instruments and a taxonomy that describes the presence of each instrument over the complete track.
The segment instrument score reaches from 0-1 where 0 (0%) indicates that the segment is unlikely to contain a given instrument and 1 (100%) indicates a high probability that the track segment contains a given instrument.
The taxonomy absent, partially, frequently and throughout describe the presence of each instrument
| Taxonomy | Description |
|---|---|
absent | Instrument has not been detected |
throughout | Instrument is detected throughout the full duration of the track |
frequently | Instrument is detected in major parts of the track |
partially | Instrument is detected in minor parts of the track. |
AudioAnalysisV7Result.advancedInstrumentPresenceAudioAnalysisV7Result.advancedInstrumentTagsAudioAnalysisV7Segments.advancedInstruments
Advanced Instruments Extended
The advanced instrument extended classifier predicts the presence of even more instruments:
acousticGuitar, bass, bassGuitar, electricGuitar, percussion, piano, synth, strings, brass, woodwinds, tuba, frenchHorn, oboe, mandolin, cello, marimba, vibraphone, electricPiano, electricOrgan, harp, ukulele, harpsichord, churchOrgan, doubleBass, glockenspiel, electronicDrums, drumKit, accordion, violin, flute, sax, trumpet, celeste, pizzicato, banjo, clarinet, bells, steelDrums, bongoConga, africanPercussion, tabla, sitar, taiko, asianFlute, asianStrings and luteOud
Like advanced instruments, it is possible to retrieve the presence of each instrument for each track segment, a list of the dominant instruments and a taxonomy that describes the presence of each instrument over the complete track.
AudioAnalysisV7Result.advancedInstrumentPresenceExtendedAudioAnalysisV7Result.advancedInstrumentTagsExtendedAudioAnalysisV7Segments.advancedInstrumentsExtended
Voiceover
The new fields available are:
- voiceoverDegree - a number showing the certainty that there is a voice over the track
- voiceoverExists - a boolean value indicating the existence of the voiceover in this track (same as voiceoverDegree > 0.5, just already calculated for convenience)
Free Genre Tags
This new field is a list of strings containing text genre descriptions for the track. The difference from the usual genre and subgenre fields we provide is that
it does not follow a fixed taxonomy and can therefore cover a broader range of musical genre description. Due to its text generator nature, no scores are returned for the tags given