Skip to main content

Changelog 2022-12-15

· 5 min read
Mika Zimina

In this post we want to make an introduction to the new audio analysis fields we have added recently.

Advanced Genre​

Similar to the V6 Genre classifier, but with more genres available. Each label has a score ranging from 0-1 where 0 (0%) indicates that the track is unlikely to represent a given genre and 1 (100%) indicates a high probability that track represents a given genre. The new genres are the following:

'afro', 'ambient', 'arab', 'asian', 'blues', 'childrenJingle', 'classical', 'electronicDance', 'folkCountry', 'funkSoul', 'indian', 'jazz', 'latin', 'metal', 'pop', 'rapHipHop', 'reggae', 'rnb', 'rock', 'singerSongwriters', 'sound', 'soundtrack', 'spokenWord'

Since music could break genre borders the genre classifier can predict multiple genres for a given song instead of only predicting one genre. A track could be classified with rapHipHop (Score: 0.9) but also reggae (Score: 0.8).

The genre can be retrieved both averaged over the whole track and segment-wise over time with 15s temporal resolution. In addition to the score, the API also exposes a list that includes the most likely genres.

Example query​

libraryTrack(id: "9411599") {
  ... on LibraryTrackNotFoundError {
    message
  }
  ... on LibraryTrack {
    id
    audioAnalysisV6 {
      ... on AudioAnalysisV6Finished {
        result {
          advancedGenre {
            afro
            ambient
            arab
            asian
            blues
            childrenJingle
            classical
            electronicDance
            folkCountry
            funkSoul
            indian
            jazz
            latin
            metal
            pop
            rapHipHop
            reggae
            rnb
            rock
            singerSongwriters
            sound
            soundtrack
            spokenWord
          }
          advancedGenreTags
        }
      }
    }
  }
}

Advanced Sub-genre​

Again, very similar to the V6 Sub-genre fields. The set of sub-genres though is the same in this case:

bluesRock, folkRock, hardRock, indieAlternative, psychedelicProgressiveRock, punk, rockAndRoll, popSoftRock, abstractIDMLeftfield, breakbeatDnB, deepHouse, electro, house, minimal, synthPop, techHouse, techno, trance, contemporaryRnB, gangsta, jazzyHipHop, popRap, trap, blackMetal, deathMetal, doomMetal, heavyMetal, metalcore, nuMetal, disco, funk, gospel, neoSoul, soul, bigBandSwing, bebop, contemporaryJazz, easyListening, fusion, latinJazz, smoothJazz, country, folk

Each label has a score ranging from 0-1 where 0 (0%) indicates that the track is unlikely to represent a given sub-genre and 1 (100%) indicates a high probability that track represents a given sub-genre.

The sub-genre can be retrieved both averaged over the whole track and segment-wise over time with 15s temporal resolution. In addition to the score, the API also exposes a list that includes the most likely sub-genres.

caution

Some tracks don't have any sub-genre. In this case the sub-genre tags is an empty array and averaged segments values are unavailable.

Example query​

libraryTrack(id: "9411599") {
  ... on LibraryTrackNotFoundError {
    message
  }
  ... on LibraryTrack {
    id
    audioAnalysisV6 {
      ... on AudioAnalysisV6Finished {
        result {
          advancedSubgenre {
            bluesRock
            folkRock
            hardRock
            indieAlternative
            psychedelicProgressiveRock
            punk
            rockAndRoll
            popSoftRock
            abstractIDMLeftfield
            breakbeatDnB
            deepHouse
            electro
            house
            minimal
            synthPop
            techHouse
            techno
            trance
            contemporaryRnB
            gangsta
            jazzyHipHop
            popRap
            trap
            blackMetal
            deathMetal
            doomMetal
            heavyMetal
            metalcore
            nuMetal
            disco
            funk
            gospel
            neoSoul
            soul
            bigBandSwing
            bebop
            contemporaryJazz
            easyListening
            fusion
            latinJazz
            smoothJazz
            country
            folk
          }
          advancedSubgenreTags
        }
      }
    }
  }
}

Advanced Instruments​

The new instrument classifier predicts the presence of the following instruments:

percussion, synth, piano, acousticGuitar, electricGuitar, strings, bass, bassGuitar, brass and woodwinds.

It is possible to retrieve the presence of each instrument for each track segment, a list of the dominant instruments and a taxonomy that describes the presence of each instrument over the complete track.

The segment instrument score reaches from 0-1 where 0 (0%) indicates that the segment is unlikely to contain a given instrument and 1 (100%) indicates a high probability that the track segment contains a given instrument.

The taxonomy absent, partially, frequently and throughout describe the presence of each instrument

TaxonomyDescription
absentInstrument has not been detected
throughoutInstrument is detected throughout the full duration of the track
frequentlyInstrument is detected in major parts of the track
partiallyInstrument is detected in minor parts of the track.

Example query​

libraryTrack(id: "9411599") {
  ... on LibraryTrackNotFoundError {
    message
  }
  ... on LibraryTrack {
    id
    audioAnalysisV6 {
      ... on AudioAnalysisV6Finished {
        result {
          advancedInstrumentPresence {
            percussion
            synth
            piano
            acousticGuitar
            electricGuitar
            strings
            bass
            bassGuitar
            brass
            woodwinds
          }
          advancedInstrumentTags
        }
      }
    }
  }
}

Voiceover​

The new fields available are:

  • voiceoverDegree - a number showing the certainty that there is a voice over the track
  • voiceoverExists - a boolean value indicating the existence of the voiceover in this track (same as voiceoverDegree > 0.5, just already calculated for convenience)

Example query​

libraryTrack(id: "9411599") {
  ... on LibraryTrackNotFoundError {
    message
  }
  ... on LibraryTrack {
    id
    audioAnalysisV6 {
      ... on AudioAnalysisV6Finished {
        result {
          voiceoverExists
          voiceoverDegree
        }
      }
    }
  }
}

Free Genre Tags​

This new field is a list of strings containing text genre descriptions for the track. The difference from the usual genre and subgenre fields we provide is that it does not follow a fixed taxonomy and can therefore cover a broader range of musical genre description. Due to its text generator nature, no scores are returned for the tags given

Example query​

libraryTrack(id: "9411599") {
  ... on LibraryTrackNotFoundError {
    message
  }
  ... on LibraryTrack {
    id
    audioAnalysisV6 {
      ... on AudioAnalysisV6Finished {
        result {
          freeGenreTags
        }
      }
    }
  }
}